Medical image semantic segmentation method and device, electronic equipment and storage medium

Abstract

The invention provides a medical image semantic segmentation method and device, electronic equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: extracting a first image feature corresponding to a first image and a plurality of second image features corresponding to at least one second image through a feature extraction network; performing feature cross alignment based on a mean value of the first image feature and the plurality of second image features to obtain a plurality of cross features; based on the similarity between the first image feature and the plurality of second image features, optimizing the plurality of cross features to obtain a plurality of optimized features; and fusing the first image feature, the plurality of optimization features and the mask matched with each optimization feature through a decoder to obtain a mask result corresponding to the first image, so as to improve the accuracy of semantic segmentation of the rare disease medical image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for semantic segmentation of medical images. Background Art

[0002] With the continuous advancement of medical imaging technology, image semantic segmentation technology is increasingly used in the field of medical diagnosis. This technology can automatically identify and segment different tissues and organs in medical images, thereby helping doctors to diagnose diseases and formulate treatment plans more accurately. In the prior art, image semantic segmentation algorithms usually rely on a large amount of labeled data to train the model in order to achieve high-precision segmentation effects. However, in the case of rare diseases, due to the limited number of cases, the available medical image samples are also relatively small, which makes it difficult for existing image semantic segmentation technologies to obtain sufficient training data, which in turn affects their performance in the diagnosis of rare diseases. Therefore, under the existing technical conditions, the image semantic segmentation effect is often unsatisfactory when processing medical images related to rare diseases. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a medical image semantic segmentation method, device, electronic device and storage medium to improve the accuracy of semantic segmentation of medical images of rare diseases.

[0004] In a first aspect, the present invention provides a method for semantic segmentation of medical images, the method comprising extracting a first image feature corresponding to a first image and a plurality of second image features corresponding to at least one second image respectively through a feature extraction network, wherein the second image is an image of a target patient taken during a historical medical visit and semantically segmented for a medical semantic label, and the first image is an image of a pixel area with a medical semantic label currently taken by the target patient; performing feature cross-alignment based on the mean of the first image feature and the plurality of second image features to obtain a plurality of cross-features; optimizing the plurality of cross-features based on the similarity between the first image feature and the plurality of second image features to obtain a plurality of optimized features; fusing the first image feature, the plurality of optimized features, and a mask matching each optimized feature through a decoder to obtain a mask result corresponding to the first image, the mask result being used to indicate a target pixel area belonging to a medical semantic label and a non-target pixel area not belonging to a medical semantic label in the first image; wherein, for the first image feature and each second image feature, the corresponding cross-feature is obtained in the following manner: For the first element value at each position in the first image feature, the product of the first element value and the second element value at the position of the second image feature is calculated as the third element value at the corresponding position of the intersection feature.

[0005] In an optional embodiment, the feature extraction network includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a fifth convolution block connected in sequence, wherein each convolution block includes a first convolution layer, an asymmetric convolution layer, and a second convolution layer connected in sequence.

[0006] In an optional implementation, for the first image feature and each second image feature, the corresponding cross feature is optimized in the following manner to obtain the corresponding optimized feature: Based on the first image feature and the current second image feature, a first element value distance feature is determined; based on the first image feature and the remaining second image features, a second element value distance feature is determined; for the third element value at each position in the current cross feature, based on the first distance value of the first element value distance feature at that position and the second distance value of the second element value distance feature at that position, it is determined whether the third element value at that position needs to be optimized; if necessary, the third element value is modified based on the second element value at that position in the current second image feature to serve as the fourth element value at that position of the corresponding optimized feature.

[0007] In an optional implementation, whether the third element value at each position needs to be optimized is determined in the following manner: Determine whether the first distance value corresponding to the position is greater than the first target value and whether the second distance value corresponding to the position is less than the second target value; if the first distance value is greater than the first target value and the second distance value is less than the second target value, determine that the third element value needs to be optimized.

[0008] In an optional implementation, the fourth element value is calculated in the following manner: The product of the third element value at the corresponding position in the current cross feature and the magnification factor is used as the fourth element value.

[0009] In an optional implementation, the first element value distance feature is determined in the following manner: Decompose the first element value at each position of the first image feature and the second element value at the corresponding position of the current second image feature, and obtain the first integer element value and the first decimal element value corresponding to the first element value, and the second integer element value and the second decimal element value corresponding to the second element value; calculate the first difference between the first integer element value and the second integer element value; calculate the second difference between the first decimal element value and the second decimal element value; calculate the sum of the absolute value of the first difference and the absolute value of the second difference as the first distance value at the corresponding position in the first element value distance feature.

[0010] In an optional implementation, the second element distance feature is determined in the following manner: For the second element values ​​at the corresponding positions of the remaining second image features, the second element mean at that position is calculated; for the first element value at each position of the first image feature and the second element mean at the corresponding position, the cosine similarity is calculated as the second distance value at the corresponding position in the second element distance feature.

[0011] In a second aspect, the present invention provides a medical image semantic segmentation device, the device comprising: An extraction module, used to extract, through a feature extraction network, first image features corresponding to a first image and multiple second image features corresponding to at least one second image, respectively, wherein the second image is an image of a target patient taken during a historical medical visit and semantically segmented for medical semantic labels, and the first image is an image of a pixel area of ​​the target patient currently taken and having medical semantic labels; An alignment module is used to perform feature cross alignment based on the first image feature and the mean of the plurality of second image features to obtain a plurality of cross features; wherein, for the first image feature and each second image feature, the corresponding cross feature is obtained by: For the first element value at each position in the first image feature, calculate the product of the first element value and the second element value at the position of the second image feature as the third element value at the corresponding position of the cross feature; An optimization module, configured to optimize the plurality of cross features respectively based on the similarity between the first image feature and the plurality of second image features to obtain a plurality of optimized features; A decoding module is used to fuse the first image feature, multiple optimized features and the mask matching each optimized feature through a decoder to obtain a mask result corresponding to the first image, and the mask result is used to indicate the target pixel area belonging to the medical semantic label and the non-target pixel area not belonging to the medical semantic label in the first image.

[0012] In a third aspect, the present invention provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of any medical image semantic segmentation method as described in the aforementioned implementation manner.

[0013] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the medical image semantic segmentation method in any of the aforementioned embodiments are executed.

[0014] The present application provides a medical image semantic segmentation method, device, electronic device and storage medium, which take the segmented images and unsegmented images of the patient with the same medical semantic labels as input, perform feature extraction, align and optimize the extracted features, further enhance the description effect of some features on the medical semantic labels, and finally decode to obtain the segmentation results of the patient's medical images, thereby improving the accuracy of medical semantic segmentation in the diagnosis and treatment of rare diseases and providing assistance for the diagnosis and treatment of patients with rare diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart of a medical image semantic segmentation method provided in an embodiment of the present application; Figure 2 A flowchart of a feature optimization step provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a medical image semantic segmentation device provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] First, the application scenario of the present application is described. The technical solution of the present application can be applied to image semantic segmentation in medical diagnosis and treatment.

[0018] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0019] Embodiment 1 In the diagnosis and treatment of patients with rare diseases, due to the complexity and individual differences of rare diseases and the scarcity of case samples, if the model is trained and constructed in a conventional way, the segmentation effect is often poor. Therefore, in one embodiment of the present application, medical images of the same patient are used, which can be MRI, CT and other medical images.

[0020] Taking the medical semantic label of tumor as an example, the MRI images taken during the patient's historical medical treatment process are first processed to generate the corresponding mask.

[0021] Figure 1 A flowchart of a medical image semantic segmentation method provided in an embodiment of the present application. Figure 1As shown, the embodiment of the present application provides a medical image semantic segmentation method, which can be executed by a computer device, and the method includes: S1. Extracting first image features corresponding to a first image and multiple second image features corresponding to at least one second image through a feature extraction network.

[0022] Among them, the second image can be a brain MRI image of the target patient taken during the historical medical treatment process and semantically segmented for medical semantic labels, and the first image is a brain MRI image of the target patient currently taken with pixel areas having medical semantic labels.

[0023] Here, a feature extraction network based on Resnet is used to extract features of the first image and the second image.

[0024] Specifically, the feature extraction network includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a fifth convolution block connected in sequence, wherein each convolution block includes a first convolution layer, an asymmetric convolution layer, and a second convolution layer connected in sequence. The convolution kernel of the first convolution block is 3×3×3, the convolution kernel of the second convolution block is 3×1×1, the convolution kernel of the third convolution block is 1×3×1, the convolution kernel of the fourth convolution block is 1×1×3, and the convolution kernel of the fifth convolution block is 1×1×1. The sizes of the first convolution layer and the second convolution layer are 3×3×3 and 1×1×1, respectively.

[0025] Here, the first image feature and the second image feature can be the features output by the fifth convolutional block. Taking the second image as an example, the first image features can be obtained respectively. , and the second image feature and .

[0026] Furthermore, in order to improve the accuracy of semantic segmentation, the features output by the third convolution block, the fourth convolution block, and the fifth convolution block can be extracted as the corresponding first image features or second image features.

[0027] S2. Perform feature cross alignment based on the first image feature and the mean of the plurality of second image features to obtain a plurality of cross features.

[0028] For the first image feature and each second image feature, the corresponding cross feature is obtained in the following manner: For the first element value at each position in the first image feature, the product of the first element value and the second element value at the position of the second image feature is calculated as the third element value at the corresponding position of the intersection feature.

[0029] First image feature and the second image feature For example, in step S2, the Hadamard product method can be used to perform feature crossover.

[0030] Calculate the first image feature The first element value at (1, 1, 1) With the second image feature The second element value at (1, 1, 1) The product between them is used as the cross feature The third element value at (1, 1, 1) in . And so on, we finally get the cross feature .

[0031] By crossing the features of the medical image to be semantically segmented with the medical image that has been semantically segmented, the description effect of some features on the medical semantic labels can be enhanced.

[0032] S3. Based on the similarity between the first image feature and the plurality of second image features, optimize the plurality of cross features respectively to obtain a plurality of optimized features.

[0033] like Figure 2 As shown, the first image feature and the second image feature For example, the corresponding cross features can be optimized in the following way to obtain the corresponding optimized features: S100: Determine a first element value distance feature based on a first image feature and a current second image feature.

[0034] In step S100, the first image feature and the second image feature For example, the corresponding first element value distance feature can be determined in the following way: The first element value at each position of the first image feature and the second element value at the corresponding position of the current second image feature are decomposed to obtain the first integer element value and the first decimal element value corresponding to the first element value, and the second integer element value and the second decimal element value corresponding to the second element value, respectively.

[0035] The first image feature The first element value at (1, 1, 1) Decomposes into the integer part as the first integer element value 1 and the fractional part as the first fractional element value 0.36.

[0036] The second image feature The second element value at (1, 1, 1) Decomposed into the integer part as the second integer element value 0 and the decimal part as the second decimal element value 0.51.

[0037] A first difference between the first integer element value and the second integer element value is calculated. A second difference between the first decimal element value and the second decimal element value is calculated. A sum of the absolute value of the first difference and the absolute value of the second difference is calculated as a first distance value at a corresponding position in the first element value distance feature.

[0038] Calculate the first distance value at (1, 1, 1) in the first element value distance feature By analogy, we get .

[0039] Here, Manhattan distance is used to calculate the pixel-level similarity between the first image feature and the second image feature, which can reduce the sensitivity to outliers and better capture the similarity between pixels.

[0040] S101. Determine a second element value distance feature based on the first image feature and the remaining second image features.

[0041] First image feature and the second image feature , For example, the corresponding second element distance feature can be determined in the following way: For the second element values ​​at the positions corresponding to the remaining second image features, the second element mean at the positions is calculated.

[0042] calculate , The second element value at (1, 1, 1) , The mean between is taken as the mean of the second element at (1, 1, 1) .

[0043] The cosine similarity is calculated for the first element value at each position of the first image feature and the second element mean value at the corresponding position to serve as the second distance value at the corresponding position in the second element distance feature.

[0044] Calculate the first image feature The first element value at (1, 1, 1) The second distance value at (1, 1, 1) in the distance feature of the second element is calculated using cosine similarity with the mean of the second element. : .

[0045] And so on, traversing all positions, we can finally get .

[0046] The distance feature of the second element value calculated by cosine similarity can reflect the directional consistency between pixels in multidimensional space and pay more attention to semantic similarity.

[0047] S102: For the third element value at each position in the current cross feature, determine whether the third element value at the position needs to be optimized based on the first distance value of the first element value from the feature at the position and the second distance value of the second element value from the feature at the position. Specifically, whether the third element value at each position needs to be optimized can be determined in the following manner: Determine whether the first distance value corresponding to the position is greater than the first target value and whether the second distance value corresponding to the position is less than the second target value. If the first distance value is greater than the first target value and the second distance value is less than the second target value, it is determined that the third element value needs to be optimized.

[0048] For a pixel feature at a certain position, if the corresponding first distance value is greater than the first target value and the second distance value is less than the second target value, the third element value is optimized to better capture the similarity of the image features.

[0049] S103: If necessary, modify the third element value based on the second element value at the position in the current second image feature to serve as the fourth element value at the position of the corresponding optimized feature.

[0050] The product of the third element value at the corresponding position in the current cross feature and the magnification coefficient is used as the fourth element value.

[0051] The amplification factor here can be preset, and the value range of the amplification factor is between 0 and 1.

[0052] In this way, optimizing the element values ​​in the features can enhance the direction of the cross features to the semantic segmentation elements to obtain more accurate semantic segmentation results.

[0053] S4. The first image feature, multiple optimized features, and the mask matching each optimized feature are fused through a decoder to obtain a mask result corresponding to the first image. The mask result is used to indicate the target pixel area belonging to the medical semantic label and the non-target pixel area that does not belong to the medical semantic label in the first image.

[0054] In step S4, a TransUnet decoder may be used. It not only provides spatial information retention capabilities, but also incorporates a Transformer mechanism to enhance global context understanding. The mask result corresponding to the first image output by the decoder can indicate the pixel portions in the image that belong to the tumor and the pixel portions that do not belong to the tumor.

[0055] A medical image semantic segmentation method provided by the present application takes both segmented images and unsegmented images of patients with the same medical semantic labels as input, performs feature extraction, aligns and optimizes the extracted features, further enhances the description effect of some features on the medical semantic labels, and finally decodes to obtain the segmentation results of the patient's medical images, thereby improving the accuracy of medical semantic segmentation in the diagnosis and treatment of rare diseases and providing assistance for the diagnosis and treatment of patients with rare diseases.

[0056] Embodiment 2 In one embodiment of the present application, a training step of a medical image semantic segmentation model is provided, specifically comprising: Obtain brain MRI images of the same patient taken at different times, with the latest image taken as the first image and the remaining images as the second images. Obtain masks corresponding to the first and second images. In the mask of the second image, the tumor part can be shown as white and the other parts can be shown as black.

[0057] The first image and the second image are subjected to feature extraction through a feature extraction network to obtain first image features and a corresponding number of second image features.

[0058] Based on the mean of the first image feature and the plurality of second image features, feature cross alignment is performed to obtain a plurality of cross features. Based on the similarity between the first image feature and the plurality of second image features, the plurality of cross features are optimized respectively to obtain a plurality of optimized features.

[0059] The first image features, multiple optimized features, and masks matching each optimized feature are fused through the decoder so that the decoder outputs a mask result corresponding to the first image. The mask result is used to indicate the target pixel area belonging to the medical semantic label and the non-target pixel area that does not belong to the medical semantic label in the first image.

[0060] Based on the deviation between the predicted mask result and the actual mask result of the first image, a loss function is calculated to adjust the parameters in the model, and finally a trained medical image semantic segmentation model is generated.

[0061] Embodiment 3 like Figure 3 As shown, based on the same inventive concept, an embodiment of the present application further provides a medical image semantic segmentation device, the device 30 comprising: An extraction module 310 is used to extract, through a feature extraction network, first image features corresponding to a first image and multiple second image features corresponding to at least one second image, respectively, wherein the second image is an image of a target patient taken during a historical medical visit and semantically segmented for a medical semantic label, and the first image is an image of a pixel area of ​​the target patient currently taken and having a medical semantic label; The alignment module 320 is used to perform feature cross alignment based on the mean of the first image feature and the plurality of second image features to obtain a plurality of cross features; wherein, for the first image feature and each second image feature, the corresponding cross feature is obtained by: For the first element value at each position in the first image feature, calculate the product of the first element value and the second element value at the position of the second image feature as the third element value at the corresponding position of the cross feature; An optimization module 330, configured to optimize the plurality of cross features respectively based on the similarities between the first image feature and the plurality of second image features to obtain a plurality of optimized features; The decoding module 340 is used to fuse the first image feature, multiple optimized features and the mask matching each optimized feature through a decoder to obtain a mask result corresponding to the first image, and the mask result is used to indicate the target pixel area belonging to the medical semantic label and the non-target pixel area that does not belong to the medical semantic label in the first image.

[0062] In a preferred embodiment, the feature extraction network includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a fifth convolution block connected in sequence, wherein each convolution block includes a first convolution layer, an asymmetric convolution layer, and a second convolution layer connected in sequence.

[0063] In a preferred embodiment, for the first image feature and each second image feature, the optimization module 330 optimizes the corresponding cross features in the following manner to obtain the corresponding optimized features: Based on the first image feature and the current second image feature, a first element value distance feature is determined; based on the first image feature and the remaining second image features, a second element value distance feature is determined; for the third element value at each position in the current cross feature, based on the first distance value of the first element value distance feature at that position and the second distance value of the second element value distance feature at that position, it is determined whether the third element value at that position needs to be optimized; if necessary, the third element value is modified based on the second element value at that position in the current second image feature to serve as the fourth element value at that position of the corresponding optimized feature.

[0064] In a preferred embodiment, the optimization module 330 determines whether the third element value at each position needs to be optimized in the following manner: Determine whether the first distance value corresponding to the position is greater than the first target value and whether the second distance value corresponding to the position is less than the second target value; if the first distance value is greater than the first target value and the second distance value is less than the second target value, determine that the third element value needs to be optimized.

[0065] In a preferred embodiment, the optimization module 330 calculates the fourth element value in the following manner: The product of the third element value at the corresponding position in the current cross feature and the magnification factor is used as the fourth element value.

[0066] In a preferred embodiment, the optimization module 330 determines the first element value distance feature in the following manner: Decompose the first element value at each position of the first image feature and the second element value at the corresponding position of the current second image feature, and obtain the first integer element value and the first decimal element value corresponding to the first element value, and the second integer element value and the second decimal element value corresponding to the second element value; calculate the first difference between the first integer element value and the second integer element value; calculate the second difference between the first decimal element value and the second decimal element value; calculate the sum of the absolute value of the first difference and the absolute value of the second difference as the first distance value at the corresponding position in the first element value distance feature.

[0067] In a preferred embodiment, the optimization module 330 determines the second element distance feature by: For the second element values ​​at the corresponding positions of the remaining second image features, the second element mean at that position is calculated; for the first element value at each position of the first image feature and the second element mean at the corresponding position, the cosine similarity is calculated as the second distance value at the corresponding position in the second element distance feature.

[0068] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0069] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 The steps of a medical image semantic segmentation method in the method embodiment shown are specifically implemented in accordance with the method embodiment and will not be described in detail here.

[0070] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of a medical image semantic segmentation method in the method embodiment shown are specifically implemented in accordance with the method embodiment and will not be described in detail here.

[0071] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0072] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0073] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0074] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0075] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0076] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0077] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A medical image semantic segmentation method, characterized in that: The method comprises: A first image feature corresponding to the first image and multiple second image features corresponding to at least one second image are respectively extracted through a feature extraction network, wherein the second image is an image of the target patient taken during a historical medical visit and semantically segmented for a medical semantic label, and the first image is an image of a pixel area of ​​the target patient currently taken and having the medical semantic label; Performing feature cross alignment based on the first image feature and the mean of the plurality of second image features to obtain a plurality of cross features; Based on the similarity between the first image feature and the plurality of second image features, respectively optimizing the plurality of cross features to obtain a plurality of optimized features; The first image feature, the multiple optimized features, and the mask matched by each optimized feature are fused through a decoder to obtain a mask result corresponding to the first image, wherein the mask result is used to indicate a target pixel area belonging to the medical semantic label and a non-target pixel area not belonging to the medical semantic label in the first image; Wherein, for the first image feature and each second image feature, the corresponding cross feature is obtained by: For the first element value at each position in the first image feature, the product of the first element value and the second element value at the position of the second image feature is calculated as the third element value at the corresponding position of the intersection feature.

2. The method according to claim 1, characterized in that The feature extraction network includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a fifth convolution block connected in sequence, wherein each convolution block includes a first convolution layer, an asymmetric convolution layer, and a second convolution layer connected in sequence.

3. The method according to claim 1, characterized in that For the first image feature and each second image feature, the corresponding cross features are optimized in the following manner to obtain corresponding optimized features: Determining a first element value distance feature based on the first image feature and the current second image feature; Determining a second element value distance feature based on the first image feature and the remaining second image features; For the third element value at each position in the current cross feature, determine whether the third element value at the position needs to be optimized based on a first distance value from the first element value to the feature at the position and a second distance value from the second element value to the feature at the position; If necessary, the third element value is modified based on the second element value at the position in the current second image feature to serve as the fourth element value at the position of the corresponding optimized feature.

4. The method according to claim 3, characterized in that Determine whether the third element value at each position needs to be optimized by: Determine whether a first distance value corresponding to the position is greater than a first target value and whether a second distance value corresponding to the position is less than a second target value; If the first distance value is greater than a first target value and the second distance value is less than a second target value; It is determined that the third element value needs to be optimized.

5. The method according to claim 3, characterized in that: The fourth element value is calculated as follows: The product of the third element value at the corresponding position in the current cross feature and the magnification coefficient is used as the fourth element value.

6. The method according to claim 3, characterized in that The first element value distance feature is determined in the following way: Decomposing the first element value at each position of the first image feature and the second element value at the position corresponding to the current second image feature, respectively obtaining a first integer element value and a first decimal element value corresponding to the first element value, and a second integer element value and a second decimal element value corresponding to the second element value; calculating a first difference between the first integer element value and the second integer element value; Calculating a second difference between the first fractional element value and the second fractional element value; The sum of the absolute value of the first difference and the absolute value of the second difference is calculated as the first distance value at the corresponding position in the first element value distance feature.

7. The method according to claim 3, characterized in that The second element distance feature is determined by: For the second element values ​​at the positions corresponding to the remaining second image features, calculate the second element mean at the positions; The cosine similarity is calculated for the first element value at each position of the first image feature and the second element mean value at the corresponding position to serve as the second distance value at the corresponding position in the second element distance feature.

8. A medical image semantic segmentation device, characterized in that: The device comprises: An extraction module, used to extract, through a feature extraction network, first image features corresponding to a first image and multiple second image features corresponding to at least one second image, respectively, wherein the second image is an image of a target patient taken during a historical medical visit and semantically segmented for a medical semantic label, and the first image is an image of a pixel area of ​​the target patient currently taken and having the medical semantic label; An alignment module is used to perform feature cross alignment based on the first image feature and the mean of the plurality of second image features to obtain a plurality of cross features; wherein, for the first image feature and each second image feature, the corresponding cross feature is obtained by: For the first element value at each position in the first image feature, calculate the product of the first element value and the second element value at the position of the second image feature as the third element value at the corresponding position of the cross feature; an optimization module, configured to optimize the plurality of cross features respectively based on the similarities between the first image feature and the plurality of second image features, so as to obtain a plurality of optimized features; A decoding module is used to fuse the first image feature, multiple optimized features and the mask matching each optimized feature through a decoder to obtain a mask result corresponding to the first image, and the mask result is used to indicate the target pixel area belonging to the medical semantic label and the non-target pixel area not belonging to the medical semantic label in the first image.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the medical image semantic segmentation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the medical image semantic segmentation method according to any one of claims 1 to 7 are executed.

Images