The invention provides an
auricle anaphora segmentation method and
system, and the method comprises the steps: inputting a to-be-segmented
human ear image and text description into an
auricle anaphora segmentation model which comprises a text coding module, a text-guided visual coding module and visual decoding module, and an angle transformation module; the text coding module obtains a text feature # imgabs0 #; a visual coding module realizes fusion of text features and image features through a structure organized into four stages, a visual
encoder in each stage generates visual features # imgabs1 #, a cross-
modal sensing module aligns # imgabs2 # and # imgabs3 # to obtain multi-
modal features # imgabs4 # and # imgabs5 #, each element in the multi-
modal features # imgabs4 # and # imgabs5 # is weighted by an attention gating module to obtain weighted multi-modal features # imgabs6 #, and the weighted multi-modal features # imgabs6 # and # imgabs5 # are weighted by an attention gating module. Adding the elements and # imgabs7 # to generate an enhanced visual feature # imgabs8 #, inputting # imgabs9 # into a visual decoding module to gradually recover the spatial resolution of the image, and further fusing the text and the visual feature to output a multi-scale feature; the angle transformation module carries out angle transformation on the multi-scale features and outputs a segmentation
mask of a text description related region. According to the invention, the
auricle can be subjected to anaphora segmentation.