The invention discloses a news customization-oriented multi-
modal abstract generation method, which comprises the following steps of: firstly, inputting pictures and text data into a BART-based multi-
modal code, realizing multi-
modal interaction in a coding process, meanwhile, adding an image-
text matching module to select a
sentence corresponding to each picture, and adding a cross attention layer in a decoder; then, constructing an abstract scoring model, and selecting an optimal abstract by calculating the similarity between the candidate abstract and the news text as a
score; and calculating a ROUGE
score between the
sentence corresponding to the picture and the optimal abstract as a text similarity
score, obtaining an image-text similarity score in combination with the picture features after multi-modal interaction, obtaining a picture selection probability, selecting the picture with the highest probability as a
model selection picture, and generating a multi-modal news abstract. According to the
system, a B / S framework is adopted, a lightweight Web framework Flask is used for building, and a Layui
open source framework is used for building an interactive interface. According to the method, the accuracy and richness of news multi-modal abstract generation are effectively improved, and the robustness is high.