The invention belongs to the technical field of travel
itinerary planning, and particularly discloses a travel
itinerary planning method and
system based on multi-
modal data, a terminal and a medium. Comprising the following steps: collecting text data,
voice data and image
data input by a user, and generating a multi-
modal feature vector; inputting the multi-
modal feature vector into a semantic analysis model, and outputting a semantic analysis result; executing corresponding
processing based on the semantic analysis result; when the semantic analysis result is a question and answer request, generating a question and answer result based on
semantic matching and knowledge reasoning; when the semantic analysis result is an accompanying interaction request, generating interaction content and driving a virtual accompanying role to output; when the semantic analysis result is an
itinerary planning request, constructing a constraint optimization model and executing
path search, and outputting at least one itinerary planning scheme;
user feedback information is received in the whole
service process, and the scheme is dynamically adjusted and iteratively updated. The problems that in the prior art,
route planning and interaction are dispersed, and anthropomorphic accompanying service cannot be simulated are solved.