Technical solutions provide extended reality based capabilities for just-in-time context-based assistance to robotic medical
system users. A processor can receive a
data stream of a medical session with a robotic medical
system. The
data stream can be captured by sensors in a medical environment. The processor can determine, based on the
data stream and via a context-identification model trained with
machine learning, a context of the medical session. The processor can determine, based on the context, an extended reality content to be presented to a user. The processor can detect, based on the data
stream and via one or more object-identification models, one or more objects of the medical environment. The processor can establish, based on the objects detected, an
interaction field in the medical environment in which to interact with the extended reality content and present the extended reality content to the user for interaction in the
interaction field.