A medical image representation learning method and system based on multi-granularity world modeling

Abstract

The application provides a medical image representation learning method and system based on multi-granularity world modeling. The method first enhances the radiograph image into first and second enhanced views with spatial overlap, and inputs a visual transformer to extract basic patch features; then constructs multi-granularity anatomical representation through hierarchical aggregation; then drives the world model to perform anatomical structure modeling, anatomical layout modeling and domain change perception modeling tasks, infers the relative spatial coordinates across views by using the overlap ratio of the overlapping area features, and modulates the input features by using the granularity perception enhancement parameters; finally, the model parameters are optimized based on a joint loss function. The application can solve the technical problems of the prior art, such as lack of unified modeling of multi-level anatomical semantics of the radiograph image, insufficient cross-view spatial layout reasoning capability, and difficulty in maintaining anatomical consistency under domain change.

Images