Multi-modal model style embedding method and system for complex electric power visual scene

Through joint learning of CLIP model and diffusion model, a domain-level style information database is built, and combined with the instance-level style feature extraction model, the problem of insufficient style portrayal ability of visual language pre-trained models in complex power visual scenes is solved, and the style is significantly improved chemistry and expression ability.

CN119295895AActive Publication Date: 2025-01-10STATE GRID ANHUI ELECTRIC POWER CO LTD ELECTRIC POWER SCI RES INST +1
View PDF 4 Cites 1 Cited by

Patent Information

Application Number
CN202411814814.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-12-11
Publication Date
2025-01-10
Estimated Expiration
2044-12-11

AI Technical Summary

Technical Problem

The existing visual language pre-trained models have weak style portrayal capabilities in complex electric visual scenes, resulting in limited generalization performance and lack of effective domain and style information modeling.

Method used

The multimodal model style embedding method is adopted, and a domain-level style information database is built through joint learning of CLIP model and diffusion model, and an instance-level style feature extraction model is established to interact across modal knowledge, and style expression ability is enhanced.

Benefits of technology

It realizes rich style expression and modeling capabilities in complex electric visual scenes, improves the style generalization performance of visual text pre-trained models, and is suitable for defect recognition and object detection tasks in electric real scenes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119295895A_ABST
    Figure CN119295895A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-modal model style embedding method and system for a complex electric power visual scene, belongs to the technical field of image processing, and solves the problem of how to enhance the style expression ability of a visual language pre-training model. According to the method, the diffusion model is utilized to assist the multi-modal model to establish the field-level style cue word of the continuous space, knowledge of the diffusion model is migrated to the multi-modal model, richer style expression and modeling ability are achieved when facing a complex electric power visual scene, and more detailed style information can be described; according to the method, the instance-level style feature extraction model is established, the instance-level style feature extraction model is aligned with the style information of the field-level style information base, efficient and accurate style cue words can be generated for single image input in the reasoning process, and the method is suitable for tasks such as defect recognition and target detection in a real power scene and has high practicability. And the style generalization performance of the visual text pre-training model in the downstream task is effectively enhanced.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Method and device for assisting in improving image classification by using multi-modal model

    CN118072104A

  • Power equipment anomaly identification method, device and equipment based on multi-modal model

    CN118228200A

  • Method and device for improving generalization ability of cross-modal image retrieval model and medium

    CN118467765A

  • Machine learning diffusion model with image encoder trained for synthetic image generation

    US20240282016A1

Cited By

  • Migration method of multi-modal pre-training model and related product

    CN120162741A