Text and image fused bimodal character classification method and device

A dual-mode, personality technology, applied in the field of artificial intelligence, can solve problems such as the inability to design humorous personality robots

Active Publication Date: 2021-06-11
SUZHOU UNIV
View PDF6 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, the current personality analysis task focuses on predicting the individual's big five personality score by constructing a regression model. In real life, this coarse-grained and abstract big five personality system has limitations in industrial applications, such as the inability to design A robot that can display a humorous personality

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text and image fused bimodal character classification method and device
  • Text and image fused bimodal character classification method and device
  • Text and image fused bimodal character classification method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0065] The specific implementation manners of the present application will be further described in detail below in conjunction with the drawings and embodiments. The following examples are used to illustrate the present application, but not to limit the scope of the present application.

[0066] First, some terms involved in this application are introduced.

[0067] Bidirectional Transformer's Encoder (Bidirectional Encoder Representation from Transformers, BERT): It is a text pre-training model, which is currently the model with the widest range of tasks in the field of natural language processing (Natural Language Processing, NLP), and has achieved very good results in various tasks. Excellent results. BERT's network architecture uses a multi-layer Transformer structure, and its biggest feature is that it abandons the traditional Recurrent Neural Network (RNN) and Convolutional Neural Networks (CNN), and uses the Attention mechanism Converting the distance of two words at ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a text and image fused bimodal character classification method and device, and belongs to the technical field of artificial intelligence. The method comprises the following steps: inputting text data and image data into a pre-trained character classification network, and obtaining a character classification result, The character classification network comprises a feature extraction network, a contrast visual attention network and a contrast perception decoding network; a text feature extraction branch in the feature extraction network is used for extracting a word embedding vector of the text dataa text feature extraction branch in the feature extraction network, and an image feature extraction branch is used for extracting an image region vector of the image dataan image feature extraction branch; a basic visual attention branch in the contrast visual attention network is used for extracting an image object aligned with the text data and calculating aligned visual representation, and an inverse visual attention branch is used for extracting an image object not aligned with the text data and calculating non-aligned visual representation the inverse visual attention branch; and the comparison perception decoding network is used for predicting character categories. The problems that the classification performance is poor and cognitive difference information cannot be captured are solved.

Description

【Technical field】 [0001] The present application relates to a dual-mode character classification method and device that integrates text and images, and belongs to the technical field of artificial intelligence. 【Background technique】 [0002] Personality is a person's long-term stable attitude towards reality, which is generally gradually formed in the practice of social life. Personality has a complex static structure, which is mainly composed of four parts: attitude characteristics, will characteristics, emotional characteristics and rational characteristics, which are related to each other and restrict each other. Attitude characteristics refer to the characteristics of how individuals deal with the relationship with society, collective, work, labor, others and themselves, such as honesty, love for the motherland, sense of responsibility, hard work and so on. Will characteristics refer to the characteristics that individuals consciously adjust their own behavior, such as...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/20G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06V10/22G06N3/045G06F18/24G06F18/253G06F18/214Y02D10/00
Inventor 王晶晶高晓雅李寿山周国栋
Owner SUZHOU UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap