Image-text double-coding implementation method and system based on CR2 neural network

A neural network and implementation method technology, applied in the field of image-text dual coding, can solve the problems of low level, difficult to obtain labeled data, stay in theoretical text expression, etc., to reduce the requirements of shooting technology, improve the matching efficiency, and meet the needs of use. The effect of experience

Active Publication Date: 2020-09-04
JINGGANGSHAN UNIVERSITY
View PDF6 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] (1) The existing method of image acquisition only considers the focal length of the image, so that the user cannot view the details of the overexposed part and / or dark part of the image; at the same time, it is difficult to obtain the captured image
[0005] (2) The manual screening method in the existing text acquisition methods is not efficient and cost-effective; the method of using supervised machine learning to evaluate the credibility of long texts is difficult to obtain labeled data, and the data and models of different platforms are difficult migrate
[0006] (3) Double coding theory is an important theory in cognitive science, but it only stays in the theoretical text expression, and the establishment of a complete mathematical model is still a difficult point

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image-text double-coding implementation method and system based on CR2 neural network
  • Image-text double-coding implementation method and system based on CR2 neural network
  • Image-text double-coding implementation method and system based on CR2 neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0078] The CR-based 2 The image-text double encoding method of neural network is as follows: figure 1 As shown, as a preferred embodiment, such as image 3 As shown, the method for obtaining an image represented by non-words in image information through an image information acquisition program provided by an embodiment of the present invention includes:

[0079] S201. Collect a plurality of image information in a predetermined area, and simultaneously collect location scene and time information corresponding to the images.

[0080] S202. Obtain target sample information, input the plurality of image information into the matching model that has been trained in advance, and match the image information with the target sample information to obtain the image information and the target sample information Information matching results.

[0081] S203, using the trained multi-layer convolutional neural network (CNN) to analyze the image information collected in the predetermined area...

Embodiment 2

[0091] The CR-based 2The implementation method of image-text double encoding of neural network is as follows: figure 1 shown, as Figure 4 As shown, as a preferred embodiment, the method for acquiring the text semantic encoding of the text information word representation through the text semantic acquisition program provided by the embodiment of the present invention includes:

[0092] S301. Obtain a natural language text related to the semantics of the text represented by the words of the text information, and perform text clipping, replacement of rare words, word segmentation, and keyword processing.

[0093] S302. Obtain training data with known long text, extract training features of the training data to construct a training feature vector set, perform unsupervised clustering on the training feature vector set, and obtain multiple training centroids.

[0094] S303. Obtain evaluation data from the long text to be evaluated through the text processing model, extract evalua...

Embodiment 3

[0101] The CR-based 2 The implementation method of image-text double encoding of neural network is as follows: figure 1 shown, as Figure 5 As shown, as a preferred embodiment, the method for associating image information and text semantic coding information through the image-text association program provided by the embodiment of the present invention includes:

[0102] S401, perform feature extraction and expression for image and text semantics, and respectively obtain feature vector spaces of semantic primitives in separate modalities of text and image.

[0103] S402, using each region in the image as a node, using various relationships between nodes as edges, and constructing an image-text semantic association model through an RBF self-increasing neural network.

[0104] S403, build a learning algorithm on the graph, and effectively spread the text semantic information corresponding to the image level to these image regions, forming a large number of semantic basic unit s...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of image-text dual coding. The invention discloses an image-text double-coding implementation method and system based on a CR2 neural network. The image-text dual-coding implementation system based on the CR2 neural network comprises an information input module, an image acquisition module, a text acquisition module, a central control module, an image-text association module, a performance test module, a data storage module and a display module. According to the system, three neural networks of CNN, RNN and RBF are utilized to form a CR2 neural network; and the system is based on the relevance between large-scale image data and text data on the Internet. Image-text double coding based on the CR2 neural network is achieved, a computer automatically learns and autonomously obtains a semantic basic concept describing the objective world, and generation of the semantic basic concept plays an important role in network content retrieval, semanticunderstanding, knowledge representation and other applications.

Description

technical field [0001] The invention belongs to the technical field of image-text double encoding, in particular to a CR-based 2 A neural network image-text double encoding method and system. Background technique [0002] At present, dual coding is a cognitive theory proposed by psychologist Pevio in 1971, which emphasizes that language and non-language information processing are equally important in the storage, processing and retrieval of information. There are two subsystems in human cognition, one is dedicated to the representation and processing of non-verbal things and events (ie images), the representational system; the other is used for language processing, the semantic system. These two subsystems are parallel to each other and interrelated. Pevio also postulates that there are two different representational units: the "picture unit" in the representational system suitable for the representation of mental images and the "linguistic unit" in the semantic system sui...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/30G06F40/289G06T9/00G06K9/62G06N3/04G06N3/08
CPCG06F40/30G06F40/289G06T9/002G06N3/08G06N3/045G06F18/23213G06F18/24Y02D10/00
Inventor 尹观海方燕红
Owner JINGGANGSHAN UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap