Self-adaptive relation modeling method for structured data

A technology of structured data and modeling methods, applied in neural learning methods, neural architecture, character and pattern recognition, etc.

Pending Publication Date: 2021-07-30
BEIJING INSTITUTE OF TECHNOLOGYGY
View PDF1 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Despite their powerful predictive capabilities, many black-box models such as DNNs model their inputs in implicit ways that are convoluted and may sometimes learn unexpected patterns

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Self-adaptive relation modeling method for structured data
  • Self-adaptive relation modeling method for structured data
  • Self-adaptive relation modeling method for structured data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0041] Embodiment 1 implements an adaptive relational modeling method for structured data described in the present invention, which can capture the intersection features between different attributes of input tuples, specifically including the following:

[0042] Use exponential neurons to model the intersection features between the structured data attributes, the index neurons are K×o, where K represents the number of attention heads, and o represents the index neurons of each attention head The number of , K and o are natural numbers; all said exponential neurons of each attention head share its bilinear attention function φ att The weight matrix W att ;

[0043] The i-th index neuron y of each attention head i Expressed as follows:

[0044]

[0045]

[0046] Among them, i, ⊙ represent Hadamard product, exp( ) function and corresponding exponent w ij apply element-wise, e j Represents the embedding vector corresponding to the jth attribute value of the structured d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a self-adaptive relation modeling method for structured data, and belongs to the technical field of artificial intelligence prediction. K*o exponential neurons are used for modeling a feature interaction relation between structured data attributes, K is the number of attention heads, and o is the number of exponential neurons of each attention head; all exponential neurons of each attention head share a bilinear attention function weight matrix W[att]; and the i exponential neuron yi of each attention head is expressed as formulas defined in the specification, a dynamic feature interaction weight w[i] is given by a formula w[i]=z[i]*v[i], z[i] is used as a gate, is dynamically generated through bilinear attention alignment scores and is expressed as formulas defined in the specification, alpha-entmax(.) is sparse softmax, and the sparsity is increased along with the increase of alpha. Compared with the prior art, the method has the advantages that input is not needed and must be positive, and the effectiveness of cross feature modeling is improved; the white box design is followed, and the modeling process is more transparent; the cross feature of any order are dynamically and selectively modeled according to the input data, so that the accuracy and the efficiency are improved; the global / local interpretability of the model is improved through the attention weight and the gating mechanism.

Description

technical field [0001] The invention relates to a data processing method, in particular to an adaptive relational modeling method for structured data, and belongs to the technical field of artificial intelligence learning and prediction. Background technique [0002] To date, most businesses have relied on structured data for data storage and predictive analytics. Relational database management system (RDBMS) has become the mainstream database system adopted by the industry. Relational database has become the standard for actually storing and querying structured data, and structured data is crucial to the operation of most businesses. Structured data often contains a large amount of information, which can often be used to make data-driven decisions or identify risks and opportunities. Extracting insights from data for decision-making requires advanced analytics, especially deep learning, which is much more complex than statistical aggregation. [0003] Deep neural networks...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/62G06N3/04G06N3/08
CPCG06N3/04G06N3/08G06F18/2415G06F18/214
Inventor 蔡少峰郑凯平陈刚张美慧
Owner BEIJING INSTITUTE OF TECHNOLOGYGY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap