Patents
Literature
Hiro is an intelligent assistant for R&D personnel, combined with Patent DNA, to facilitate innovative research.
Hiro

39 results about "Kullback–Leibler divergence" patented technology

In mathematical statistics, the Kullback–Leibler divergence (also called relative entropy) is a measure of how one probability distribution is different from a second, reference probability distribution. Applications include characterizing the relative (Shannon) entropy in information systems, randomness in continuous time-series, and information gain when comparing statistical models of inference. In contrast to variation of information, it is a distribution-wise asymmetric measure and thus does not qualify as a statistical metric of spread (it also does not satisfy the triangle inequality). In the simple case, a Kullback–Leibler divergence of 0 indicates that the two distributions in question are identical. In simplified terms, it is a measure of surprise, with diverse applications such as applied statistics, fluid mechanics, neuroscience and machine learning.

Vehicle-mounted CAN bus network abnormity detection method and system

The invention, which belongs to the technical field of vehicle-mounted network, discloses a vehicle-mounted CAN bus network abnormity detection method and system. CAN bus abnormity detection based on a relative entropy is performed on an identifier ID; a sliding window with a fixed message number is employed; messages are paired based on a relationship between a message sensing sequence and a sending number, relative entropies of the paired messages and relative entropies of all IDs and normal distribution are calculated, and whether abnormity occurs is determined based on the two kinds of relative entropies; a replay attack and a denial of service attack are detected; CAN bus network abnormity detection based on a message data domain is performed on a data domain; features, including a constant value feature, a cyclic value feature, and a multi-value feature, of the message data domain are extracted; and a normal message model is established based on the extracted features and the message abnormity is detected. Therefore, the replay attack, the denial of service attack, the tampering attack and the forgery attack can be detected effectively and efficiently; more abnormal information is provided; and thus subsequent protection can be performed well.
Owner:XIDIAN UNIV

Personalized webpage recommendation method based on topic and relative entropy

The present invention discloses a personalized webpage recommendation method based on a topic and a relative entropy. According to the method, firstly, an LDA (latent dirichlet allocation) model is adopted to carry out topic mining on webpage content and user reading behaviors and to calculate a webpage semantic feature vector and a user interest feature vector based on the topic; and then a similarity measuring formula based on the concept of the relative entropy is utilized to calculate similarity between a webpage-to-be-recommended semantic feature vector and the user interest feature vector, and the obtained similarity is used as a decision basis for personalized webpage recommendation. According to the personalized webpage recommendation method based on the topic, a great deal of computing cost based on a collaborative filtering method is avoided; and meanwhile, the topic, instead of a keyword, is adopted to represent webpage content, and thus, the recommendation process and the recommendation results can more comprehensively and accurately reflect conceal information and deep semantic features of the webpage content.
Owner:SOUTHEAST UNIV

Ocular fundus image registration method estimated based on distance transformation parameter and rigid transformation parameter

The invention provides an ocular fundus image registration method estimated based on distance transformation parameter and rigid transformation parameter, comprising the following main steps: (1) ocular fundus images are loaded; (2) the optic disk center is extracted to estimate image translation parameters; (3) gradient vectors of pixel points in the adjacent zone of the optic disk are calculated, vessel segmentation is carried out, vessel distribution probability characteristics are calculated, and the estimation of image rotation parameters are obtained by minimizing two probability distribution relative entropies (Kullback-Leibler Divergence); (4) the Euclidean distance transformation of vessels segmented in step 3 is calculated; (5) accurate registration of images is carried out. The invention is a quick, precise, robust and automatic ocular fundus image registration algorithm, and has great application value on the aspect of ocular fundus image registration.
Owner:INST OF AUTOMATION CHINESE ACAD OF SCI

Automatic driving system and method based on relative-entropy deep and inverse reinforcement learning

The invention relates to an automatic driving system based on relative-entropy deep and inverse reinforcement learning. The system comprises a client, a driving basic data collection sub-system and astorage module, wherein the client displays a driving strategy; the driving basic data collection sub-system collects road information; the storage module is connected with the client and the drivingbasic data collection sub-system and stores the road information collected by the driving basic data collection sub-system. The driving basic data collection sub-system collects the road information and transmits the road information to the client and the storage module; the storage module receives the road information, stores a piece of continuous road information into a historical route, conducts analysis and calculation according to the historical route so as to simulate the driving strategy, and transmits the driving strategy to the client so that a user can select the driving strategy; the client receives the road information and implements automatic driving according to the selection of the user. In the automatic driving system, the relative-entropy deep and inverse reinforcement learning algorithm is adopted, so that automatic driving under the model-free condition is achieved.
Owner:POLIXIR TECH LTD

Image incremental learning method based on dynamic correction vector

The invention discloses an image incremental learning method based on a dynamic correction vector. The method comprises the following steps: S1, a backbone network with a ResNet-32 network layer structure as a model is constructed, an Adam training optimizer is adopted, and meanwhile, a basic classification loss function is a Kullback-Leibler Diverge relative entropy loss function; s2, knowledge distillation is introduced into a loss function, a classification loss function is combined, a new model is helped to learn knowledge in an old category, and the catastrophic forgetting problem is relieved; s3, a ResNet-32 model is trained in a training mode by adopting a representative memory method and a dynamic correction vector method; and S4, the optimal model trained in the previous incremental stage is reloaded for repeating the steps S2 to S3, and the performance on all test sets is evaluated until all incremental data is trained. According to the method, the incremental learning task recognition capability is improved, and the method has high practical value.
Owner:ZHEJIANG UNIV OF TECH

Sequential accelerated degradation test optimal design method based on relative entropy

The invention discloses a sequential accelerated degradation test optimal design method based on relative entropy, comprising the specific steps of: step 1, utilizing a Bayesian theory to establish an accelerated degradation test optimal design method based on the relative entropy; step 2, establishing a sequential truncation judging method; and step 3, carrying out sequential accelerated degradation test based on the relative entropy. According to the method disclosed by the invention, a 'sequential design' is introduced to an optimal design of the accelerated degradation test for the first time and the sequential accelerated degradation test optimal design method is provided. With the adoption of the 'sequential design', not only can prior information before the test be sufficiently utilized, but also performance degradation information obtained in the test is gradually utilized, and a test design error caused by that the deviation between the prior information and a product real condition is larger is reduced, so that compared with a partial optimal design and Bayesian optimal design of the accelerated degradation test, the sequential accelerated degradation test optimal design method based on the relative entropy has the greater advantage.
Owner:BEIHANG UNIV

Method and device for defending interest flooding attacks in information centric network

The invention provides a method and device for defending interest flooding attacks in an information centric network and relates to the field of network security. The method includes: statistically counting the entropy of the names of interest packets received by a router in the information centric network after different moments according to a preset window; using a cumulative sum algorithm to process the obtained entropy to obtain the accumulative values of the entropy at different moments; judging whether the accumulative values are smaller than a preset attack detecting threshold or not, if not, judging that interest flooding attacks are detected, and using a prefix determining algorithm based on relative entropy to search the prefix set of the names of the interest packets so as to obtain attack prefixes; generating data packets containing the attack prefixes according to the attack prefixes, and transmitting the data packets to the access router where an attacker is located according to the router information of the interest packets, containing the attack prefixes, recorded in the pending interest table of the router so as to allow the access router to perform corresponding access limitation on the received interest packets according to the attack prefixes in the data packets.
Owner:INST OF INFORMATION ENG CAS

Recurrent neural network language model training method and device, equipment and medium

ActiveCN108960407AGuaranteed accuracyRealize knowledge distillationNeural architecturesRecurrent neural network language modelsAlgorithm
The embodiment of the invention discloses a recurrent neural network language model (RNNLM) training method and device, equipment and a medium. The method includes: inputting language texts in the corpus into a trained high-rank NNLM and a to-be -rained lightweight RNNLM respectively; and carrying out iteration of parameters of the lightweight RNNLM and minimizing a weighted sum of a cross-entropyloss and Kullback-Leibler divergence to complete training of the lightweight RNNLM. The cross-entropy loss is one of the output vector of the lightweight RNNLM relative to a training data label vector; and the Kullback-Leibler divergence is one of the output vector of the lightweight RNNLM relative to the output vector of the high-rank RNNLM. Therefore, the RNNLM scale is reduced effectively.
Owner:MOBVOI INFORMATION TECH CO LTD

Prioritization of reports using content data change from baseline

The raw data for a plurality of numerical reports (distributions or histograms) concerning malware infection in a computer network are stored in a data source. The data source is queried to produce any number of reports. Each report's content comes from a distribution of data within a time interval, and a baseline distribution is formed for comparison by the corresponding historical data. The shape change for the distributions is determined by using Kullback-Leibler divergence. The change of volume (i.e., total sample count) for the distributions is determined using the L1 norm ratio. A cutoff threshold is determined for the K-L divergence and the volume ratio threshold is determined for the count change. A measure value for each report is determined by multiplying the shape change by the volume change (modified by raising it to a particular power). The reports are ranked based upon their measure values. A report is determined to be important if its shape change is greater than the cutoff threshold, if it's volume change is greater than the count ratio threshold, or if the measure value is greater than a measure threshold. The invention can be applied to all kinds of reports suitable for a distribution or histogram, and also provides one approach to detect anomalous behaviors.
Owner:TREND MICRO INC

Vehicle-mounted CAN bus anomaly detection method based on a relative entropy

The invention discloses a vehicle-mounted CAN bus anomaly detection method based on a relative entropy. The method comprises the following steps: after a vehicle is powered on in an interrupt processing program, starting fixed-time interrupt and fixed-time triggering of a detection node to realize the calculation of the relative entropy of a message, triggering an alarm if the calculated result ofthe relative entropy exceeds a set threshold, otherwise, executing the operations of counting data clearing and window switching; and during calculation of the relative entropy of the message, usingtwo variable-length time windows alternately, which are recorded as a window 1 and a window 2 respectively, performing growth, and performing counting with two independent counters. An anomaly detection algorithm based on the relative entropy is adopted and improved, thereby improving the detection accuracy and the detection granularity. Through combination of anomaly detection based on a white list and a time interval, a better detection effect is achieved.
Owner:SOUTHEAST UNIV

Behavior prediction apparatus and method

A behavior estimation apparatus inputs a prediction period, an annual consumption expenditure, extracts, from behavioral statistical data including an occurrence probability of each of behaviors of human during each time period, the occurrence probability of each behavior within the prediction period as a prior probability distribution, obtains a consumption time ratio between replaceable behaviors, and calculates estimated occurrence probabilities of the behaviors which minimize a Kullback-Leibler divergence with respect to the prior probability distribution and satisfy a condition that a ratio between estimated occurrence probabilities of the replaceable behaviors within the prediction period equals a consumption time ratio between the replaceable behaviors, a condition that a sum of expenditures of the behaviors per unit time equals an expenditure per unit time calculated from the annual consumption expenditure, and a condition that a sum of the occurrence probabilities and a sum of the estimated occurrence probabilities are “1”.
Owner:KK TOSHIBA

Apparatus and method for ionospheric anomaly monitoring using kullback-leibler divergence metric for gbas

InactiveUS20170192098A1Affect safetyEnhanced time-to-flagSatellite radio beaconingIonosphereGround system
Provided is an apparatus for monitoring an ionospheric anomaly using a Kullback-Leibler divergence (KLD) metric based on a ground-based augmentation system (GBAS) ground system. The ionospheric anomaly monitoring apparatus includes a filter configured to embed a pseudo range measurement value and a carrier-phase measurement value measured at the GBAS, and to provide a KLD metric; and a determiner configured to determine a presence or an absence of the ionospheric anomaly using the KLD metric for GBA.
Owner:CHO JEONGHO +1

Coding and decoding of source signals using constrained relative entropy quantization

Methods and devices for encoding and decoding are provided. A source signal value is encoded by a quantization index determined using a partition into quantization cells. Decoding of the quantization index takes place by sampling a reconstruction probability distribution, thereby obtaining a reconstructed signal value, such that the reconstructed signal value lies in the same quantization cell as the source signal value. In one embodiment, encoding and decoding are such that their succession preserves the source signal distribution. In another embodiment, the partition and the reconstruction probability distribution are determined in such manner that the quantization error is minimized subject to a constraint on the relative entropy between the source signal and the reconstructed signal.
Owner:GOOGLE LLC

Fault diagnosis method and system for rolling bearing based on relative entropy and k-nearest neighbor algorithm

The invention discloses a fault diagnosis method for a rolling bearing based on a relative entropy and k-nearest neighbor algorithm. The method comprises the following steps: acquiring vibration datagenerated by a bearing running under various fault states and vibration data of the bearing running under a health state; calculating a relative entropy vector sequence between the vibration data generated by running under the health state and vibration signals generated by running under the fault states according to a division result; taking the fault types as training samples to obtain a trainedclassifying model; acquiring a relative entropy vector between vibration data generated by running under an unknown state and the vibration signals generated by running under the fault states; and taking the acquired relative entropy vector as a test sample of the classifying model, and testing the test sample by using the classifying model to continually diagnose the fault of the rolling bearingto obtain the diagnosis result. Through adoption of the method, the difference between the vibration signals generated by the bearings under different states is measured by adopting the relative entropy; different feature indexes do not need to be calculated and optimally combined; and the distribution of an original vibration signal can be utilized directly.
Owner:HANGZHOU ANMAISHENG INTELLIGENT TECH CO LTD

Passive cooperative positioning method for predicting fast covariance interactive fusion on basis of information theory

The invention discloses a passive cooperative positioning method for predicting fast covariance interactive fusion on the basis of an information theory. The method includes the steps that 1, a mathematical model of a passive cooperative positioning system for two receiving stations and one external radiation source is constructed; 2, the two receiving stations separately perform prediction and updating on the state of a target according to an extended Kalman filtering algorithm; 3, according to an information theory optimization criterion and by taking original fusion state estimation into account, a K-L divergence (Kullback-Leibler Divergence) between error covariances can be obtained, and a calculation method for predicting a fast covariance interactive fusion algorithm weight is proposed; 4, an algorithm for predicting the fast covariance interactive fusion is proposed to obtain state estimation of the target. Under a passive positioning environment of the receiving stations and the external radiation source, the fusion effect can be improved, the target tracking precision is improved, and the complexity of the algorithm is low.
Owner:HANGZHOU DIANZI UNIV

Regional growth image segmentation method based on local information

The invention discloses a regional growth image segmentation method based on local information, and the method comprises the following steps: firstly selecting a seed point in an image target region;then, taking the seed point as a starting point, carrying out multi-scale region growth on the image by utilizing pixel local information and adopting growth criteria of different scales, and dynamically updating the growth criteria by utilizing R, G and B color mean values and standard deviation of an intermediate result in each region growth; and finally, calculating a probability distribution difference between two adjacent region growth results by adopting Kullback-Leibler divergence, and when a difference value is greater than a preset threshold value, taking a previous region growth result as a final segmentation result. The method is insensitive to noise, can effectively segment images with fuzzy target boundaries, uneven gray levels and rich textures, and is high in segmentation precision and robustness.
Owner:HUNAN UNIV OF SCI & TECH

Confident deep learning ensemble method and apparatus based on specialization

Disclosed herein are a confident deep learning ensemble method and apparatus based on specialization. In one aspect, a confident deep learning ensemble method based on specialization proposed by the present invention includes the steps of generating a target function of maximizing entropy by minimizing Kullback-Leibler divergence with a uniform distribution with respect to the not-classified data of models for image processing and generating general features by sharing features between the models and performing learning for image processing using the general features.
Owner:KOREA ADVANCED INST OF SCI & TECH

Network intrusion detection method and system based on partial least squares

InactiveCN105516206AAccurate and fast intrusion detectionData switching networksData setRegression analysis
The invention provides a network intrusion detection method based on partial least squares. The method comprises the steps that an anomaly detection model of network data is built according to a partial least squares regression equation; attribute domain mapping is carried out on unlabelled data, and the anomaly detection model is utilized for classifying the data to obtain different types of access behaviors. According to the network intrusion detection method, when the network is subjected to external intrusion, the intrusion data can be regarded as a nonlinear perturbation superposed on the normal network flow, the perturbation strength is affected by intrusion time and intrusion data traffic, the anomaly detection model of the network nonlinear data is built through the nonlinear theory, abnormal data traffic is found through parameter fitting, intrusion behavior detection is carried out by means of nonlinear theory and the partial least squares method in the prediction theory, Kullback Leibler divergence is adopted as determining criterions of normal and abnormal behaviors for convergence conditions of the partial least squares method, and therefore detection of intrusion behaviors in the network is more accurate and rapider.
Owner:SOUTHWEST UNIV

A design method for on-line fault detection of high-speed railway traction system

The invention discloses a design method for on-line fault detection of a high-speed railway traction system, comprising an off-line stage and an on-line stage. Off-line phase: (1) storing the operation data of high-speed railway traction system under normal operation conditions; (2) Principal Component Analysis (PCA) is used to process the stored data, which is mapped to principal component spaceand residual space; (3) In principal component space, ICA is used to extract features from non-Gaussian components; (4) According to the Kullback-Leibler divergence, the statistic is defined for the data processed by ICA and the data of residual space; (5) Determining the threshold of statistics. On-line stage: (6) reading on-line data; (7) processing the on-line data according to the offline stages (2) and (3); (8) calculating the value of on-line data statistics; (9) judging whether the fault occurs or not. This method can detect the minor faults in real-time and effectively when the model of high-speed railway traction system is unknown.
Owner:NANJING UNIV OF AERONAUTICS & ASTRONAUTICS

Relative entropy prediction method for relative complexity degree of mine unexploited area structure

The invention discloses a relative entropy prediction method for a relative complexity degree of a mine unexploited area structure. The relative entropy prediction method comprises the steps of: acquiring evaluation index data of an exploited area, wherein the evaluation index data comprises three structural type index data including fold type, fault amount and fault scale; calculating a relative entropy actual value S of a relative complexity degree of the exploited area mainly according to the structural type index data; substituting the relative entropy actual value S of the relative complexity degree of the exploited area and non-structural type index data into a formula S=beta<0>+beta<1>x<1>+...+ betax, so as to calculate a relationship parameter beta and a relationship constant beta<0>; verifying the relationship parameter beta, and then substituting the relationship parameter beta, the relationship constant beta<0> and non-structural type index data X of a mine well field unexploited area into a formula S'= beta<0>+beta<1>X<1>+...+ betaX, so as to calculate relative entropy S' of the relative complexity degree of the mine well field unexploited area; and finally judging a complexity degree level corresponding to the relative entropy S' of the relative complexity degree of the mine well field unexploited area by utilizing a relationship comparison table of structural relative complexity degrees and relative entropy. The relative entropy prediction method is simple in operation, and has good prediction effect.
Owner:XIAN UNIV OF SCI & TECH +1

Detection method for micro faults in chemical process

The invention relates to a detection method for micro faults in a chemical process. The method comprises the following steps: normalization processing is carried out on training data and then an LGPCA (local-global principal component analysis) model is established, a local-global feature is extracted from the training data and serves as a score vector, the mean value and the variance of the score vector of the training data are calculated through a sliding window, a training KLD (Kullback Leibler Divergence) component is obtained on the basis, further, main component space statistics T2 and residual space statistics SPE are calculated based on the training KLD component, and a corresponding control limit is determined; test data is collected, a corresponding main component vector and a residual vector are extracted by utilizing the LGPCA model, the mean value and the variance of the test data score vector are calculated by utilizing the sliding window, an online KLD component is further obtained, the main component space statistics T2 and the residual space statistics SPE are calculated on the basis of the online KLD component, and the control limit is used for monitoring. According to the method in the invention, the KLD is introduced into the traditional LGPCA method, the probability information contained in the chemical process data can be fully utilized, and the micro fault detection rate is improved.
Owner:CHINA UNIV OF PETROLEUM (EAST CHINA)

Relative-entropy-criterion-based external radiation source selection method of passive MIMO radar

The invention, which belongs to the field of signal processing, discloses a relative-entropy-criterion-based external radiation source selection method of a passive MIMO radar and particularly relatesto a method for selection of a passive MIMO radar transmitting station so as to solve a problem of external radiation source selection in the field of radar signal processing technology. A matched filter outputs a plan of opportunistic irradiation source selection of a passive MIMO radar by using a relative entropy as a selection standard under two kinds of hypothesis testing. Therefore, an EI selection combination having an optimal or suboptimum detection performance can be obtained in a range allowed by resources, so that a problem of too high complexity in a receiver in the prior is solved.
Owner:UNIV OF ELECTRONICS SCI & TECH OF CHINA

A method and system for identifying abnormal Weibo users

The invention relates to a method for identifying abnormal microblog users. The method comprises the steps of obtaining a plurality of users' microblog data, storing the microblog data into a database, taking statistical distribution of time intervals of user behaviors as behavior time characteristics of the users according to the microblog data of the users, generating behavior time characteristic vectors and defined parameters, calculating Kullback-Leibler divergence between the behavior time characteristic vectors of the normal users and the behavior time characteristic vectors of the users to be detected, judging the users to be detected with the calculated Kullback-Leibler divergence exceeding the defined parameters as the abnormal users, and extracting and showing keywords of contents of the abnormal users. The invention further provides a system for identifying the abnormal microblog users corresponding to the method. According to the method and system, the keywords of the blog article contents of the abnormal users can be extracted quickly, promulgators of junk information such as marketing and advertisements can be identified accurately, and the method and the system are applicable to detection of multiple microblog service platforms, and has the advantages of high accuracy and efficiency and wide applicability.
Owner:INST OF INFORMATION ENG CHINESE ACAD OF SCI

Defense method and device for interest packet flooding attack in content-centric network

The invention provides a method and device for defending interest flooding attacks in an information centric network and relates to the field of network security. The method includes: statistically counting the entropy of the names of interest packets received by a router in the information centric network after different moments according to a preset window; using a cumulative sum algorithm to process the obtained entropy to obtain the accumulative values of the entropy at different moments; judging whether the accumulative values are smaller than a preset attack detecting threshold or not, if not, judging that interest flooding attacks are detected, and using a prefix determining algorithm based on relative entropy to search the prefix set of the names of the interest packets so as to obtain attack prefixes; generating data packets containing the attack prefixes according to the attack prefixes, and transmitting the data packets to the access router where an attacker is located according to the router information of the interest packets, containing the attack prefixes, recorded in the pending interest table of the router so as to allow the access router to perform corresponding access limitation on the received interest packets according to the attack prefixes in the data packets.
Owner:INST OF INFORMATION ENG CHINESE ACAD OF SCI

Online variational Gaussian process method for time series data

*The invention discloses an online variational Gaussian process method for time series data, which comprises the following steps of: S1, determining a data set and an observation value by adopting a regression model of a Gaussian process framework; s2, solving variation free energy to perform single data processing, and calculating distribution of corresponding variation lower limits; s3, aiming at the stream data condition, solving the problem by adopting online variational reasoning; s4, solving the model posterior probability by solving the new relative entropy divergence; s5, converting the new relative entropy divergence into minimum variational free energy, and correspondingly solving factor distribution q * (b); s6, solving a variation lower limit to obtain variation free energy; and S7, according to the variational distribution obtained by solving, calculating prediction distribution and a prediction result at any test point. According to the method, the training complexity and the prediction complexity of a traditional Gaussian process algorithm are reduced, the calculation cost is reduced, and the method has the performance equivalent to that of a traditional sparse Gaussian process approximation method.
Owner:YANGTZE DELTA REGION INST OF UNIV OF ELECTRONICS SCI & TECH OF CHINE HUZHOU +2

Coding and decoding of source signals using constrained relative entropy quantization

Methods and devices for encoding and decoding are provided. A source signal value is encoded by a quantization index determined using a partition into quantization cells. Decoding of the quantization index takes place by sampling a reconstruction probability distribution, thereby obtaining a reconstructed signal value, such that the reconstructed signal value lies in the same quantization cell as the source signal value. In one embodiment, encoding and decoding are such that their succession preserves the source signal distribution. In another embodiment, the partition and the reconstruction probability distribution are determined in such manner that the quantization error is minimized subject to a constraint on the relative entropy between the source signal and the reconstructed signal.
Owner:GOOGLE LLC
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products