85 results about "Unlabelled data" patented technology

Techniques for creating training sets for predictive modeling are provided. In one aspect, a method for generating training data from an unlabeled data set is provided which includes the following steps. A small initial set of data is selected from the unlabeled data set. Labels are acquired for the initial set of data selected from the unlabeled data set resulting in labeled data. The data in the unlabeled data set is clustered using a semi-supervised clustering process along with the labeled data to produce data clusters. Data samples are chosen from each of the clusters to use as the training data. The selecting, presenting, clustering and choosing steps are repeated with one or more additional sets of data selected from the unlabeled data set until a desired amount of training data has been obtained, wherein at each iteration an amount of the labeled data is increased.

A network system for actively controlling congestion to optimize throughput is provided. The network system includes a sending host which is configured to send packet traffic at a set rate. The network system also includes a sending switch for receiving the packet traffic. The sending switch includes an input buffer for receiving the packet traffic at the set rate where the input buffer is actively monitored to ascertain a capacity level. The sending switch also includes code for setting a probability factor that is correlated to the capacity level where the probability factor increases as the capacity level increases and decreases as the capacity level decreases. The sending switch also has code for randomly generating a value where the value is indicative of whether packets being sent by the sending switch are to be marked with a congestion indicator. The sending switch also includes transmit code that forwards the packet traffic out of the sending switch where the packet traffic includes one of marked packets and unmarked packets. The network system also has a receiving end which is the recipient of the packet traffic and also generates acknowledgment packets back to the sending host where the acknowledgment packets are marked with the congestion indicator when receiving marked packets and are not marked with the congestion indicator when receiving unmarked packets. In another example, the sending host is configured to monitor the acknowledgment packets and to adjust the set rate based on whether the acknowledgment packets are marked with the congestion indicator. In a further example, the set rate is decreased every time one of the marked packets is detected and increased when no marked packets are detected per round trip time (PRTT).

The spectral kernel machine combines kernel functions and spectral graph theory for solving problems of machine learning. The data points in the dataset are placed in the form of a matrix known as a kernel matrix, or Gram matrix, containing all pairwise kernels between the data points. The dataset is regarded as nodes of a fully connected graph. A weight equal to the kernel between the two nodes is assigned to each edge of the graph. The adjacency matrix of the graph is equivalent to the kernel matrix, also known as the Gram matrix. The eigenvectors and their corresponding eigenvalues provide information about the properties of the graph, and thus, the dataset. The second eigenvector can be thresholded to approximate the class assignment of graph nodes. Eigenvectors of the kernel matrix may be used to assign unlabeled data to clusters, merge information from labeled and unlabeled data by transduction, provide model selection information for other kernels, detect novelties or anomalies and/or clean data, and perform supervised learning tasks such as classification.

A system and method for placement of sharing physical buffer lists in RDMA communication. According to one embodiment, a network adapter system for use in a computer system includes a host processor and host memory and is capable for use in network communication in accordance with a direct data placement (DDP) protocol. The DDP protocol specifies tagged and untagged data movement into a connection-specific application buffer in a contiguous region of virtual memory space of a corresponding endpoint computer application executing on said host processor. The DDP protocol specifies the permissibility of memory regions in host memory and specifies the permissibility of at least one memory window within a memory region. The memory regions and memory windows have independently definable application access rights, the network adapter system includes adapter memory and a plurality of physical buffer lists in the adapter memory. Each physical buffer list specifies physical address locations of host memory corresponding to one of said memory regions. A plurality of steering tag records are in the adapter memory, each steering tag record corresponding to a steering tag. Each steering tag record specifies memory locations and access permissions for one of a memory region and a memory window. Each physical buffer list is capable of having a one to many correspondence with steering tag records such that many memory windows may share a single physical buffer list. According to another embodiment, each steering tag record includes a pointer to a corresponding physical buffer list.

A local-neighborhood Laplacian Eigenmap (LNLE) algorithm is provided for methods and systems for semi-supervised learning on manifolds of data points in a high-dimensional space. In one embodiment, an LNLE based method includes building an adjacency graph over a dataset of labelled and unlabelled points. The adjacency graph is then used for finding a set of local neighbors with respect to an unlabelled data point to be classified. An eigen decomposition of the local subgraph provides a smooth function over the subgraph. The smooth function can be evaluated and based on the function evaluation the unclassified data point can be labelled. In one embodiment, a transductive inference (TI) algorithmic approach is provided. In another embodiment, a semi-supervised inductive inference (SSII) algorithmic approach is provided for classification of subsequent data points. A confidence determination can be provided based on a number of labeled data points within the local neighborhood. Experimental results comparing LNLE and simple LE approaches are presented.

The invention relates to a realization method and system for electronic medical record post-structuring and auxiliary diagnosis. A combination mode of multiple types of distance measurement is used: a character string editing distance refers to a minimum number of replacement, insertion and deletion operations required for converting a character into another character string; a Jaro-Winkler distance measures similarity between two character strings and is used for repeated recording detection; a geometric mean value of a Chinese character distance and a Chinese character input method is adopted as comprehensive similarity measurement for measuring similarity between characteristic texts; characteristic ranking is realized by using a TF-IDF method and is used for assessing the importance of characteristic terms relative to documents in a file set or a corpus library, and the importance of the characteristic terms is in direct proportion to an occurrence frequency in the documents and is in inverse proportion to an occurrence document in the corpus library; and files are converted to be in a file format of PU learning of a positive example data set and an unlabelled data set according to the generated characteristic terms, and through the PU learning, the system automatically recommends related diagnoses for clinical medical personnel to refer.

While deep learning is used for classification, multi-modal information with rich samples and classification contribution variability of each modality are considered, and the problem of insufficient samples is solved by using a semi supervision method. Data of different modalities of a hyperspectral image is sent into a deep neural network, the semi supervision method is used and a large number ofunlabeled samples are utilized, and the deep neural network based on self-encoding is used for feature learning. All labeled and unlabeled data are sent into the self-encoding deep neural network tocarry out learning, similar networks are designed for different modalities, a respective initialization parameter is obtained through self-encoding reconstruction, and hidden attributive classification of labeled samples is obtained through a clustering method. For the unlabeled data, a deep characteristic is calculated through a multi-target deep network, then a similar marked sample is searchedbased on a clustering label, and finally, labels of the unlabeled samples are predicted according to the label information of the labeled samples.

The invention discloses an intrusion detection method based on semi-supervised learning. The method comprises the steps of selecting an initial mixed sample set with samples with labels and unlabeled samples to be tested, calculating information gain of each characteristic value in a characteristic space, and completing characteristic selection based on information entropy; then, screening the samples with the labels based on the characteristic selection of the information entropy, using new screened training data for semi-supervised training of a classifier based on LapSVM, and utilizing the classifier after training is finished to classify the unlabeled samples to be tested; according to a detection index, determining the best evaluation value of the detection index, and outputting a classification result corresponding to the best evaluation value of the detection index. According to the intrusion detection method based on semi-supervised learning, the characteristic selection method is adopted to deal with redundancy phenomena easily occurring in network environment data, a semi-supervised learning model is established by utilizing a small number of samples with labels and a large amount of unlabeled data, the false alarm rate is reduced, and the detection rate is increased; meanwhile, the data redundancy can be reduced, and the detection efficiency is improved.

The present invention relates to a semi-supervised biomedicine event extraction method based on co-training. Automatic extraction of biomedicine events greatly interests people with rapid increasing of biomedicine literatures. The scale of the marked biomedicine event corpus is small to influence the performances of the classification algorithm and even cause overfitting. The method provided by the invention identifies more accurate positive instances from unmarked data to enlarge a marked training set. The method comprises the steps of: designing abundant features for usage of an SVM; learning short sentences based on word embedding from Word2vec and Pubmed; further extending the short sentences to dependent short sentences between triggering words and parameters, and inputting the dependent short sentences into a CNN; and finally, performing backfill of samples, meeting conditions, predicted by the SVM and the CNN in the unmarked corpus into the training set, incrementally extendingthe training set. Lots of experiment results show that the new semi-supervised biomedicine event extraction method can effectively extract events.

The invention relates to an active learning sample selection strategy integrated with a confidence criterion and a diversity criterion. The active learning sample selection strategy comprises the following steps: training a model Mt based on an existing labeled data set DL; predicting a current unlabelled data set DU by using the Mt to obtain a predicted vector set Pt; calculating an information entropy of each sample according to the Pt, and selecting front K samples each having a largest entropy; extracting feature representations of K unlabelled samples according to the Mt to obtain a feature vector set Ft; performing density peaks clustering on the Ft, respectively selecting corresponding proportion and number of samples from a center of a cluster generated by the density peaks clustering, and an edge point and an outlier of the cluster, handing the samples to an expert for labeling, adding the labeled data set DL, and simultaneously deleting corresponding samples from the unlabelled data set DU; updating the Mt by using the current labeled data set DL to obtain Mt + 1; and repeating the above steps till labeling of all samples is ended or reaches to a designated number of iteration times to complete a whole algorithm flow.

The invention relates to the technical field of image processing, and provides an image classification method based on active semi-supervised learning, comprising the following steps: randomly selecting part of labeled samples and all unlabeled samples for training semi-supervised dictionary learning components in a model; The criterion based on predicting the probability of classification iteratively selects the unlabeled samples which contain the most information from the unlabeled dataset, namely the most detailed samples. A user is introduced to tag the most informative samples, and then the most informative samples that have completed the tagging are added to the tagged dataset for training the active learning components in the model. steps are repeated to iteratively update the modeluntil the algorithm finally converges or reaches a certain number of iterations; The model is used to classify the images of the test samples. The invention solves the problem of poor expression ability between classes, combines semi-supervised learning and active learning, effectively utilizes all training data, and improves the performance of the algorithm model.

The invention discloses a building method and device of an anomaly detection training set. The method comprises the steps as follows: an acquired sampled data set is determined as a current data set; labelled data is acquired in the current data set according to each received current labelling instruction, the labelled data is added into a first data set, and unlabelled data forms a second data set; and whether the number of outlier data reaches a preset value is determined, if yes, a training set is generated according to the labelled data and the unlabelled data, otherwise, the outlier probability of the unlabelled data is computed according to the first data set, the unlabelled data is ordered according to the outlier probability and determined as the current data set, and each current labelling instruction is acquired by returning for execution. Compared with the single computation of the outlier probability in the prior art, the method utilizes the labelled data to recalculate the outlier probability of the unlabeled data; and on the basis that the outlier ordering shifts forwards after the outlier probability ordering, labelling times can be reduced, and building efficiency of the training set is improved.

A migration retrieval method based on a semi-supervised countermeasure generation network is provided. A countermeasure generation network is designed to retrieve hashes across data domains, and the goal is to map the original and target datasets into a common Hamming space, so that the image retrieval in a particular scene can be migrated to a retrieval image of another scene through the learningof the semi-supervised antagonism generation network. Therefore, the problem that the unlabeled data can not be fully utilized and the retrieval model is only suitable for a single scene in the era of big data is solved. The invention effectively improves the automatic and intelligent level of image retrieval.

The invention provides an image recognition method and device based on non-negative low-rank representation and semi-supervised learning. The method includes the following steps that: an image data set is obtained, wherein the data set contains marked data and unmarked data; an objective function is obtained according to a Gaussian field, a harmonic function and a low-rank representation function,non-negative constraint is performed on the coefficient of the low-rank representation function, the objective function is converted into a Lagrangian function, and variables, Lagrangian multipliersand a penalty factor in the Lagrangian function are updated; and iterative updating is carried out continuously until the method terminates, and the label matrix of the image data set is outputted, and test data are classified and identified according to the label matrix. According to the image recognition method and device of the invention, the semi-supervised learning and the low-rank representation are combined, and therefore, global structure information and local structure information can be well utilized, and the corruption of samples can be effectively eliminated or mitigated. The method and device have high robustness to noises and can obtain high classification performance regardless of whether training samples or test samples are damaged.

A system and methods are provided in which an artificial intelligence inference module identifies targeted information in large-scale unlabeled data, wherein the artificial intelligence inference module autonomously learns hierarchical representations from large-scale unlabeled data and continually self-improves from self-labeled data points using a teacher model trained to detect known targets from combined inputs of a small hand labeled curated dataset prepared by a domain expert together with self-generated intermediate and global context features derived from the unlabeled dataset by unsupervised and self-supervised processes. The trained teacher model processes further unlabeled data to self-generate new weakly-supervised training samples that are self-refined and self-corrected, without human supervision, and then used as inputs to a noisy student model trained in a semi-supervised learning process on a combination of the teacher model training set and new weakly-supervised training samples. With each iteration, the noisy student model continually self-optimizes its learned parameters against a set of configurable validation criteria such that the learned parameters of the noisy student surpass and replace the learned parameter of the prior iteration teacher model, with these optimized learned parameters periodically used to update the artificial intelligence inference module.