Kernels (206) for use in learning machines, such as support vector machines, and methods are provided for selection and construction of such kernels are controlled by the nature of the data to be analyzed (203). In particular, data which may possess characteristics such as structure, for example DNA sequences, documents; graphs, signals, such as ECG signals and microarray expression profiles; spectra; images; spatio-temporal data; and relational data, and which may possess invariances or noise components that can interfere with the ability to accurately extract the desired information. Where structured datasets are analyzed, locational kernels are defined to provide measures of similarity among data points (210). The locational kernels are then combined to generate the decision function, or kernel. Where invariance transformations or noise is present, tangent vectors are defined to identify relationships between the invariance or noise and the data points (222). A covariance matrix is formed using the tangent vectors, then used in generation of the kernel.