3 results about "Sparse data sets" patented technology

This disclosure provides a data-driven and scalable method to discover cause-and-effect relationships in data from natural systems that include sparse data sets. This technique can learn a causal graph from heterogenous data sources by combining embeddings from real data and embeddings from simulated data generated by process-based models. The causal graph is used for what-if analysis in out-of-distribution settings. One application is understanding the factors that affect soil carbon. A causal model created by these techniques can be used to discover cause-and-effect relationships that affect soil carbon. This model has applications such as forecasting soil carbon for a future time point to help inform farm practices. Farm practices, like tilling, may be modified in response to predictions provided by the model.

This disclosure provides a mechanism for the enrichment of sparse datasets using language models. By training language models on the specific distribution of known values in a dataset, missing values can be predicted, and the predicted values added, thereby resulting in a more complete dataset. This method also facilitates the enhancement and augmentation of datasets by predicting values for new properties that were not previously available. The approach proves particularly effective at scale, transforming large sparse datasets into more complete and enhanced datasets. Masking language modeling may be employed to train language models capable of generating representations of technical data. Training data includes corpuses of technical data that may be represented as text strings. These pretrained models are fine-tuned to predict various properties. The resulting models can predict missing values in large technical datasets, providing valuable data for guiding scientific research.

The application discloses a kind of unmanned aerial vehicle hyperspectral non-imaging paradigm's oil slick detection method, comprising: parallel push-scan type scanning is carried out to offshore oil spill area, and the hyperspectral original data of measurement sea area is obtained;Spectral mixing matrix pixel splicing method is used to seamlessly splice the hyperspectral original data into image to obtain clean sea area data, other coastline and land non-oil slick area hyperspectral pixel cube;Full-spectral data set obtained in oil slick area and suspected oil slick area is regarded as foreground, and corresponding area relative position information is retained, other clean sea area data, other coastline and land non-oil slick area data are marked as background;The background is carried out hyperspectral cube sparse representation, and spectral segment sparse line compression is carried out using matrix mapping mode to obtain sparse data set, and the feature data set of oil slick area and sparse data set are used as transmission data;Containing relative position information oil slick area is marked to background image, and oil slick area focusing result is marked on map.