Large-scale unbalanced diabetes electronic medical record parallel classification neighborhood evidence Spark method

Abstract

The invention provides a large-scale unbalanced diabetes electronic medical record parallel classification neighborhood evidence Spark method. The method comprises the steps: reading diabetes data on a main node, and dividing the diabetes data into a training set and a test set according to a ratio of 4: 1; carrying out Spark parallel undersampling on the diabetes training set on the child nodes to obtain a plurality of new training subsets; obtaining pathological feature reduction subsets on the sub-nodes through a Spark parallel pathological feature reduction device, updating pathological feature sets of the training subsets and the test subsets on each sub-node, obtaining prediction category label sets of the test subsets on the sub-nodes through a neighborhood evidence Spark parallel classifier, and obtaining prediction category label sets of the test subsets on the sub-nodes; and obtaining a final prediction category label on the main node according to a voting mechanism. The method has the beneficial effects that redundant attributes in large-scale data are removed, the calculation efficiency is improved, support information among samples is fully utilized, and the efficiency and precision of diabetes data classification are improved.

Description

technical field [0001] The invention relates to the technical field of medical information intelligent processing, in particular to a large-scale unbalanced diabetes electronic medical record parallel classification neighborhood evidence Spark method. Background technique [0002] Diabetes mellitus (DM) refers to the sugar, protein, fat, water and electrolytes caused by the decline of islet function and insulin resistance caused by various pathogenic factors such as genetic factors, endocrine, and dietary imbalance after dysfunction. A series of metabolic disorder syndromes. At the same time, there are more types of complications caused by diabetes, and doctors cannot effectively and accurately determine whether a patient has diabetes only by relying on the patient's physical signs. [0003] At present, the effective way to judge the condition of diabetes is through the pathological characteristic experiment of the etiology and pathogenesis of diabetes, but the experiment n...

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to provide a large-scale unbalanced diabetes electronic medical record parallel classification neighborhood evidence Spark method, which solves the problem that the existing effective way to judge the state of diabetic lesions is to pass the pathological characteristic experiment of the etiology and pathogenesis of diabetes, resulting in experimental test data Too many attributes and a large amount of data will increase the workload of doctors in judging the pathological conditions of diabetic patients

Images