Parallel attribute reduction method for diabetes based on coevolutionary discrete particle swarm optimization

Abstract

The present invention provides a diabetes parallel attribute reduction method based on collaborative evolution discrete particle swarm optimization, which belongs to the field of medical electronic medical records technology. It solves the technical problem that the diabetic electronic medical record data is large in dimension and redundant, which leads to doctors' wrong judgment. Its technical solution is: including the following steps: S1: storing diabetes data in a distributed file system; S2: computer nodes read the data of blocks in HDFS; S3: after the master node obtains the summarized data key-value pairs; S4: the master node performs the operation of step S3<key,value> The key-value pair data is broadcast to each child node; S5: the master node models the obtained attribute evaluation function in combination with the CQBPSO algorithm. The beneficial effects of the present invention are: the present invention combines the rough set theory and the Spark distributed computing platform to screen out the most representative and critical attributes from the diabetes data set.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic medical record data processing, and in particular to a diabetes parallel attribute reduction method based on co-evolutionary discrete particle swarm optimization. Background Art

[0002] Diabetes is a common, complex, chronic metabolic disease, the most important feature of which is high blood sugar. High blood sugar can damage various systems and organs of the body, such as the cardiovascular, kidney, eye and nervous systems, and in severe cases may lead to fatal complications. However, the characteristics of diabetes are diverse and may affect multiple systems and organs of the body, which shows that early diagnosis of diabetic patients is very important.

[0003] However, due to the diversity of diabetes and the complexity of related factors, it is difficult to establish effective classification and regression models. Yang Meijie and other scholars used the backward greedy search strategy in the literature "Preprocessing of Diabetes Electronic Medical Record Data" to achieve attribute simplification of diabetes electronic medical records. This search strategy works well when processing low-dimensional data, but when facing large-scale diabetes electronic medical record data, the search efficiency drops sharply, and the global optimal attribute subset cannot be obtained. In addition, there is a large amount of uncertainty in the diabetes electronic medical record data, which will cause information loss during processing; Zhou Weiguang, a scholar, combined the taboo search algorithm with rough set theory in "Research and Application of Rough Set Theory in Processing Massive Electronic Medical Records" and proposed a new attribute simplification method, which increases the probability of obtaining a better global optimal subset. However, this method has a strong dependence on the initial solution. It is difficult to find a good initial solution in practical applications, which often makes the search difficult and cannot reach the global optimal. In addition, the iterative search process is serial, which is only a single state movement, not a parallel search.

[0004] In summary, the existing methods can meet the attribute simplification requirements of large-scale diabetes electronic medical record datasets, but they have problems such as long processing time, low accuracy, and high resource consumption. Therefore, a new method is urgently needed to quickly and effectively extract key information from diabetes electronic medical records to help doctors better analyze patient symptoms. Summary of the invention

[0005] The purpose of the present invention is to provide a diabetes parallel attribute reduction method based on collaborative evolutionary discrete particle swarm optimization, which solves the problem that the electronic medical record data of diabetes has large dimensions and high redundancy, leading to doctors' misjudgment, and improves the efficiency and accuracy of data processing.

[0006] The inventive idea of ​​the present invention is: the present invention performs parallel calculation on data through the Spark platform, which greatly reduces the resources and time of data processing, and at the same time combines the rough set attribute reduction method with the binary co-evolutionary particle swarm optimization algorithm, which greatly accelerates the search for the global optimal solution, can effectively reduce data redundancy, accelerate convergence speed, and improve data interpretability. It can effectively solve a series of difficult problems in electronic medical record data mining and disease diagnosis of diabetes in the medical field, and has broad application prospects.

[0007] The present invention is achieved by the following measures: a diabetes parallel attribute reduction method based on co-evolutionary discrete particle swarm optimization, comprising the following steps:

[0008] S1: Store diabetes data in a distributed file system, read data through the textfile method in Spark, set the appropriate file path and number of partitions, make full use of cluster resources, and greatly improve data processing efficiency;

[0009] S2: The computer node reads the data of the block in HDFS and converts the data into<key,value> In the form of key-value pairs,<key,value> The key-value pair representation effectively reduces repeated calculations between numerous conditional attributes of data and saves computing resources; the key is the conditional attribute value of diabetes, the value is the decision attribute value of diabetes, and the converted data key-value pairs are sent to the master node for aggregation;

[0010] S3: After the master node obtains the summarized data key-value pairs, for the key-value pairs with the same key and value, only one is retained, and the value of the remaining key-value pair is increased by 1. For the key-value pairs with the same key but different values, the data with smaller value is deleted and the data with larger value is retained. The master node performs data deduplication based on the value to reduce redundant data, making the rough set modeling more concise and effective.

[0011] S4: The master node will perform the operation in step S3.<key,value> The key-value pair data is broadcast to each child node. Each child node performs rough set modeling on the data, calculates the upper and lower approximate space, attribute importance, attribute dependency, and attribute correlation. The child node sends the calculated local attribute importance, local attribute dependency, and local attribute correlation to the main node for summary. By allowing each child node to calculate local information and summarizing global information at the main node, the cluster computing capability is fully utilized and the amount of calculation is reduced. The main node obtains the sum of the local information of each node and obtains the global attribute evaluation function value.

[0012] S5: The master node combines the obtained attribute evaluation function with the CQBPSO algorithm to model, and uses the collaborative evolution discrete particle swarm algorithm in conjunction with the rough set attribute evaluation function to locate the attribute subset with the maximum separation ability and achieve accurate attribute simplification.

[0013] As a further optimization scheme of the diabetes parallel attribute reduction method based on collaborative evolutionary discrete particle swarm optimization provided by the present invention, the specific steps of step S2 are as follows:

[0014] Step S2.1: According to the characteristics of electronic medical record data and the needs of disease diagnosis, select appropriate condition attributes and decision attributes to generate data set S, where S is defined as follows:

[0015] S = {U, C∪D, V, f}, where U is the original data set to be processed, U = {x 1 ,x 2 ,...,x i ,...,x n} represents the set of all patients, x i represents the i-th patient, n represents the number of all patients; C represents the non-empty finite conditional attribute set of the patient's electronic medical record, and D represents the non-empty finite decision attribute set of the patient's electronic medical record. V is a description of the elements in U, and f is a mapping from U to a target set, f = U × (C ∪ D) → V;

[0016] Step S2.2: Use the map method in SparkCore to transform the data into<key,value> The RDD format of key-value pairs. RDD is a distributed elastic dataset and a data type in Spark. The key is a Tuple tuple. The elements in the tuple are the conditional attribute sets of the object. If the conditional attributes exceed 22, the data type needs to be converted to ArrayBuffer instead of Tuple. The value is the decision attribute set of the object.

[0017] As a further optimization scheme of the diabetes parallel attribute reduction method based on collaborative evolutionary discrete particle swarm optimization provided by the present invention, the specific steps of step S3 are as follows:

[0018] Step S3.1: Preprocess the data by using the reduceByKey method in SparkCore. For data with the same key value but different value, only keep the one with the larger value. For data with the same key and value, only keep one and perform the value+1 operation.

[0019] Step S3.2: Use the distributed computing framework and parallel processing method provided by the Spark platform to divide the original data into equivalence classes. There is an equivalence relation IND(B) defined as follows:

[0020] An equivalence relation is an indiscernibility relation, where the equivalence class of an object x to an attribute set B is represented by U / IND(B) or U / B;

[0021] Due to the distributed computing framework and parallel processing capabilities provided by the Spark platform, the data set S is divided into several data subsets S′. In the Spark framework, the communication between the master node and the partition node executor is established through the built-in Netty communication protocol. The master node divides each data subset S i ' Passed to executor i On the node, each partition is calculated in parallel, the attribute values ​​of each object are divided, and an equivalence class partition table is generated.

[0022] As a further optimization scheme of the diabetes parallel attribute reduction method based on collaborative evolutionary discrete particle swarm optimization provided by the present invention, the specific steps of step S4 are as follows:

[0023] Step S4.1: Each child node calculates the upper and lower approximation, attribute importance, attribute dependency, and attribute correlation evaluation index of each partition attribute in parallel and generates a local evaluation function index;

[0024] The upper and lower approximations are defined as:

[0025]

[0026]

[0027] in R (X) is the lower approximation, is an upper approximation, Y is a subset of the conditional attribute set, X is the domain, is an empty set, where the upper approximate set represents the set of objects in U that may belong to X, and the lower approximate set represents the set of objects that belong to X. The lower approximate set is also called the positive domain, denoted by POS R (X);

[0028] Attribute Dependency Defined as:

[0029]

[0030] Among them A m is the attribute, m is the index of the attribute, For attribute A mThe corresponding positive domain, |·| represents the cardinality, is the cardinality of the positive domain, |U| represents the cardinality of the domain;

[0031] The attribute correlation ρ is defined as:

[0032]

[0033] Where P is a subset of the conditional attribute set C, |P| represents the cardinality of P, and γ Am (D) is attribute A m Attribute dependence on the decision attribute set D;

[0034] The importance of attribute subset P is σ P is defined as:

[0035]

[0036] where γ P (D) represents the dependency of attribute subset P on D, Indicates that attribute subset P excludes attribute A m The latter is about the dependence of D;

[0037] Step S4.2: When the child node completes the computing task, the master node aggregates the computing results of each executor node to the master node through the collect method in SparkCore;

[0038] Step S4.3: The master node obtains the local evaluation function of the attribute and combines it to generate the global evaluation function Γ, which is expressed as:

[0039] Γ=ωρ+λ(1-ω)γ P (D)+(1-λ)(1-ω)σ(D,A) (6)

[0040] Among them, ω, λ are two hyperparameters and ω, λ∈(0,1), and the generated global evaluation function Γ is used as the objective function value of the CQBPSO algorithm.

[0041] As a further optimization scheme of the diabetes parallel attribute reduction method based on collaborative evolutionary discrete particle swarm optimization provided by the present invention, the specific steps of step S5 are as follows:

[0042] Step S5.1: Initialize the parameters such as particle position and velocity, and use binary coding to discretize the data. The particle position is represented by a binary bit string of length N, where N is the number of attributes. The attribute value consists of '0' and '1', and each position vector is an attribute subset;

[0043] The velocity v of each particle is represented by a positive integer and satisfies v∈[1,Vmax ],V max It is a custom integer, which means how many bits of the particle's characteristics should be changed at a specific moment;

[0044] Step S5.2: Update the local optimal position and global optimal position of each particle according to the fitness function (6);

[0045] Step S5.3: Update the speed and position of each particle according to the speed update formula (7) and the position update formula (8);

[0046]

[0047]

[0048] Where t represents the number of iterations, j represents the j-th dimension component of the i-th particle, j = 1, 2, 3, ..., n, represents the personal optimal position of the j-dimensional component of the i-th particle, represents the global optimal position of the j-dimensional component of the ith particle, represents the particle position of the j-th dimension component of the ith particle, rand 1 (),rand 2 () is a random function with a value between [0,1], δ is the inertia weight, c 1 ,c 2 are personal learning factors and social learning factors respectively, and c is updated according to the nonlinear adjustment strategy in the co-evolution theory. 1 and c 2 ;

[0049]

[0050]

[0051] where c 1max ,c 1min ,c 2max ,c 2min Indicates c 1 ,c 2 The maximum and minimum values ​​of c 1max ,c 2max ∈[2,3], c 1min ,c 2min ∈[0,1], take c 1max =2.5,c 1min =0.5,c 2max =2.5,c 2min =0.5, iter is the number of iterations, iter max is the maximum number of iterations;

[0052] Update δ according to the adaptive inertia weight update formula:

[0053]

[0054] δ max ,δ min To set the maximum and minimum values ​​of δ;

[0055] In a certain dimension of the particle, the velocity vector represents the probability that the position vector becomes '1'. The following sigmoid function is used to convert all continuous values ​​of this dimension in the velocity vector into probability values ​​in the range of [0,1];

[0056]

[0057] The update calculation of the particle position is as follows:

[0058]

[0059] Where r is a random number in [0,1], s(·) represents the position The probability distribution of , z is the independent variable;

[0060] Step S5.4: Calculate the evaluation function for the particle according to the above formula (6). For a binary-coded position vector, a value of '0' represents that the attribute is not selected, and a value of '1' represents that the attribute is selected.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. Aiming at the problem that the features of large-scale electronic medical record data of diabetes are diverse and redundant, the present invention uses the rough set method to select features, remove redundancy to obtain typical features, and assist doctors in making more accurate medical diagnoses for patients.

[0063] 2. The present invention combines rough set attribute reduction with binary co-evolution particle swarm algorithm, which greatly accelerates the search for the global optimal solution and improves the convergence speed. Due to the introduction of the concept of co-evolution, the algorithm has a good search ability for both the global optimal solution and the local optimal solution, and is not prone to fall into the local optimal solution;

[0064] 3. The present invention constructs data in RDD format through the Spark parallel computing framework, which saves a lot of computing time resources and has high fault tolerance. Lost RDD can be regenerated, which improves the accuracy of the entire simplification process and has important guiding significance for the medical diagnosis of diabetes. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0066] Figure 1 This is an overall framework diagram of the parallel attribute reduction method for diabetes based on collaborative evolutionary discrete particle swarm optimization of the present invention.

[0067] Figure 2 It is an overall flow chart of the diabetes parallel attribute reduction method based on collaborative evolutionary discrete particle swarm optimization of the present invention.

[0068] Figure 3 This is a Spark architecture diagram of the diabetes parallel attribute reduction method based on collaborative evolutionary discrete particle swarm optimization of the present invention.

[0069] Figure 4 Feature strategy diagram selected for binary encoding in the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Of course, the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0071] Example 1

[0072] See also Figures 1 to 3 The present invention provides a technical solution, which is a diabetes parallel attribute reduction method based on co-evolutionary discrete particle swarm optimization, comprising the following steps:

[0073] S1: Read the diabetes electronic medical record data from HDFS, process the data, and convert the data into a four-tuple information decision system S = {U, C∪D, V, f}, where U = {x 1 ,x 2 ,x 3 ,...,x n} represents a non-empty finite set of electronic medical records of diabetes, n is the number of patients, C = {c 1 ,c 2 ,c 3 ,...,c m} represents the non-empty finite conditional attribute set of the diabetes electronic medical record data set, m represents the number of diabetes attributes, D = {d} represents the non-empty finite decision attribute set of the diabetes electronic medical record data set, and V is a description of the elements in U, f is a mapping of the domain U to a target set, and f = U × (C ∪ D) → V represents the information function;

[0074]

[0075]

[0076] S2: Transform decision information systems into<key,value> In the form of a key-value pair, the key value is a tuple consisting of elements of the object's condition attribute set, and the value value is the object's decision attribute value;

[0077]

[0078]

[0079] S3.1: The master node obtains the data calculated by each slave node through the collect method.<key,value> The key-value pair results are preprocessed. For data with the same key value but different value, the data with smaller value is deleted and the larger value is retained. For data with the same key value and value, only one of them is retained and the value is +1. For objects with conditional attribute set (1,3,5,7,9), the corresponding decision attribute values ​​are 1, 0, 1, and 2 respectively. According to the above rules, only x1 is left: ((1,3,5,7,9),3). The preprocessed data set is:

[0080] U Key Value RDD <![CDATA[x 1 ]]> (1,3,5,7,9) 3 ((1,3,5,7,9),3) <![CDATA[x 2 ]]> (2,4,6,8,10) 3 ((2,4,6,8,10),3) <![CDATA[x 3 ]]> (1,4,9,16,25) 3 ((1,4,9,16,25),3) <![CDATA[x 4 ]]> (2,2,5,1,7) 4 ((2,2,5,1,7),4) <![CDATA[x 5 ]]> (1,4,8,9,6) 3 ((1,4,8,9,6),3) <![CDATA[x 6 ]]> (2,4,5,9,0) 4 ((2,4,5,9,0),4)

[0081] S3.2: Compute equivalence classes by decision attribute values We get the equivalence class U / D = {D 1 ,D 2}, where D 1 ={x 1 ,x 2 ,x 3 ,x 5}, D 2 ={x 4 ,x 6};

[0082] S4.1: Calculate the upper and lower approximation sets of each attribute in the equivalence class by the formula:

[0083] For example, given a sample X = {x 1 ,x 4 ,x 6}, corresponding to R (X) = {D 2}={x 4 ,x 6},

[0084] The evaluation function of the attribute is calculated according to the attribute dependency, relevance, and importance formulas:

[0085]

[0086]

[0087]

[0088] S4.2: Wait for the calculation to complete and aggregate the results to the master node through the collect function;

[0089] S5.1: Generate particles and initialize the number, speed, position and other parameters of the particles, and represent the position of the particles as a binary string;

[0090] S5.2: Calculate the fitness of each particle and update the local best position and the global best position;

[0091] Fitness=ωΓ relev +λ(1-ω)γ P (D)+(1-λ)(1-ω)σ(D,A)

[0092] S5.3: Update the particle's velocity, inertia weight, and learning factor:

[0093]

[0094]

[0095]

[0096]

[0097] S5.4: Update the position of the particle:

[0098]

[0099] Iterate sequentially while satisfying the iteration conditions, update each parameter each time, and convert the global optimal position into the feature subset S after reaching the maximum number of iterations reduct , S reduct This is the reduced set obtained. The element with a value of 1 is the selected attribute. The element index of the selected attribute is converted into the reduced attribute set output. The final reduced result is S reduct ={c 1 ,c 2 ,c 3 ,c 4}.

[0100] Example 2

[0101] See also Figures 1 to 3 The present invention provides a technical solution, which is a diabetes parallel attribute reduction method based on co-evolutionary discrete particle swarm optimization, comprising the following steps:

[0102] S1: Read data from another diabetes electronic medical record from HDFS, process the data, and convert it into a four-tuple information decision system S = {U, C∪D, V, f}, where U = {x 1 ,x 2 ,x 3 ,...,x n} represents a non-empty finite set of electronic medical records of diabetes, n is the number of patients, C = {c 1 ,c 2 ,c 3 ,...,c m} represents the non-empty finite conditional attribute set of the diabetes electronic medical record data set, m represents the number of diabetes attributes, D = {d} represents the non-empty finite decision attribute set of the diabetes electronic medical record data set, and V is a description of the elements in U, f is a mapping of the domain U to a target set, and f = U × (C ∪ D) → V represents the information function;

[0103] U <![CDATA[c 1 ]]> <![CDATA[c 2 ]]> <![CDATA[c 3 ]]> <![CDATA[c 4 ]]> <![CDATA[c 5 ]]> <![CDATA[c 6 ]]> <![CDATA[c 7 ]]> D <![CDATA[x 1 ]]> 80 0 1 0 25.19 6.6 140 0 <![CDATA[x 2 ]]> 54 0 0 0 27.32 6.6 80 0 <![CDATA[x 3 ]]> 28 0 0 0 27.32 5.7 158 1 <![CDATA[x 4 ]]> 36 0 0 1 23.45 5 155 1 <![CDATA[x 5 ]]> 76 1 1 1 20.14 4.8 155 0 <![CDATA[x 6 ]]> 20 0 0 0 27.32 6.6 85 0 <![CDATA[x 7 ]]> 44 0 0 0 19.31 6.5 200 1 <![CDATA[x 8 ]]> 79 0 0 0 23.86 5.7 85 0 <![CDATA[x 9 ]]> 42 0 0 0 33.64 4.8 145 0 <![CDATA[x 10 ]]> 32 0 0 0 27.32 5 100 1 <![CDATA[x 11 ]]> 53 0 0 0 27.32 6.1 85 0 <![CDATA[x 12 ]]> 54 0 0 1 54.7 6 100 1 <![CDATA[x 13 ]]> 78 0 0 1 36.05 5 130 1 <![CDATA[x 14 ]]> 67 0 0 0 25.69 5.8 200 1 <![CDATA[x 15 ]]> 76 0 0 0 27.32 5 160 0

[0104] S2: Transform decision information systems into<key,value> In the form of a key-value pair, the key value is a tuple consisting of elements of the object's condition attribute set, and the value value is the object's decision attribute value;

[0105]

[0106]

[0107] S3.1: The master node obtains the data calculated by each slave node through the collect method.<key,value> The key-value pair results are used for data preprocessing. For data with the same key value but different value, the data with smaller value is deleted and the larger value is retained. For data with the same key value and value, only one is retained and the value is increased by 1. Due to the diversity of the diabetes data set in Example 2, the data set in the data preprocessing stage is not changed.

[0108] S3.2: Compute equivalence classes by decision attribute values We get the equivalence class U / D = {D 1 ,D 2}, where D 1 ={x 1 ,x 2 ,x 5 ,x6 ,x,D 2 ={x 3 ,x 4 ,x 7 ,x 10 ,x 12 ,x 13 ,x 14};

[0109] S4.1: Calculate the upper and lower approximation sets of each attribute in the equivalence class by the formula:

[0110] For example, given a sample X = {x 1 ,x 4 ,x 6}, corresponding to R (X) = {D 2}={x 4 ,x 6},

[0111] The evaluation function of the attribute is calculated according to the attribute dependency, relevance, and importance formulas:

[0112]

[0113]

[0114]

[0115] S4.2: Wait for the calculation to complete and aggregate the results to the master node through the collect function;

[0116] S5.1: Generate particles and initialize the number, speed, position and other parameters of the particles, and represent the position of the particles as a binary string;

[0117] S5.2: Calculate the fitness of each particle and update the local best position and the global best position;

[0118] Fitness=ωΓ relev +λ(1-ω)γ P (D)+(1-λ)(1-ω)σ(D,A)

[0119] S5.3: Update the particle's velocity, inertia weight, and learning factor:

[0120]

[0121]

[0122]

[0123]

[0124] S5.4: Update the position of the particle:

[0125]

[0126] Iterate sequentially while satisfying the iteration conditions, update each parameter each time, and convert the global optimal position into the feature subset S after reaching the maximum number of iterations reduct , S reduct This is the reduced set obtained. The element with a value of 1 is the selected attribute. The element index of the selected attribute is converted into the reduced attribute set output. The final reduced result is S reduct ={c 6}.

[0127] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A parallel attribute reduction method for diabetes based on co-evolutionary discrete particle swarm optimization, characterized by: The following steps are involved: S1: Store the diabetes data in a distributed file system, read the data through the textfile method in Spark, and set the file path and the number of partitions; S2: The computer node reads the data of the block in HDFS and converts the data into<key,value> In the form of key-value pairs, key is the condition attribute value of diabetes, value is the decision attribute value of diabetes, and the converted data key-value pairs are sent to the master node for aggregation; The specific steps of step S2 are as follows: Step S2.1: According to the characteristics of electronic medical record data and the needs of disease diagnosis, select appropriate condition attributes and decision attributes to generate data set S, where S is defined as follows: S = {U, C∪D, V, f}, where U is the original data set to be processed, U = {x1, x2, ..., x i ,...,x n } represents the set of all patients, x i represents the i-th patient, n represents the number of all patients; C represents the non-empty finite conditional attribute set of the patient's electronic medical record, and D represents the non-empty finite decision attribute set of the patient's electronic medical record. V is a description of the elements in U, and f is a mapping from U to a target set, f = U × (C ∪ D) → V; Step S2.2: Use the map method in SparkCore to transform the data into<key,value> The RDD format of key-value pairs. RDD is a distributed elastic dataset and is a data type in Spark. The key is a Tuple tuple. The elements in the tuple are the conditional attribute sets of the object. If the conditional attributes n are greater than or equal to 22, the data type is converted to ArrayBuffer instead of Tuple, and the value is the decision attribute set of the object. S3: After the master node obtains the summarized data key-value pairs, it retains one of the key-value pairs with the same key and value, and increases the value of the remaining key-value pair by 1. For the key-value pairs with the same key but different values, it deletes the data with the smaller value and retains the data with the larger value. The specific steps of step S3 are as follows: Step S3.1: Preprocess the data by using the reduceByKey method in SparkCore. For data with the same key value but different value, only keep the one with the larger value. For data with the same key and value, only keep one and perform the value+1 operation. Step S3.2: Use the distributed computing framework and parallel processing method provided by the Spark platform to divide the original data into equivalence classes. There is an equivalence relation IND(B) defined as follows: An equivalence relation is an indiscernibility relation, where the equivalence class of an object x to an attribute set B is represented by U / IND(B) or U / B; Due to the distributed computing framework and parallel processing capabilities provided by the Spark platform, the data set S is divided into several data subsets S′. In the Spark framework, the communication between the master node and the partition node executor is established through the built-in Netty communication protocol. The master node divides each data subset S i ' Passed to executor i On the node, each partition is calculated in parallel, the attribute values ​​of each object are divided, and an equivalence class partition table is generated; S4: The master node will perform the operation in step S3.<key,value> The key-value pair data is broadcast to each child node. Each child node performs rough set modeling on the data, calculates the upper and lower approximate space, attribute importance, attribute dependency, and attribute relevance. The child node sends the calculated local attribute importance, local attribute dependency, and local attribute relevance to the main node for aggregation. The main node obtains the local information of each node and sums it up to obtain the global attribute evaluation function value. The specific steps of step S4 are as follows: Step S4.1: Each child node calculates the upper and lower approximation, attribute importance, attribute dependency, and attribute correlation evaluation index of each partition attribute in parallel and generates a local evaluation function index; The upper and lower approximations are defined as: in R (X) is the lower approximation, is an upper approximation, Y is a subset of the conditional attribute set, X is the domain, is an empty set, where the upper approximate set represents the set of objects in U that may belong to X, and the lower approximate set represents the set of objects that belong to X. The lower approximate set is also called the positive domain, denoted by POS R (X); Attribute Dependency Defined as: Among them A m is the attribute, m is the index of the attribute, For attribute A m The corresponding positive domain, |·| represents the cardinality, is the cardinality of the positive domain, |U| represents the cardinality of the domain; The attribute correlation ρ is defined as: Where P is a subset of the conditional attribute set C, |P| represents the cardinality of P, and γ Am (D) is attribute A m Attribute dependence on the decision attribute set D; The importance of attribute subset P is σ P is defined as: where γ P (D) represents the dependency of attribute subset P on D, Indicates that attribute subset P excludes attribute A m The latter is about the dependence of D; Step S4.2: When the child node completes the computing task, the master node aggregates the computing results of each executor node to the master node through the collect method in SparkCore; Step S4.3: The master node obtains the local evaluation function of the attribute and combines it to generate the global evaluation function Γ, which is expressed as: C=ωr+λ(1-ω)c P (D)+(1-λ)(1-ω)σ(D,A) (6) Where ω, λ are two hyperparameters and ω, λ∈(0,1), the generated global evaluation function Γ is used as the objective function value of the CQBPSO algorithm; S5: The main node combines the obtained attribute evaluation function with the CQBPSO algorithm to model and achieve the simplification of diabetes attributes; The specific steps of step S5 are as follows: Step S5.1: Initialize the particle position and velocity parameters, and use binary coding to discretize the data. The particle position is represented by a binary bit string of length N, where N is the number of attributes. The attribute value consists of '0' and '1', and each position vector is an attribute subset; The velocity v of each particle is represented by a positive integer and satisfies v∈[1,V max ],V max For custom integers, how many bits of the particle's characteristics should be changed at a specific moment; Step S5.2: Update the local optimal position and global optimal position of each particle according to the fitness function (6); Step S5.3: Update the speed and position of each particle according to the speed update formula (7) and the position update formula (8); Where t represents the number of iterations, j represents the j-th dimension component of the i-th particle, j = 1, 2, 3, ..., n, represents the personal optimal position of the j-dimensional component of the i-th particle, represents the global optimal position of the j-dimensional component of the ith particle, represents the particle position of the j-dimensional component of the ith particle, rand1() and rand2() are random functions with values ​​between [0,1], δ is the inertia weight, c1 and c2 are individual learning factors and social learning factors respectively, and c1 and c2 are updated according to the nonlinear adjustment strategy in the co-evolution theory; where c 1max ,c 1min ,c 2max ,c 2min Indicates the maximum and minimum values ​​of c1 and c2, and iter is the number of iterations. max is the maximum number of iterations; Update δ according to the adaptive inertia weight update formula: δ max ,δ min To set the maximum and minimum values ​​of δ; In a certain dimension of the particle, the velocity vector represents the probability that the position vector becomes '1'. The following sigmoid function is used to convert all continuous values ​​of this dimension in the velocity vector into probability values ​​in the range of [0,1]; The update calculation of the particle position is as follows: Where r is a random number in [0,1], s(·) represents the position The probability distribution of , z is the independent variable; Step S5.4: Calculate the evaluation function for the particle according to the above formula (6). For a binary-coded position vector, a value of '0' represents that the attribute is not selected, and a value of '1' represents that the attribute is selected.

Images