A sarcopenia risk factor analysis device based on a DQN network

Abstract

The application belongs to the field of data technology processing, and particularly relates to a sarcopenia risk factor analysis device based on a DQN network. The device comprises a collection device for collecting sarcopenia risk factor characteristic data, a computer for receiving the sarcopenia patient risk factor characteristic data, the computer uses mRMR for feature selection first to obtain a candidate feature subset, trains a DQN network based on the candidate feature subset to screen out sarcopenia risk factors, and in the DQN network training process, an association analysis strategy is introduced to select features that may exist in association, accelerate the convergence of the DQN network, and improve the accuracy of the finally obtained risk factor results.

Description

Technical Field

[0001] This invention belongs to the field of data technology processing, specifically a sarcopenia risk factor analysis device based on DQN network. Background Technology

[0002] Sarcopenia is a systemic skeletal muscle disorder characterized by muscle loss associated with aging, and is a progressive disease affecting older adults. The occurrence of sarcopenia in older adults is associated with multiple risk factors, including lack of physical activity with increasing age, decreased secretion of growth hormone and sex hormones, reduced vitamin D levels, malnutrition, and changes in gut microbiota. Analyzing these risk factors is crucial not only for sarcopenia screening and early prevention but also for providing insights into patient prognosis. With advancements in data mining techniques, feature selection has achieved significant results in risk factor analysis. Algorithms can be used to select the most relevant risk factors for the disease, enabling more accurate decision-making by physicians.

[0003] Feature selection can be categorized into filtering, wrapping, and embedding methods. Filtering methods rank features based on relevance, ignoring feature dependencies and interactions. Wrapping methods use prediction accuracy as the objective function, achieving high accuracy, but for high-dimensional data, the feature subspace grows exponentially, leading to excessive computational cost and a susceptibility to overfitting. Embedding methods integrate feature selection into the training process, reducing computational cost but making it difficult to construct the optimization function.

[0004] Reinforcement learning can interact with the environment, learn from action rewards, and seek long-term optimal decisions, offering great potential for feature subspace exploration. Chinese patent CN112488188 discloses a feature selection method based on deep reinforcement learning. This algorithm selects a feature subset based on an ε-greedy policy, then feeds the subset into a classifier for evaluation, and finally uses deep reinforcement learning to optimize the action selection strategy to obtain the optimal feature subset. However, this method only considers the reward assignment of individual features and does not consider features with joint effects. These features are easily overlooked, thus affecting the obtained feature subset. Furthermore, this method has low efficiency in processing high-dimensional datasets and cannot adapt to real-world medical big data environments. Summary of the Invention

[0005] The purpose of this invention is to provide a sarcopenia risk factor analysis device based on DQN network, which generates a subset of candidate features through the mRMR algorithm, effectively reducing the data dimensionality; and introduces a joint effect analysis strategy to select features that may have joint effects, thereby accelerating the convergence of DQN network and improving the accuracy of the final risk factor analysis results.

[0006] To achieve the above objectives, the present invention adopts the following technical solution;

[0007] A device for analyzing sarcopenia risk factors based on a DQN network, comprising a data acquisition device for collecting risk factor characteristic data of sarcopenia, and a computer for receiving the risk factor characteristic data of sarcopenia patients, the computer being programmed to perform the following steps:

[0008] Step 1: Collect hazard factor characteristic data and preprocess the collected hazard factor characteristic data;

[0009] Step 2: Use a feature selection algorithm to sort the preprocessed risk factor feature data, select the top N most relevant and least redundant features to obtain a candidate feature subset R;

[0010] Step 3: Construct a risk factor analysis model based on DQN, which includes an active action network, a target network, and a data pool;

[0011] Step 4: Input the candidate feature subset R as the initial feature space S into the DQN-based hazard factor analysis model to train the DQN-based hazard factor analysis model;

[0012] Step 5: Repeat step 4 to train the DQN-based hazard factor analysis model multiple times. Select the feature subset obtained in the round with the highest reward value during training as the final feature subset output to obtain the final selected hazard factors.

[0013] Furthermore, the data collected in step 1 includes five aspects of risk factor characteristics: demographic characteristics, related disease characteristics, lifestyle characteristics, scale characteristics, physical muscle characteristics, and biochemical indicator characteristics.

[0014] Among them, demographic characteristics include: gender, height, age, weight, and body mass index;

[0015] Lifestyle characteristics include: smoking history, drinking history, and nutritional status;

[0016] Related disease characteristics include: type 2 diabetes, hypertension, chronic kidney disease, heart failure, and osteoporosis in the elderly;

[0017] Muscle characteristics include: muscle strength, bodily function, and muscle mass;

[0018] Biochemical indicators include: blood glucose, uric acid, hemoglobin, triglycerides, and cholesterol.

[0019] Furthermore, the data preprocessing in step 1 includes the following steps:

[0020] Step 1.1: Delete risk factors with many missing values ​​and delete obviously abnormal data;

[0021] Step 1.2: Use statistical interpolation to fill in missing data; for discrete features in the data, use the feature value with the highest frequency in all data to fill in the missing values; for continuous features in the data, use the overall average value of all data to fill in the missing values.

[0022] Step 1.3: Standardize the data obtained in Step 1.2 using the min-max method.

[0023] Furthermore, in step 1.3, one-hot encoding is used to map discrete features in the data; and continuous features in the data are directly mapped to the [0,1] interval to eliminate the dimensional influence between different risk factors.

[0024] Furthermore, the feature selection algorithm used in step 2 is the mRMR algorithm.

[0025] Furthermore, step 4 includes the following steps:

[0026] Step 4.1: Initialize the data pool according to the actual capacity, initialize the main action network and the target network, randomly initialize the parameters θ of the main action network, and the parameters θ*=θ of the target network;

[0027] Step 4.2: Extract features S from the initial feature space S. t The input is fed into the main action network, which generates action sequence A. t ';

[0028] Step 4.3: Determine if the threshold ε is less than the set value. If so, then use the DQN combined action analysis strategy to analyze the action sequence A obtained in step 4.2. t 'Update the sequence to obtain the action sequence A to be executed.' t If not: then the action sequence A obtained in step 4.2 will be... t 'As the sequence of actions to be performed, A' t Calculate the sequence of actions A t The subsequent reward value R t Then for the next feature S t+1 Make a selection and (S) t A t ,R t ,S t+1 ) is stored as a set of experience data in a data pool;

[0029] Step 4.4: Repeat steps 4.2 to 4.3 iteratively until the data pool reaches its maximum capacity. Then, randomly select a specified number of empirical data points, and denote one of them as {S}. j A j ,R j ,S (j+1)};

[0030] Step 4.5: Transfer the data {S} j A j ,R j ,S (j+1) The target network is provided with the information to calculate the target network's performance on feature S. j The highest reward value y obtained by performing the following action j ;

[0031] Step 4.6: Calculate the predicted reward value R of the active action network. j With the target network reward value y j The error between them is used to calculate the loss function of the active action network and update the parameters θ of the active action network; after the target network executes C steps, let θ* = θ and update the parameters of the target network once.

[0032] Furthermore, step 4.2 generates action sequence A using the following steps. t ':

[0033] Execute the ε-greedy policy to explore action sequence A t ':

[0034] Let the threshold of the ε-greedy policy be ε, and extract features S from the initial feature space S. t A random number m is generated according to the ε-greedy strategy; if m is less than or equal to the set threshold ε, then each feature in the initial feature space randomly selects an action, and the actions corresponding to all features constitute the action sequence A. t '; ε is initially set to 1. After each round of feature selection, it decays at a rate of ε = 0.998ε. When ε = 0.05, the decay stops; Action sequence A t ' is represented as:

[0035]

[0036] Among them, A t ' is a vector representing a sequence of actions, and N is the number of actions. Represents the elements that make up the action sequence, when =1 indicates that the corresponding feature is selected. This indicates that the corresponding feature has been deselected;

[0037] If m is greater than the set threshold ε, then S tThe input is fed into a DQN network for training to obtain action sequence A. t '.

[0038] Furthermore, in step 4.3, the DQN joint action analysis strategy is used to update the action sequence A′. t The following steps are included:

[0039] Let action sequence A t 'For the current action sequence A' t The previous action sequence was

[0040] The two action sequences are compared to identify two similar features.

[0041] Determine whether the reward value obtained when both features are selected is better than other cases. If so, then for action sequence A... t and action sequence A t-1 The corresponding features are jointly distributed and matched; the other cases refer to the reward values ​​calculated when one feature is selected, the other is not selected, or neither feature is selected.

[0042] Based on the joint distribution matching results, the action sequence A obtained in step 4.2 is... t 'Update the feature subset to obtain S' t In this case, the sequence of actions to be executed

[0043] Furthermore, step 4.3 involves calculating the reward value using the following steps:

[0044] Let FS be denoted as based on action sequence A. t The selected feature subset, |FS|, is the number of features selected from this feature subset. FS is input into a support vector machine for training, and the action A is calculated. t Reward value R t The specific settings for the reward value are as follows:

[0045] (1) Calculate the classification accuracy A respectively. CC Feature subset FS correlation R v And the redundancy R of the feature subset FS d :

[0046] The classification accuracy A CC The number of risk factors correctly predicted by the support vector machine is compared with |FS|.

[0047] The feature subset FS correlation R v The calculation formula is:

[0048]

[0049] In the formula, L is the label vector, i.e., whether or not sarcopenia exists, and x i The i-th feature in the subset FS represents the feature i. i The formula for calculating mutual information is: (L)

[0050] In the formula x i y j Let represent the i-th and j-th features in the feature subset FS, respectively; p(x, y) is the joint distribution of x and y; and p(x) and p(y) are the marginal distributions of x and y, respectively.

[0051] The feature subset FS redundancy R d The calculation formula is:

[0052] In the formula, x i y j Let represent the i-th and j-th features in the feature subset FS, respectively; p(x, y) is the joint distribution of x and y; and p(x) and p(y) are the marginal distributions of x and y, respectively.

[0053] (2) Based on classification accuracy A CC Feature subset FS correlation R v and feature subset FS redundancy R d The reward value is calculated using the following formula:

[0054] R = Acc + αR v +βR d

[0055] The values ​​of α and β change with |FS|:

[0056]

[0057]

[0058] In the formula, N is the number of candidate features obtained by the feature selection algorithm in step 2; when the number of selected candidate features is too large, the redundancy of the feature subset FS will play a greater reward role; when the number of feature subset FS is small, the relevance of the feature subset FS will play a greater role.

[0059] This invention provides a sarcopenia risk factor analysis device based on DQN network. First, mRMR is used for feature selection to obtain a subset of candidate features, which improves the efficiency of subsequent network model construction. Second, a joint feature analysis strategy is added during the training process of DQN network, which can guide the network to consider features with joint effects during training, improve the training efficiency of DQN network, help select the optimal feature subset, and improve the accuracy of risk factor output results. Attached Figure Description

[0060] Figure 1 This is a diagram of the overall network structure.

[0061] Figure 2 The diagram shows the structure of the main action network and the target network. Detailed Implementation

[0062] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0063] Before elaborating on the detailed technical solution, we define Z = {z1, z2, ..., zM} as the set of risk factors for sarcopenia, where M is the total number of risk factors, and zi (i = 1, ..., M) represents the i-th risk factor. For ease of description, we will refer to the risk factors as characteristics below.

[0064] This embodiment provides a sarcopenia risk factor analysis device based on a DQN network. The device includes a data acquisition unit for collecting risk factor characteristic data of sarcopenia, and a computer for receiving the risk factor characteristic data of sarcopenia patients. The computer is programmed to perform the following steps:

[0065] Step 1: Data collection and preprocessing:

[0066] The collected data included five risk factor characteristics of patients with sarcopenia: demographic characteristics, related disease characteristics, lifestyle characteristics, scale characteristics, and body muscle characteristics. Demographic characteristics included: sex, height, age, weight, and body mass index; lifestyle characteristics included: smoking history, alcohol consumption history, and nutritional status; related disease characteristics included: type 2 diabetes in the elderly, hypertension, chronic kidney disease, heart failure, and osteoporosis; body muscle characteristics included: muscle strength, physical function, and muscle mass; and biochemical indicators included: blood glucose, uric acid, hemoglobin, triglycerides, and cholesterol.

[0067] The collected data is preprocessed, including the following steps:

[0068] Step 1.1: Delete risk factors with many missing values ​​and delete obviously abnormal data; after deletion, this embodiment finally obtained data from 335 patients with sarcopenia, which included 38 risk factors.

[0069] Step 1.2: Use statistical interpolation to fill in missing data; for discrete features in the data, use the feature value that appears most frequently in all data to fill in the missing values; for continuous features in the data, use the overall average value of all data to fill in the missing values.

[0070] Step 1.3: Standardize the data obtained in Step 1.2 using the min-max method. During standardization, one-hot encoding is used for discrete features; continuous features are directly mapped to the [0,1] interval to eliminate the dimensional influence between different risk factors.

[0071]

[0072] Step 2: Use the mRMR algorithm to sort the preprocessed hazard factor feature data, select the top N most relevant and least redundant features to obtain a candidate feature subset R = {r1, r2, ..., r...} N}; where N is half of the entire feature set, r i (i = 1...N) represents the i-th most important feature selected by the mRMR algorithm.

[0073] Step 3: Construct a hazard factor analysis model based on DQN. This model includes an active action network, a target network, and a data pool. The active action network and the target network have the same network structure, both including an input layer, a hidden layer, and an output layer. The DQN network structure diagram, the active action network structure, and the target network structure are shown below. Figure 1 , 2 As shown.

[0074] Data pool D stores records (S) t A t ,R t ,S t+1 ), where S t For the selected state subspace (1, 0, 1, ..., 1) N*1 , represents the selected feature set (1 means this feature is selected, 0 means this feature is not selected), A t N*1 For each feature's execution action (1 for selecting the feature, 0 for deselecting the feature), R t The reward S obtained for this feature subspace t+1 To execute A tThe next state subspace is then determined. The data pool capacity is set according to requirements, but it should not be too small. If it is set too small, newly generated samples will be continuously discarded; if it is set too large, the randomness during ε-greddy filling of the data pool will be too high, affecting the training of subsequent networks. In this embodiment, the data pool capacity is set to 2000.

[0075] Here is an example of a data pool:

[0076]

[0077] The DQN network obtains the reward value corresponding to each action under given state conditions, and selects the best reward value for continuous updating and training.

[0078] Step 4: Input the candidate feature subset R as the initial feature space S into the DQN-based hazard factor analysis model to train the DQN-based hazard factor analysis model; the training steps are as follows:

[0079] Step 4.1: Initialize the data pool according to the actual capacity, initialize the main action network and the target network, randomly initialize the parameters θ of the main action network, and the parameters θ*=θ of the target network;

[0080] Step 4.2: Extract features S from the initial feature space S. t The input is fed into the main action network, which generates action sequence A. t The specific operation includes the following steps:

[0081] Execute the ε-greedy policy to explore action sequence A t ':

[0082] Let the threshold of the ε-greedy policy be ε, and extract features S from the initial feature space S. t A random number m is generated according to the ε-greedy strategy; if m is less than or equal to the set threshold ε, then each feature in the initial feature space randomly selects an action, and the actions corresponding to all features constitute the action sequence A. t '; ε is initially set to 1. After each round of feature selection, it decays at a rate of ε = 0.998ε. When ε = 0.05, the decay stops; Action sequence A t ' is represented as:

[0083]

[0084] Among them, A t ' is a vector representing a sequence of actions, and N is the number of actions. Represents the elements that make up the action sequence, when =1 indicates that the corresponding feature is selected. This indicates that the corresponding feature has been deselected;

[0085] If m is greater than the set threshold ε, then S t The input is used to train a DQN-based hazard factor analysis model to obtain action sequence A. t The process is as follows:

[0086] The first layer is the input layer, with input S. t The hidden vector of the first layer is obtained as follows:

[0087] h1 = ReLU(W1S) t +b1)

[0088] Among them, W1 N*N and The first layer consists of the weight parameters and bias vector of the input layer; the second layer is a hidden layer, and the hidden vector of the second layer is obtained as follows:

[0089] h2 = ReLU(W2h1 + b2)

[0090] in and These are the weight parameters and bias vector for the second hidden layer;

[0091] The third layer is the output layer, which has N neurons, each representing a feature. The calculation formula is as follows:

[0092] y i =ReLU(W 2i h2+b 2i )

[0093] Where y i ={Q(S t ,a i1 ),Q(S t ,a i0 )} 2*1 Q(S) t ,a i1 Q(S) represents the Q-value for selecting the i-th feature in the current state. t ,a i0 ) represents the Q value for not selecting this feature. and These represent the weight and bias value corresponding to the i-th feature, respectively.

[0094] The output action sequence is as follows:

[0095] A t '={max(y1),max(y2),....max(y N )} N*1 , where: max(y i)=max(Q(S t ,a i1 ),Q(S t ,a i0 ))

[0096] Step 4.3: Executing joint strategy optimization requires a certain number of records in the data pool. Therefore, it is necessary to determine whether the threshold ε is less than the set value. If so, the DQN joint action analysis strategy is used to optimize the action sequence A obtained in step 4.2. t 'Update the sequence to obtain the action sequence A to be executed.' t If not: then the action sequence A obtained in step 4.2 will be... t 'As the sequence of actions to be performed, A' t Calculate the sequence of actions A t The subsequent reward value R t Then for the next feature S t+1 Make a selection and (S) t A t ,R t ,S t+1 ) is stored as a set of experience data in the data pool.

[0097] This embodiment utilizes the combined effect present in the features to achieve the desired action sequence A. t Optimize. The optimization process is as follows:

[0098] Action sequence A t 'As the current action sequence A' t The previous action sequence was Sequence A t-1 and A t If there is a change in action, that is Where i∈{1,2,...,N} are the indices corresponding to the features, the feature indices are added to the set F. During each training process, the set F stores features that may have joint effects. For example, if F={3,5,8,…,N} represents the 3rd, 5th, 8th,…,Nth features satisfying the conditions respectively.

[0099] Initialize a zero matrix JF, whose dimension is the number of features in set F, and the diagonal elements are denoted as ∞. The remaining elements JF[i][j] represent the action sequence similarity between feature i and feature j in set F, and are calculated as follows:

[0100] Randomly select T<=300 data points from the data pool, and denote the action sequence of feature i as α. i ={α i1 ,α i2 ,α i3 ,,α iT} T*1, represents the action corresponding to each of the T data points for feature i, for example: α i ={1,1,0,...0} T*1 Then the action sequence similarity between feature i and feature j

[0101] If we select the features i and j corresponding to the minimum value in JF, then the sum of the rewards when features i and j corresponding to the minimum value in JF are selected together is expressed as:

[0102]

[0103] R t This represents the reward value corresponding to each record, which can be obtained directly from the records in the data pool.

[0104] The sum of rewards when features i and j are not jointly selected is represented as:

[0105]

[0106] remember

[0107]

[0108]

[0109] like Therefore, it is assumed that the average reward is greater when the action sequences are similar and common choices are made, and these two features are considered to have a joint effect. For the action sequence A obtained in step 4.2... t 'Updating, action coming soon' and Change to Where i and j represent the i-th and j-th features selected from matrix JF, respectively, and t represents the t-th training action, resulting in the final feature subset S. t In this case, the sequence of actions to be executed

[0110] In this embodiment, the execution action sequence A is calculated. t The subsequent reward value R t The process is as follows:

[0111] Let FS be denoted as based on action sequence A. t The selected feature subset, |FS|, is the number of features selected from this feature subset. FS is input into a support vector machine for training, and the action A is calculated. t Reward value R t The specific settings for the reward value are as follows:

[0112] (1) Calculate the classification accuracy A respectively. CCFeature subset FS correlation R v And the redundancy R of the feature subset FS d :

[0113] The classification accuracy A CC The number of risk factors correctly predicted by the support vector machine is compared with |FS|.

[0114] The feature subset FS correlation R v The calculation formula is:

[0115]

[0116] In the formula, L is the label vector, i.e., whether or not sarcopenia exists, and x i The i-th feature in the feature subset FS. I(x) i The formula for calculating mutual information is: (L)

[0117]

[0118] In the formula x i y j Let represent the i-th and j-th features in the feature subset FS, respectively; p(x, y) is the joint distribution of x and y; and p(x) and p(y) are the marginal distributions of x and y, respectively.

[0119] The feature subset FS redundancy R d The calculation formula is:

[0120] In the formula, x i y j Let represent the i-th and j-th features in the feature subset FS, respectively; p(x, y) is the joint distribution of x and y; and p(x) and p(y) are the marginal distributions of x and y, respectively.

[0121] (2) Based on classification accuracy A CC Feature subset FS correlation R v and feature subset FS redundancy R d The reward value is calculated using the following formula:

[0122] R = Acc + αR v +βR d

[0123] The values ​​of α and β change with |FS|:

[0124]

[0125]

[0126] In the formula, N is the number of candidate features obtained by the feature selection algorithm in step 2; when the number of selected candidate features is too large, the redundancy of the feature subset FS will play a greater reward role; when the number of feature subset FS is small, the relevance of the feature subset FS will play a greater role.

[0127] Step 4.4: Repeat steps 4.2 to 4.3 iteratively until the data pool reaches its maximum capacity. Then, randomly select a specified number of empirical data points, and denote one of them as {S}. j A j ,R j ,S (j+1)};

[0128] Step 4.5: Transfer the data {S} j A j ,R j ,S (j+1) The target network is provided with the information to calculate the target network's performance on feature S. j The highest reward value y obtained by performing the following action j ;, highest reward value y j The calculation formula is:

[0129] y j =R j +γmax A* Q * (S j+1 A * ;θ * )where y j Represents the target network Q … In state S j The highest reward value obtained from performing the following action, R j Indicates that in state S j The reward value obtained from the action sequence is then processed through the network. γ is a decay factor, which is set to 0.99 in this embodiment. A… Q … (S j+1 A … ;θ * ) indicates that through the target network Q … The maximum value that can be obtained subsequently. A * Indicates that in state S j+1 The action performed to obtain the maximum reward under certain circumstances.

[0130] Step 4.6: Calculate the predicted reward value R of the active action network. j With the target network reward value y j The error between them is used to calculate the loss function of the active action network and update the parameters θ of the active action network; after the target network executes C steps, let θ* = θ and update the parameters of the target network once.

[0131] loss(θ) = (y j -Q(S j A j ,θ)) 2

[0132] The θ parameter is updated as follows:

[0133]

[0134] Where δ represents the learning rate, set to δ = 0.005. This indicates the derivative with respect to the parameter.

[0135] Step 5: Repeat step 4 to iterate the training of the DQN-based hazard factor analysis model multiple times. In this embodiment, the training iteration is 500 rounds. Select the feature subset obtained from the round with the highest reward value in these 500 rounds and output it as the final feature subset to obtain the final selected hazard factors.

Claims

1. A sarcopenic muscular dystrophy risk factor analysis device based on a DQN network, the device comprising a data acquisition unit for collecting risk factor characteristic data of sarcopenic muscular dystrophy, and a computer for receiving the risk factor characteristic data of sarcopenic muscular dystrophy patients, characterized in that: The computer is programmed to perform the following steps: Step 1: Collect hazard factor characteristic data and preprocess the collected hazard factor characteristic data; Step 2: Use a feature selection algorithm to sort the preprocessed risk factor feature data, select the top N most relevant and least redundant features to obtain a candidate feature subset R; Step 3: Construct a risk factor analysis model based on DQN, which includes an active action network, a target network, and a data pool; Step 4: Input the candidate feature subset R as the initial feature space S into the DQN-based hazard factor analysis model to train the DQN-based hazard factor analysis model; specifically including: Step 4.1: Initialize the data pool according to the actual capacity, initialize the main action network and the target network, randomly initialize the parameters θ of the main action network and the parameters θ*=θ of the target network; Step 4.2: Extract features S from the initial feature space S. t The input is fed into the main action network, which generates action sequence A. t '; Step 4.3: Determine if the threshold ε is less than the set value. If so, then use the DQN combined action analysis strategy to analyze the action sequence A obtained in step 4.

2. t 'Update the sequence to obtain the action sequence A to be executed.' t If not: then the action sequence A obtained in step 4.2 will be... t 'As the sequence of actions to be performed, A' t Calculate the sequence of actions A t The subsequent reward value R t Then for the next feature S t+1 Make a selection and (S) t A t , R t , S t+1 The data is stored as a set of experience data in the data pool; the action sequence is updated using a DQN joint action analysis strategy. The following steps are included: Let action sequence A t 'For the current action sequence A' t The previous action sequence was ; The two action sequences are compared to identify two similar features. Determine whether the reward value obtained when both features are selected is better than other cases. If so, then for action sequence A... t and action sequence A t-1 The corresponding features are jointly distributed and matched; the other cases refer to the reward values ​​calculated when one feature is selected, the other is not selected, or neither feature is selected. Based on the joint distribution matching results, the action sequence A obtained in step 4.2 is... t 'Update the feature subset to obtain S' t In this case, the sequence of actions to be executed ; Step 4.4: Repeat steps 4.2 to 4.3 iteratively until the data pool reaches its maximum capacity. Then, randomly select a specified number of empirical data points, and denote one of them as {S}. j A j ,R j ,S (j+1) }; Step 4.5: Transfer the data {S} j A j ,R j ,S (j+1) The target network is provided with the information to calculate the target network's performance on feature S. j The highest reward value y obtained by performing the following action j ; Step 4.6: Calculate the predicted reward value R of the active action network. j With the target network reward value y j The error between them is used to calculate the loss function of the main action network and update the parameters θ of the main action network; after the target network executes C steps, let θ*=θ and update the parameters of the target network once. Step 5: Repeat step 4 to train the DQN-based hazard factor analysis model multiple times. Select the feature subset obtained in the round with the highest reward value during training as the final feature subset output to obtain the final selected hazard factors.

2. The sarcopenia risk factor analysis device based on DQN network as described in claim 1, characterized in that: The data collected in step 1 includes: Risk factors are characterized in five aspects: demographic characteristics, related disease characteristics, lifestyle characteristics, scale characteristics, physical muscle characteristics, and biochemical indicators. Among them, demographic characteristics include: gender, height, age, weight, and body mass index; Lifestyle characteristics include: smoking history, drinking history, and nutritional status; Related disease characteristics include: type 2 diabetes, hypertension, chronic kidney disease, heart failure, and osteoporosis in the elderly; Muscle characteristics include: muscle strength, bodily function, and muscle mass; Biochemical indicators include: blood glucose, uric acid, hemoglobin, triglycerides, and cholesterol.

3. The sarcopenia risk factor analysis device based on DQN network as described in claim 1, characterized in that: The data preprocessing in step 1 includes the following steps: Step 1.1: Delete risk factors with many missing values ​​and delete obviously abnormal data; Step 1.2: Use statistical interpolation to fill in missing data; for discrete features in the data, use the feature value with the highest frequency in all data to fill in the missing values; for continuous features in the data, use the overall average value of all data to fill in the missing values. Step 1.3: Standardize the data obtained in Step 1.2 using the min-max method.

4. The sarcopenia risk factor analysis device based on DQN network as described in claim 3, characterized in that: In step 1.3, One-hot encoding is used to map discrete features in the data; continuous features are directly mapped to the [0,1] interval to eliminate the dimensional influence between different risk factors.

5. The sarcopenia risk factor analysis device based on DQN network as described in claim 1, characterized in that: The feature selection algorithm used in step 2 is the mRMR algorithm.

6. The sarcopenia risk factor analysis device based on DQN network as described in claim 1, characterized in that: In step 4.2, the action sequence A is generated using the following steps. t ': Execute the ε-greedy policy to explore action sequence A t ': Let the threshold of the ε-greedy policy be ε, and extract features S from the initial feature space S. t A random number m is generated according to the ε-greedy strategy; if m is less than or equal to the set threshold ε, then each feature in the initial feature space randomly selects an action, and the actions corresponding to all features constitute the action sequence A. t '; ε is initially set to 1. After each round of feature selection, it decays at a rate of ε = 0.998ε. When ε = 0.05, the decay stops. Action Sequence A t ' is represented as: Among them, A t ' is a vector representing a sequence of actions, and N is the number of actions. Represents the elements that make up the action sequence, when =1 indicates that the corresponding feature is selected. =0 indicates that the corresponding feature is deselected; If m is greater than the set threshold ε, then S t The input is fed into a DQN network for training to obtain action sequence A. t '.

7. The sarcopenia risk factor analysis device based on DQN network as described in claim 1, characterized in that: Let FS be denoted as based on action sequence A. t The selected feature subset, |FS|, is the number of features selected from this feature subset. FS is input into a support vector machine for training, and the action A is calculated. t Reward value R t The specific settings for the reward value are as follows: (1) Calculate the classification accuracy A respectively. CC Feature subset FS correlation And the redundancy R of the feature subset FS d : The classification accuracy A CC The number of risk factors correctly predicted by the support vector machine is compared with |FS|. The feature subset FS correlation The calculation formula is: In the formula, L is the label vector, i.e., whether or not sarcopenia exists. The i-th feature in the subset FS represents the feature. Mutual information calculation formula: In the formula x i y j Let represent the i-th and j-th features in the feature subset FS, respectively; p(x, y) is the joint distribution of x and y; and p(x) and p(y) are the marginal distributions of x and y, respectively. The feature subset FS redundancy R d The calculation formula is: In the formula, y j Let represent the i-th and j-th features in the feature subset FS, respectively; p(x, y) is the joint distribution of x and y; and p(x) and p(y) are the marginal distributions of x and y, respectively. (2) Based on the classification accuracy A CC Feature subset FS correlation and feature subset FS redundancy R d The reward value is calculated using the following formula: in and The value changes with |FS|: In the formula, N is the number of candidate features obtained by the feature selection algorithm in step 2; when the number of selected candidate features is too large, the redundancy of the feature subset FS will play a greater reward role; when the number of feature subset FS is small, the relevance of the feature subset FS will play a greater role.

Images