Method, device, equipment and medium for improving random forest prediction results

Abstract

This application discloses a method, apparatus, device, and medium for improving random forest prediction results. The method includes: obtaining an original sample set D; extracting a training sample set with replacement from the original sample set D and randomly extracting a feature subset; generating a decision tree based on the training sample set and the feature subset; calculating the out-of-bag error rate of the decision tree based on the out-of-bag data of the decision tree, and calculating the weight of the decision tree based on the out-of-bag error rate; and weighting the weights of the decision trees to calculate the prediction results of the random forest prediction model. This application significantly improves the prediction accuracy of the random forest.

Description

Technical Field

[0001] The present application relates to the fields of data processing technology and digital medicine, and in particular to a method, apparatus, device, and medium for improving random forest prediction results. Background Art

[0002] Random forest prediction models are widely used in many technologies and fields, such as text classification, sentiment classification, semantic recognition, and intent recognition.

[0003] For example, in the online consultation system of digital medicine, auxiliary models are often configured to identify patient intentions. The random forest prediction model can be selected as an auxiliary model to help doctors complete online consultations.

[0004] However, the final prediction result output by the random forest prediction model is the mean or mode of the prediction results of all decision trees. Even if the accuracy of different decision trees is different, the decision trees with high accuracy and low accuracy have the same weight, and this part of the information is ignored. At the same time, due to the use of random sampling with replacement, some samples in the training set are never selected, resulting in the loss of sample information and the failure of the model to be fully trained. Summary of the Invention

[0005] In response to the above situation, the embodiments of the present application propose a method, device, equipment and medium for improving random forest prediction results to overcome or at least partially overcome the shortcomings of the existing technology.

[0006] In a first aspect, an embodiment of the present application provides a method for improving random forest prediction results, the method comprising:

[0007] Get the original sample set D;

[0008] Extracting a training sample set with replacement from the original sample set D and randomly extracting a feature subset, and generating a decision tree based on the training sample set and the feature subset;

[0009] Calculating an out-of-bag error rate of the decision tree based on out-of-bag data of the decision tree, and calculating a weight of the decision tree according to the out-of-bag error rate;

[0010] The weights of the decision trees are weighted and integrated to calculate the prediction results of the random forest prediction model.

[0011] In a second aspect, an embodiment of the present application further provides a device for improving random forest prediction results, the device comprising:

[0012] Acquisition module, used to obtain the original sample set D;

[0013] The sampling module is used to extract the training sample set with replacement from the original sample set D and randomly extract the feature subset;

[0014] A decision tree generation module, configured to generate a plurality of decision trees based on the training sample set and the feature subset;

[0015] A weight acquisition module, configured to calculate an out-of-bag error rate of the decision tree based on out-of-bag data of the decision tree, and calculate a weight of the decision tree according to the out-of-bag error rate;

[0016] The prediction result calculation module is used to weight the weights of the decision trees and calculate the prediction result of the random forest prediction model.

[0017] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer-executable instructions, wherein the executable instructions, when executed, enable the processor to perform any of the above methods.

[0018] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes any of the above methods.

[0019] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0020] After building all the decision tree models, this application uses each decision tree to predict the out-of-bag data, that is, the data that has not been sampled, so as to calculate the out-of-bag error rate of each decision tree, use the error rate to calculate the weight of each decision tree model, and then comprehensively obtain the prediction results of the random forest prediction model under the weighted condition. In the context of random forest using random replacement sampling method to obtain the training set of decision trees, this application uses the out-of-bag data that has not been sampled to calculate the out-of-bag error rate of each decision tree, ensuring the full use of the sample information in the original training set; using the out-of-bag error rate to calculate the weight of each decision tree, so as to ensure that the weak learner with higher prediction accuracy has a higher voice in the final integrated learning; while retaining the advantages of high efficiency and strong generalization ability of parallel computing, there is no need to use the previous iteration results to update the weight of the learner and the distribution of each sample point as in the boosting algorithm, which greatly saves computing power requirements and greatly improves the prediction accuracy of the random forest prediction results. It can be applied to various scenarios and has a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 A schematic diagram illustrating a process of improving a random forest prediction result according to an embodiment of the present application;

[0023] Figure 2 A schematic diagram illustrating the structure of an apparatus for improving random forest prediction results according to an embodiment of the present application is shown;

[0024] Figure 3 This is a structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0027] Random forests are used in various fields, such as finance and digital medicine. For example, in online consultation systems for digital medicine, auxiliary models are often configured to identify patient intentions. The random forest prediction model can be selected as an auxiliary model to help doctors complete online consultations.

[0028] Random forest is a strong learner based on the bagging method that comprehensively considers the training results of several decision trees. Bagging is entirely based on the bootstrapping principle, using random sampling with replacement to obtain training data before each iteration. This directly reflects a major feature of bagging and is the biggest difference between bagging and boosting methods: each iteration does not rely on the previously established model. In other words, the generated weak models are uncorrelated, thus fully achieving data parallel training.

[0029] Compared with the Boosting method, Bagging has many advantages: since each time a weak learner is constructed, it does not need to consider the previous learning results, so the structure is simple and easy to implement. At the same time, parallel training can be achieved, and the computational efficiency is higher. At the same time, randomly extracting the training set with replacement and randomly selecting some features to construct a weak learner makes the final generated strong learner have strong noise resistance, small variance, not easy to fall into overfitting, and the classification accuracy is also greatly improved.

[0030] However, the random forest prediction model using bagging methods does have room for improvement. First, the final prediction output of the random forest prediction model is the mean or mode of the prediction results of all decision trees. Even if the accuracy of different decision trees varies, high-accuracy and low-accuracy decision trees are given the same weight, and this information is ignored. Furthermore, because random sampling with replacement is used, some samples in the training set are never selected, resulting in a loss of sample information and an inability to fully train the model.

[0031] The prediction result finally output by the random forest prediction model is the mean or mode of the prediction results of all decision trees. Even if the accuracy of different decision trees is different, decision trees with high accuracy and low accuracy have the same weight. This part of the information is ignored. At the same time, due to the use of random sampling with replacement, some samples in the training set are never selected, resulting in the loss of sample information and the failure of the model to be fully trained. In order to solve the above technical problems, this application adopts a method of adjusting weights and using out-of-bag data. After all decision tree models are built, the out-of-bag data of each decision tree, that is, the data that has not been sampled, is used for prediction, thereby calculating the out-of-bag error rate of each decision tree, and using the error rate to calculate the weight of each decision tree model, and then comprehensively obtains the prediction result of the random forest prediction model under the weighted condition.

[0032] The construction process of the random forest prediction model can be roughly expressed in words as follows: Assuming the original sample set D and the sample set size is |D|, n decision trees will be established, and each time m features are selected from the entire feature set M to construct a decision tree. Taking the prediction of abdominal aortic aneurysms in the field of digital medicine as an example, in this scenario, many existing studies have applied artificial intelligence technology to the analysis and prediction of vascular surgery. For example, ultrasound screening and computed tomography (CT) images have been used for the analysis of abdominal aortic aneurysms. However, these methods only focus on the analysis after the abdominal aortic aneurysm forms, and ignore how to effectively predict the abdominal aortic aneurysm before the abdominal aortic aneurysm forms. The solution of this application can not only effectively predict abdominal aortic aneurysms, but also improve the reliability of the prediction results.

[0033] In this regard, this application proposes a method to improve the random forest prediction results, please refer to Figure 1 , including the following steps:

[0034] S10, obtaining the original sample set D;

[0035] First, obtain the original sample set and record it as D.

[0036] The random forest model can be used in various scenarios, such as for auxiliary prediction of various tumors in digital medicine. For the convenience of explanation, the following example uses the random forest model to improve the early prediction results of abdominal aortic aneurysm. It should be noted that this scenario does not constitute any limitation to this application and is only used as an illustrative example.

[0037] In some embodiments of the present application, a random forest prediction model can be used to solve the problem of how to effectively predict abdominal aortic aneurysm before it forms. The original sample set D is used to obtain characteristic information of patients with abdominal aortic aneurysm, where the characteristic information includes age, gender, smoking, obesity, hypertension, and family history of abdominal aortic aneurysm; the characteristic information of the patient of the present invention, such as age, gender, smoking, obesity, hypertension, and family history of abdominal aortic aneurysm, is used as the original sample set D.

[0038] A dataset for training AI models was created based on some characteristics of patients with abdominal aortic aneurysms, including age, smoking, male sex, obesity, hypertension, and family history of aneurysms. In order to facilitate the subsequent random sampling or extraction of feature information in the dataset, the definition generally includes abdominal aortic aneurysms.

[0039]

[0040] The dataset of patient characteristic information is expressed in the form of a data matrix. For example, in the formula, m represents the number of columns of the matrix, n represents the number of rows of the matrix, x represents the characteristic information of the original sample set, y represents the label information of the original sample set D, and S represents the original sample set D. Specifically, in the actual calculation process, in order to obtain the final prediction probability, the characteristic information is represented in binary. For example, for the age feature of a patient, the preset age threshold is 60 years old. For patients aged 60 or above, of course, the preset age threshold can also be 55 years old, 50 years old, etc. There is no specific restriction here. The characteristic information of age is represented in binary. The information is represented by 1. For another example, for the patient's smoking characteristics, 0 represents non-smoking and 1 represents smoking. Of course, it can also be 1 for non-smoking and 0 for smoking. For example, 1 represents smoking, and the average daily smoking amount * age is > 400, and 0 represents non-smoking or average daily smoking amount * age is < 400; 1 represents drinking, and the average weekly drinking amount is > 7 standard drinking units, and 0 represents non-drinking or average weekly drinking amount is < 7 standard drinking units; 1 represents a history of abdominal aortic aneurysm within three generations, and 0 represents no history of abdominal aortic aneurysm within three generations; the remaining feature information is similar and will not be described in further detail here.

[0041] S20 , extracting a training sample set with replacement from the original sample set D and randomly extracting a feature subset, and generating multiple decision trees based on the training sample set and the feature subset.

[0042] The steps include:

[0043] Step S210: Using the Bootstrapping method, |D| training samples are extracted from the original sample set D as the training set d of the current decision tree.

[0044] In the present application, the Bootstrapping method extracts the sample set D, specifically including taking out |D| training samples from the original sample set D with replacement each time to form a new training set as the training set d of the current decision tree.

[0045] Step S220: randomly extract m features from the entire feature set M to construct a feature subset.

[0046] Among them, all features of the original sample set D are used as the overall feature set M.

[0047] Step S230: generating a plurality of decision trees based on the training sample set and the feature subset.

[0048] Step S240: Generate a random forest prediction model consisting of n independent decision trees.

[0049] Specifically, it includes repeating the above steps S210 to S230 n times to generate n independent decision trees, and finally forming a random forest prediction model containing n independent decision trees. For example, the random forest prediction model can be an abdominal aortic aneurysm prediction model.

[0050] The above steps S210 to S240 disclose the steps of constructing a random forest prediction model including n unrelated decision trees.

[0051] Step S30 , calculating the out-of-bag error rate of the decision tree based on the out-of-bag data of the decision tree, and calculating the weight of the decision tree according to the out-of-bag error rate.

[0052] The decision tree generation process of generating a feature subset based on a classification regression algorithm includes: using a CART regression tree construction method to generate a decision tree with a sample set d and features being the feature subset extracted in step S220.

[0053] Considering the reliability of the prediction results for patients with abdominal aortic aneurysms, this embodiment uses the out-of-bag data of each decision tree to make predictions after all decision trees are built, thereby calculating the out-of-bag error rate of each decision tree. Out-of-bag data (OOB) refers to the sample set that was not sampled during the training of the decision tree model. Since the out-of-bag data (OOB) does not participate in the fitting of the training set model, it can be used to test the model effect. The out-of-bag data refers to the samples that were not sampled during the training of the decision tree, and the out-of-bag data of each decision tree is independent of each other.

[0054] The out-of-bag error (OOB error) is an unbiased estimate of the generalization error of random forests. Its result is similar to the k-fold cross-validation that requires a lot of calculations. Therefore, the out-of-bag error rate is often used to measure the prediction effect of random forests.

[0055] When making a prediction for a sample set, which category the sample set belongs to depends on the judgment of each decision tree. The random forest prediction model itself is composed of multiple classifiers, and the final prediction result is obtained through voting and averaging. Compared with the prediction algorithms in existing technologies (such as neural networks and decision trees), the random forest algorithm has good generalization performance. Doctors can understand the entire process of the prediction algorithm for abdominal aortic aneurysm prediction, thereby establishing a trustworthy relationship between doctors and artificial intelligence systems.

[0056] However, in this application, since the purpose is to use out-of-bag data to adjust the weights of each weak learner (decision tree), the out-of-bag error rate of each decision tree will be calculated for the OOB samples of each decision tree, so as to make full use of all samples in the original data set.

[0057] Step S30, calculating the out-of-bag error rate of the decision tree based on the out-of-bag data of the decision tree, and calculating the weight of the decision tree according to the out-of-bag error rate. Specifically, the steps include:

[0058] Step S300 , each time a decision tree for a feature subset is generated, the out-of-bag error rate (OOB error) of the decision tree needs to be calculated based on the out-of-bag data of the decision tree;

[0059] For the kth decision tree model G k (x):X→Y, where G k (x) is called a decision tree or decision tree model. Assume that there are m out-of-bag data: {(x1,y1),(x2,y2),…,(x m ,y m )}, referring to the error rate calculation method of the Adaboost model, the out-of-bag error rate calculation formula is E=y i -G k (x i ),i=1,2,…,m

[0060] Each decision tree in a random forest generates out-of-bag data (OOB samples). The OOB samples for each decision tree are independent of each other. Using this unselected out-of-bag data to calculate the out-of-bag error rate for each decision tree ensures full utilization of the sample information in the original training set.

[0061] Step S310, finding the maximum out-of-bag error rate of the decision tree.

[0062] Maximum out-of-bag error rate: E k =max|y i -G k (x i )|,i=1,2,…,m.

[0063] Step S320: Calculate the relative error of each decision tree based on the maximum out-of-bag error rate.

[0064] The calculation formula for relative error is:

[0065] Step S330: Calculate the error rate of the current learning rate of each decision tree based on the relative error.

[0066] The calculation formula for the error rate of the current learning rate of each decision tree is:

[0067]

[0068] Step S340 , the error rate of the current learning rate of each decision tree is used to calculate the weight of each decision tree.

[0069] The kth decision tree G k Weight of (x):

[0070] Step S40: weighting the weights of the decision trees to calculate the prediction result of the random forest prediction model.

[0071] The weighted integration of each decision tree gives the final prediction result of the random forest prediction model:

[0072]

[0073] This application obtains the final prediction result of the abdominal aortic aneurysm prediction model through the above method, and outputs the prediction result of the patient to be predicted. After building all the decision tree models, this solution uses each decision tree to predict the out-of-bag data, that is, the data that has not been sampled, thereby calculating the out-of-bag error rate of each decision tree, and using the error rate to calculate the weight of each decision tree model, and then comprehensively obtains the prediction result of the random forest prediction model under the weighted condition. The above technical solution is used to make the prediction result of the abdominal aortic aneurysm prediction model have a higher prediction accuracy, and its prediction process is transparent to the doctor. Therefore, this embodiment proposes a trustworthy prediction model for the early prediction of abdominal aortic aneurysm. The doctor can understand the entire process of the prediction algorithm for abdominal aortic aneurysm prediction, improve the prediction result of the abdominal aortic aneurysm prediction model, and thus establish a trustworthy relationship between the doctor and the artificial intelligence system.

[0074] from Figure 1 It can be seen from the method shown that after building all the decision tree models, the present application uses each decision tree to predict the out-of-bag data, that is, the data that has not been sampled, so as to calculate the out-of-bag error rate of each decision tree, use the error rate to calculate the weight of each decision tree model, and then comprehensively obtain the prediction results of the random forest prediction model under the weighted condition. In the context of random forest using random replacement sampling method to obtain the training set of decision trees, the present application uses the out-of-bag data that has not been sampled to calculate the out-of-bag error rate of each decision tree, ensuring the full utilization of the sample information in the original training set; using the out-of-bag error rate to calculate the weight of each decision tree, thereby ensuring that the weak learner with higher prediction accuracy has a higher voice in the final integrated learning; while retaining the advantages of high efficiency and strong generalization ability of parallel computing, there is no need to use the previous iteration results to update the weight of the learner and the distribution of each sample point as in the boosting algorithm, which greatly saves computing power requirements and greatly improves the prediction accuracy of the random forest prediction results. It can be applied to various scenarios and has a wide range of applications.

[0075] Figure 2 A schematic diagram showing the structure of an apparatus for improving random forest prediction results according to an embodiment of the present application is shown. Figure 2 It can be seen that the device 300 for improving the random forest prediction result includes:

[0076] An acquisition module 310 is configured to acquire an original sample set D;

[0077] Sampling module 320, for extracting a training sample set with replacement from the original sample set D and randomly extracting a feature subset;

[0078] A decision tree generation module 330 is configured to generate multiple decision trees based on the training sample set and the feature subset;

[0079] A weight acquisition module 340 is configured to calculate an out-of-bag error rate of the decision tree based on out-of-bag data of the decision tree, and calculate a weight of the decision tree according to the out-of-bag error rate;

[0080] The prediction result calculation module 350 weights the weighted integration decision trees and calculates the prediction result of the random forest prediction model.

[0081] In some embodiments of the present application, in the above apparatus, the sampling module 320 is configured to extract |D| training samples from the original sample set D using a Bootstrapping method as a training set d of the current decision tree.

[0082] In some embodiments of the present application, in the above apparatus, the sampling module 320 is configured to extract |D| training samples from the original sample set D using a Bootstrapping method as a training set d for the current decision tree;

[0083] Randomly extract m features from the entire feature set M to construct a feature subset;

[0084] generating a decision tree based on the training sample set and the feature subset;

[0085] Generate a random forest prediction model consisting of n unrelated decision trees.

[0086] In some embodiments of the present application, in the above-mentioned device, the out-of-bag data is a sample set that has not been sampled during the training of the decision tree, and the out-of-bag data of each decision tree are independent of each other.

[0087] In some embodiments of the present application, in the above-mentioned device, the weight acquisition module 340 is used to calculate the out-of-bag error rate of the decision tree based on the out-of-bag data of the decision tree each time a decision tree for a feature subset is generated; find the maximum out-of-bag error rate of the decision tree; calculate the relative error of each decision tree based on the maximum out-of-bag error rate; calculate the error rate of the current learning rate of each decision tree based on the relative error; and calculate the weight of each decision tree according to the error rate of the current learning rate of each decision tree.

[0088] In some embodiments of the present application, in the above device, the maximum out-of-bag error rate formula is: E k =max|y i -G k (x i )|,i=1,2,…,m;

[0089] The calculation formula of the relative error is:

[0090] The calculation formula of the error rate of the current learning rate of each decision tree is:

[0091]

[0092] Among them, the kth decision tree G k Weight of (x):

[0093] In some embodiments of the present application, in the above-mentioned device, the weighted integration of the weights of the decision trees and calculation of the prediction result of the random forest prediction model include:

[0094] The prediction result is determined according to the following formula:

[0095]

[0096] Where K is the number of decision trees.

[0097] In some embodiments of the present application, in the above apparatus, the sampling module 320 is configured to take out |D| training samples from the original sample set D with replacement each time as the training set d of the current decision tree.

[0098] It should be noted that the above-mentioned device for improving the random forest prediction results can implement the above-mentioned method based on improving the random forest prediction results one by one, and will not be repeated here.

[0099] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 3 At the hardware level, the electronic device includes a processor and, optionally, an internal bus, a network interface, and memory. The memory may include internal memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for its services.

[0100] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0101] The memory is used to store programs. Specifically, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0102] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming a device for improving the random forest prediction results at the logical level. The processor executes the program stored in the memory and is specifically used to perform the aforementioned method.

[0103] The above application Figure 2 The method performed by the apparatus for improving the random forest prediction results disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor or software instructions. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0104] The electronic device may also perform Figure 2 A method for improving the random forest prediction result device execution, and realizing the device for improving the random forest prediction result in Figure 2 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0105] The embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, can enable the electronic device to execute Figure 2 The method performed by the device for improving the random forest prediction results in the illustrated embodiment is specifically used to perform the aforementioned method.

[0106] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0107] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0108] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0110] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0111] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0112] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0113] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0114] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for improving random forest prediction results, characterized in that: The prediction result is a prediction result of whether the patient to be predicted suffers from abdominal aortic aneurysm, including: Obtaining an original sample set D, where the original sample set D is used to obtain characteristic information of patients with abdominal aortic aneurysm; Extracting a training sample set with replacement from the original sample set D and randomly extracting a feature subset, and generating multiple decision trees based on the training sample set and the feature subset; Calculating an out-of-bag error rate of the decision tree based on out-of-bag data of the decision tree, and calculating a weight of the decision tree according to the out-of-bag error rate; The weights of the decision trees are weighted and integrated to calculate the prediction results of the random forest prediction model; The step of calculating the out-of-bag error rate of the decision tree based on the out-of-bag data of the decision tree, and calculating the weight of the decision tree according to the out-of-bag error rate, includes: Each time a decision tree for a feature subset is generated, the out-of-bag error rate of the decision tree is calculated based on the out-of-bag data of the decision tree; Finding the maximum out-of-bag error rate of the decision tree; Calculating the relative error of each decision tree based on the maximum out-of-bag error rate; Calculating the error rate of the current learning rate of each decision tree based on the relative error; Calculating the weight of each decision tree according to the error rate of the current learning rate of each decision tree; Among them, the maximum value formula of the out-of-bag error rate is: E k =max|y i -G k (x i )|,i=1,2,…,m; The calculation formula of the relative error is: The calculation formula of the error rate of the current learning rate of each decision tree is: Among them, the kth decision tree G k Weight of (x): The weighted integration of the weights of the decision trees and calculation of the prediction results of the random forest prediction model include: The prediction result is determined according to the following formula: Where K is the number of decision trees.

2. The method according to claim 1, wherein The step of extracting a training sample set with replacement from the original sample set D and randomly extracting a feature subset, and generating a plurality of decision trees based on the training sample set and the feature subset, comprises: Use the Bootstrapping method to extract |D| training samples from the original sample set D as the training set d of the current decision tree; Randomly extract m features from the entire feature set M to construct a feature subset; generating a decision tree based on the training sample set and the feature subset; Generate a random forest prediction model consisting of n unrelated decision trees.

3. The method according to claim 1, wherein The out-of-bag data is a sample set that has not been sampled during the decision tree training process, and the out-of-bag data of each decision tree are independent of each other.

4. The method according to claim 2, wherein The Bootstrapping method is used to extract |D| training samples from the original sample set D as the training set d of the current decision tree, including: Each time, |D| training samples are taken from the original sample set D with replacement as the training set d of the current decision tree.

5. A device for improving random forest prediction results, characterized in that: The device comprises: An acquisition module is used to acquire an original sample set D, where the original sample set D is used to acquire characteristic information of patients with abdominal aortic aneurysms; The sampling module is used to extract the training sample set with replacement from the original sample set D and randomly extract the feature subset; A decision tree generation module, configured to generate a plurality of decision trees based on the training sample set and the feature subset; A weight acquisition module, configured to calculate an out-of-bag error rate of the decision tree based on out-of-bag data of the decision tree, and calculate a weight of the decision tree according to the out-of-bag error rate; A prediction result calculation module is used to weight the weights of the decision trees and calculate the prediction result of the random forest prediction model, wherein the prediction result is a prediction result of whether the patient to be predicted suffers from abdominal aortic aneurysm; The weight acquisition module is specifically configured to calculate the out-of-bag error rate of a decision tree based on out-of-bag data of the decision tree each time a decision tree for a feature subset is generated; Finding the maximum out-of-bag error rate of the decision tree; Calculating the relative error of each decision tree based on the maximum out-of-bag error rate; Calculating the error rate of the current learning rate of each decision tree based on the relative error; Calculating the weight of each decision tree according to the error rate of the current learning rate of each decision tree; Among them, the maximum value formula of the out-of-bag error rate is: E k =max|y i -G k (x i )|,i=1,2,…,m; The calculation formula of the relative error is: The calculation formula of the error rate of the current learning rate of each decision tree is: Among them, the kth decision tree G k Weight of (x): The weighted integration of the weights of the decision trees and calculation of the prediction results of the random forest prediction model include: The prediction result is determined according to the following formula: Where K is the number of decision trees.

6. An electronic device comprising: processor; as well as A memory arranged to store computer-executable instructions, which, when executed, cause the processor to execute the improved random forest prediction result method according to any one of claims 1 to 5.

7. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device comprising a plurality of application programs, enables the electronic device to execute the improved random forest prediction result method according to any one of claims 1 to 5.

Images