Processing method and device for performing input feature dimension reduction on target algorithm model

Abstract

The embodiment of the invention relates to a processing method and device for performing input feature dimension reduction on a target algorithm model. The method comprises the following steps: confirming the target algorithm model and identifying a classification threshold value of model prediction data; collecting input and output data of the target algorithm model in a real application environment; fitting an input-output mapping relation of the target algorithm model based on a GBDT model, training the GBDT model based on the observation sample set, and after training is finished, performing sub-feature importance sorting based on the GBDT model and the observation sample set; and carrying out necessary sub-feature selection on the input feature set of the target algorithm model according to the classification threshold, the observation sample set, the GBDT model and the sub-feature sorting, and reconstructing the input features based on a selection result to obtain a corresponding reconstructed feature set and a standby algorithm model. According to the invention, the model operation performance can be improved, and the vehicle driving / control potential safety hazard is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a processing method and device for reducing the dimension of input features of a target algorithm model. Background Art

[0002] The decision-making and control algorithm models of intelligent connected vehicles can be roughly divided into single-vehicle version and multi-vehicle version. Each type of single-vehicle version algorithm model is used to calculate the vehicle’s feature data set X. S Predict a certain type of decision / control parameters of the vehicle, the feature dataset X mentioned here S Based on the characteristic data d of multiple vehicles s Each type of multi-vehicle version algorithm model is used to collect feature data sets X of multiple connected vehicles near a certain traffic location (such as the ramp merging location entering the main road, the main road diverging location entering the ramp, etc.). M Predict a certain type of decision / control parameter of a connected car that is about to / is entering / leaving the location. The feature dataset X mentioned here M The characteristic data of multiple connected cars c composition.

[0003] Both the single-vehicle and multi-vehicle algorithm models should have high computing performance, otherwise it will be difficult to meet the vehicle's safe driving requirements. However, in actual applications, we found that some single-vehicle or multi-vehicle algorithm models did not fully consider the problem of feature dimension surge caused by complex traffic conditions in their early design, and naturally did not provide an adaptive feature dimension reduction strategy. In this case, once the feature dimension increases to a certain extent, it may cause the computing performance of the algorithm model to drop sharply, thereby causing adverse effects on vehicle driving safety. For example, for the single-vehicle algorithm model, the number of perceived objects around the vehicle will surge under complex traffic conditions, and the corresponding feature data d s If no adaptive feature dimension reduction strategy is given, the computing performance of the algorithm model used by the vehicle will inevitably be significantly reduced, resulting in a decrease in the response performance of the vehicle and a negative impact on the driving safety of the vehicle. For example, for the multi-vehicle version of the algorithm model, the number of vehicles near the ramp merging position (or main road diverging position) will increase sharply under complex traffic conditions, and the corresponding feature data d c The number of features will also increase rapidly; if no adaptive feature dimension reduction strategy is given, the computing performance of the algorithm model used in this position will inevitably be significantly reduced, resulting in a decrease in the ability to coordinate control of vehicles that are about to / are entering / exiting this position, causing adverse effects on road traffic safety.

[0004] To solve this problem, it is necessary to provide a processing mechanism that can reduce the dimension of the input features of the target algorithm model, which is also the technical problem that the present invention needs to solve. Summary of the invention

[0005] The purpose of the present invention is to provide a method, device, electronic device and computer-readable storage medium for reducing the dimension of input features of a target algorithm model in view of the defects of the prior art. The present invention records an algorithm or model that needs to reduce the dimension of input features as the corresponding target algorithm model; and sets an invalid / valid data threshold for distinguishing the validity of the predicted data output by the target algorithm model in the actual application environment to identify whether the invalid / valid data threshold is set for distinguishing the validity of the data. If so, the currently set invalid / valid data threshold is recorded as the corresponding classification threshold y hold ; and collect the input and output data of the target algorithm model in the real application environment to obtain the corresponding observation sample set; and design a GBDT model to fit the input-output mapping relationship of the target algorithm model, and train the GBDT model based on the observation sample set, and after the model training is completed, sort the importance of the sub-features in the input features of the target algorithm model based on the GBDT model and the observation sample set; and according to the classification threshold y hold , observation sample set, GBDT model and sub-feature sorting to select necessary sub-features of the input feature set of the target algorithm model and reconstruct the input feature set based on the selection results to obtain the corresponding reconstructed feature set. The present invention can identify necessary features of the input features of the target algorithm model and can reconstruct the overall input features by reducing the dimensionality based on the necessary feature identification results. Through the present invention, a backup model with fewer input dimensions can be reconstructed based on the actual observation data of any target algorithm model. When the complexity of traffic conditions increases, the original target algorithm model can be replaced based on the backup algorithm model provided by the present invention to achieve the purpose of improving model operation performance and reducing vehicle driving / control safety hazards.

[0006] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present invention provides a method for processing input feature dimensionality reduction of a target algorithm model, the method comprising:

[0007] An algorithm or model that needs to reduce the dimension of input features is recorded as the corresponding target algorithm model; and in the actual application environment, whether an invalid / valid data threshold for identifying the validity of the predicted data output by the target algorithm model is set, and if so, the currently set invalid / valid data threshold is recorded as the corresponding classification threshold y hold ;

[0008] Collecting input and output data of the target algorithm model in a real application environment to obtain a corresponding first observation sample set;

[0009] Design a GBDT model to fit the input-output mapping relationship of the target algorithm model; and train the GBDT model based on the first observation sample set; and after the model training is completed, sort the importance of sub-features in the input features of the target algorithm model based on the GBDT model and the first observation sample set to obtain a corresponding first sub-feature sorting;

[0010] According to the classification threshold y hold , the first observed sample set, the GBDT model and the first sub-feature sorting perform necessary sub-feature selection on the input feature set of the target algorithm model and reconstruct the input feature set based on the selection result to obtain the corresponding reconstructed feature set and the backup algorithm model.

[0011] Preferably, the expression of the target algorithm model is y=f target (x); x is the input feature of the target algorithm model, y is the predicted data of the target algorithm model, f target () is the algorithm function of the target algorithm model; the input feature x is specifically a feature set {d i}, composed of N sub-features d i Composition, 1≤sub-feature index i≤N, the total number of sub-features N is a positive integer greater than 1;

[0012] The first observation sample set includes a plurality of first observation samples a j , 1≤sample index j≤S, the total number of observed samples S is a positive integer greater than 1; the first observed sample a j Including the first input feature x j and the first prediction data y j ; The first input feature x j Includes N first sub-features d j,i ;

[0013] The first sub-feature sorting is formed by sorting the N first sub-feature identifiers in sequence;

[0014] The reconstruction feature set is the reconstruction feature set {d k}, 1≤k≤K, 1≤K≤N, K is the total number of necessary sub-features.

[0015] Preferably, the GBDT model is used to fit the mapping relationship between the input feature x of the target algorithm model and the predicted data y; the GBDT model is composed of M decision trees, and the total number of decision trees M is a positive integer greater than 1; when the GBDT model fits the mapping relationship between the input feature x and the predicted data y, it is achieved through step-by-step iterative prediction of the M decision trees;

[0016] The overall mapping relationship function expression of the GBDT model is:

[0017]

[0018] The single-step iterative mapping relationship function expression of the GBDT model is:

[0019] F m (x) = F m-1 (x)+α m h m (x; β m );

[0020] F() is the overall mapping relationship function of the GBDT model; F m () is the single-step iterative mapping relationship function of the GBDT model; 1≤decision tree index m≤M; α m is the weight parameter of the mth decision tree; h m () is the model function of the mth decision tree; β m is the decision tree parameter of the mth decision tree; the overall model parameter set of the GBDT model is

[0021] The loss function L used by the GBDT model during training G is an arbitrarily differentiable loss function.

[0022] Preferably, the training of the GBDT model based on the first observation sample set specifically includes:

[0023] Step 41: S first prediction data y of the first observation sample set j Perform mean calculation and predict the S step 0 data based on the calculation results Perform initialization settings; and initialize the decision tree index m to 1;

[0024] The predicted data The setting formula is:

[0025] The predicted data is the predicted value of the jth observation sample in step 0;

[0026] Step 42, calculate the decision tree residual of the j-th observation sample in the m-th step to obtain the corresponding residual g m,j ;

[0027] The residual g m,j The calculation formula is:

[0028] is the predicted value of the jth observed sample in the m-1th step; when the decision tree index m is 1, each is the predicted data of the jth observation sample in step 0

[0029] Step 43: The S predicted data of step m-1 are The S residuals g of the mth step m,j , the first observation sample set S for the first input feature x j and the first prediction data y j Bring in the preset m-th step fitting objective function group; and adjust the weighting parameter α corresponding to the m-th decision tree in the direction that makes the m-th step fitting objective function group reach the minimum value. m and the decision tree parameter β m Perform parameter estimation;

[0030] The m-th step fitting objective function group is:

[0031]

[0032]

[0033] If the loss function L G is the square error loss function, then the m-th step fitting objective function group is simplified to:

[0034]

[0035] Step 44, the weighted parameter α corresponding to the mth decision tree m and the decision tree parameter β m After confirmation, the first input features x of the first observation sample set are j The corresponding m-th step prediction data Perform calculations;

[0036] The m-th step prediction data y j m The calculation formula is:

[0037] Step 45, identifying whether the decision tree index m is greater than the total number of decision trees M; if the decision tree index m is less than or equal to the total number of decision trees M, adding 1 to the decision tree index m and returning to step 42 to continue iteration; if the decision tree index m is greater than the total number of decision trees M, going to step 46;

[0038] Step 46, the weighted parameter α of the obtained M groups is m and the decision tree parameter β mThe overall model parameter set that constitutes the GBDT model is And confirm that the model training is completed.

[0039] Preferably, the step of ranking the importance of sub-features in the input features of the target algorithm model based on the GBDT model and the first observation sample set to obtain a corresponding first sub-feature ranking specifically includes:

[0040] Step 51: S first prediction data y of the first observation sample set j Perform mean calculation and predict the S step 0 data based on the calculation results Perform initialization settings;

[0041] Step 52: each of the first input features x j Input the GBDT model and perform M-step iterative calculation to obtain the corresponding M-step prediction data set

[0042] The M-step prediction dataset From M single-step prediction data composition;

[0043] Step 53: according to the loss function L G And each of the first input features x j The corresponding prediction data and the M-step prediction dataset For each of the first input features x j Each of the first sub-features d j,i The loss reduction at the mth step is estimated to obtain the corresponding single-step loss reduction los j,i,m ; and for each of the first input features x j Each of the first sub-features d j,i The M single-step loss reduction amount los j,i,m The total amount of loss reduction dl corresponding to the first sub-feature is calculated by summing up j,i ; and the sample index j is used as the column index and the sub-feature index i is used as the row index, and the total amount dl of all the first sub-feature losses obtained is reduced j,i Form a two-dimensional matrix A with a shape of S columns × N rows;

[0044] The two-dimensional matrix A is composed of S×N first matrix units, and the first matrix unit and the first sub-feature loss reduce the total amount dl j,i One to one correspondence;

[0045] Step 54, calculate the mean of the S first matrix elements in each row of the two-dimensional matrix A and use the calculation result as the corresponding sub-feature importance parameter v i ; and the obtained N sub-feature importance parameters v are calculated in descending order i Sorting to obtain the corresponding first sequence; and calculating the importance parameter v of the sub-feature at each sequence index position of the first sequence i The corresponding sub-feature index i is used as a corresponding first sub-feature identifier; and all the obtained first sub-feature identifiers are sorted according to the sorting order of the first sequence to form a corresponding first sub-feature sorting.

[0046] Preferably, the classification threshold y hold , the first observation sample set, the GBDT model and the first sub-feature sorting perform necessary sub-feature selection on the input feature set of the target algorithm model and reconstruct the input feature set based on the selection result to obtain the corresponding reconstructed feature set and the backup algorithm model, specifically including:

[0047] Step 61, set the first value P to a preset initial value k 0 ;

[0048] The initial value k 0 is a positive integer, 1 <k 0 <N;

[0049] Step 62: extract the first sub-feature identifiers that are ranked in the first P positions in the first sub-feature ranking to form a corresponding necessary sub-feature identifier set; and add the feature set {d i}, the P sub-features d that match the necessary sub-feature identifier set i Extract the necessary sub-feature sets to form the corresponding ones;

[0050] Step 63: According to the necessary sub-feature set, each of the first input features x of the first observation sample set is j The corresponding necessary sub-features are extracted and the P first sub-features d are extracted. j,i The corresponding second input feature x ’ j ; and each of the second input features x j ’ and the corresponding first prediction data y j forming a corresponding first data record; and forming a corresponding first data set from all the first data records obtained;

[0051] Step 64, adaptively adjusting the model input structure and the corresponding model internal structure of the target algorithm model based on the necessary sub-feature set to obtain a corresponding adjusted model; and training the adjusted model based on the first data set;

[0052] Step 65, after the model training of the adjusted model is completed, the classification threshold y hold Is it empty for identification? If the classification threshold y hold is empty, go to step 66; if the classification threshold y hold If it is not empty, go to step 67;

[0053] Step 66: each of the second input features x of the first data set ’ j Input the adjusted model for processing to obtain the corresponding second prediction data y ’ j ; and all the first prediction data y j And all the second prediction data y ’ j Substitute the preset first evaluation function F A Calculate and obtain the corresponding first evaluation value; and go to step 68;

[0054] The first evaluation function F A for:

[0055] Step 67: based on the preset data validity identification rules and the corresponding classification threshold y hold For each of the first prediction data y j The data validity is screened to obtain the corresponding first screening type r j ; and based on the data validity identification rules and the corresponding classification threshold y hold For each of the second prediction data y ’ j The data validity is screened to obtain the corresponding second screening type r j ’ ; and all the first prediction data y j All the second prediction data y ’ j All the first identification types r j and all the second screening types r ’ j Substitute the preset second evaluation function F B Performing calculation to obtain the corresponding first evaluation value;

[0056] The first identification type rj and the second screening type r ’ j are all binary data, the first identification type r j Or the second screening type r ’ j When it is 0, it means the data is invalid, and when it is 1, it means the data is valid;

[0057] The second evaluation function F B for:

[0058] Step 68, save the model structure and model parameters of the current adjusted model as the corresponding first standby model; and form a corresponding first evaluation record with the current first evaluation value, the corresponding first value P and the necessary sub-feature set and save it; and identify whether the current first evaluation value meets the preset first evaluation value range; if the first evaluation value meets the first evaluation value range, go to step 69; if the first evaluation value does not meet the first evaluation value range, add 1 to the first value P, and identify whether the first value P after adding 1 is greater than the total number of sub-features N, if so, go to step 69, if not, return to step 62;

[0059] Step 69: extract the first value P and the necessary sub-feature set of the first evaluation record that is most recently saved as the corresponding total number of necessary sub-features K and the reconstructed feature set {d k}; and reconstruct the current feature set {d k}The corresponding first backup model is used as the corresponding backup algorithm model.

[0060] The second aspect of the embodiment of the present invention provides a device for implementing the method for processing the input feature dimensionality reduction of the target algorithm model described in the first aspect, the device comprising: a target algorithm model confirmation module, an observation sample collection module, a sub-feature importance identification module and a target algorithm model reconstruction module;

[0061] The target algorithm model confirmation module is used to record an algorithm or model that needs to perform input feature dimensionality reduction as the corresponding target algorithm model; and to identify whether an invalid / valid data threshold for distinguishing the validity of the predicted data output by the target algorithm model is set in the actual application environment. If so, the currently set invalid / valid data threshold is recorded as the corresponding classification threshold y hold ;

[0062] The observation sample collection module is used to collect input and output data of the target algorithm model in a real application environment to obtain a corresponding first observation sample set;

[0063] The sub-feature importance identification module is used to design a GBDT model to fit the input-output mapping relationship of the target algorithm model; and train the GBDT model based on the first observation sample set; and after the model training is completed, the sub-feature importance in the input features of the target algorithm model is sorted based on the GBDT model and the first observation sample set to obtain the corresponding first sub-feature sorting;

[0064] The target algorithm model reconstruction module is used to reconstruct the target algorithm model according to the classification threshold y hold , the first observed sample set, the GBDT model and the first sub-feature sorting perform necessary sub-feature selection on the input feature set of the target algorithm model and reconstruct the input feature set based on the selection result to obtain the corresponding reconstructed feature set and the backup algorithm model.

[0065] A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0066] The processor is used to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps described in the first aspect above;

[0067] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0068] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed by a computer, the computer executes the instructions of the method described in the first aspect above.

[0069] The embodiment of the present invention provides a processing method, device, electronic device and computer-readable storage medium for reducing the dimension of input features of a target algorithm model. As can be seen from the above invention content, the embodiment of the present invention records an algorithm or model that needs to reduce the dimension of input features as the corresponding target algorithm model; and sets an invalid / valid data threshold for distinguishing the validity of the prediction data output by the target algorithm model in the actual application environment to identify whether it is used to identify the invalid / valid data threshold. If so, the currently set invalid / valid data threshold is recorded as the corresponding classification threshold y hold ; and collect the input and output data of the target algorithm model in the real application environment to obtain the corresponding observation sample set; and design a GBDT model to fit the input-output mapping relationship of the target algorithm model, and train the GBDT model based on the observation sample set, and after the model training is completed, sort the importance of the sub-features in the input features of the target algorithm model based on the GBDT model and the observation sample set; and according to the classification threshold y hold, observation sample set, GBDT model and sub-feature sorting to select necessary sub-features for the input feature set of the target algorithm model and reconstruct the input feature set based on the selection results to obtain the corresponding reconstructed feature set. The embodiment of the present invention can perform necessary feature identification on the input features of the target algorithm model and can perform dimensionality reduction reconstruction on the overall input features based on the necessary feature identification results. Through the embodiment of the present invention, a backup model with fewer input dimensions can be reconstructed based on the actual observation data of any target algorithm model. When the complexity of traffic conditions increases, replacing the original target algorithm model based on the backup algorithm model provided by the embodiment of the present invention not only improves the model operation performance, but also reduces the potential safety hazards of vehicle driving / control. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic diagram of a processing method for reducing the dimension of input features of a target algorithm model provided in Embodiment 1 of the present invention;

[0071] Figure 2 A module structure diagram of a processing device for reducing the dimension of input features of a target algorithm model provided in Embodiment 2 of the present invention;

[0072] Figure 3 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] A method for reducing the dimension of input features of a target algorithm model is provided in a first embodiment of the present invention, such as Figure 1 A schematic diagram of a method for reducing the dimension of input features of a target algorithm model provided in the first embodiment of the present invention includes the following main steps:

[0075] Step 1: record an algorithm or model that needs to reduce the dimension of input features as the corresponding target algorithm model; and identify whether an invalid / valid data threshold is set for the prediction data output by the target algorithm model in the actual application environment to identify its data validity. If so, record the currently set invalid / valid data threshold as the corresponding classification threshold y hold .

[0076] Here, the target algorithm model of the embodiment of the present invention includes but is not limited to the single-vehicle version decision and control algorithm model and the multi-vehicle version decision and control algorithm model mentioned above.

[0077] The expression of the target algorithm model is y = f target (x); where x is the input feature of the target algorithm model, y is the predicted data of the target algorithm model, and f target () is the algorithm function of the target algorithm model; the input feature x is specifically a feature set {d i}, composed of N sub-features d i The sub-feature index i is 1≤N, and the total number of sub-features N is a positive integer greater than 1.

[0078] It should be noted that in the actual application environment, the prediction data of some target algorithm models do not need to be screened for data validity, while the prediction data of some target algorithm models will be screened for data validity by subsequent steps in the application scenario; for example, the prediction data output by the target algorithm model used to predict the vehicle's driving control parameters are generally not screened for validity by subsequent steps, such as the throttle / brake opening and closing degree, steering wheel angle, etc.; for another example, the prediction data output by the target algorithm model used to predict the vehicle's planned trajectory are generally screened for validity by subsequent steps, such as whether the trajectory coordinates are drivable coordinates, if so, confirming that the predicted trajectory coordinates are valid data, otherwise confirming that the predicted trajectory coordinates are invalid data. In this case, there will be a pre-set or dynamic invalid / valid data threshold, that is, the classification threshold y hold To identify the data.

[0079] Step 2, collecting input and output data of the target algorithm model in the actual application environment to obtain a corresponding first observation sample set;

[0080] The first observation sample set includes a plurality of first observation samples a j , 1≤sample index j≤S, the total number of observed samples S is a positive integer greater than 1; the first observed sample a j Including the first input feature x j and the first prediction data y j ; The first input feature x j Includes N first sub-features d j,i .

[0081] Step 3, design a GBDT model to fit the input-output mapping relationship of the target algorithm model; and train the GBDT model based on the first observation sample set; and after the model training is completed, sort the importance of sub-features in the input features of the target algorithm model based on the GBDT model and the first observation sample set to obtain the corresponding first sub-feature sorting;

[0082] Specifically, the method includes: step 31, designing a GBDT model to fit the input-output mapping relationship of the target algorithm model;

[0083] Here, the GBDT model of the embodiment of the present invention is used to fit the mapping relationship between the input feature x of the target algorithm model and the predicted data y; the GBDT model is composed of M decision trees, and the total number of decision trees M is a positive integer greater than 1; when the GBDT model fits the mapping relationship between the input feature x and the predicted data y, it is achieved through step-by-step iterative prediction of the M decision trees;

[0084] The overall mapping relationship function expression of the GBDT model is:

[0085]

[0086] The single-step iterative mapping relationship function expression of the GBDT model is:

[0087] F m (x) = F m-1 (x)+α m h m (x; β m );

[0088] Among them, F() is the overall mapping relationship function of the GBDT model; F m () is the single-step iterative mapping relationship function of the GBDT model; 1≤decision tree index m≤M; α m is the weight parameter of the mth decision tree; h m () is the model function of the mth decision tree; β m is the decision tree parameter of the mth decision tree; the overall model parameter set of the GBDT model is

[0089] The loss function L used in the training process of the GBDT model in the embodiment of the present invention is G is an arbitrarily differentiable loss function;

[0090] Step 32, training the GBDT model based on the first observation sample set;

[0091] Specifically, step 321 includes: j Perform mean calculation and predict the S step 0 data based on the calculation results Perform initialization settings; and initialize the decision tree index m to 1;

[0092] Among them, the forecast data The setting formula is:

[0093]

[0094] Here, the forecast data is the predicted value of the jth observation sample in step 0;

[0095] Step 322, calculate the decision tree residual of the j-th observation sample in step m to obtain the corresponding residual g m,j ;

[0096] Among them, the residual g m,j The calculation formula is:

[0097]

[0098] Here, the formula is the predicted value of the jth observation sample in the m-1th step; when the decision tree index m is 1, each It is actually the predicted data of the jth observation sample in step 0

[0099] Step 323: The S prediction data of step m-1 are The S residuals g of the mth step m,j , the first observation sample set S for the first input feature x j and the first prediction data y j Bring in the preset m-th step fitting objective function group; and adjust the weight parameter α corresponding to the m-th decision tree in the direction that makes the m-th step fitting objective function group reach the minimum value. m and decision tree parameter β m Perform parameter estimation;

[0100] Among them, the m-th step fitting objective function group is:

[0101]

[0102]

[0103] Here, if the loss function L G If the square error loss function is the above step m fitting objective function group, it can be simplified into a single objective function as shown below:

[0104]

[0105] Step 324, the weighted parameter α corresponding to the mth decision tree m and decision tree parameter β m After confirmation, for each first input feature x of the first observation sample set j The corresponding m-th step prediction data Perform calculations;

[0106] Among them, the m-th step prediction data The calculation formula corresponds to the single-step iterative mapping relationship function of the GBDT model, specifically:

[0107]

[0108] Step 325, identifying whether the decision tree index m is greater than the total number of decision trees M; if the decision tree index m is less than or equal to the total number of decision trees M, then adding 1 to the decision tree index m and returning to step 322 to continue iteration; if the decision tree index m is greater than the total number of decision trees M, then going to step 326;

[0109] Step 326, the obtained M groups of weighted parameters α m and decision tree parameter β m The overall model parameter set that constitutes the GBDT model is And confirm that the model training is completed;

[0110] Step 33, after the model training is completed, the importance of sub-features in the input features of the target algorithm model is sorted based on the GBDT model and the first observation sample set to obtain the corresponding first sub-feature sorting;

[0111] The first sub-feature ranking is formed by sorting the N first sub-feature identifiers in order;

[0112] Specifically, step 331 includes: j Perform mean calculation and predict the S step 0 data based on the calculation results Perform initialization settings;

[0113] Step 332: each first input feature x of the first observation sample set j Input the GBDT model and perform M-step iterative calculations to obtain the corresponding M-step prediction data set

[0114] Among them, the M-step prediction dataset From M single-step prediction data composition;

[0115] Step 333, according to the loss function L G And each first input feature x j Corresponding prediction data and M-step prediction dataset For each first input feature x j Each first sub-feature d j,i The loss reduction at the mth step is estimated to obtain the corresponding single-step loss reduction los j,i,m; and for each first input feature x j Each first sub-feature d j,i The M single-step loss reduction los j,i,m The total amount of loss reduction dl corresponding to the first sub-feature is calculated by summing up j,i ; With sample index j as column index and sub-feature index i as row index, the total amount dl is reduced by all the first sub-feature losses obtained j,i Form a two-dimensional matrix A with a shape of S columns × N rows;

[0116] Among them, the two-dimensional matrix A is composed of S×N first matrix units, and the first matrix unit and the first sub-feature loss reduce the total amount dl j,i One to one correspondence;

[0117] Step 334, calculate the mean of the S first matrix elements in each row of the two-dimensional matrix A and use the calculation result as the corresponding sub-feature importance parameter v i ; and the N sub-feature importance parameters v are obtained in descending order i Sort the first sequence to obtain the corresponding first sequence; and calculate the importance parameter v of the sub-feature at each sequence index position of the first sequence i The corresponding sub-feature index i is used as a corresponding first sub-feature identifier; and all the obtained first sub-feature identifiers are sorted according to the sorting order of the first sequence to form a corresponding first sub-feature sorting.

[0118] Step 4: According to the classification threshold y hold , the first observed sample set, the GBDT model and the first sub-feature sorting perform necessary sub-feature selection on the input feature set of the target algorithm model and reconstruct the input feature set based on the selection result to obtain the corresponding reconstructed feature set and the backup algorithm model;

[0119] Specifically, step 41: setting the first value P to a preset initial value k 0 ;

[0120] Among them, the initial value k 0 is a positive integer, 1 <k 0 <N;

[0121] Step 42, extract the first sub-feature identifiers ranked in the top P positions in the first sub-feature sorting to form a corresponding necessary sub-feature identifier set; and add the feature set {d i}, the P sub-features d that match the necessary sub-feature identifier set i Extract the necessary sub-feature sets to form the corresponding ones;

[0122] Step 43: According to the necessary sub-feature set, each first input feature x of the first observation sample set is jThe corresponding necessary sub-features are extracted and the P first sub-features d are extracted. j,i The corresponding second input feature x ’ j ; and each second input feature x ’ j and the corresponding first prediction data y j forming a corresponding first data record; and forming a corresponding first data set from all the obtained first data records;

[0123] Here, the first data set consists of S first data records; each first data record consists of the second input feature x j ’ and the first prediction data y j composition;

[0124] Step 44, adaptively adjusting the model input structure and the corresponding model internal structure of the target algorithm model based on the necessary sub-feature set to obtain a corresponding adjusted model; and training the adjusted model based on the first data set;

[0125] Here, when the embodiment of the present invention adaptively adjusts the model input structure of the target algorithm model and the corresponding model internal structure based on the necessary sub-feature set, the adjustment can be performed automatically based on the machine configuration method or manually based on the manual configuration method; when the embodiment of the present invention trains the adjusted model based on the first data set, the first prediction data y in the data set is used. j To verify the labels, the adjusted model is trained in a supervised manner;

[0126] Step 45, after the model training of the adjusted model is completed, the classification threshold y hold Is it empty for identification? If the classification threshold y hold is empty, go to step 46; if the classification threshold y hold If it is not empty, go to step 47;

[0127] Step 46: each second input feature x of the first data set ’ j Input the adjusted model for processing to obtain the corresponding second prediction data y ’ j ; and all the first prediction data y j And all the second prediction data y ’ j Substitute the preset first evaluation function F A Calculate and obtain the corresponding first evaluation value; and go to step 48;

[0128] Among them, the first evaluation function F A for:

[0129] Here, the first evaluation function F A It is actually a mean square error evaluation function; because the classification threshold y hold If it is empty, it means that the prediction data of the current target algorithm model does not need to be screened for data validity. In this case, as long as the mean square error of the prediction results of the adjusted model and the target algorithm model is within a small error range, the target algorithm model can be replaced based on the adjusted model.

[0130] Step 47: Based on the preset data validity identification rules and the corresponding classification threshold y hold For each first prediction data y j The data validity is screened to obtain the corresponding first screening type r j ; and based on the data validity identification rules and the corresponding classification threshold y hold For each second prediction data y j ’ The data validity is screened to obtain the corresponding second screening type r ’ j ; and all the first prediction data y j , all second prediction data y ’ j , all first screening types r j and all second discrimination types r ’ j Substitute the preset second evaluation function F B Performing calculation to obtain a corresponding first evaluation value;

[0131] Among them, the data validity identification rule is a pre-set data identification rule, which can be customized based on the actual application scenario of the current target algorithm model; the first identification type r j and the second discrimination type r ’ j Both are binary data. The first type of identification is r j or the second screening type r ’ j When it is 0, it means the data is invalid, and when it is 1, it means the data is valid;

[0132] The second evaluation function F B for:

[0133]

[0134] Here, the second evaluation function F B It is actually a summed mean square error evaluation function; because the classification threshold y holdIf it is not empty, it means that the prediction data of the current target algorithm model will be used for data validity screening in the subsequent steps of the application scenario. Therefore, when evaluating the adjusted model, it is necessary not only to evaluate the mean square error of the prediction results between it and the target algorithm model, but also to evaluate the validity screening effect of the prediction data of the two.

[0135] Step 48, save the model structure and model parameters of the current adjusted model as the corresponding first standby model; and form a corresponding first evaluation record composed of the current first evaluation value, the corresponding first value P and the necessary sub-feature set and save it; and identify whether the current first evaluation value meets the preset first evaluation value range; if the first evaluation value meets the first evaluation value range, go to step 49; if the first evaluation value does not meet the first evaluation value range, add 1 to the first value P, and identify whether the first value P after adding 1 is greater than the total number of sub-features N, if so, go to step 49, if not, return to step 42;

[0136] Here, the first evaluation value range is a preset evaluation value range;

[0137] Step 49: extract the first value P and the necessary sub-feature set of the most recently saved first evaluation record as the corresponding total number of necessary sub-features K and the reconstructed feature set {d k}; and the current reconstruction feature set {d k The corresponding first backup model is used as the corresponding backup algorithm model;

[0138] Here, the reconstructed feature set obtained is the reconstructed feature set {d k}, 1≤k≤K, 1≤K≤N, K is the total number of necessary sub-features.

[0139] Figure 2 The module structure diagram of a processing device for reducing the dimension of input features of a target algorithm model provided in the second embodiment of the present invention, the device is a terminal device or server that implements the aforementioned method embodiment, and can also be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiment, for example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 2 As shown, the processing device for reducing the dimension of input features of the target algorithm model provided in the second embodiment of the present invention includes: a target algorithm model confirmation module 201, an observation sample collection module 202, a sub-feature importance identification module 203 and a target algorithm model reconstruction module 204.

[0140] The target algorithm model confirmation module 201 is used to record an algorithm or model that needs to perform input feature dimensionality reduction as the corresponding target algorithm model; and to identify whether an invalid / valid data threshold for distinguishing the validity of the predicted data output by the target algorithm model in the actual application environment is set, and if so, the currently set invalid / valid data threshold is recorded as the corresponding classification threshold y hold .

[0141] The observation sample collection module 202 is used to collect input and output data of the target algorithm model in the actual application environment to obtain a corresponding first observation sample set.

[0142] The sub-feature importance identification module 203 is used to design a GBDT model to fit the input-output mapping relationship of the target algorithm model; and to train the GBDT model based on the first observation sample set; and after the model training is completed, the importance of sub-features in the input features of the target algorithm model is sorted based on the GBDT model and the first observation sample set to obtain the corresponding first sub-feature sorting.

[0143] The target algorithm model reconstruction module 204 is used to reconstruct the target algorithm model according to the classification threshold y. hold , the first observed sample set, the GBDT model and the first sub-feature sorting perform necessary sub-feature selection on the input feature set of the target algorithm model and reconstruct the input feature set based on the selection results to obtain the corresponding reconstructed feature set and the backup algorithm model.

[0144] An embodiment of the present invention provides a processing device for reducing the dimension of input features of a target algorithm model, which can execute the method steps in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0145] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the target algorithm model confirmation module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above determination module. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0146] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0147] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the above method embodiments are generated. The above computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.) methods. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The above-mentioned available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0148] Figure 3 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. The electronic device may be a terminal device or a server for implementing the method of the aforementioned embodiment, or may be a terminal device or a server for implementing the method of the aforementioned embodiment connected to the aforementioned terminal device or server. Figure 3 As shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver 303. Various instructions may be stored in the memory 302 to complete various processing functions and implement the processing steps described in the aforementioned embodiment method. Preferably, the electronic device involved in the embodiment of the present invention also includes: a power supply 304, a system bus 305 and a communication port 306. The system bus 305 is used to realize the communication connection between components. The above-mentioned communication port 306 is used for connecting and communicating between the electronic device and other peripherals.

[0149] exist Figure 3The system bus 305 mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize the communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM), and may also include non-volatile memory (Non-Volatile Memory), such as at least one disk storage.

[0150] The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (Network Processor, NP), a graphics processing unit (Graphics Processing Unit, GPU), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0151] It should be noted that an embodiment of the present invention further provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the method and processing process provided in the above embodiments.

[0152] The embodiment of the present invention provides a processing method, device, electronic device and computer-readable storage medium for reducing the dimension of input features of a target algorithm model. As can be seen from the above invention content, the embodiment of the present invention records an algorithm or model that needs to reduce the dimension of input features as the corresponding target algorithm model; and sets an invalid / valid data threshold for distinguishing the validity of the prediction data output by the target algorithm model in the actual application environment to identify whether it is used to identify the invalid / valid data threshold. If so, the currently set invalid / valid data threshold is recorded as the corresponding classification threshold y hold ; and collect the input and output data of the target algorithm model in the real application environment to obtain the corresponding observation sample set; and design a GBDT model to fit the input-output mapping relationship of the target algorithm model, and train the GBDT model based on the observation sample set, and after the model training is completed, sort the importance of the sub-features in the input features of the target algorithm model based on the GBDT model and the observation sample set; and according to the classification threshold y hold, observation sample set, GBDT model and sub-feature sorting to select necessary sub-features for the input feature set of the target algorithm model and reconstruct the input feature set based on the selection results to obtain the corresponding reconstructed feature set. The embodiment of the present invention can perform necessary feature identification on the input features of the target algorithm model and can perform dimensionality reduction reconstruction on the overall input features based on the necessary feature identification results. Through the embodiment of the present invention, a backup model with fewer input dimensions can be reconstructed based on the actual observation data of any target algorithm model. When the complexity of traffic conditions increases, replacing the original target algorithm model based on the backup algorithm model provided by the embodiment of the present invention not only improves the model operation performance, but also reduces the potential safety hazards of vehicle driving / control.

[0153] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0154] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0155] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for reducing the dimension of input features of a target algorithm model, characterized in that: The method comprises: An algorithm or model that needs to reduce the dimension of input features is recorded as the corresponding target algorithm model; and in the actual application environment, whether an invalid / valid data threshold for identifying the validity of the predicted data output by the target algorithm model is set, and if so, the currently set invalid / valid data threshold is recorded as the corresponding classification threshold y hold ; Collecting input and output data of the target algorithm model in a real application environment to obtain a corresponding first observation sample set; Design a GBDT model to fit the input-output mapping relationship of the target algorithm model; and train the GBDT model based on the first observation sample set; and after the model training is completed, sort the importance of sub-features in the input features of the target algorithm model based on the GBDT model and the first observation sample set to obtain a corresponding first sub-feature sorting; According to the classification threshold y hold , the first observed sample set, the GBDT model and the first sub-feature sorting perform necessary sub-feature selection on the input feature set of the target algorithm model and reconstruct the input feature set based on the selection result to obtain the corresponding reconstructed feature set and the backup algorithm model.

2. The method for reducing the dimension of input features of a target algorithm model according to claim 1, characterized in that: The expression of the target algorithm model is y=f target (x); x is the input feature of the target algorithm model, y is the predicted data of the target algorithm model, f target () is the algorithm function of the target algorithm model; the input feature x is specifically a feature set {d i }, composed of N sub-features d i Composition, 1≤sub-feature index i≤N, the total number of sub-features N is a positive integer greater than 1; The first observation sample set includes a plurality of first observation samples a j , 1≤sample index j≤S, the total number of observed samples S is a positive integer greater than 1; the first observed sample a j Including the first input feature x j and the first prediction data y j ; The first input feature x j Includes N first sub-features d j,i ; The first sub-feature sorting is formed by sorting the N first sub-feature identifiers in sequence; The reconstruction feature set is the reconstruction feature set {d k }, 1≤k≤K, 1≤K≤N, K is the total number of necessary sub-features.

3. The method for reducing the dimension of input features of a target algorithm model according to claim 2, characterized in that: The GBDT model is used to fit the mapping relationship between the input feature x of the target algorithm model and the predicted data y; the GBDT model is composed of M decision trees, and the total number of decision trees M is a positive integer greater than 1; when the GBDT model fits the mapping relationship between the input feature x and the predicted data y, it is achieved through step-by-step iterative prediction of the M decision trees; The overall mapping relationship function expression of the GBDT model is: The single-step iterative mapping relationship function expression of the GBDT model is: F m (x)=F m-1 (x)+a m h m (x;b m ); F() is the overall mapping relationship function of the GBDT model; F m () is the single-step iterative mapping relationship function of the GBDT model; 1≤decision tree index m≤M; α m is the weight parameter of the mth decision tree; h m () is the model function of the mth decision tree; β m is the decision tree parameter of the mth decision tree; the overall model parameter set of the GBDT model is The loss function L used by the GBDT model during training G is an arbitrarily differentiable loss function.

4. The method for reducing the dimension of input features of a target algorithm model according to claim 3, characterized in that: The training of the GBDT model based on the first observation sample set specifically includes: Step 41: S first prediction data y of the first observation sample set j Perform mean calculation and predict the S step 0 data based on the calculation results Perform initialization settings; and initialize the decision tree index m to 1; The predicted data The setting formula is: The predicted data is the predicted value of the jth observation sample in step 0; Step 42, calculate the decision tree residual of the j-th observation sample in the m-th step to obtain the corresponding residual g m,j ; The residual g m,j The calculation formula is: is the predicted value of the jth observed sample in the m-1th step; when the decision tree index m is 1, each is the predicted data of the jth observation sample in step 0 Step 43: The S predicted data of step m-1 are The S residuals g of the mth step m,j , the first observation sample set S for the first input feature x j and the first prediction data y j Bring in the preset m-th step fitting objective function group; and adjust the weighting parameter α corresponding to the m-th decision tree in the direction that makes the m-th step fitting objective function group reach the minimum value. m and the decision tree parameter β m Perform parameter estimation; The m-th step fitting objective function group is: If the loss function L G is the square error loss function, then the m-th step fitting objective function group is simplified to: Step 44, the weighted parameter α corresponding to the mth decision tree m and the decision tree parameter β m After confirmation, the first input features x of the first observation sample set are j The corresponding m-th step prediction data Perform calculations; The m-th step prediction data The calculation formula is: Step 45, identifying whether the decision tree index m is greater than the total number of decision trees M; if the decision tree index m is less than or equal to the total number of decision trees M, adding 1 to the decision tree index m and returning to step 42 to continue iteration; if the decision tree index m is greater than the total number of decision trees M, going to step 46; Step 46, the weighted parameter α of the obtained M groups is m and the decision tree parameter β m The overall model parameter set that constitutes the GBDT model is And confirm that the model training is completed.

5. The method for reducing the dimension of input features of a target algorithm model according to claim 2, characterized in that: The step of sorting the importance of sub-features in the input features of the target algorithm model based on the GBDT model and the first observation sample set to obtain a corresponding first sub-feature sorting specifically includes: Step 51: S first prediction data y of the first observation sample set j Perform mean calculation and predict the S step 0 data based on the calculation results Perform initialization settings; Step 52: each of the first input features x j Input the GBDT model and perform M-step iterative calculation to obtain the corresponding M-step prediction data set The M-step prediction dataset From M single-step prediction data composition; Step 53: according to the loss function L G And each of the first input features x j The corresponding prediction data and the M-step prediction dataset For each of the first input features x j Each of the first sub-features d j,i The loss reduction at the mth step is estimated to obtain the corresponding single-step loss reduction los j,i,m ; and for each of the first input features x j Each of the first sub-features d j,i The M single-step loss reduction amount los j,i,m The total amount of loss reduction dl corresponding to the first sub-feature is calculated by summing up j,i ; and the sample index j is used as the column index and the sub-feature index i is used as the row index, and the total amount dl of all the first sub-feature losses obtained is reduced j,i Form a two-dimensional matrix A with a shape of S columns × N rows; The two-dimensional matrix A is composed of S×N first matrix units, and the first matrix unit and the first sub-feature loss reduce the total amount dl j,i One to one correspondence; Step 54, calculate the mean of the S first matrix elements in each row of the two-dimensional matrix A and use the calculation result as the corresponding sub-feature importance parameter v i ; and the obtained N sub-feature importance parameters v are calculated in descending order i Sorting to obtain the corresponding first sequence; and calculating the importance parameter v of the sub-feature at each sequence index position of the first sequence i The corresponding sub-feature index i is used as a corresponding first sub-feature identifier; and all the obtained first sub-feature identifiers are sorted according to the sorting order of the first sequence to form a corresponding first sub-feature sorting.

6. The method for reducing the dimension of input features of a target algorithm model according to claim 2, characterized in that: The classification threshold y hold , the first observation sample set, the GBDT model and the first sub-feature sorting perform necessary sub-feature selection on the input feature set of the target algorithm model and reconstruct the input feature set based on the selection result to obtain the corresponding reconstructed feature set and the backup algorithm model, specifically including: Step 61, setting the first value P to a preset initial value k0; The initial value k0 is a positive integer, 1 <k0<N; Step 62: extract the first sub-feature identifiers that are ranked in the first P positions in the first sub-feature ranking to form a corresponding necessary sub-feature identifier set; and add the feature set {d i }, the P sub-features d that match the necessary sub-feature identifier set i Extract the necessary sub-feature sets to form the corresponding ones; Step 63: According to the necessary sub-feature set, each of the first input features x of the first observation sample set is j The corresponding necessary sub-features are extracted and the P first sub-features d are extracted. j,i The corresponding second input feature x ’ j ; and each of the second input features x j ’ and the corresponding first prediction data y j forming a corresponding first data record; and forming a corresponding first data set from all the first data records obtained; Step 64, adaptively adjusting the model input structure and the corresponding model internal structure of the target algorithm model based on the necessary sub-feature set to obtain a corresponding adjusted model; and training the adjusted model based on the first data set; Step 65, after the model training of the adjusted model is completed, the classification threshold y hold Is it empty for identification? If the classification threshold y hold is empty, go to step 66; if the classification threshold y hold If it is not empty, go to step 67; Step 66: each of the second input features x of the first data set ’ j Input the adjusted model for processing to obtain the corresponding second prediction data y ’ j ; and all the first prediction data y j And all the second prediction data y ’ j Substitute the preset first evaluation function F A Calculate and obtain the corresponding first evaluation value; and go to step 68; The first evaluation function F A for: Step 67: based on the preset data validity identification rules and the corresponding classification threshold y hold For each of the first prediction data y j The data validity is screened to obtain the corresponding first screening type r j ; and based on the data validity identification rules and the corresponding classification threshold y hold For each of the second prediction data y ’ j The data validity is screened to obtain the corresponding second screening type r j ’ ; and all the first prediction data y j All the second prediction data y ’ j All the first identification types r j and all the second screening types r ’ j Substitute the preset second evaluation function F B Performing calculation to obtain the corresponding first evaluation value; The first identification type r j and the second screening type r ’ j are all binary data, the first identification type r j Or the second screening type r ’ j When it is 0, it means the data is invalid, and when it is 1, it means the data is valid; The second evaluation function F B for: Step 68, save the model structure and model parameters of the current adjusted model as the corresponding first standby model; and form a corresponding first evaluation record with the current first evaluation value, the corresponding first value P and the necessary sub-feature set and save it; and identify whether the current first evaluation value meets the preset first evaluation value range; if the first evaluation value meets the first evaluation value range, go to step 69; if the first evaluation value does not meet the first evaluation value range, add 1 to the first value P, and identify whether the first value P after adding 1 is greater than the total number of sub-features N, if so, go to step 69, if not, return to step 62; Step 69: extract the first value P and the necessary sub-feature set of the first evaluation record that is most recently saved as the corresponding total number of necessary sub-features K and the reconstructed feature set {d k }; and reconstruct the current feature set {d k }The corresponding first backup model is used as the corresponding backup algorithm model.

7. A device for executing the method for processing input feature dimensionality reduction of a target algorithm model according to any one of claims 1 to 6, characterized in that: The device comprises: a target algorithm model confirmation module, an observation sample collection module, a sub-feature importance identification module and a target algorithm model reconstruction module; The target algorithm model confirmation module is used to record an algorithm or model that needs to perform input feature dimensionality reduction as the corresponding target algorithm model; and to identify whether an invalid / valid data threshold for distinguishing the validity of the predicted data output by the target algorithm model is set in the actual application environment. If so, the currently set invalid / valid data threshold is recorded as the corresponding classification threshold y hold ; The observation sample collection module is used to collect input and output data of the target algorithm model in a real application environment to obtain a corresponding first observation sample set; The sub-feature importance identification module is used to design a GBDT model to fit the input-output mapping relationship of the target algorithm model; and train the GBDT model based on the first observation sample set; and after the model training is completed, the sub-feature importance in the input features of the target algorithm model is sorted based on the GBDT model and the first observation sample set to obtain the corresponding first sub-feature sorting; The target algorithm model reconstruction module is used to reconstruct the target algorithm model according to the classification threshold y hold , the first observed sample set, the GBDT model and the first sub-feature sorting perform necessary sub-feature selection on the input feature set of the target algorithm model and reconstruct the input feature set based on the selection result to obtain the corresponding reconstructed feature set and the backup algorithm model.

8. An electronic device, characterized in that: include: memory, processors, and transceivers; The processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the method according to any one of claims 1 to 6; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a computer, the computer is enabled to execute the method according to any one of claims 1 to 6.

Images