A method, device, computer equipment and storage medium for feature dimension screening

Abstract

This disclosure provides a feature dimension filtering method, apparatus, computer device, and storage medium. The method includes: acquiring a sample dataset; wherein each sample data in the sample dataset contains feature data under multiple feature dimensions to be filtered; training a logistic regression model based on the feature data of each sample data under multiple feature dimension combinations, and determining the classification performance index value corresponding to the feature dimension combination used in each training session; the classification performance index value is used to characterize the classification ability of the logistic regression model for each sample data; and determining a target feature dimension combination based on the classification performance index value; wherein each feature data of each sample data under the target feature dimension combination is used for neural network training.

Description

Technical Field

[0001] This disclosure relates to the field of neural network technology, and more specifically, to a feature dimension selection method, apparatus, computer device, and storage medium. Background Technology

[0002] Feature engineering is an engineering activity that aims to extract feature data from raw data for use by algorithms or models.

[0003] In related technologies, when performing feature engineering, developers often choose the processing method for the raw data based on their development experience, and manually filter and verify the feature dimensions of the obtained feature data to obtain the feature data under the required target feature dimensions. This process requires a lot of manpower and has low filtering efficiency. In addition, due to the limitations of the developers' personal experience, the selected target feature dimensions may not meet the actual needs, which may lead to poor training results of the subsequent network model. Summary of the Invention

[0004] This disclosure provides at least one feature dimension filtering method, apparatus, computer device, and storage medium.

[0005] In a first aspect, embodiments of this disclosure provide a feature dimension filtering method, including:

[0006] Obtain a sample dataset; wherein each sample data in the sample dataset contains feature data under multiple feature dimensions to be filtered;

[0007] Based on the feature data of each sample data under multiple feature dimension combinations, the logistic regression model is trained, and the classification performance index value corresponding to the feature dimension combination used in each training is determined; the classification performance index value is used to characterize the classification ability of the logistic regression model for each sample data.

[0008] Based on the classification performance index value, the target feature dimension combination is determined; wherein, each feature data of each sample data under the target feature dimension combination is used for neural network training.

[0009] Secondly, embodiments of this disclosure also provide a feature dimension filtering device, comprising:

[0010] An acquisition module is used to acquire a sample dataset; wherein each sample data in the sample dataset contains feature data under multiple feature dimensions to be filtered.

[0011] The training module is used to train the logistic regression model based on the feature data of each sample data under multiple feature dimension combinations, and to determine the classification performance index value corresponding to the feature dimension combination used in each training session; the classification performance index value is used to characterize the classification ability of the logistic regression model for each sample data.

[0012] The determination module is used to determine the target feature dimension combination based on the classification performance index value; wherein, each feature data of each sample data under the target feature dimension combination is used for neural network training.

[0013] Thirdly, embodiments of this disclosure also provide a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the first aspect above, or any possible implementation of the first aspect, are performed.

[0014] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the first aspect or any possible implementation of the first aspect.

[0015] The feature dimension filtering method, apparatus, computer device, and storage medium provided in this disclosure train a logistic regression model based on the feature data of each sample data in the sample dataset under multiple feature dimension combinations. Based on the classification performance index value corresponding to the feature dimension combination used in each training process, the target feature dimension combination that can be used for neural network training is automatically determined, realizing automatic feature dimension filtering and improving the efficiency of feature dimension filtering. Furthermore, based on the classification performance index value during the training process, feature dimension combinations that result in better classification performance can be filtered out, improving the quality of the filtered feature dimensions and further benefiting the improvement of subsequent neural network training effects.

[0016] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a feature dimension filtering method provided by an embodiment of this disclosure is shown;

[0019] Figure 2 This diagram illustrates a round of training of a logistic regression model in the feature dimension filtering method provided in this embodiment of the present disclosure.

[0020] Figure 3a The diagram illustrates the first round of training in two consecutive rounds of training in the feature dimension filtering method provided in this embodiment of the present disclosure.

[0021] Figure 3b This illustration shows a schematic diagram of the second round of training in a series of two consecutive rounds of training in the feature dimension filtering method provided in this embodiment of the present disclosure;

[0022] Figure 4 The diagram illustrates a feature dimension filtering method provided in this embodiment of the present disclosure.

[0023] Figure 5 This diagram illustrates the architecture of a feature dimension filtering device provided in an embodiment of the present disclosure.

[0024] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0026] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0027] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0028] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0029] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0030] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0031] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0032] Research has found that when performing feature engineering, developers often choose the processing method for the raw data based on their development experience, and manually filter the feature dimensions of the obtained feature data to obtain the feature data under the required target feature dimensions. However, this may lead to the following problems:

[0033] 1. Manually selected feature dimensions may not meet the usage requirements.

[0034] Here, due to the limitations of the developers' personal experience, the selected target feature dimensions may be difficult to meet the actual needs, and may even lead to problems such as poor training effect of subsequent network models.

[0035] 2. Manually selecting feature dimensions consumes a lot of manpower.

[0036] If developers manually select the target feature dimensions, they will need to spend a lot of time on data analysis and verifying the selected target feature dimensions. These processes may require a lot of manpower and time.

[0037] Based on the above research, this disclosure provides a feature dimension selection method, apparatus, computer device, and storage medium. It trains a logistic regression model based on the feature data of each sample in a sample dataset under multiple feature dimension combinations. Based on the classification performance index values ​​corresponding to the feature dimension combinations used in each training process, it automatically determines the target feature dimension combinations that can be used for neural network training, thus achieving automatic feature dimension selection and improving the efficiency of feature dimension selection. Furthermore, based on the classification performance index values ​​during the training process, it can select feature dimension combinations that result in better classification performance, improving the quality of the selected feature dimensions and further contributing to the improvement of subsequent neural network training effects.

[0038] To facilitate understanding of this embodiment, a feature dimension filtering method disclosed in this disclosure will first be described in detail. The execution entity of the feature dimension filtering method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this feature dimension filtering method can be implemented by a processor calling computer-readable instructions stored in memory.

[0039] See Figure 1 The diagram shows a flowchart of a feature dimension filtering method provided in an embodiment of this disclosure. The method includes steps S101 to S103, wherein:

[0040] S101: Obtain a sample dataset; wherein each sample data in the sample dataset contains feature data under multiple feature dimensions to be filtered.

[0041] S102: Based on the feature data of each sample data under multiple feature dimension combinations, train the logistic regression model and determine the classification performance index value corresponding to the feature dimension combination used in each training; the classification performance index value is used to characterize the classification ability of the logistic regression model for each sample data.

[0042] S103: Based on the classification performance index value, determine the target feature dimension combination; wherein, each feature data of each sample data under the target feature dimension combination is used for neural network training.

[0043] The following is a detailed explanation of the steps described above.

[0044] For S101,

[0045] Here, a sample data can consist of feature data under different feature dimensions; the feature dimensions can be used to characterize the attributes of the feature data, such as product type, object length, moving speed, etc., and a sample data can be, for example, composed of feature data under dimensions such as object length, object width, object height, and moving speed of object A.

[0046] In one possible implementation, sample data in the sample dataset can be generated according to the following steps A1-A2:

[0047] A1: Obtain field information of the original data under at least one field, wherein the field has a corresponding feature dimension.

[0048] Here, the original data can be composed of field information under multiple fields, and the relationship between fields and feature dimensions can be one-to-one. For example, the field "product type" can correspond to the feature dimension "product type".

[0049] A2: For any of the fields mentioned, according to the data processing method that matches the field type, the field value corresponding to the field is processed to obtain the feature data under the feature dimension corresponding to the field.

[0050] Here, the field type can include numeric fields, character fields, etc.

[0051] Specifically, data processing methods that match numeric fields can include any of the following:

[0052] Method 1: Take the logarithm of the field value of the numeric field.

[0053] Here, when processing the field values ​​of numeric fields, the logarithm of the field values ​​can be taken according to a preset base to obtain the feature data under the corresponding feature dimension of the field.

[0054] For example, taking a preset base of 2 and a field value of 8 for a numeric field, the logarithm of the field value 8 of the numeric field is taken according to the preset base of 2 to obtain the processed feature data 3.

[0055] Method 2: Perform a division operation on the field value of the numeric field and obtain the quotient.

[0056] Here, when processing the field values ​​of numeric fields, a division operation can be performed on the field values ​​of numeric fields according to a preset divisor, and the quotient obtained after the division operation can be used as the feature data under the feature dimension corresponding to the field.

[0057] For example, if the preset divisor is 10 and the field value of the numeric field is 135, then by performing a division operation on the field value of the numeric field 135 according to the preset divisor 10, the quotient of the division operation is 13, and the processed feature data can be determined to be 13.

[0058] Method 3: Perform numerical calculations based on the field values ​​of different numeric fields.

[0059] Here, when processing the field values ​​of numeric fields, the field values ​​of the first numeric field and at least one second numeric field can be calculated according to a preset numeric calculation formula, and the calculation result is used as the feature data under the feature dimension corresponding to the first numeric field; wherein, both the first numeric field and the second numeric field can be randomly extracted from the fields contained in the original data.

[0060] For example, if the value of the first numeric field is 10 and the value of the second numeric field is 20, and the numerical calculation formula is to add the first numeric field and the second numeric field together, then according to the preset numerical calculation formula, the calculation result is 30. Therefore, the feature data under the feature dimension corresponding to the processed first numeric field is 30.

[0061] In this way, by processing the field values ​​corresponding to the numeric fields using any of the above data processing methods, the format of the field values ​​in the original data can be converted into a unified data format, thereby obtaining feature data that meets the actual usage requirements.

[0062] Specifically, data processing methods that match character fields can include any of the following:

[0063] Method 1: Count the number of characters contained in the field value of a character field.

[0064] Here, when processing the field values ​​of character fields, we can count the number of characters contained in the field values ​​to obtain the feature data under the corresponding feature dimension of the field.

[0065] For example, if the field value of a character field is A0B0C1D0E0, the number of characters corresponding to A0B0C1D0E0 is 10. Therefore, the feature data under the feature dimension corresponding to this character field is 10, which is used to represent that the number of characters contained in the character field is 10.

[0066] Method 2: Perform word segmentation on the field values ​​of character fields.

[0067] Here, when processing the field values ​​of a character field, the field values ​​of the character field can be segmented into words, and the resulting segmented words can be used as feature data under the feature dimension corresponding to the character field.

[0068] For example, taking the value of a character field as "one shirt", after word segmentation of the field value, we can obtain "one" and "shirt". We can choose "shirt" as the feature data under the feature dimension corresponding to the character field to represent the item type; or we can choose "one" as the feature data under the feature dimension corresponding to the character field to represent the item quantity. This embodiment of the disclosure does not limit how to perform word segmentation or which word segmentation result to choose as the feature data, but only to meet the needs of actual application.

[0069] Method 3: Count browsing data that matches the field value of a character field.

[0070] Here, the browsing data may include exposure data, click data, and other data that can characterize browsing activity.

[0071] Specifically, when processing the values ​​of character fields, you can count the browsing data that matches the values ​​of the character fields and use the matching browsing data as feature data under the feature dimension corresponding to the character field.

[0072] For example, taking a character field as the product type, and the field value of the character field can be top, pants, or dress, then when the field value of the character field is "top", the browsing data that matches "top" can be used as feature data under the feature dimension corresponding to the character field.

[0073] In this way, by processing the field values ​​corresponding to character fields using any of the above data processing methods, the format of the field values ​​in the original data can be converted into a unified data format, thereby obtaining feature data that meets the actual usage requirements.

[0074] For S102 and S103,

[0075] Here, the feature dimension combination includes at least one feature dimension; the logistic regression model is used to perform a logistic regression task, and in the process of performing the logistic regression task, it can classify the input data to determine the category corresponding to the input data; the classification performance index value can be, for example, the area under the curve (AUC), which can be calculated by the area under the receiver operating characteristic curve (ROC) and is used to characterize the classification performance of the logistic regression model.

[0076] In practical applications, the sample data in the sample dataset can contain feature data under multiple feature dimensions. However, the role of feature data under different feature dimensions in the feature expression can be different. For example, in terms of whether or not one will choose to travel by plane, the feature data under the feature dimension "trip distance" is obviously more effective in determining whether or not one will travel by plane than the feature data under the feature dimension "number of trips". Therefore, after obtaining the sample data, the sample data can be filtered to reduce the number of feature dimensions used in subsequent feature dimension filtering and improve the efficiency of feature dimension filtering.

[0077] In one possible implementation, before training the logistic regression model based on the feature data of each sample data under multiple feature dimensions, the feature data under the multiple feature dimensions can also be filtered.

[0078] When filtering data, the following parameters can be used to filter feature data:

[0079] Parameter 1, Information Value Data

[0080] Here, the information value data corresponding to the feature data is used to characterize the degree of correlation between the feature data and the sample category.

[0081] Specifically, for any feature dimension, the information value (IV) corresponding to that feature dimension can be determined by the weight of evidence (WOE) corresponding to each feature data under that feature dimension; the WOE is used to describe the relationship between predictable variables (i.e., the current feature dimension) and binary variables (i.e., the output of the logistic regression model), and the WOE can be determined by the number of positive and negative samples under each value of the feature dimension, and the relationship between the number of positive and negative samples under the feature dimension.

[0082] For example, the formula for calculating the weight of evidence (WOE) can be:

[0083]

[0084] Where i represents the i-th value (i.e., feature data) in the feature dimension; Neg represents a negative sample; Pos represents a positive sample; and T represents a feature. This represents the ratio of the negative sample data corresponding to the i-th value of the feature dimension to all negative sample data under the feature dimension (i.e., the marginal negative sample ratio). This represents the ratio of the positive sample data corresponding to the i-th value of the feature dimension to all positive sample data under the feature dimension (i.e., the marginal positive sample ratio).

[0085] Furthermore, after calculating the weight of evidence (WOE), the information value data (IV) can be obtained by weighted summation of the corresponding WOE values. This embodiment of the present disclosure does not limit the specific calculation formula for the information value data.

[0086] Specifically, the information value data corresponding to each feature data can be determined, and each feature data can be filtered based on a preset information value threshold.

[0087] For example, with an information value threshold of 0.6, the information value data corresponding to feature data 1 to 5 under feature dimension 1 are 0.7, 0.5, 0.8, 0.4, and 0.6, respectively. Since the information value data corresponding to feature data 2 and feature data 4 are both less than the information value threshold, it can be determined that feature dimension 1 does not meet the information value requirement, or that feature data 2 and feature data 4 do not meet the information value requirement.

[0088] It should be noted that the above method of filtering each feature data based on the information value threshold includes deleting the feature data or deleting the feature data under the entire feature dimension when the information value data corresponding to the feature data is lower than the information value threshold. However, the above method is only exemplary. This disclosure does not limit how to filter each feature data based on the information value threshold, and the implementation shall prevail.

[0089] Parameter 2, Feature Enumeration Value

[0090] Here, the feature enumeration value is used to characterize the number of value types of feature data under the feature dimension. Taking the feature dimension as "product type" as an example, if the value of the feature data under the feature dimension can be "top", "pants" or "dress", then the feature enumeration value corresponding to the feature dimension "product type" is 3, which is used to characterize that there are 3 possible value types of feature data under this feature dimension.

[0091] Specifically, the feature enumeration value corresponding to each feature data is determined, and each feature data is filtered based on a preset enumeration value threshold.

[0092] For example, if the enumeration value threshold is 5, and the feature enumeration value corresponding to the feature dimension "product type" is 4, then it can be determined that the feature dimension "product type" does not meet the requirement of the number of enumeration values, and the feature data under the feature dimension "product type" is deleted.

[0093] In this way, by filtering the sample data used to train the logistic regression model in the above manner, the number of feature dimensions used when filtering feature dimensions can be reduced, thereby improving the efficiency of feature dimension filtering.

[0094] In one possible implementation, the training of the logistic regression model can be performed using the following steps B1 to B4:

[0095] B1: Each feature dimension among the multiple feature dimensions to be screened is taken as a candidate feature dimension combination. Based on the feature data of the sample data under each candidate feature dimension combination, the logistic regression model is trained to determine the classification performance index value corresponding to each candidate feature dimension combination.

[0096] Here, the training process of the logistic regression model may include multiple rounds of training. In the first round of training, each feature dimension in the feature dimension to be screened can be used as a candidate feature dimension combination, and the classification performance index value corresponding to each candidate feature dimension combination is determined during the training of the logistic regression model.

[0097] In one possible implementation, when training the logistic regression model based on the feature data of the sample data under each of the candidate feature dimension combinations, the following steps B11-B12 can be used:

[0098] B11: Based on the number of feature dimensions in the candidate feature dimension combination used in the current round of training, the sample data used in the current round is split to obtain multiple sample data.

[0099] Here, the number of segments to be segmented in the sample dataset can be determined based on the number of feature dimensions in the candidate feature dimension combination used in the current round of training. The sample data can then be segmented according to the number of segments to obtain multiple sets of sample data.

[0100] Specifically, when determining the number of segments based on the number of feature dimensions, the following formula can be used:

[0101] B = floor(log2A)

[0102] K=2 B

[0103] Where B is an intermediate parameter used to characterize the number of segments; floor represents the floor operation; A represents the number of feature dimensions; and K represents the number of segments.

[0104] For example, taking 100 sample data as an example, each sample data contains feature data under 5 feature dimensions. After splitting the sample data using the number of splits of 4, we can get 4 sample data, each of which contains 25 sample data, and each of which contains feature data under 5 feature dimensions.

[0105] B12: Determine the sample data used for each training session in this round of training from the multiple sample data, and train the logistic regression model based on the determined sample data; wherein, the number of sample data used for each training session increases with the increase of the number of training sessions, and the number of candidate feature dimension combinations used for each training session decreases with the increase of the number of training sessions.

[0106] In one possible implementation, in each round of training, the candidate feature combination used for each training session can be determined according to the following steps B121-B122;

[0107] B121: Select all candidate feature dimension combinations as candidate feature dimension combinations used for the first training.

[0108] B122: After obtaining the classification performance index values ​​of each candidate feature dimension combination in the current training, sort the candidate feature dimension combinations in descending order of classification performance index values, and select the candidate feature dimension combination that ranks first according to a preset ratio as the candidate feature dimension combination to be used in the next training.

[0109] Here, the number of sample data used in each training session can be doubled, and the number of candidate feature dimension combinations can be reduced by half (when the result of reducing by half is not an integer, it can be rounded down).

[0110] For example, a diagram illustrating one round of training of a logistic regression model can be shown as follows: Figure 2 As shown, Figure 2 In this training round, the number of candidate feature dimension combinations is 5. Based on the formula above, the number of segments is 4. Therefore, the training sample data can be divided into 4 parts. In the first training round, one set of sample data can be used to train the logistic regression model. Based on the AUC corresponding to each candidate feature dimension combination, the candidate feature dimension combinations corresponding to features 4 and 5 are eliminated to reduce the number of candidate feature dimension combinations in subsequent training. In the second training round, two sets of sample data can be used to train the logistic regression model. Based on the AUC corresponding to each candidate feature dimension combination, the candidate feature dimension combination corresponding to feature 3 is eliminated to reduce the number of candidate feature dimension combinations in subsequent training. In the third training round, four sets of sample data can be used to train the logistic regression model. Based on the AUC corresponding to each candidate feature dimension combination, the candidate feature dimension combination corresponding to feature 2 is eliminated, thus ending this training round.

[0111] In this way, since the differences between candidate feature dimension combinations are relatively large in the early stage of a training round, only a small amount of sample data is needed to distinguish the candidate feature dimension combinations. In subsequent training, candidate feature dimension combinations with lower AUC are eliminated to reduce the number of candidate feature dimension combinations. As the number of candidate feature dimension combinations gradually decreases, and the AUC values ​​of each candidate feature dimension combination may become closer and closer in subsequent training, the sample data used in each training session can be doubled in order to better distinguish the impact of each candidate feature dimension combination on the classification ability of the logistic regression model in subsequent training. This allows for faster training speed in the early training stage when it is easy to distinguish each candidate feature dimension combination, thereby accelerating the selection efficiency of feature dimensions. By gradually using more sample data for training as the number of training sessions increases, the accuracy of selecting candidate feature dimension combinations based on AUC values ​​is ensured.

[0112] B2: Based on the classification performance index value corresponding to each candidate feature dimension combination, determine the candidate feature dimension combination with the highest classification performance index value from each candidate feature dimension combination, and use it as the intermediate feature dimension combination.

[0113] For example, Figure 2 Feature 1 (feature dimension 1) is the candidate feature dimension combination with the highest classification performance index value, and feature 1 can be used as an intermediate feature dimension combination.

[0114] B3: Add each feature dimension other than the intermediate feature dimension combination to the multiple feature dimensions to be screened, to the intermediate feature dimension combination, to obtain each updated candidate feature dimension combination.

[0115] For example, if the feature dimensions to be filtered are feature dimension 1 to feature dimension 5, and the determined intermediate feature dimension combination is feature dimension 1, then the updated feature dimension combination is feature dimension 1-feature dimension 2, feature dimension 1-feature dimension 3, feature dimension 1-feature dimension 4, and feature dimension 1-feature dimension 5.

[0116] B4: The step of training the logistic regression model based on the feature data of the sample data under each of the candidate feature dimension combinations, until a preset cutoff condition is reached, and the final intermediate feature dimension combination is taken as the target feature dimension combination; wherein, the cutoff condition includes returning the number of rounds of training to reach a preset number of rounds, and / or, the difference between the classification performance index value of the intermediate feature dimension combination obtained in the current round and the classification performance index value of the intermediate feature dimension combination obtained in the previous round is less than a set threshold.

[0117] Here, the preset number of rounds can be matched with the number of feature dimensions to be screened. For example, if the number of feature dimensions to be screened is 5, then the preset number of rounds can be 3. The difference between the classification performance index value of the intermediate feature dimension combination and the classification performance index value of the intermediate feature dimension combination obtained in the previous round is used to characterize the gain in classification ability brought to the logistic regression model after adding intermediate feature dimensions. If the difference between the classification performance index value of the intermediate feature dimension combination and the classification performance index value of the intermediate feature dimension combination obtained in the previous round is less than a set threshold, it indicates that adding the intermediate feature dimension combination has not brought a significant improvement in classification ability. If the intermediate feature dimension combination with the highest classification performance index value cannot bring a significant improvement in classification ability, then the remaining feature dimension combinations are also unlikely to bring a significant improvement in classification ability. Therefore, this can be used as a cutoff condition.

[0118] For example, a diagram illustrating two consecutive rounds of training can be shown as follows: Figure 3a and Figure 3b As shown, where, Figure 3a In the first round of two consecutive training rounds, the new intermediate feature determined by the AUC corresponding to each candidate feature dimension combination is feature dimension 2, and the new intermediate feature dimension combination is feature dimension 1 and feature dimension 2. Figure 3b In the second round of two consecutive training rounds, the new intermediate feature determined by the AUC corresponding to each candidate feature dimension combination is feature dimension 3. The difference between the AUC corresponding to the new intermediate feature dimension combination feature dimension 1-feature dimension 2-feature dimension 3 and the AUC corresponding to the intermediate feature dimension combination feature dimension 1-feature dimension 2 in the previous round is 0.03, which is greater than the set threshold of 0.01 and has not reached the preset number of rounds. Then, the subsequent training steps can continue.

[0119] In this way, by selecting new intermediate feature dimensions through classification performance index values ​​in each round of training, and recombining the training data based on the combination of intermediate feature dimensions containing the new intermediate feature dimensions, intermediate feature dimensions that can improve classification ability can be obtained through multiple rounds of training of the logistic regression model.

[0120] In one possible implementation, when determining the target feature dimension combination, the candidate feature dimension combination with the highest classification performance index value can be determined from the candidate feature dimension combinations based on the classification performance index value corresponding to each candidate feature dimension combination in the last round of training, and this combination can be used as the target feature dimension combination.

[0121] For example, with Figure 3bTaking the last round of training as an example, the feature dimension 1-feature dimension 2-feature dimension 3 with the highest classification performance index value can be used as the target feature dimension combination selected from feature dimension 1 to feature dimension 5.

[0122] Below, we will combine Figure 4 The feature dimension filtering method provided in this disclosure embodiment is described below. The feature dimension filtering method provided in this disclosure embodiment may include the following steps:

[0123] Step 1: Process the raw data to generate candidate feature data corresponding to the feature dimensions to be screened.

[0124] Step 2: Filter the candidate feature data to obtain the initial feature data.

[0125] Step 3: Train the logistic regression model based on the initial feature data. In each round of training, determine a new intermediate feature dimension from the combination of multiple feature dimensions to be screened based on the classification performance index value.

[0126] Step 4: Based on the combination of intermediate feature dimensions formed by each intermediate feature dimension, determine the target feature dimension combination to complete the screening process for multiple feature dimensions to be screened.

[0127] For details on the specific steps mentioned above, please refer to the relevant descriptions, which will not be repeated here.

[0128] The feature dimension filtering method provided in this disclosure trains a logistic regression model based on the feature data of each sample in the sample dataset under multiple feature dimension combinations. Based on the classification performance index value corresponding to the feature dimension combination used in each training process, it automatically determines the target feature dimension combination that can be used for neural network training, thereby realizing automatic feature dimension filtering and improving the efficiency of feature dimension filtering. Furthermore, based on the classification performance index value during the training process, it can filter out feature dimension combinations that result in better classification performance, improving the quality of the filtered feature dimensions and further benefiting the improvement of subsequent neural network training effects.

[0129] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0130] Based on the same inventive concept, this disclosure also provides a feature dimension filtering device corresponding to the feature dimension filtering method. Since the principle of the device in this disclosure is similar to the feature dimension filtering method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0131] Reference Figure 5 The diagram shown is an architectural schematic of a feature dimension filtering device provided in an embodiment of this disclosure. The device includes: an acquisition module 501, a training module 502, and a determination module 503; wherein,

[0132] The acquisition module 501 is used to acquire a sample dataset; wherein each sample data in the sample dataset contains feature data under multiple feature dimensions to be filtered.

[0133] Training module 502 is used to train the logistic regression model based on the feature data of each sample data under multiple feature dimension combinations, and to determine the classification performance index value corresponding to the feature dimension combination used in each training; the classification performance index value is used to characterize the classification ability of the logistic regression model for each sample data.

[0134] The determination module 503 is used to determine the target feature dimension combination based on the classification performance index value; wherein, each feature data of each sample data under the target feature dimension combination is used for neural network training.

[0135] In one possible implementation, the training module 502, when training the logistic regression model based on feature data of each sample data under multiple feature dimension combinations, is used to:

[0136] Each feature dimension among the multiple feature dimensions to be screened is taken as a candidate feature dimension combination. Based on the feature data of the sample data under each candidate feature dimension combination, the logistic regression model is trained to determine the classification performance index value corresponding to each candidate feature dimension combination.

[0137] Based on the classification performance index value corresponding to each candidate feature dimension combination, the candidate feature dimension combination with the highest classification performance index value is determined from each candidate feature dimension combination and used as the intermediate feature dimension combination.

[0138] Each feature dimension other than the intermediate feature dimension combination is added to the intermediate feature dimension combination from the multiple feature dimensions to be screened, to obtain each updated candidate feature dimension combination.

[0139] The steps include: returning feature data based on the sample data under each of the candidate feature dimension combinations; training the logistic regression model until a preset cutoff condition is reached; and using the final intermediate feature dimension combination as the target feature dimension combination. The cutoff condition includes returning a preset number of training rounds, and / or the difference between the classification performance index value of the intermediate feature dimension combination obtained in the current round and the classification performance index value of the intermediate feature dimension combination obtained in the previous round is less than a set threshold.

[0140] In one possible implementation, the training module 502, when training the logistic regression model based on the feature data of the sample data under each combination of candidate feature dimensions, is used to:

[0141] Based on the number of feature dimensions in the candidate feature dimension combination used in the current round of training, the sample data used in the current round is split to obtain multiple sample data;

[0142] The sample data used in each training session during this round of training is determined from the multiple sample data sets, and the logistic regression model is trained based on the determined sample data. The number of sample data sets used in each training session increases with the number of training sessions, and the number of candidate feature dimension combinations used in each training session decreases with the number of training sessions.

[0143] In one possible implementation, in each round of training, the training module 502 is used to determine the candidate feature combination to be used in each training session according to the following steps;

[0144] All candidate feature dimension combinations are selected as the candidate feature dimension combinations used in the first training;

[0145] After obtaining the classification performance index values ​​of each candidate feature dimension combination in the current training, the candidate feature dimension combinations are sorted in descending order of classification performance index values. The candidate feature dimension combinations that are ranked first are selected according to a preset ratio as the candidate feature dimension combinations to be used in the next training.

[0146] In one possible implementation, the training module 502, when determining the target feature dimension combination based on the classification performance index value, is used to:

[0147] In the final round, based on the classification performance index values ​​corresponding to each candidate feature dimension combination, the candidate feature dimension combination with the highest classification performance index value is determined from all candidate feature dimension combinations and used as the target feature dimension combination.

[0148] In one possible implementation, the acquisition module 501 is further configured to generate sample data in the sample dataset according to the following steps:

[0149] Obtain field information of the original data under at least one field, wherein the field has a corresponding feature dimension;

[0150] For any of the fields, data processing is performed on the field value corresponding to the field according to the data processing method that matches the field type, to obtain the feature data under the feature dimension corresponding to the field.

[0151] In one possible implementation, if the field type includes a numeric field, the data processing method matching the field type includes at least one of the following: taking the logarithm of the field value of the numeric field, performing a division operation on the field value of the numeric field to obtain the quotient, and performing numerical operations based on the field values ​​of different numeric fields.

[0152] If the field type includes a character field, the data processing method matching the field type includes at least one of the following: counting the number of characters contained in the field value of the character field, performing word segmentation on the field value of the character field, and counting browsing data that match the field value of the character field.

[0153] In one possible implementation, before training the logistic regression model based on the feature data of each sample data under multiple feature dimension combinations, the acquisition module 501 is further configured to:

[0154] The feature data under the multiple feature dimensions is filtered according to the following steps:

[0155] The information value data corresponding to each feature data is determined, and each feature data is filtered based on a preset information value threshold, wherein the information value data is used to characterize the degree of correlation between the feature data and the sample category; and / or,

[0156] The feature enumeration value corresponding to each feature data is determined, and each feature data is filtered based on a preset enumeration value threshold. The feature enumeration value is used to characterize the number of value types of feature data under the feature dimension.

[0157] The feature dimension filtering device provided in this embodiment trains a logistic regression model based on the feature data of each sample data in the sample dataset under multiple feature dimension combinations. Based on the classification performance index value corresponding to the feature dimension combination used in each training process, it automatically determines the target feature dimension combination that can be used for neural network training, realizing automatic feature dimension filtering and improving the feature dimension filtering efficiency. Furthermore, based on the classification performance index value during the training process, it can filter out feature dimension combinations that have better classification performance, improving the quality of the filtered feature dimensions and further benefiting the improvement of subsequent neural network training effects.

[0158] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0159] Based on the same technical concept, this disclosure also provides a computer device. (See also...) Figure 6 The diagram shows the structure of a computer device 600 provided in this embodiment of the present disclosure, including a processor 601, a memory 602, and a bus 603. The memory 602 stores execution instructions and includes main memory 6021 and external memory 6022. The main memory 6021, also called internal memory, is used to temporarily store computational data in the processor 601 and data exchanged with external memory 6022 such as a hard disk. The processor 601 exchanges data with the external memory 6022 through the main memory 6021. When the computer device 600 is running, the processor 601 and the memory 602 communicate through the bus 603, causing the processor 601 to execute the following instructions:

[0160] Obtain a sample dataset; wherein each sample data in the sample dataset contains feature data under multiple feature dimensions to be filtered;

[0161] Based on the feature data of each sample data under multiple feature dimension combinations, the logistic regression model is trained, and the classification performance index value corresponding to the feature dimension combination used in each training is determined; the classification performance index value is used to characterize the classification ability of the logistic regression model for each sample data.

[0162] Based on the classification performance index value, the target feature dimension combination is determined; wherein, each feature data of each sample data under the target feature dimension combination is used for neural network training.

[0163] In one possible implementation, the instruction of the processor 601, which involves training the logistic regression model based on feature data from multiple feature dimension combinations of each sample data, includes:

[0164] Each feature dimension among the multiple feature dimensions to be screened is taken as a candidate feature dimension combination. Based on the feature data of the sample data under each candidate feature dimension combination, the logistic regression model is trained to determine the classification performance index value corresponding to each candidate feature dimension combination.

[0165] Based on the classification performance index value corresponding to each candidate feature dimension combination, the candidate feature dimension combination with the highest classification performance index value is determined from each candidate feature dimension combination and used as the intermediate feature dimension combination.

[0166] Each feature dimension other than the intermediate feature dimension combination is added to the intermediate feature dimension combination from the multiple feature dimensions to be screened, to obtain each updated candidate feature dimension combination.

[0167] The steps include: returning feature data based on the sample data under each of the candidate feature dimension combinations; training the logistic regression model until a preset cutoff condition is reached; and using the final intermediate feature dimension combination as the target feature dimension combination. The cutoff condition includes returning a preset number of training rounds, and / or the difference between the classification performance index value of the intermediate feature dimension combination obtained in the current round and the classification performance index value of the intermediate feature dimension combination obtained in the previous round is less than a set threshold.

[0168] In one possible implementation, the instructions of the processor 601, wherein training the logistic regression model based on the feature data of the sample data under each combination of the candidate feature dimensions, includes:

[0169] Based on the number of feature dimensions in the candidate feature dimension combination used in the current round of training, the sample data used in the current round is split to obtain multiple sample data;

[0170] The sample data used in each training session during this round of training is determined from the multiple sample data sets, and the logistic regression model is trained based on the determined sample data. The number of sample data sets used in each training session increases with the number of training sessions, and the number of candidate feature dimension combinations used in each training session decreases with the number of training sessions.

[0171] In one possible implementation, the instructions of the processor 601 include, in each round of training, determining the candidate feature combination to be used for each training session according to the following steps;

[0172] All candidate feature dimension combinations are selected as the candidate feature dimension combinations used in the first training;

[0173] After obtaining the classification performance index values ​​of each candidate feature dimension combination in the current training, the candidate feature dimension combinations are sorted in descending order of classification performance index values. The candidate feature dimension combinations that are ranked first are selected according to a preset ratio as the candidate feature dimension combinations to be used in the next training.

[0174] In one possible implementation, the instruction of the processor 601, the step of determining the target feature dimension combination based on the classification performance index value, includes:

[0175] In the final round, based on the classification performance index values ​​corresponding to each candidate feature dimension combination, the candidate feature dimension combination with the highest classification performance index value is determined from all candidate feature dimension combinations and used as the target feature dimension combination.

[0176] In one possible implementation, the instructions of the processor 601 further include generating sample data from the sample dataset according to the following steps:

[0177] Obtain field information of the original data under at least one field, wherein the field has a corresponding feature dimension;

[0178] For any of the fields, data processing is performed on the field value corresponding to the field according to the data processing method that matches the field type, to obtain the feature data under the feature dimension corresponding to the field.

[0179] In one possible implementation, if the field type in the instructions of the processor 601 includes a numeric field, the data processing method matching the field type includes at least one of the following: taking the logarithm of the field value of the numeric field, performing a division operation on the field value of the numeric field to obtain the quotient, and performing numerical operations based on the field values ​​of different numeric fields.

[0180] If the field type includes a character field, the data processing method matching the field type includes at least one of the following: counting the number of characters contained in the field value of the character field, performing word segmentation on the field value of the character field, and counting browsing data that match the field value of the character field.

[0181] In one possible implementation, before training the logistic regression model based on the feature data of each sample data under multiple feature dimension combinations, the instructions of the processor 601 further include:

[0182] The feature data under the multiple feature dimensions is filtered according to the following steps:

[0183] The information value data corresponding to each feature data is determined, and each feature data is filtered based on a preset information value threshold, wherein the information value data is used to characterize the degree of correlation between the feature data and the sample category; and / or,

[0184] The feature enumeration value corresponding to each feature data is determined, and each feature data is filtered based on a preset enumeration value threshold. The feature enumeration value is used to characterize the number of value types of feature data under the feature dimension.

[0185] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the feature dimension filtering method described in the above method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0186] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the feature dimension filtering method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0187] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0188] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0189] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0190] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0191] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0192] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A feature dimension selection method, applied to feature engineering, characterized in that, include: Obtain a sample dataset; wherein each sample data in the sample dataset contains feature data under multiple feature dimensions to be filtered, and the multiple feature dimensions respectively indicate multiple different attributes of the product; Based on the feature data of each sample data under multiple feature dimension combinations, the logistic regression model is trained, and the classification performance index value corresponding to the feature dimension combination used in each training is determined; the classification performance index value is used to characterize the classification ability of the logistic regression model for each sample data. Based on the classification performance index value, the target feature dimension combination is determined; wherein, the feature data of each sample data under the target feature dimension combination is used for neural network training. The step of training the logistic regression model based on feature data from each sample under multiple feature dimension combinations includes: Each of the multiple feature dimensions to be screened is taken as a candidate feature dimension combination. Based on the feature data of the sample data under each candidate feature dimension combination, the logistic regression model is trained to determine the classification performance index value corresponding to each candidate feature dimension combination. Based on the classification performance index value corresponding to each of the candidate feature dimension combinations, the candidate feature dimension combination with the highest classification performance index value is determined from each of the candidate feature dimension combinations and used as the intermediate feature dimension combination. Each feature dimension other than the intermediate feature dimension combination is added to the intermediate feature dimension combination from the multiple feature dimensions to be screened, to obtain each updated candidate feature dimension combination. The steps include: returning feature data based on the sample data under each of the candidate feature dimension combinations; training the logistic regression model until a preset cutoff condition is reached; and using the final intermediate feature dimension combination as the target feature dimension combination. The cutoff condition includes returning a preset number of training rounds, and / or the difference between the classification performance index value of the intermediate feature dimension combination obtained in the current round and the classification performance index value of the intermediate feature dimension combination obtained in the previous round is less than a set threshold. The method further includes generating sample data in the sample dataset according to the following steps: Obtain field information of the original data under at least one field, wherein the field has a corresponding feature dimension; For any of the fields, according to the data processing method that matches the field type, the field value corresponding to the field is processed to obtain the feature data under the feature dimension corresponding to the field; Wherein, if the field type includes a numeric field; the data processing method matching the field type includes at least one of the following: taking the logarithm of the field value of the numeric field, performing a division operation on the field value of the numeric field to obtain the quotient, and performing numerical operations based on the field values ​​of different numeric fields; If the field type includes a character field, the data processing method matching the field type includes at least one of the following: counting the number of characters contained in the field value of the character field, performing word segmentation on the field value of the character field, and counting browsing data that match the field value of the character field.

2. The method according to claim 1, characterized in that, The training of the logistic regression model based on the feature data of the sample data under each of the candidate feature dimension combinations includes: Based on the number of feature dimensions in the candidate feature dimension combination used in the current round of training, the sample data used in the current round is split to obtain multiple sample data; The sample data used in each training session during the current round of training is determined from the multiple sample data sets, and the logistic regression model is trained based on the determined sample data. The number of sample data sets used in each training session increases with the number of training sessions, and the number of candidate feature dimension combinations used in each training session decreases with the number of training sessions.

3. The method according to claim 2, characterized in that, In each round of training, the combination of candidate feature dimensions to be used for each training session is determined according to the following steps; All candidate feature dimension combinations are selected as the candidate feature dimension combinations used in the first training; After obtaining the classification performance index values ​​of each candidate feature dimension combination in the current training, the candidate feature dimension combinations are sorted in descending order of classification performance index values. The candidate feature dimension combinations that are ranked first are selected according to a preset ratio as the candidate feature dimension combinations to be used in the next training.

4. The method according to claim 1, characterized in that, The step of determining the target feature dimension combination based on the classification performance index value includes: The candidate feature dimension combination with the highest classification performance index value determined from each candidate feature dimension combination in the last round is taken as the target feature dimension combination.

5. The method according to claim 1, characterized in that, Before training the logistic regression model based on the feature data of each sample data under multiple feature dimension combinations, the method further includes: The feature data under the multiple feature dimensions is filtered according to the following steps: The information value data corresponding to each feature data is determined, and each feature data is filtered based on a preset information value threshold, wherein the information value data is used to characterize the degree of correlation between the feature data and the sample category; and / or, The feature enumeration value corresponding to each feature data is determined, and each feature data is filtered based on a preset enumeration value threshold. The feature enumeration value is used to characterize the number of value types of feature data under the feature dimension.

6. A feature dimension filtering device, applied to feature engineering, characterized in that, include: The acquisition module is used to acquire a sample dataset; wherein each sample data in the sample dataset contains feature data under multiple feature dimensions to be filtered, and the multiple feature dimensions respectively indicate multiple different attributes of the product; The training module is used to train the logistic regression model based on the feature data of each sample data under multiple feature dimension combinations, and to determine the classification performance index value corresponding to the feature dimension combination used in each training session; the classification performance index value is used to characterize the classification ability of the logistic regression model for each sample data. The determination module is used to determine the target feature dimension combination based on the classification performance index value; wherein, the feature data of each sample data under the target feature dimension combination is used for neural network training; The training module, when training the logistic regression model based on feature data from multiple feature dimensions of each sample data, is used for: Each of the multiple feature dimensions to be screened is taken as a candidate feature dimension combination. Based on the feature data of the sample data under each candidate feature dimension combination, the logistic regression model is trained to determine the classification performance index value corresponding to each candidate feature dimension combination. Based on the classification performance index value corresponding to each of the candidate feature dimension combinations, the candidate feature dimension combination with the highest classification performance index value is determined from each of the candidate feature dimension combinations and used as the intermediate feature dimension combination. Each feature dimension other than the intermediate feature dimension combination is added to the intermediate feature dimension combination from the multiple feature dimensions to be screened, to obtain each updated candidate feature dimension combination. The steps include: returning feature data based on the sample data under each of the candidate feature dimension combinations; training the logistic regression model until a preset cutoff condition is reached; and using the final intermediate feature dimension combination as the target feature dimension combination. The cutoff condition includes returning a preset number of training rounds, and / or the difference between the classification performance index value of the intermediate feature dimension combination obtained in the current round and the classification performance index value of the intermediate feature dimension combination obtained in the previous round is less than a set threshold. The acquisition module is further configured to generate sample data in the sample dataset according to the following steps: Obtain field information of the original data under at least one field, wherein the field has a corresponding feature dimension; For any of the fields, according to the data processing method that matches the field type, the field value corresponding to the field is processed to obtain the feature data under the feature dimension corresponding to the field; Wherein, if the field type includes a numeric field; the data processing method matching the field type includes at least one of the following: taking the logarithm of the field value of the numeric field, performing a division operation on the field value of the numeric field to obtain the quotient, and performing numerical operations based on the field values ​​of different numeric fields; If the field type includes a character field, the data processing method matching the field type includes at least one of the following: counting the number of characters contained in the field value of the character field, performing word segmentation on the field value of the character field, and counting browsing data that match the field value of the character field.

7. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the feature dimension filtering method as described in any one of claims 1 to 5 are performed.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the feature dimension filtering method as described in any one of claims 1 to 5.

Images