Mixed-type data sampling processing device and method

Abstract

The present invention relates to a mixed-type data sampling processing device and method. According to one embodiment of the present invention, the mixed-type data sampling processing device filters minority class sample data to be synthesized and generates, with respect to the minority class sample data to be synthesized, synthetic samples in consideration of both numerical variables and categorical variables, thereby enabling local oversampling to be implemented for mixed-type data.

Description

Mixed-type data sampling processing device and method

[0001] The present invention relates to a mixed-type data sampling processing apparatus and method, and more specifically, to a technique for implementing local oversampling for mixed-type data by filtering minority class sample data to be synthesized and generating a synthetic sample by considering both numerical and categorical variables for the minority class sample data to be synthesized.

[0002] Generally, classification learning refers to predicting the corresponding class for given input data.

[0003] However, if the data used for classification learning is severely imbalanced, poor classification performance occurs.

[0004] Class imbalance issues arise from the difference in the number of data points corresponding to each class of the target variable in the data.

[0005] In this case, the model fails to properly identify the minority class, and there is also a possibility of model overfitting.

[0006] Class imbalance is commonly observed in real-world data, so it is necessary to address it appropriately.

[0007] Research to address class imbalance issues has been actively conducted for some time, and among these, data-level methods have been extensively studied due to their ease and flexibility of application.

[0008] Among data-level oversampling methods, local oversampling is a method of synthesizing new samples by utilizing the local characteristics of minority classes, and a representative local oversampling technique is SMOTE (Synthetic Minority Oversampling Technique).

[0009] SMOTE (Synthetic Minority Oversampling Technique) is a representative local oversampling algorithm that synthesizes new samples between minority class samples using interpolation.

[0010] KNNOR (k-Nearest Neighbors OveRsampling) is a method that determines a subset of minority class samples to be used for synthesis based on the distance of minority class samples to their k-nearest neighbor samples, and synthesizes new samples using the selected minority class samples and their neighbor samples.

[0011] The existing KNNOR local oversampling algorithm is an oversampling method that does not consider mixed data. Therefore, there is a need for improvement so that it can be applied to mixed data as well.

[0012] NROMM (Noise Robust Oversampling for Multi-class and Mixed-type Data) is a new oversampling technique that extends the application scope of local oversampling algorithms from binary classification problems of numerical data to multi-class classification problems of mixed-type data.

[0013] To summarize, local oversampling is a method that increases the number of minority class samples by synthesizing new samples using local information from the minority class samples of the original dataset.

[0014] A representative local oversampling algorithm is SMOTE, which has the advantage of augmenting data without distorting the data distribution because it generates new samples that are locally similar to minority class samples within the existing data.

[0015] However, existing local oversampling algorithms have primarily been studied for numerical data, and research on mixed data is still lacking.

[0016] The present invention aims to implement local oversampling for mixed data by filtering minority class sample data to be synthesized and generating synthetic samples by considering both numerical and categorical variables for the minority class sample data to be synthesized.

[0017] The present invention aims to process the sampling of mixed data so that model training can proceed with data having an equal number of classes by filtering sample data of a minority class to be synthesized based on a distance measurement method for mixed data and generating synthetic samples by considering both numerical and categorical variables.

[0018] The present invention aims to implement HEEM (Heterogeneous Euclidean-Entropy Metric), a distance measurement method that considers both categorical and numerical variables, by performing Euclidean distance calculations for numerical variables and calculating entropy-based distances for categorical variables.

[0019] According to one embodiment of the present invention, a mixed-type data sampling processing device may include: a filtering processing unit that performs a distance measurement considering numerical variables and categorical variables with respect to the sample target minority class sample data and the sample target minority class sample data in the original data set to be sampled, and filters at least one neighbor class minority sample data according to the distance measurement performed; a sample synthesis processing unit that generates a sample synthesis data considering numerical variables and categorical variables with respect to the sample target minority sample data and the at least one neighbor minority class sample data; and a validation unit that calculates neighbor sample data by applying a distance based on the distance measurement to the sample synthesis data and validates the sample synthesis data based on the class of the neighbor sample data to process sampling for the sample target.

[0020] The filtering processing unit determines a plurality of neighbor class prime sample data located at an arbitrary number (k-th) close to the synthesis target prime class sample data according to the distance measurement performed above, sorts the plurality of neighbor class prime sample data according to the distance size based on the distance measurement, and can filter at least one neighbor class prime sample data located within a distance range within a preset threshold value by considering noise or outliers.

[0021] The filtering processing unit above can perform distance measurement using Euclidean distance for numerical variables and using entropy-based distance for categorical variables in the distance measurement performed above.

[0022] The sample synthesis processing unit above can generate the synthesized sample data by using interpolation in the case of the numerical variable and using the mode of the at least one neighboring minority class sample data in the case of the categorical variable for the synthesized minority sample data and the at least one neighboring minority class sample data.

[0023] The sample synthesis processing unit above can randomly select one neighbor prime class sample data from among the nearest neighbor prime class sample data of an arbitrary number (k-th) for the case of the numeric variable, and generate the synthesized sample data at some point between the prime class sample data to be synthesized and the selected neighbor prime class sample data.

[0024] The above sample synthesis processing unit can generate the above synthetic sample data selected as the variable value of the mode by comparing the category value of the neighboring minority class sample data at an arbitrary number (k-th) for the above categorical variable.

[0025] The above validation unit can calculate neighbor sample data by applying a distance based on the distance measurement to the above synthetic sample data, and process sampling for the sampling target by excluding the above synthetic sample data from the minority class sample data of the synthesis target when a majority class is identified in the class of the above neighbor sample data.

[0026] According to one embodiment of the present invention, a mixed-type data sampling processing method may include: a step in a filtering processing unit of performing a distance measurement considering numerical variables and categorical variables with respect to the sample target minority class sample data and the sample target minority class sample data in the sample target original data set, and filtering at least one neighbor class minority sample data according to the performed distance measurement; a step in a sample synthesis processing unit of generating a composite sample data considering numerical variables and categorical variables with respect to the sample target minority sample data and the at least one neighbor minority class sample data in the sample synthesis processing unit; and a step in a validation unit of calculating neighbor sample data by applying a distance based on the distance measurement to the composite sample data and validating the composite sample data based on the class of the neighbor sample data to process sampling for the sample target.

[0027] The step of filtering at least one neighbor class minority sample data according to the distance measurement performed above may include determining a plurality of neighbor class minority sample data located at an arbitrary number (k-th) close to the synthesis target minority class sample data according to the distance measurement performed above, sorting the plurality of neighbor class minority sample data according to a distance magnitude based on the distance measurement, and filtering the at least one neighbor class minority sample data located within a distance range within a preset threshold value by considering noise or outliers.

[0028] The step of generating the synthetic sample data may include the step of generating the synthetic sample data by using interpolation in the case of the numerical variable and using the mode of the at least one neighboring minority class sample data for the case of the synthetic target minority sample data and the at least one neighboring minority class sample data.

[0029] The step of processing sampling for the above-mentioned sampling target may include the step of calculating neighbor sample data by applying a distance based on the distance measurement to the above-mentioned synthetic sample data, and excluding the above-mentioned synthetic sample data from the above-mentioned minority class sample data when a majority class is identified in the class of the neighbor sample data to process sampling for the above-mentioned sampling target.

[0030] The present invention can implement local oversampling for mixed data by filtering minority class sample data to be synthesized and generating synthetic samples by considering both numerical and categorical variables for the minority class sample data to be synthesized.

[0031] The present invention can process the sampling of mixed data to enable model training with data having an equal number of classes by filtering minority class sample data to be synthesized based on a distance measurement method for mixed data and generating synthetic samples by considering both numerical and categorical variables.

[0032] The present invention can implement HEEM (Heterogeneous Euclidean-Entropy Metric), a distance measurement method that considers both categorical and numerical variables, by performing Euclidean distance calculations for numerical variables and calculating entropy-based distances for categorical variables.

[0033] FIG. 1 is a drawing illustrating a mixed-type data sampling processing device according to an embodiment of the present invention.

[0034] FIGS. 2a to 2d are drawings illustrating the sampling simulation results of a mixed-type data sampling processing device according to an embodiment of the present invention.

[0035] FIG. 3 is a diagram illustrating a mixed-type data sampling processing method according to an embodiment of the present invention.

[0036] Hereinafter, various embodiments of this document are described with reference to the attached drawings.

[0037] The embodiments and terms used therein are not intended to limit the technology described in this document to specific embodiments and should be understood to include various modifications, equivalents, and / or substitutions of said embodiments.

[0038] In describing various embodiments below, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the invention, such detailed description will be omitted.

[0039] Furthermore, the terms described below are defined considering their functions in various embodiments, and these may vary depending on the intentions or practices of the user or operator. Therefore, their definitions should be based on the content throughout this specification.

[0040] In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0041] A singular expression may include a plural expression unless the context clearly indicates otherwise.

[0042] In this document, expressions such as "A or B" or "at least one of A and / or B" may include all possible combinations of the items listed together.

[0043] Expressions such as "first," "second," "first," or "second" may modify the corresponding components regardless of order or importance, and are used merely to distinguish one component from another without limiting the components.

[0044] Where it is stated that a certain (e.g., first) component is "(functionally or telecommunicationally) connected" or "connected" to another (e.g., second) component, said certain component may be directly connected to said other component or connected through another component (e.g., third component).

[0045] In this specification, "configured to" may be used interchangeably with, depending on the context, for example, in hardware or software, "suitable for," "capable of," "modified to," "made to," "capable of," or "designed to."

[0046] In some situations, the expression "device configured to..." may mean that the device is "able to..." together with other devices or parts.

[0047] For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing said operations (e.g., an embedded processor), or a general-purpose processor capable of performing said operations by executing one or more software programs stored in a memory device (e.g., a CPU or an application processor).

[0048] Also, the term 'or' means an inclusive or rather an exclusive or.

[0049] That is, unless otherwise noted or is not clear from the context, the expression 'x uses a or b' means any one of the natural inclusive permutations.

[0050] Terms such as '..bu', '..gi' used below refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or a combination of hardware and software.

[0051] FIG. 1 is a drawing illustrating a mixed-type data sampling processing device according to an embodiment of the present invention.

[0052] FIG. 1 illustrates the components of a mixed-type data sampling processing device according to an embodiment of the present invention.

[0053] Referring to FIG. 1, a mixed-type data sampling processing device (100) according to one embodiment of the present invention includes a filtering processing unit (110), a sample synthesis processing unit (120), and a validity verification unit (130).

[0054] According to one embodiment of the present invention, the filtering processing unit (110) performs distance measurement considering numerical variables and categorical variables for the sample data of the minority class of the synthesis target and the sample data of the minority class of the synthesis target in the original data set to be sampled.

[0055] Additionally, the filtering processing unit (110) filters at least one neighbor class minority sample data according to the distance measurement performed.

[0056] For example, the filtering processing unit (110) determines a plurality of neighboring class prime sample data located at an arbitrary number (kth) close to the synthesis target prime class sample data based on the distance measurement performed.

[0057] Additionally, the filtering processing unit (110) sorts multiple neighbor class minority sample data according to a distance size based on distance measurement, and can filter at least one neighbor class minority sample data located within a distance range within a preset threshold value by considering noise or outliers.

[0058] According to one embodiment of the present invention, the filtering processing unit (110) can perform distance measurement using Euclidean distance for numerical variables in the performed distance measurement, and can perform distance measurement using entropy-based distance for categorical variables.

[0059] For example, the filtering processing unit (110) performs distance measurement, which is a HEEM (Heterogeneous Euclidean-Entropy Metric) measurement, using the following mathematical formula 1.

[0060] [Mathematical Formula 1]

[0061]

[0062] Mathematical Equation 1 exemplifies the calculation of distance using entropy-based distance for categorical variables.

[0063] In mathematical formula 1, the distance calculation result is represented using the entropy of variable and the average entropy weight of variable.

[0064] Since local oversampling generates new samples between existing prime class samples and their neighboring prime class samples, the core of this algorithm lies in determining the appropriate prime class samples to use for generation and properly defining their neighbor samples.

[0065] The filtering processing unit (110) introduces a distance measurement method (HEEM) for mixed data to determine a subset of minority class data to be used for generation, defines each neighbor sample, and calculates the final distance using Euclidean distance for numerical variables and entropy-based distance for categorical variables.

[0066] The filtering processing unit (110) calculates the distance to the k-th closest neighboring prime class sample for every prime class sample in the data.

[0067] At this time, the HEEM distance measurement method for mixed data is used when calculating the distance to the k-th neighboring prime class sample.

[0068] After sorting the distances calculated for all prime class samples in order of size, only the prime class samples that satisfy the distance within the threshold set by the algorithm are finally synthesized.

[0069] At this time, noise or outliers located far from the cluster where minority class samples are gathered can be filtered out.

[0070] For example, the sample synthesis processing unit (120) can generate synthesized sample data by considering numerical variables and categorical variables for the sample data to be synthesized and at least one neighboring minority class sample data.

[0071] A sample synthesis processing unit (120) according to an embodiment of the present invention can generate the synthesized sample data by using interpolation in the case of a numerical variable and using the mode of the at least one neighboring minority class sample data for the case of a categorical variable.

[0072] For example, the sample synthesis processing unit (120) can randomly select one neighboring prime class sample data from among the nearest neighboring prime class sample data of an arbitrary number (k-th) in the case of a numeric variable and generate synthesis sample data at some point between the prime class sample data to be synthesized and the selected neighboring prime class sample data.

[0073] A sample synthesis processing unit (120) according to one embodiment of the present invention can generate synthetic sample data selected as the variable value of the mode by comparing the category value of the neighboring minority class sample data of an arbitrary number (k-th) in the case of a categorical variable.

[0074] For example, the sample synthesis processing unit (120) can process sample synthesis using mathematical formula 2.

[0075] [Mathematical Formula 2]

[0076]

[0077] In mathematical formula 2, sample synthesis is performed using neighbors of the minority class and k-nearest neighbors.

[0078] Most local oversampling methods use interpolation, SMOTE's sample synthesis method, to synthesize new samples between minority class samples.

[0079] Since this method is designed for numerical variables (continuous variables), it cannot be hastily applied to categorical variables.

[0080] This is because it is difficult to define the median for categorical variables (e.g., it is impossible to define the median for men and women).

[0081] SMOTE can actually distort the distribution of the variable or generate meaningless data.

[0082] Therefore, the proposed algorithm performs sample synthesis by adopting the interpolation method for numerical types and the SMOTE-NC synthesis method, which selects the mode of neighboring samples, for categorical types, depending on the variable type.

[0083] A sample synthesis processing unit (120) according to an embodiment of the present invention finds, for every prime class sample, k neighbor samples that are closest in distance and have the same class among the samples adjacent to it.

[0084] The sample synthesis processing unit (120) uses the sample synthesis method of SMOTE-NC introduced for mixed-type data during the process of synthesizing samples to take different synthesis methods depending on the variable type.

[0085] For numerical variables, the sample synthesis processing unit (120) randomly selects a neighbor sample from among the minority class sample and the k nearest neighbor samples corresponding thereto and synthesizes a new sample at some point between the two minority class samples.

[0086] The sample synthesis processing unit (120) compares the category values ​​of k nearest neighbor samples in the case of a categorical variable and selects the category value that appears most frequently as the value of the variable.

[0087] For example, the validation unit (130) can calculate neighbor sample data by applying a distance based on distance measurement to the synthetic sample data, and process sampling for the sampling target by validating the synthetic sample data based on the class of the neighbor sample data.

[0088] A validation unit (130) according to an embodiment of the present invention can calculate neighbor sample data by applying a distance based on distance measurement to synthetic sample data, and can process sampling for a sampling target by excluding synthetic sample data from the minority class sample data of the synthesis target when a majority class is identified in the class of neighbor sample data.

[0089] That is, the validation unit (130) examines the class (label) of all neighbor samples, and if any of the k nearest neighbor samples contains a sample of the majority class, the corresponding synthetic sample is removed from the synthetic minority class sample candidates.

[0090] The present invention discloses HEEM (Heterogeneous Euclidean-Entropy Metric), a distance measurement method that considers both numerical and categorical variables for mixed data.

[0091] SMOTE-NC (SMOTE for Nominal and Continuous features) is an algorithm in the SMOTE family designed to be applied to mixed data that includes both numerical and categorical variables.

[0092] Neighbor samples are obtained by considering the variable types, and a sample synthesis method specified according to the variable types is used.

[0093] First, a distance measurement method that considers both categorical and numerical variables was utilized. Using HEEM, a distance measurement method proposed by NROMM that considers both types of variables, Euclidean distance is calculated for numerical variables, and entropy-based distance is calculated for categorical variables.

[0094] Secondly, we adopted the SMOTE-NC sample synthesis method that considers both categorical and numerical variables. For numerical variables, sample synthesis is performed using SMOTE based on HEEM distance. For categorical variables, samples are synthesized using the mode of the categorical values ​​of neighboring minority class samples.

[0095] Accordingly, the present invention can implement local oversampling for mixed data by filtering minority class sample data to be synthesized and generating synthetic samples by considering both numerical and categorical variables for the minority class sample data to be synthesized.

[0096] FIGS. 2a to 2d are drawings illustrating the sampling simulation results of a mixed-type data sampling processing device according to an embodiment of the present invention.

[0097] FIGS. 2a to 2d illustrate the sampling processing results of a mixed-type data sampling processing device according to an embodiment of the present invention.

[0098] Referring to FIG. 2a, the graph (200) shows the distribution of the original data before applying filtering processing, showing the majority class as negative and the minority class as positive.

[0099] Referring to FIG. 2b, the graph (210) represents a prime class sample to be finally used for synthesis through a filtering process.

[0100] It can be seen that the bolded "X" indicates a selected sample among the minority class samples, and through filtering, samples located relatively far from the region where the minority class is clustered are identified as noise or outliers and are filtered out so as not to be used in sample synthesis.

[0101] Referring to FIG. 2c, the graph (220) shows the sample synthesis process and the validation process integrated.

[0102] Data points marked with a "star" refer to data points newly generated during the sample synthesis process.

[0103] At this time, some stars represent synthetic samples rejected during the validation process, and some stars represent final synthetic prime class samples that passed the validation process.

[0104] Looking at the distribution of some stars, one can see that the minority and majority classes are very close together and are mainly located between these classes.

[0105] During the validation process, these samples are identified as noise and removed, thereby enabling the safer synthesis of new minority class samples.

[0106] Referring to FIG. 2d, graph (230) shows the augmented data distribution after filtering, and it can be seen that the number of minority class samples is equal to the number of majority class samples.

[0107] Two source code files (KNNOR_HEEM.py, KNNOR_dist_metric.py), class imbalance data, and the smote_variant.py file are required.

[0108] After separating the prepared data into X (independent variable) and y (dependent variable, target variable), the KNNOR_HEEM() oversampling algorithm of KNNOR_HEEM is loaded.

[0109] Afterwards, if you run KNNOR_HEEM().sample(X, y), you can obtain the equilibrium data resulting from oversampling.

[0110] At this time, the algorithm's hyperparameters can be modified and used in KNNOR_HEEM.

[0111] The process of performing oversampling in the invented algorithm can be seen in the figure above. Oversampling for mixed data is performed in three stages: filtering, augmentation, and validation, and the result can be obtained as shown in the graph (230).

[0112] Accordingly, the present invention can implement HEEM (Heterogeneous Euclidean-Entropy Metric), a distance measurement method that considers both categorical and numerical variables, by performing Euclidean distance calculations for numerical variables and calculating entropy-based distances for categorical variables.

[0113] FIG. 3 is a diagram illustrating a mixed-type data sampling processing method according to an embodiment of the present invention.

[0114] FIG. 3 illustrates a procedure for implementing local oversampling for mixed data by a mixed data sampling processing method according to an embodiment of the present invention, which filters minority class sample data to be synthesized and generates a synthetic sample by considering both numerical variables and categorical variables for the minority class sample data to be synthesized.

[0115] Referring to FIG. 3, in step (S301), the mixed data sampling processing method according to an embodiment of the present invention filters the minority sample data to be synthesized and at least one neighbor class minority sample data.

[0116] That is, a mixed-type data sampling processing method according to one embodiment of the present invention can perform distance measurements considering numerical variables and categorical variables on the sampled minority class sample data and the sampled minority class sample data in the original data set to be sampled, and filter at least one neighboring minority class sample data according to the distance measurements performed.

[0117] Step (S301) is a filtering step, and a subset to be used for sample synthesis is obtained from the entire set of prime class samples.

[0118] In this process, the distance to the k-th neighbor is used to measure the density of minority class samples, and HEEM, a distance measurement method for mixed data, is used.

[0119] HEEM distance calculates Euclidean distance for numerical variables and entropy-based distance for categorical variables.

[0120] In step (S302), the mixed-type data sampling processing method according to an embodiment of the present invention generates sample composite data by considering at least one variable among numerical variables and categorical variables.

[0121] That is, a mixed-type data sampling processing method according to one embodiment of the present invention can generate synthetic sample data by considering numerical variables and categorical variables for the minority sample data to be synthesized and at least one neighboring minority class sample data.

[0122] Step (S302) synthesizes a new sample using a type-specific method, taking into account the type of the variable.

[0123] In this case, sample synthesis for numerical variables is performed through SMOTE with HEEM distance applied, while sample synthesis for categorical variables proceeds using the mode of the categorical values ​​of neighboring minority class samples.

[0124] In step (S303), the mixed-type data sampling processing method according to an embodiment of the present invention performs sampling processing by checking the validity of the synthetic sample data.

[0125] That is, a mixed-type data sampling processing method according to one embodiment of the present invention can process sampling for a sampling target by applying a distance based on distance measurement to synthetic sample data to calculate neighbor sample data, and checking the validity of the synthetic sample data based on the class of the neighbor sample data.

[0126] Step (S303) is a validation step, and validates the samples synthesized in the previous process. During validation, the classes of the k nearest neighbor samples obtained by HEEM distance are checked, and only synthetic samples having only minority classes as neighbors are determined as the final synthetic samples to be used.

[0127] Accordingly, the present invention can process the sampling of mixed data to enable model training with data having an equal number of classes by filtering minority class sample data to be synthesized based on a distance measurement method for mixed data and generating synthetic samples by considering both numerical and categorical variables.

[0128] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0129] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0130] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0131] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. A filtering processing unit that performs a distance measurement considering numerical variables and categorical variables with respect to the sample data of the minority class to be synthesized and the sample data of the minority class to be synthesized in the original dataset to be sampled, and filters at least one neighboring minority class sample data according to the distance measurement performed; A sample synthesis processing unit that generates synthesized sample data by considering the numerical variable and the categorical variable for the above-mentioned prime number sample data and the above-mentioned at least one neighboring prime number class sample data; and Characterized by including a validation unit that calculates neighbor sample data by applying a distance based on the distance measurement to the synthetic sample data, and processes sampling for the sampling target by verifying the validity of the synthetic sample data based on the class of the neighbor sample data. Mixed-type data sampling processing device.

2. In Paragraph 1, The filtering processing unit determines a plurality of neighbor class prime sample data located close to the synthesis target prime class sample data by an arbitrary number (k-th) based on the distance measurement performed above, sorts the plurality of neighbor class prime sample data according to the distance magnitude based on the distance measurement, and filters at least one neighbor class prime sample data located within a distance range within a preset threshold value by considering noise or outliers. Mixed-type data sampling processing device.

3. In Paragraph 2, The filtering processing unit is characterized by performing distance measurement using Euclidean distance for numerical variables and using entropy-based distance for categorical variables in the distance measurement performed above. Mixed-type data sampling processing device.

4. In Paragraph 1, The sample synthesis processing unit is characterized by generating the synthesized sample data by using interpolation in the case of the numerical variable and using the mode of the at least one neighboring prime class sample data in the case of the categorical variable, for the prime sample data to be synthesized and the at least one neighboring prime class sample data. Mixed-type data sampling processing device.

5. In Paragraph 4, The sample synthesis processing unit is characterized by randomly selecting one neighbor prime class sample data from among the nearest neighbor prime class sample data of an arbitrary number (k-th) for the case of the numeric variable, and generating the synthesis sample data at some point between the prime class sample data to be synthesized and the selected neighbor prime class sample data. Mixed-type data sampling processing device.

6. In Paragraph 4, The above sample synthesis processing unit is characterized by generating the above synthetic sample data selected as the variable value of the mode by comparing the category value of the neighboring minority class sample data at an arbitrary number (k-th) for the above categorical variable. Mixed-type data sampling processing device.

7. In Paragraph 1, The above validation unit calculates neighbor sample data by applying a distance based on the distance measurement to the above synthetic sample data, and is characterized by processing sampling for the sampling target by excluding the above synthetic sample data from the minority class sample data to the synthesis target when a majority class is identified in the class of the above neighbor sample data. Mixed-type data sampling processing device.

8. In the filtering processing unit, a distance measurement considering numerical variables and categorical variables is performed on the synthesis target minority class sample data and the synthesis target class minority sample data in the original dataset to be sampled, and at least one neighboring class minority sample data is filtered according to the performed distance measurement; In a sample synthesis processing unit, a step of generating synthetic sample data by considering the numerical variable and the categorical variable with respect to the prime sample data to be synthesized and the at least one neighboring prime class sample data; and The validation unit is characterized by including the step of calculating neighbor sample data by applying a distance based on the distance measurement to the synthetic sample data, and processing sampling for the sampling target by validating the synthetic sample data based on the class of the neighbor sample data. Mixed data sampling processing method.

9. In Paragraph 8, The step of filtering at least one neighbor class minority sample data according to the distance measurement performed above is: The method is characterized by including the step of determining a plurality of neighbor class prime sample data located close to the synthesis target prime class sample data by an arbitrary number (k-th) based on the distance measurement performed above, sorting the plurality of neighbor class prime sample data according to the distance magnitude based on the distance measurement, and filtering the at least one neighbor class prime sample data located within a distance range within a preset threshold value by considering noise or outliers. Mixed data sampling processing method.

10. In Paragraph 8, The step of generating the above synthetic sample data is, The method includes the step of generating the synthetic sample data by using interpolation for the numerical variable and using the mode of the at least one neighboring prime class sample data for the synthetic target prime sample data and the at least one neighboring prime class sample data, and for the categorical variable. The step of processing sampling for the above-mentioned sampling target is, The method is characterized by including the step of calculating neighbor sample data by applying a distance based on the distance measurement to the synthetic sample data, and processing sampling for the sampling target by excluding the synthetic sample data from the minority class sample data to the synthesis target when a majority class is identified in the class of the neighbor sample data. Mixed data sampling processing method.

Images