Method and system for data normalization for training using tabular data

Abstract

The present invention relates to a method and a system for data normalization for training using tabular data, and provides a B-spline-based method and system for data normalization for efficient deep neural network (DNN) training using tabular data.

Description

Data Normalization Method and System for Learning Tabular Data

[0001] The present invention relates to a data normalization method and system for learning tabular data, and provides a B-spline-based data normalization method and system for efficient Deep Neural Network (DNN) learning in tabular data.

[0002] Input normalization of numerical features plays a crucial role in improving the performance and training stability of neural networks. This is particularly important for tabular data, which is widely used across many industries.

[0003] A key characteristic of tabular data is its high heterogeneity, which means that features often have different scales and distributions. Therefore, it is essential to apply custom normalization to each input feature.

[0004] Unlike image or text data, tabular data generally has non-uniform input features; therefore, more precise normalization techniques are required to effectively handle diverse feature distributions. In this regard, various normalization techniques have been proposed to date, but they are often specialized for specific input feature distributions and have limitations in that they fail to consider the neural network learning process itself. In other words, they are not optimized for neural network training.

[0005] Therefore, there is still a need for new normalization methods for efficient neural network training on tabular data.

[0006] The present invention aims to provide a data normalization method and system for learning tabular data that enables an artificial neural network to understand tabular data well.

[0007] More specifically, the present invention aims to provide a data normalization method and system for learning tabular data that can improve the learning efficiency of an artificial neural network for tabular data by performing normalization on tabular data.

[0008] In addition, the present invention aims to provide a data normalization method and system for learning tabular data that can effectively handle the heterogeneity of tabular data.

[0009] Furthermore, the present invention aims to provide a data normalization method and system for learning tabular data that enables an artificial neural network to efficiently utilize limited capacity.

[0010] To solve the problem described above, a data normalization method for learning tabular data according to the present invention, which is computerized and comprises the following steps: receiving table data configured to include a plurality of different features and a plurality of samples having values ​​corresponding to each of the plurality of features; obtaining a plurality of normalized features by applying a curve-parameterized normalization function to each of the plurality of features included in the table data; performing training of an artificial neural network using each of the normalized plurality of features; and optimizing the normalization function based on the progress of the training of the artificial neural network.

[0011] In an embodiment, the method further includes the step of applying a min-max transformation to each of the plurality of features included in the table data, and in the step of obtaining the normalized plurality of features, the normalization function can be applied to each of the plurality of features to which the min-max transformation has been applied to obtain the normalized plurality of features.

[0012] In an embodiment, the step of training the artificial neural network includes the step of calculating a learning loss between a learning target value corresponding to the table data and an output value for the normalized plurality of features using a preset loss function, and the step of training the artificial neural network so that the calculated learning loss becomes smaller, and in the optimization step, the normalization function can be optimized based on the calculated learning loss.

[0013] In an embodiment, the step of calculating the learning loss is a step of calculating the learning loss between the output values ​​for a plurality of samples including the learning target value and the value corresponding to each of the plurality of normalized features using the loss function, and in the optimization step, at least one parameter of the normalization function can be optimized simultaneously with the learning of the artificial neural network.

[0014] In the embodiment, in the optimization step, the parameters of the normalization function are optimized simultaneously with the learning of the artificial neural network according to the importance specified for each of the plurality of samples, and the importance may be specified based on the learning loss calculated through the loss function.

[0015] In an embodiment, to specify the importance of each of the plurality of samples, the method further includes the step of analyzing the learning difficulty of each of the plurality of samples based on the learning loss calculated through the loss function, and the step of specifying the importance of each of the plurality of samples based on the analyzed learning difficulty, wherein the optimizing step may be the step of adjusting the slope of the normalization function according to the importance of each of the plurality of samples.

[0016] In an embodiment, the learning difficulty includes at least one of a first learning difficulty related to the learning loss satisfying a first criterion and a second learning difficulty related to the learning loss satisfying a second criterion, and in the step of specifying the importance, in order to adjust the slope of the normalization function, the importance of a first sample satisfying the first learning difficulty among the plurality of samples may be specified as having the first importance, and the importance of a second sample satisfying the second learning difficulty among the plurality of samples may be specified as having the second importance.

[0017] In the embodiment, in the optimization step, for the first sample having the first importance among the plurality of samples, the magnitude of the slope of the normalization function can be adjusted to increase.

[0018] In the embodiment, in the optimization step, for the second sample having the second importance among the plurality of samples, the magnitude of the slope of the normalization function can be adjusted to be reduced.

[0019] In an embodiment, the optimizing step is a step of adjusting the parameters of the normalization function according to the learning difficulty of each of the plurality of samples using the loss function, and the loss function may be defined to adjust the slope of the normalization function according to the importance of each of the plurality of samples.

[0020] In an embodiment, the importance includes at least one of a first importance and a second importance determined according to the learning loss, and in the optimization step, for a first sample having the first importance among the plurality of samples, the parameters of the normalization function can be adjusted using the loss function so that the magnitude of the gradient of the normalization function increases.

[0021] In the embodiment, in the optimization step, the parameters of the normalization function can be adjusted using the loss function so that, for the second sample having the second importance among the plurality of samples, the magnitude of the slope of the normalization function is reduced.

[0022] In an embodiment, the optimizing step may be a step of updating the parameters of the normalization function based on the plurality of sample-specific learning losses.

[0023] In an embodiment, when the parameters of the normalization function are updated, the values ​​of each of the normalized plurality of features may be updated.

[0024] In an embodiment, the loss function may be defined such that the learning process of the artificial neural network and the parameter optimization process of the regularization function are performed separately.

[0025] In an embodiment, the normalization function may be generated for each of the plurality of features included in the table data.

[0026] A data normalization system for learning tabular data according to the present invention, comprising a memory configured to store executable instructions and one or more processors configured to perform operations by executing one or more instructions, receives table data configured to include a plurality of different features and a plurality of samples having values ​​corresponding to each of the plurality of features, obtains a plurality of normalized features by applying a normalization function parameterized by a curve to each of the plurality of features included in the table data, performs learning of an artificial neural network using each of the plurality of normalized features, and can optimize the normalization function based on the progress of the learning of the artificial neural network.

[0027] A program according to the present invention is a program that is executed by one or more processes in an electronic device and can be stored on a computer-readable recording medium, and may include instructions for performing the steps of: receiving table data configured to include a plurality of different features and a plurality of samples having values ​​corresponding to each of the plurality of features; obtaining a plurality of normalized features by applying a normalization function parameterized by a curve to each of the plurality of features included in the table data; performing training of an artificial neural network using each of the normalized features; and optimizing the normalization function based on the progress of training of the artificial neural network.

[0028] As described above, according to the data normalization method and system for learning tabular data of the present invention, by identifying the importance of tabular data (i.e., input data) during the learning process of an artificial neural network and applying a normalization function accordingly, it is possible to support the artificial neural network in efficiently allocating capacity to input values ​​of high importance and to support efficient learning of the artificial neural network regarding tabular data.

[0029] In addition, according to the data normalization method and system for learning tabular data of the present invention, tabular data can be normalized using a B-spline-based normalization technique, and an artificial neural network can be trained using a loss function configured to facilitate learning of the normalized tabular data and difficult input sections, while simultaneously compressing already learned sections. Through this, the present invention can improve the performance of the artificial neural network in difficult samples by adjusting the transformation according to the difficulty of each data point. That is, in the case of data points that are difficult to learn, the present invention allocates a wider range in the normalized tabular data, thereby allowing the artificial neural network to efficiently distribute capacity to data points that are difficult to learn, and thus improving learning efficiency.

[0030] Furthermore, according to the data normalization method and system for learning tabular data of the present invention, by parameterizing the tabular data into curves and optimizing them together with the learning of an artificial neural network, adaptive normalization is enabled for each feature of the tabular data, thereby effectively handling the heterogeneity of the tabular data.

[0031] Furthermore, according to the data normalization method and system for learning tabular data of the present invention, by training an artificial neural network using a new loss function that dynamically adjusts B-spline parameters according to the difficulty of each data point, the artificial neural network can efficiently allocate capacity to difficult data points and ensure optimized transformation for both difficult and easy data.

[0032] Thus, the present invention enables an artificial neural network to learn optimal parameters that improve learning efficiency and performance by modeling feature transformations through a B-spline-based normalization technique customized for learning tabular data. Furthermore, the adaptability of the artificial neural network can be enhanced through a new loss function that dynamically stores input transformations according to the learning difficulty of specific data points.

[0033] FIG. 1 is a conceptual diagram illustrating a data normalization system for learning tabular data according to the present invention.

[0034] FIGS. 2a, FIGS. 2b, and FIGS. 2c are flowcharts illustrating a data normalization method for learning tabular data according to the present invention.

[0035] FIGS. 3a, FIGS. 3b, FIGS. 4, and FIGS. 5 are conceptual diagrams illustrating a data normalization method for learning tabular data according to the present invention.

[0036] FIGS. 6, FIGS. 7, and FIGS. 8 are formulas related to a data normalization method for learning tabular data according to the present invention.

[0037] FIG. 9 is a table showing an example of the learning result of an artificial neural network learned using a data normalization method for learning tabular data according to the present invention.

[0038] FIG. 10 illustrates an example of a block diagram of a computing system in which the present invention can be implemented.

[0039] FIG. 11 illustrates an example of a block diagram of a computing device that may be included in a user computing device, a server computing system, and a training computing system, as an embodiment of a computing system in which the present invention can be implemented.

[0040] FIG. 12 illustrates an example of a block diagram from another perspective of a computing device, which is one of the components of a computing system.

[0041] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components are assigned the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not have distinct meanings or roles in themselves. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention.

[0042] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0043] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0044] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0045] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0046] The present invention relates to a data normalization method and system for learning tabular data that enables an artificial neural network to better understand tabular data. More specifically, the present invention relates to a data normalization method and system for learning tabular data that can improve the learning efficiency of an artificial neural network regarding tabular data by performing normalization on the tabular data.

[0047] In the present invention, the term "table data" may also be referred to as "table data," "table-type data," or "table-type data." Table data may be structured in a table form that includes row information representing information on various multiple heterogeneous features of individual samples and column information representing information on one feature of multiple samples.

[0048] Such tabular data may possess both categorical features, such as gender and country, which are difficult to represent numerically and can be expressed through combinations of characters, nouns, or specific words, and numerical features, such as height, weight, and age, which can be expressed numerically. Additionally, tabular data may include various types of information related to diverse industrial sectors. In this case, the tabular data may contain numerical information that can be expressed as continuous numbers, and categorical information that is difficult to represent numerically and can be expressed through combinations of characters, nouns, or specific words.

[0049] In one embodiment, the table data relates to personal information such as the age, height, weight, and gender of multiple people, and may include information in which various characteristic information regarding multiple samples is structured in a table form. Here, the multiple samples may correspond to multiple people.

[0050] In another embodiment, the table data is information related to products held by the store and may include numerical information such as the product's year, weight, and length, and categorical information such as the product's brand name and color.

[0051] In another embodiment, the table data is information related to an item produced in a manufacturing plant and may include numerical information such as the height, width, and depth of the item, and categorical information such as manufacturing process line information and manufacturing process management supervisor information.

[0052] As such, table data may include at least one of numeric information and categorical information, and may be data of information structured in the form of a table containing various types of information regarding multiple samples. Additionally, table data may include information regarding various types of samples related to various industrial fields.

[0053] Furthermore, the present invention allows for the adaptive modification of a normalization function based on the importance of input data (or input values, samples, features, etc.) to efficiently utilize an artificial neural network in tabular data. Since each component of the input value in tabular data requires a different normalization function, the importance of the normalization function is higher compared to artificial neural network training in general data. To achieve efficient normalization in such tabular data, the present invention identifies the importance of input values ​​during the training process of the artificial neural network and adaptively modifies the normalization function accordingly, thereby enabling the artificial neural network to efficiently allocate capacity to more important input values ​​for training. More specific details will be described later.

[0054] Meanwhile, the present invention can be usefully utilized in various situations. More specifically, the data normalization method and system for learning tabular data according to the present invention can be applied to various industries and services and utilized effectively. For example, referring to FIG. 1, in the manufacturing industry, major decisions such as process optimization, quality control, equipment maintenance, and production forecasting are made based on tabular data (200) composed of sensor data, equipment logs, process variables, etc. The present invention can be usefully utilized by applying it to the manufacturing industry that utilizes the vast amount of tabular data (200) generated from such manufacturing processes and inspections. However, the fields in which the present invention is utilized are not necessarily limited to the fields mentioned above, and it is obvious that it can be applied and utilized in various fields other than those mentioned (e.g., medical field, financial field, scientific field, urban analysis, etc.).

[0055] Hereinafter, the present invention will be examined in more detail with reference to the attached drawings. FIG. 1 is a conceptual diagram illustrating a data normalization system for learning tabular data according to the present invention. FIGS. 2a, 2b, and 2c are flowcharts illustrating a data normalization method for learning tabular data according to the present invention. FIGS. 3a, 3b, 4, and 5 are conceptual diagrams illustrating a data normalization method for learning tabular data according to the present invention, and FIGS. 6, 7, and 8 are formulas related to a data normalization method for learning tabular data according to the present invention. Furthermore, FIG. 9 is a table showing an example of the learning result of an artificial neural network trained using the data normalization method for learning tabular data according to the present invention. In addition, FIG. 10 illustrates an example of a block diagram of a computing system in which the present invention can be implemented. FIG. 11 illustrates an example of a block diagram of a computing device that may be included in a user computing device, a server computing system, and a training computing system, as an embodiment of a computing system in which the present invention can be implemented. FIG. 12 illustrates an example of a block diagram of a computing device in another aspect that is one of the components of a computing system.

[0056] Meanwhile, as illustrated in FIG. 1, a data normalization system for learning tabular data according to the present invention (hereinafter referred to as the “data normalization system,” 100) may include at least one of an input unit (110), an output unit (120), a communication unit (130), a storage unit (140), a judgment unit (150), a data processing unit (160), an artificial neural network (170), and a control unit (180).

[0057] A data normalization system (100) according to the present invention may include at least one processor and at least one memory containing computer program code. In this case, the memory may perform the role of the storage unit (140) above. In the present invention, the memory and the program code may cooperate with the processor to perform a series of processes described below.

[0058] Although not illustrated, the data normalization system (100) according to the present invention may include one or more processors, and such processors may include one or more general-purpose processors and / or one or more special-purpose processors (e.g., digital signal processor, tensor processing unit (TPU), graphics processing unit (GPU), neural network processing unit (NPU), application integrated circuit, application semiconductor (ASIC), field programmable gate array (FPGA), quantum processing unit (or quantum processor, QPU), etc.). One or more processors may be configured to execute instructions, computer-readable instructions, and / or other instructions described herein that are stored (or included) in the storage unit (140). The data normalization method and system according to the present invention may perform the data processing described below in cooperation with memory and at least one processor. The processor may perform a series of operations and data processing using data and information stored in memory. In this case, memory may be a component of the storage unit (140).

[0059] In addition, the data normalization system (100) according to the present invention can perform data processing and computation processes using quantum gates, quantum entanglement, and quantum superposition states, taking into consideration implementation in a quantum computer environment. For example, the present invention can perform parallel computations based on qubits, and such quantum computations can operate complementarily with existing classical computers.

[0060] Such quantum computers may include parallel computation using qubits and high-speed data processing devices utilizing quantum entanglement, and hardware-based computational optimization using FPGAs and ASICs is possible. In addition, quantum computers may utilize quantum processors capable of qubit-based parallel computation, and data processing efficiency can be improved through a hybrid structure with existing classical computers.

[0061] Meanwhile, the input unit (110) can be configured in various ways as a means of data input. For example, the input unit (110) can be configured to receive user input. The input unit (110) can be configured to receive user input from a user terminal (10). Here, “receiving input” may mean receiving an input signal (or selection signal) corresponding to the user’s input based on input made by the user through the input unit configuration provided in the user terminal (10).

[0062] Here, the user terminal (10) may include at least one of a mobile phone, a smartphone, a notebook computer, a laptop computer, a slate PC, a tablet PC, an ultrabook, a desktop computer, a digital broadcasting terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, and a wearable device (e.g., a smartwatch, a smart glass, a head-mounted display).

[0063] In addition, the input unit (110) in the present invention does not necessarily mean a hardware means, but can be understood as a channel for receiving input from a user.

[0064] The input unit (110) may also be referred to as a user interface module. The input unit (110) may include a touch screen, a computer mouse, a keyboard, a keypad, a touchpad, a trackball, a joystick, a voice recognition module, or other similar devices. However, the present invention does not limit the type of input unit (110).

[0065] Here, user input may include documents, text, images (or videos), voice, etc. In this case, the data normalization system (100) may further include a module that converts voice into text.

[0066] Next, the output unit (120) can output information through an output unit configuration (e.g., a display unit, a touch screen, a speaker, etc.) provided in a user terminal (10) linked to the data normalization system (100) according to the present invention. For example, the output unit (120) can output a page (or service page) linked to the data normalization system (100) according to the present invention to the display unit of the user terminal. In addition, the output unit (120) does not necessarily mean a hardware means, but can be understood as a channel for outputting results to a user.

[0067] Next, the communication unit (130) may be connected via a wireless or wired network to a user terminal, a server (e.g., a central server, an external server, etc.), a device, and at least one network, etc., to receive or transmit overall data and information necessary for the operation of the data normalization system (100) according to the present invention.

[0068] The communication unit (130) can support various communication methods depending on the communication standard of the communicating device.

[0069] For example, the communication unit (130) may be configured to communicate with a communication target using at least one of the following technologies: WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth (Bluetooth™ RFID (Radio Frequency Identification), Infrared Communication (Infrared Data Association; IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus).

[0070] Next, the storage unit (140, or memory) serves to store various data related to the present invention and may include one or more non-transient computer-readable storage media that can be read and / or accessed by at least one of one or more processors.

[0071] One or more computer-readable storage media may include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage devices. In some examples, the storage unit (140) may be implemented using a single physical device (e.g., one optical, magnetic, organic, or other memory or disk storage device), whereas in other examples, the storage unit (140) may be implemented using two or more physical devices.

[0072] The storage unit (140) may include computer-readable instructions and additional data. The storage unit (140) may include a storage necessary to perform at least some of the methods, scenarios, and techniques described herein and / or at least some of the functions of the device and network.

[0073] Furthermore, at least a portion of the storage unit (140) may be a cloud storage or a cloud server. At least a portion of the data corresponding to user input received from the input unit (110) (e.g., table data (200)) and training data may be stored in the storage unit (140).

[0074] That is, the storage unit (140) is sufficient as a space where information necessary for the operation of the data normalization system (100) according to the present invention is stored, and it can be understood that there are no restrictions on the physical space.

[0075] Furthermore, the storage unit (140) may store a computer program including computer program instructions. Furthermore, the storage unit (140) may store a computer program including computer program instructions that control the operation of the system (100) or control the operation of the control unit (180) when loaded into the processor of the system (100).

[0076] Next, the judgment unit (150) may be configured to perform the role of judging (or determining) the processing method of the table data (200). The judgment unit (150) may determine the processing method of the table data (200) according to a pre-set standard (e.g., processing based on columns or processing based on rows, etc.) and / or the structure of the table data (200) (e.g., whether it is structured around rows or structured around columns, etc.).

[0077] In one embodiment, the judgment unit (150) is included in a part of the artificial neural network (170), and when table data (200) is input to the artificial neural network (170), it can determine the processing method of the table data (200) according to a pre-set standard and / or the structure of the table data (200). However, it is obvious that in the present invention, the determination of the processing method of the table data (200) may be determined by the artificial neural network (170) itself, which is trained with a large amount of table data.

[0078] Next, the data processing unit (160) can perform preprocessing on the table data (200). The data processing unit (160) can perform normalization on the table data (200) in order to train an artificial neural network (170) using the table data (200). In this case, the data processing unit (160) can perform normalization on the table data (200) based on the processing method of the table data (200) determined by the judgment unit (150).

[0079] In one embodiment, let us assume that the processing method for the table data (200) determined by the judgment unit (150) is determined to be centered on “features (or columns).” The data processing unit (160) may apply a min-max transformation to each of the plurality of features included in the table data (200), and apply a normalization function to each of the plurality of features to which the min-max transformation has been applied to obtain a plurality of normalized features. More specific details regarding this will be described later.

[0080] Next, the artificial neural network (170) is a neural network having multiple hidden layers (or hidden layers, hidden layers, etc.) and can be constructed to solve complex non-linear problems. Each layer of the artificial neural network (170) learns a high-level representation of data, and such layers may include at least one of an input layer (or input layer), multiple hidden layers, and an output layer (or output layer). Additionally, each layer of the artificial neural network (170) is composed of neurons or nodes, which exist connected to each other through weights and activation functions. That is, the artificial neural network is configured to include multiple hidden layers between the input layer and the output layer, and can model complex non-linear relationships.

[0081] In the present invention, the term artificial neural network may also be referred to as an “artificial intelligence model,” a “deep neural network (DNN),” a “neural network,” or a “neural network.”

[0082] Next, the control unit (180) can perform the role of controlling the overall operation of the data normalization system (100) related to the present invention. The control unit (180) can process signals, data, information, etc. that are input or output through the components of the data normalization system (100) described above, or perform a series of data processing to provide or process appropriate information and functions to the user. The control unit (180) can be physically implemented by the processor described above.

[0083] Meanwhile, the present invention is intended to provide a data normalization method and system for learning tabular data that enables an artificial neural network to understand tabular data well. More specifically, the present invention is intended to provide a data normalization method and system for learning tabular data that can improve the learning efficiency of an artificial neural network for tabular data by performing normalization on the tabular data. Below, we will examine in more detail a method for training an artificial neural network (170) through efficient normalization of tabular data.

[0084] Meanwhile, as illustrated in FIG. 2a, the data normalization method for learning tabular data according to the present invention can improve the learning efficiency and performance of an artificial neural network (170) for tabular data by: receiving tabular data configured to include a plurality of different features and a plurality of samples having values ​​corresponding to each of the plurality of features (S210); obtaining a plurality of normalized features by applying a normalization function parameterized by a curve to each of the plurality of features included in the tabular data (S220); learning an artificial neural network using each of the normalized features (S230); and optimizing the normalization function based on the learning of the artificial neural network (S240).

[0085] Table data can be structured to include rows (or records, tuples, samples, etc.) and columns (or features, columns, fields, characteristics, attributes, variables, predictors, etc.). For example, a table refers to a tabular data structure composed of rows and columns, and it can be a unit for storing and managing data in a relational database. Tables can be used to systematically organize and manage data, such as Excel sheets or database tables. Here, a column refers to a set of data arranged vertically in a table, and each column can represent a specific attribute (or field) of the table. Additionally, a row refers to a single line of data arranged horizontally in a table, and each row can represent a single record or tuple. In other words, each row represents a single record in the table, and each column can represent a specific attribute of that record.

[0086] The features included in such tabular data are heterogeneous, requiring different normalization for each input feature. However, conventional normalization techniques are often specialized for specific input feature distributions, and more importantly, they are not optimized for neural network learning.

[0087] To this end, the present invention proposes a B-spline-based learnable input normalization method (or technique). In the present invention, transformations are parameterized into curves and can be optimized together with the learning of a neural network. Since input normalization is applied individually to each feature, this can be viewed as a curve fitting problem of a specific dimension (e.g., 1-dimensional). More specifically, the present invention utilizes B-splines to model these transformations.

[0088] The B-spline used in the present invention is a mathematical method (or technique) for defining a smooth and continuous curve for a given input value. The B-spline defines a smooth and continuous curve using multiple control points, and generates the entire curve by using different polynomials for each segment and connecting them. The B-spline is defined by an order (n+1), a knot vector (T), and control points (P). The B-spline curve is expressed as a weighted sum of control points, where the weights are determined by basis functions defined by the order and the knot vector. In general, a B-spline corresponding to an order of n+1 can approximate any n-times differentiable curve by adjusting the knot vector and control points. Furthermore, since the B-spline can control both the start and end points of the curve, it can be ideal for input normalization, which maps arbitrary inputs (values ​​with varying ranges) to a predefined range.

[0089] In this regard, the present invention allows for the adaptive modification of a normalization function based on the importance of input data (or input values, samples, features, etc.) to efficiently utilize an artificial neural network in table data. Since each component of the input value in table data requires a different normalization function, the importance of the normalization function is higher compared to artificial neural network training in general data. To achieve efficient normalization in such table data, the present invention identifies the importance of input values ​​during the training process of the artificial neural network and adaptively modifies the normalization function accordingly, thereby enabling the artificial neural network to efficiently allocate capacity to more important input values ​​for training.

[0090] The normalization function (or input normalization function) used in the present invention is a transformation function applied to map raw data (raw data, or raw input data, original table data, etc.) to a predefined range (e.g., [0, 1]), and can be defined in a form that is individually learnable by considering different scales and distributions for each input feature. More specifically, in the present invention, a B-spline (or B-spline curve) may be used to define the normalization function, and the normalization function may be defined as a curve function parameterized based on the B-spline (i.e., the normalization function may be modeled as a B-spline (or B-spline) curve). In the present invention, a learnable normalization function parameterized as a curve (e.g., a B-spline curve) can be defined (or set, implemented, etc.) using a B-spline. Such a regularization function may also be referred to as a “B-spline-based regularization function,” a “B-spline-based learnable regularization function,” a “regularization function modeled with B-splines,” a “parameterized regularization function modeled with B-splines,” or a “learnable regularization function modeled with B-splines.”

[0091] Meanwhile, as illustrated in FIG. 3a, the control unit (150) may receive (or acquire) at least one (or multiple) table data (300) from a user terminal (10) or from the outside. For example, the control unit (150) may receive the table data (300) directly from the outside, or may extract the table data (300) from document data (e.g., papers, books, patent documents, reports, etc.) received from the outside. The control unit (150) may extract table data (300) structured in a table form from the input document data based on a machine learning model and / or a document understanding model included in the data augmentation system (100).

[0092] Such table data (300) may be configured to include rows (or records, tuples, samples, etc.) and columns (or features, columns, fields, features, characteristics, attributes, variables, predictors, etc.). For example, a table refers to a table-shaped data structure composed of rows and columns, and may be a unit for storing and managing data in a relational database. A table may be used to systematically organize and manage data, such as an Excel sheet or a database table. Here, a column refers to a set of data arranged vertically in a table, and each column may represent a specific attribute (or field) of the table. Additionally, a row refers to a line of data arranged horizontally in a table, and each row may represent a single record or tuple in the table. That is, each row represents a single record of the table, and each column may represent a specific attribute of that record.

[0093] Based on the receipt of table data (300), the control unit (180) can determine the processing method of the table data (300) using the judgment unit (150). The judgment unit (150) can determine the processing method of the table data (300) according to a pre-set standard (e.g., processing based on columns or processing based on rows, etc.) and / or the structure of the table data (300) (e.g., whether it is structured around rows or structured around columns, etc.).

[0094] In one embodiment, the judgment unit (150) can determine a processing method for the table data (300) based on the analysis result of the table data (300), that the structure of the table data (300) is structured around columns, so that the table data (300) is processed based on (or centered on) the features (columns).

[0095] In another embodiment, let us assume that a preset standard related to the processing method of table data is set to process based on columns. In this case, the judgment unit (150) can determine the processing method of table data (300) based on the preset standard so that the table data (300) is processed based on features.

[0096] Once the processing method of the table data (300) is determined, the control unit (180) can perform preprocessing on the table data (300) using the data processing unit (160). The data processing unit (160) can perform normalization on the table data (300) in order to input the table data (300) into the artificial neural network (170). This can also be understood as an input normalization process that converts the original table data (300) into a form suitable for learning by the artificial neural network (170).

[0097] In the present invention, input normalization can be defined as a transformation function that maps raw input values ​​(i.e., input values ​​included in original table data (300)) to a preset normalized range. In the present invention, such a transformation function is defined as a parameterized B-spline, and the B-spline can be defined by an order (n+1), a knot vector (T), and a control point (P).

[0098] Here, the degree can be a parameter that determines the smoothness and shape of a B-spline curve. This degree is defined in the form n+1, where n represents how many times the curve is differentiable. For example, the higher the degree, the smoother the curve becomes and the more complex it can represent. In this case, the degree can determine how smoothly the normalization function (or transformation function) should change when normalizing numerical features included in tabular data.

[0099] Additionally, a knot vector can be a set of boundary values ​​that divide a curve into multiple intervals. The knot vector values ​​must be non-decreasing, which allows the continuity and smoothness of the curve to be maintained. For example, if the position of the knot changes, the range affected by the curve changes, and the knot vector can determine which polynomial defines a B-spline curve in which interval. In this case, the knot vector can determine where the curve changes by dividing (or splitting) the range of data values ​​(such as feature values ​​or sample values) included in the tabular data into fixed intervals. In other words, the knot vector can serve as a reference point where the normalization function changes (e.g., if the distribution of a feature value is 0–100, setting the knot vector to [0, 20, 50, 100] allows different transformations (curve shapes) to be applied in each interval).

[0100] Furthermore, control points can be a set of points that directly determine the shape of a curve. A B-spline curve is defined as a weighted sum of control points, where each weight can be a value of a basis function calculated by the knot vector and the degree. For example, control points do not directly pass through the curve but can define its overall shape. As the number of control points increases, the curve can express various forms more flexibly. In this case, control points can determine the actual location where data values ​​(input values) contained in the table data are mapped to normalized output values. In other words, they can determine how much to compress or increase the data around specific values ​​contained in the table data.

[0101] And, the output (i.e., curve) of the B-spline is calculated as a weighted sum of control points, where the weights can be determined by the knot vector and the degree of the B-spline. A B-spline with a degree of “(n+1)” can be represented as in Fig. 6(a). Here, the i-th element (or component) of the knot vector (T) can be represented as in Fig. 6(b), and the i-th element (or component) of the control point (P) can be represented as in Fig. 6(c). Likewise, the derivative of the B-spline, which is important for understanding how the transformation is adjusted across the input range, can be represented as in Fig. 6(d).

[0102] Furthermore, in the present invention, the order (n+1) is fixed as a hyperparameter, and the knot vector (T) and control point (P) can be learned. The knot vector must be non-decreasing according to the definition of a B-spline. To ensure the continuity and smoothness of the B-spline, each knot vector value must be greater than or equal to the preceding consecutive value. This ensures that the input normalization function maintains a one-to-one relationship, so that each unique input is mapped to a unique output without reversing the order of the input values. These constraints are achieved through a combination of softmax and cumulative sum functions, thereby ensuring that the knot vector and control point remain non-decreasing while maintaining learnable differentiability. This can be represented as shown in FIG. 7 (a) and (b).

[0103] Again, the data processing unit (160) can apply a min-max transformation to each of the plurality of features (310, 320, 330, 340, 350, 360, 370, 380, 390) included in the table data (300) based on the processing method of the table data (300) determined by the judgment unit (150) (see S301 in FIG. 2b). For example, since (a) and (b) of FIG. 7 restrict the knot vector to a value between 0 and 1, the data processing unit (160) can apply a minimum-maximum transformation to each of the multiple features (310, 320, 330, 340, 350, 360, 370, 380, 390) included in the table data (300) (or per multiple features) before passing the table data (300) through the B-spline.

[0104] In this case, the process of applying a min-max transformation to the table data (300) is such that for each of the multiple features (310, 320, 330, 340, 350, 360, 370, 380, 390) included in the original table data (300), the value of each of the multiple samples (311, 312, 313, 321, 322, 323, 331, 332, 333, 341, 342, 343, 351, 352, 353, 361, 362, 363, 371, 372, 373, 381, 382, ​​383, 391, 392, 393, etc.) corresponding to each of the multiple features (310, 320, 330, 340, 350, 360, 370, 380, 380, 390) satisfies a preset range (e.g., a value between 0 and 1), for each of the multiple features It can also be understood as the process of applying a min-max transformation. This min-max transformation can be understood as performing scaling on each of the multiple features included in the table data to linearly transform the data values ​​contained in the table data into a pre-set range (i.e., through scaling, the shape of the data distribution is preserved while only the range is adjusted).

[0105] And, as illustrated in FIG. 2b, the data processing unit (160) can obtain (or calculate, collect, etc.) a plurality of normalized features by applying a normalization function to each of the plurality of features to which a min-max transformation has been applied (S303). In this case, the process of obtaining the plurality of features S303 can also be understood as a process of obtaining a plurality of samples containing values ​​corresponding to each of the plurality of normalized features. Here, applying a normalization function can also be understood as performing normalization of table data by feature (column) or row using a B-spline curve. Such a B-spline curve has the characteristic that all curves can be approximated by changing parameters.

[0106] In addition, normalization functions can be generated for each of the multiple features included in the table data. For example, if the number of multiple features is “10,” the number of normalization functions can be generated in a number corresponding to the number of multiple features (e.g., 10). This can also be understood as B-spline curves being generated for each of the multiple features in the table data (i.e., if there are 10 features, there are also 10 B-spline curves).

[0107] Furthermore, in the present invention, the final input normalization function (i.e., the normalization function parameterized by a curve) can be represented as shown in (c) of FIG. 7. Here, M(x) represents the min-max transformation applied to the original table data (300) (see FIG. 7 (c)), and the parameters of the normalization function (see FIG. 7 (e)) can be learned through backpropagation. In this case, the parameters of the normalization function may include at least one of a first parameter and a second parameter. At this time, the first parameter can be understood as a knot vector of a B-spline, and the second parameter can be understood as a control point of a B-spline. In the present invention, the first parameter (knot vector (T)) and the second parameter (control point (P)) are each set as learnable parameters so that they can be optimized (or updated) during the learning process of the artificial neural network (170).

[0108] However, in the embodiment described above, the process of performing normalization based on features included in the table data (or by feature unit) was explained; however, the method of processing table data in the present invention is not necessarily limited to this, and it is also possible to perform normalization based on rows (or by row unit). The method of processing table data in the present invention may vary depending on the structure (or configuration) of the table data. More specifically, in the present invention, table data can be processed differently depending on whether the table data is configured based on features (i.e., columns) or rows (or samples). For example, in the present invention, analyzing the structure of the table data and determining the method of processing the table data (e.g., whether to process by column unit or by row unit, etc.) based on the analyzed result can be performed by the judgment unit (150) described above.

[0109] As previously discussed, in the case of table data organized based on (or centered on) features, attributes exist included within the features, and data (or data values) corresponding to the features may be included in the samples (see Fig. 3a).

[0110] On the other hand, table data organized based on (or centered on) rows has the opposite structure, where attributes can be arranged on a row-by-row basis (see FIG. 3b). In this case, the data normalization system (100) according to the present invention can perform normalization based on (or row-by-row) the rows included in the table data.

[0111] In one embodiment, as illustrated in FIG. 3b, the judgment unit (150) can determine the processing method of the table data (300) such that the table data (300) is processed based on (or centered on) the rows, based on the structural analysis result of the table data (400) and the fact that the structure of the table data (400) is structured around the rows.

[0112] In another embodiment, let us assume that a preset standard related to the processing method of table data is set to process based on rows. In this case, the judgment unit (150) can determine the processing method of table data (400) so that the table data (400) is processed based on rows based on the preset standard.

[0113] The data processing unit (160) can apply a min-max transformation to each of the multiple rows (411, 412, 413, 421, 422, 423, 431, 432, 433, 441, 442, 443, 451, 452, 453, 461, 462, 463, 471, 472, 473) included in the table data (400) based on the processing method of the table data (400) determined by the judgment unit (150). That is, the data processing unit (160) can apply a min-max conversion to each of the multiple rows (411, 412, 413, 421, 422, 423, 431, 432, 433, 441, 442, 443, 451, 452, 453, 461, 462, 463, 471, 472, 473) included in the table data (400) before passing the table data (400) through the B-spline (or per multiple rows).

[0114] Furthermore, the data processing unit (160) can obtain (or calculate, collect, etc.) multiple normalized rows by applying a normalization function to each of the multiple rows to which a min-max transformation has been applied.

[0115] In this way, the present invention can perform normalization on a column-by-column or row-by-row basis having the same attribute (or characteristic, meaning, etc.) so that the characteristics of the table data are preserved. As an example, the normalization process in the present invention may be a process of processing a column or row having a value of 100 to have a value of 1.0 or 0.99, assuming that there are columns or rows having values ​​from 1 to 100 in the table data.

[0116] Meanwhile, in the present invention, during the learning process of the artificial neural network (170), a loss function defined to make learning easier in input sections with high learning difficulty while compressing already learned sections can be used.

[0117] Here, the input interval may refer to the range of original values ​​(the interval from the minimum value to the maximum value) that each of the multiple features included in the table data can have. This input interval becomes the target for mapping to a pre-set range (e.g., [0, 1]) during the normalization process, and the learning performance and stability of the neural network may vary depending on how a specific input interval is distributed and / or transformed. For example, the input interval may refer to the range of values ​​in which the original table data is distributed before the min-max transformation. In the present invention, this input interval is mapped to a normalization function (or a B-spline-based normalization function) to perform normalization according to the learning difficulty of each data point (e.g., adaptively redistributing by spreading out difficult input intervals more widely for normalization and compressing easy input intervals more narrowly for normalization). That is, the input interval refers to the range of values ​​in which the original input values ​​are distributed, and may refer to the range that serves as the basic unit of the normalization transformation.

[0118] Specifically, as discussed above, input normalization may be a process of distributing a normalized input of a pre-set range across the raw input (e.g., original table data). In one embodiment, for data points (or samples) that are difficult to learn, if a wider range is allocated in the normalized input, the artificial neural network (170) can utilize more capacity for these data points, making learning easier. Conversely, a smaller range is allocated to data points that are easy to learn. This can be achieved by increasing the derivative of the curve for data points with large loss values. As a result, a small interval of the raw input corresponds to a larger interval of the normalized input. Conversely, for data points with small loss values, the derivative of the curve is decreased so that a larger interval of the raw input maps to a smaller interval of the normalized input.

[0119] That is, in the present invention, by utilizing a loss function that optimizes (or updates, adjusts, etc.) the parameters of a normalization function (i.e., the parameters of a B-spline) according to the difficulty of each sample included in the table data in the learning process of an artificial neural network (170), the artificial neural network (170) can allocate more capacity to samples with high learning difficulty and can guarantee an optimized transformation for both samples with high learning difficulty and samples with low learning difficulty. More specific details regarding this will be explained below.

[0120] Meanwhile, the control unit (180) can perform training of the artificial neural network (170) using each of the normalized features. At this time, the control unit (180) can perform optimization of the normalization function based on the progress of training of the artificial neural network (170). This may mean optimizing at least one parameter of the normalization function simultaneously with the training of the artificial neural network (170) (or while the training of the artificial neural network (170) is in progress).

[0121] To this end, the control unit (180) first processes a plurality of normalized features as inputs to the artificial neural network (170) and can obtain an output value (or predicted value) for each of the plurality of features from the artificial neural network (170).

[0122] For example, the output value may include a prediction result in which a plurality of normalized input features each pass through a plurality of computational layers of the artificial neural network (170). The output value represents a prediction result (or a result of a functional mapping learned by the artificial neural network (170) based on the correlation between the plurality of normalized features and learned weights) calculated by the artificial neural network (170), and the form of the output value may differ depending on the purpose of the problem or the type of task (e.g., classification, regression, prediction, etc.). In the case of a regression problem (or task), the output value may be represented as a continuous numerical value. Also, in the case of a classification problem, the output value may be represented as a probability value of belonging to each class or a corresponding score. Additionally, in a specific task (e.g., time series prediction, image restoration, anomaly detection, etc.), the output value may be in the form of at least one of a sequence, an image, or a specific score depending on the problem definition. That is, the output value represents the prediction result calculated by the artificial neural network (170) through learning for each of the multiple normalized features, and can take various forms depending on the type of task. This output value is used in the calculation of the loss function by comparing it with the learning target value (ground truth) and serves as a criterion for optimizing parameters during the learning process of the artificial neural network (170). Therefore, the output value is the result of the neural network's interpretation of the input data and can be utilized as a key indicator for evaluating and improving the performance of the neural network during the learning and inference process.

[0123] And, as illustrated in FIG. 2b, the control unit (180) can calculate the learning loss (or loss value) between the output value of a plurality of normalized features and the learning target value (target value) corresponding to the table data using a preset loss function (S305), and can proceed with the learning of the artificial neural network (170) so that the calculated learning loss becomes smaller (S307).

[0124] In this regard, the pre-configured loss function according to the present invention may be defined (or configured) to consider the learning difficulty in different input intervals. For example, referring to FIGS. 7(f) and (g), the derivative may indicate how much the normalized input increases (or expands, becomes larger, etc.) or is compressed compared to the raw input. By increasing the derivative for data points with high (or large) loss values, the neural network can support efficient learning by allocating more capacity to samples with high learning difficulty (i.e., making learning for difficult samples easier). Here, the neural network may be represented as in FIG. 7(h), the original input (or raw input) may be represented as in FIG. 8(a), and the learning target value corresponding to the original input may be represented as in FIG. 8(b). In this case, the numerator in FIG. 7(g) may represent the learning loss for the original input (i.e., the original table data) (see FIG. 8(c)). This formula allows the parameters of the B-spline (i.e., the normalization function) to be adjusted (see (d) in Fig. 8), thereby allowing the neural network to allocate more capacity to data points with high learning difficulty, which can improve learning efficiency.

[0125] In this case, in order to prevent the loss function from interfering with the parameter learning of the artificial neural network (170) during the learning process of the artificial neural network (170), the present invention utilizes a specific pre-set function (e.g., the .detach() function of PyTorch) to ensure that the loss function does not interfere with the learning process of the artificial neural network (170). That is, the pre-set loss function according to the present invention can be defined so that the learning process of the artificial neural network (170) and the parameter optimization process of the regularization function are performed (or proceed) separately. Such a pre-set loss function can be represented as shown in (e) and (f) of FIG. 8.

[0126] In one embodiment, the control unit (180) can calculate a learning loss between a learning target value corresponding to table data and an output value for each of a plurality of normalized features using a preset loss function, and proceed with training of the artificial neural network (170) so that the calculated learning loss becomes smaller. In this case, the process of calculating the learning loss may be a step of calculating a learning loss between an output value for a plurality of samples including a learning target value and a value corresponding to each of a plurality of normalized features using a preset loss function. Furthermore, training the artificial neural network (170) so that the calculated learning loss becomes smaller may mean that the parameters (e.g., weights, biases, etc.) of the artificial neural network (170) are repeatedly updated in a direction that minimizes the loss function representing the error between the output value output by the artificial neural network (170) and the actual correct answer.

[0127] At this time, while the artificial neural network (170) is being trained, the control unit (180) can optimize the normalization function based on the calculated learning loss. The control unit (180) can optimize the parameters of the normalization function simultaneously with the training of the artificial neural network (170) according to the importance specified for each of the plurality of samples (see S309 in FIG. 2b).

[0128] In the present invention, importance can be determined based on the learning loss calculated through a loss function. More specifically, as illustrated in FIG. 2c, the control unit (180) analyzes the learning difficulty of each of the plurality of samples based on the learning loss calculated through a loss function to determine the importance of each of the plurality of samples (S311), and determines the importance of each of the plurality of samples based on the analyzed learning difficulty (S313).

[0129] Here, the learning difficulty may include various difficulty levels. The learning difficulty may include at least one of a first learning difficulty related to the calculated learning loss satisfying a first criterion and a second learning difficulty related to the calculated learning loss satisfying a second criterion. In this case, the first criterion means that the calculated learning loss (or loss value) is high (i.e., means that the degree of loss is high), and the second criterion may mean that the calculated learning loss (or loss value) is low (i.e., means that the degree of loss is low).

[0130] In this case, for a sample where the calculated learning loss satisfies the first criterion, the learning difficulty of that sample can be specified as having the first learning difficulty. Alternatively, for a sample where the calculated learning loss satisfies the second criterion, the learning difficulty of that sample can be specified as having the second learning difficulty.

[0131] Specifically, based on the results of the learning difficulty analysis for each of the plurality of samples, the control unit (180) can determine the importance of each of the plurality of samples based on the analyzed learning difficulty. Based on the learning loss calculated for each of the plurality of samples, the control unit (180) can determine at least some (or at least one) first samples satisfying the first learning difficulty and at least some (or at least one) second samples satisfying the second learning difficulty among the plurality of samples.

[0132] That is, in order to adjust the gradient of the normalization function according to the learning difficulty of each of the multiple samples, the control unit (180) can specify the samples among the multiple samples that satisfy the first learning difficulty as the first samples and the samples among the multiple samples that satisfy the second learning difficulty as the second samples, based on the learning loss calculated for each of the multiple samples.

[0133] Accordingly, the control unit (180) can specify that the importance of the first sample satisfying the first learning difficulty is the first importance. Additionally, the control unit (180) can specify that the importance of the second sample satisfying the second learning difficulty is the second importance.

[0134] Furthermore, as illustrated in FIG. 2c, the control unit (180) can adjust the parameters of the normalization function using a loss function so that the magnitude of the slope of the normalization function increases or decreases according to the importance of each of the specified plurality of samples (S315).

[0135] In the present invention, optimizing the normalization function can also be understood as adjusting (or updating, optimizing, etc.) the slope of the normalization function according to the importance of each of the plurality of samples.

[0136] Alternatively, it can be understood as a process of adjusting the parameters of the regularization function according to the learning difficulty of each of the multiple samples using a loss function. As discussed above, the loss function can be defined to adjust the gradient of the regularization function according to the importance of each of the multiple samples.

[0137] The process of training the artificial neural network (170) and optimizing the regularization function using such loss functions is such that the first loss (e.g., training loss (or base loss)) is used to train only the artificial neural network (170), and the second loss (spline loss) is used to update only the parameters of the regularization function (B-spline). In this case, the first loss is a loss to increase the prediction accuracy of the artificial neural network (170), and the second loss may be an auxiliary loss to optimize the regularization function (B-spline curve) according to the difficulty per sample. This separate learning may mean that different optimization algorithms can be used to independently update the parameters of the artificial neural network (170) and the parameters of the regularization function. For example, for the parameters of the regularization function, a stable update of the regularization function is performed using an optimization algorithm (e.g., Adam) with a fixed learning rate of 0.01. On the other hand, the parameters of the artificial neural network (170) can be optimized using different appropriate algorithms depending on the task.

[0138] The control unit (180) can adjust the magnitude of the slope of the normalization function so that it increases in the case of the first sample having a first importance among the plurality of samples. More specifically, the control unit (180) can adjust the parameters of the normalization function using a loss function so that the magnitude of the slope of the normalization function increases in the case of the first sample having a first importance among the plurality of samples. Here, increasing the magnitude of the slope may mean increasing the derivative value of the normalization function (or normalization curve, B-spline curve, B-spline normalization curve, transformation curve, etc.) defined in the input interval, thereby causing a small change in the original input value (original table data) to be reflected as a large change in the normalized output value (multiple normalized features). In this case, a wider normalization interval is allocated to the first sample, which has a relatively high learning difficulty, so that the artificial neural network (170) can intensively utilize the capacity to distinguish and learn the first sample more finely.

[0139] Additionally, the control unit (180) can adjust the magnitude of the slope of the normalization function so that it is reduced in the case of a second sample having a second importance among multiple samples. More specifically, the control unit (180) can adjust the parameters of the normalization function using a loss function so that the magnitude of the slope of the normalization function is reduced in the case of a second sample having a second importance among multiple samples. Here, reducing the magnitude of the slope may mean making the derivative value of the normalization function (or normalization curve, B-spline curve, transformation curve, etc.) defined in the input range smaller so that a large change in the original input value is reflected as a small change in the normalized output value. In this case, the second sample, which has a relatively low learning difficulty, is mapped to a narrow normalization range, and the artificial neural network (170) can allocate more resources to an important area (an area including a first sample with high difficulty) without unnecessarily consuming capacity.

[0140] That is, as seen above, the process of optimizing the normalization function in the present invention may be a process of updating the parameters of the normalization function based on multiple sample-specific learning losses.

[0141] In this regard, FIGS. 4 and 5 illustrate the evolution of the learned B-spline transform (i.e., the regularization function) and the learning loss corresponding to each sample throughout the learning process. FIG. 4 visualizes the learned B-spline transform, and FIG. 5 illustrates the learning loss of individual samples. FIG. 4(a) and FIG. 5(a) represent the initial stage of learning, at which point the B-spline takes on a form very similar to a min-max transform. This shows the result of uniformly high learning loss occurring in most samples because the artificial neural network (170) could not distinguish based on the difficulty of the samples.

[0142] As the artificial neural network (170) progresses in learning (see FIG. 4 (b) and (c) and FIG. 5 (b) and (c)), the artificial neural network (170) begins to distinguish between second samples with low learning difficulty and first samples with high learning difficulty. For first samples with high learning loss, the B-spline allocates a wider input range to effectively store gradients, thereby providing more capacity to the first samples. On the other hand, second samples exhibiting lower learning loss are compressed into smaller input ranges. This adaptive learning behavior causes the B-spline transform to prioritize the most difficult regions in the input space, thereby improving performance by allowing the artificial neural network (170) to dynamically allocate resources where needed. For example, first samples with high learning loss are concentrated approximately around raw input values ​​of 0.15, 0.4, and 0.8. Correspondingly, referring to FIG. 4, the B-spline curve exhibits steeper gradients in these regions. This demonstrates that the loss function and the learnable regularization function according to the present invention are operating with optimal results, and may mean that the artificial neural network (170) is allocating more capacity to samples with high learning difficulty by increasing the gradient of the corresponding region. This operation demonstrates the effectiveness of dynamically adjusting regularization according to the difficulty of each of the multiple samples.

[0143] That is, in the present invention, since the part (or region, interval, etc.) with high loss is a part that the artificial neural network (170) finds difficult to learn, the part is enlarged so that the artificial neural network (170) can learn intensively. As shown in FIGS. 4 and 5, as the artificial neural network (170) learns, a result is produced regarding the degree of loss for each learning data (table data) used for learning, and in the present invention, a B-spline is represented according to the loss. That is, a B-spline curve (or B-spline normalization curve) is represented according to the learning data, and the B-spline curve may also be changed as learning progresses. At this time, the change of the B-spline curve can also be interpreted as meaning that the normalization function is changed (or optimized).

[0144] Furthermore, when the parameters of the normalization function are updated, the values ​​of each of the multiple normalized features may be updated together. For example, the control unit (180) may update multiple samples containing values ​​corresponding to each of the multiple normalized features based on the update of the normalization function. That is, the values ​​of the data being normalized, such as columns or rows, may also be updated according to the B-spline curve.

[0145] Thus, in the present invention, for data samples with a large learning loss during the learning process of the artificial neural network (170), the slope of the input normalization curve (i.e., B-spline curve) is significantly adjusted so that the corresponding data interval is represented more widely in the normalization space, thereby increasing learning efficiency. Conversely, for data samples with a small learning loss, they are placed in a relatively narrow interval so that the use of unnecessary learning resources can be reduced (i.e., the waste of learning resources is minimized).

[0146] That is, in the present invention, by simultaneously optimizing the parameters of the artificial neural network (170) and the parameters of the normalization function, the overall learning performance and safety can be improved. For example, FIG. 9 presents a comparison of the performance of different input normalization techniques in various neural network architectures. The results assume that the most effective normalization technique varies depending on the structure of the neural network, and it can be confirmed that the artificial neural network (170) trained with the normalization method according to the present invention exhibits the highest performance.

[0147] Meanwhile, in addition to the loss function and proposed method discussed above, there are important hyperparameters for learnable regularization. For example, the spline degree can be set to “n=2” so that the transformation is differentiable in all regions. Additionally, the number of control points can be set to “16”. Furthermore, since the knot vector generated through Fig. 7(a) cannot represent values ​​of 0 and 1, this implies that the minimum and maximum values ​​of the raw input are not directly represented. To address this, the present invention applies a margin to the min-max transformation so that the original data after transformation falls within a pre-set range (see Fig. 8(g)) (in this case, the margin can be set to 0.1). Finally, to handle values ​​that fall outside the range in the test set when the min-max transformation is applied to the training data, linear extrapolation is applied to the B-spline for values ​​exceeding the [0.1] range. Through this, the neural network can process inputs that exceed the learned range in a linear manner. Computer code reflecting this content can be implemented as shown in the example below (see Table 1).

[0148] def bspline(p: int,X: Tensor,k: Tensor,t: Tensor,eps: float = 1e-6,) -> Tensor:"""Computes the coordinates of points on multiple B-spline curves using PyTorch.Parameters:p (int): The degree of the B-spline.X (Tensor): Control points, a 2D tensor of shape [L, N], where L is the number of B-spline curves, and N is the number of control points per curve.k (Tensor): Knot vectors, a 2D tensor of shape [L, N - p + 1]. The function clamps each curve by duplicating the first and last knot values p times.t (Tensor): Parameter values, a 2D tensor of shape [B, L], where L is the number of B-spline curves, and B is the number of parameter values to evaluate per curve.Returns:Tensor: Coordinates of the points on the B-spline curves at the given parameter values, a tensor of shape [B, L]."""# Input validationassert p >= 1, f"B-spline degree p must be at least 1, but got p={p}"assert X.ndim == 2, f"X must be a 2D tensor [L, N], but got shape {X.shape}"assert k.ndim == 2, f"k must be a 2D tensor [L, N - p + 1], but got shape {k.shape}"assert k.shape[1] == X.shape[1] - p + 1assert (k == k.sort().values).all(), "Knot vectors must be sorted in non-decreasing order"assert t.ndim == 2, f"t must be a 2D tensor [B, L], but got shape {t.shape}"assert (X.shape[0] == k.shape[0] == t.shape[1]), f"Mismatch in num curves: X.shape[0]={X.shape[0]}, k.shape[0]={k.shape[0]}, t.shape[1]={t.shape[1]}"X = X.unsqueeze(2) # Shape [L, N, 1]t = t.T.unsqueeze(2) # Transpose and shape [L, B, 1] for easier batch processing# Backup original t values and clamp them to the range [0, 1 - eps]t_unclamp = torch.clone(t)t = torch.clamp(t, min=0, max=1 - eps)# Clamp the knot vectors by duplicating the first and last values `p` timesk = torch.cat([k[:, 0].unsqueeze(1).expand(-1, p), # Clamp startk,k[:, -1].unsqueeze(1).expand(-1, p), # Clamp end],dim=1,)k= k.unsqueeze(1) # Reshape to [L, 1, N + 2p]# Initialize basis function and derivative function valuesb = X.new_zeros((X.shape[0], t.shape[1], X.shape[1] + p))d = X.new_zeros((X.shape[0], t.shape[1], X.shape[1] + p))# Basis functions for degree 0for i in range(X.shape[1] + p):b[..., i] = (k[..., i] <= t[..., 0]) & (t[..., 0] < k[..., i + 1])# Iteratively compute basis functions and derivatives for degrees 1 to pfor i in range(1, p + 1):left_denom = k[..., i:-1] - k[..., : -i - 1] # Denominator for the left termright_denom = k[..., 1 + i :] - k[..., 1:-i] # Denominator for the right term# Avoid NaNs by setting zero denominators to 1left_denom[left_denom == 0] = 1right_denom[right_denom == 0] = 1# Recurrence relation for derivativesd_term1 = (t - k[..., : -i - 1]) / left_denom * d[..., :-1]d_term2 = (k[..., i + 1 :] - t) / right_denom * d[..., 1:]d_term3 = 1 / left_denom * b[..., :-1]d_term4 = 1 / right_denom * b[..., 1:]d = d_term1 + d_term2 + d_term3 - d_term4# Recurrence relation for B-spline basis functionsb_term1 = (t - k[..., : -i - 1]) / left_denom * b[..., :-1]b_term2 = (k[..., i + 1 :] - t) / right_denom * b[..., 1:]b = b_term1 + b_term2# Compute points on the B-spline curvepoint = b @ Xderiv = d @ X# Handle out-of-bound `t` values by extrapolating using linear approximationk = k.unsqueeze(2) # Shape [L, 1, N + 2p, 1]X = X.unsqueeze(2) # Shape [L, N, 1, 1]left_out_mask = t_unclamp < k[..., 0]right_out_mask = t_unclamp >= k[..., -1]# Linear extrapolation for out-of-bound valuesleft_extrap = X[:, 0] + deriv * (t_unclamp - t)right_extrap = X[:, -1] + deriv * (t_unclamp - t)# Replace out-of-bound points with extrapolated valuespoint = torch.where(left_out_mask, left_extrap, point)point = torch.where(right_out_mask, right_extrap, point)point = point.squeeze(2).T # Reshape to [B, L]deriv = deriv.squeeze(2).T # Reshape to [B, L]return point, derivdef cumsum_softmax(v, dim=1):v = F.softmax(v, dim)v = v.cumsum(dim)shape = list(v.shape)shape[dim] = 1z = v.new_zeros(shape)v = torch.cat([z, v], dim)return vnode_vector = cumsum_softmax(node_logits)knot_vector = cumsum_softmax(knot_logits)Yp, Yd = bspline(spline_order, node_vector, knot_vector, X).

[0149] Meanwhile, as discussed above, the present invention proposes a learnable numerical input normalization based on B-splines, specifically performing normalization individually for each feature to handle the heterogeneity of each input feature in learning tabular data. This invention can be usefully utilized in various situations. More specifically, the invention can be applied to various industries and services and utilized effectively. In one embodiment, the invention can be usefully utilized in the field of appearance inspection machines and defective product inspection in manufacturing processes.

[0150] In the manufacturing process, visual inspection and defect detection are based on various sensor data. For example, this data may include pixel values ​​generated by imaging sensors, extracted values ​​for specific defect features, and location data. Since this data is highly heterogeneous and even small variations in defects can lead to major quality issues, detailed normalization is required.

[0151] According to one embodiment of the present invention, a neural network can be configured to learn a B-spline normalization curve for each sensor feature. For example, the sensitivity of defect detection can be improved by learning a normalization curve for the pixel brightness values ​​of a specific defect. In this case, the slope of the B-spline curve is adjusted during the learning process so that a wider normalization range is assigned to defect data points that are difficult to detect, and a narrow range is assigned to defect data points that are easy to detect.

[0152] For example, the present invention allows B-splines to steepen their curves to concentrate more resources on high-difficulty defect patterns (such as small and faint scratches). This improves the detection sensitivity of the visual inspection system and increases the probability of detecting defective products. In other words, it can be effective in reducing false positives and omissions.

[0153] Furthermore, unexpected defects may occur during the manufacturing process. In such cases, the present invention can also be utilized for adaptive normalization of outliers. The present invention can distinguish pattern differences between existing outlier data and normal data through the differentiation of B-spline curves, thereby significantly improving the performance of models that detect process abnormalities in real time. For example, as subtle pattern variations that are not easily detected under normal conditions become more likely to be classified as outliers within a wider normalization range, defective products can be detected at an early stage.

[0154] In another embodiment, the present invention can be applied and utilized in the field of new material development and research.

[0155] Research on novel materials deals with precise experimental data regarding the physical and chemical properties of materials (e.g., strength, thermal stability, electrical conductivity, etc.). These properties exhibit non-linear data distributions depending on the experimental environment, the material's microstructure, and molecular composition. In particular, since some properties respond very sensitively within specific ranges, it is important to properly normalize them.

[0156] Accordingly, the present invention applies B-spline normalization, which allows for individual learning of each physical and chemical property, to construct a normalization curve that reflects the non-linear distribution of each property. For example, in the case of a novel material whose electrical conductivity changes rapidly within a specific temperature range, a wider normalization range can be allocated to that range to enable the model to learn it more precisely. Furthermore, by adjusting the derivative values ​​of the B-spline, the present invention can allocate more resources to sections that are difficult to predict during experiments, thereby enabling more accurate prediction of the properties of the novel material. This can play a crucial role in the optimal design of the novel material or in predicting its performance improvement.

[0157] Furthermore, it is important to detect unexpected changes in properties as outliers when they occur during the experimentation of new materials. In this invention, outliers can be detected based on the property distribution using B-splines, thereby enabling the early detection and analysis of abnormal data generated during research on new materials. For example, when the physical changes of a material under specific environmental conditions differ significantly from existing predictions, the slope of the B-spline is significantly adjusted in this section, thereby providing the effect of identifying it as an outlier.

[0158] Meanwhile, the data normalization system (100) according to the present invention described above can be implemented through a computing device described below and can perform data processing related to the aforementioned data normalization method.

[0159] Meanwhile, FIG. 10 illustrates an example of a block diagram of a computing system in which the present invention can be implemented.

[0160] Referring to FIG. 10, a computing system (10000) that performs data normalization based on a B-spline for efficient artificial neural network learning in table data according to one embodiment of the present invention may include at least one computing device. At this time, the at least one computing device may be a single processor or a multi-processor computing device.

[0161] The components of at least one computing device of the present invention may include various hardware components such as one or more processors, memory, other hardware, and a system bus (not shown) that connects various system components so that they can transmit and receive data to and from each other (e.g., telecommutatively connected, physically connected, electrically connected), and the components of at least one computing device are not limited thereto and may be very diverse.

[0162] Meanwhile, at least one computing device included in a computing system (10000) that performs data normalization based on B-splines for efficient artificial neural network learning on table data may be connected to communicate via a network (1070). For example, at least one computing device included in the computing system (10000) may be clustered or part of a local area network (LAN). Additionally, at least one computing device may be part of a wide area network (WAN) or connected to at least one of a client-server network and a peer-to-peer network within the cloud.

[0163] Meanwhile, when at least one computing device is used in at least one of a network environment and a cloud computing environment, the at least one computing device may be connected to at least one of a public and private network through a network interface or adapter. In one embodiment, other communication connection devices, such as a modem, may be used to establish communication through the network. The modem may be at least one of an internal modem and an external modem, and may be connected to a system bus through a network interface or a specific mechanism, etc. A wireless network component consisting of an interface and an antenna may be coupled to the network through a device such as an access point, a peer computer, etc. In the present invention, the method of connecting at least one computing device to communicate through the network (1070) is not limited, and it may be connected to communicate in a manner different from the described example.

[0164] Furthermore, other computer-type devices and / or systems not shown in FIG. 10 may also interact technically with at least one computing device or other system through one or more connections to the network (1070) via a network interface. Here, the network interface may include network interface equipment such as a physical network interface controller (NIC) or a virtual network interface (VIF).

[0165] The network (1070) of the present invention may include various forms such as the Internet, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth (Bluetooth™ RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus), etc., and in the present invention, data transmission may be performed based on standard communication protocols such as TCP / IP, HTTP, SSL, etc.

[0166] A computing system (10000) that performs data normalization based on a B-spline for efficient artificial neural network learning in table data according to the present invention may include at least one of a user computing device (1010), a training computing system (1050), and a server computing system (1030).

[0167] A user computing device (1010) according to the present invention may be understood as a computing device comprising at least one processor (1011) and a memory (1012) that perform data normalization based on a B-spline for efficient artificial neural network learning in table data. For example, the user computing device (1010) may include at least one computing device among a smartphone, a smart TV, a laptop computer, a desktop computer, a digital broadcasting terminal, a PDA (personal digital assistant), a PMP (portable multimedia player), a navigation device, a slate PC, a tablet PC, an ultrabook, a wearable device (e.g., a smartwatch, a smart glass, and a head-mounted display).

[0168] At least one processor (1011) constituting the user computing device (1010) may include one or more general-purpose processors and / or one or more special-purpose processors. For example, at least one processor (1011) constituting the user computing device (1010) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), an arithmetic logic unit (ALU), a floating-point arithmetic unit (FPU), an application integrated circuit, an application semiconductor (ASIC), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions.

[0169] Furthermore, at least one processor (1011) may be configured to execute computer-readable instructions contained in memory (1012) and / or other instructions described herein.

[0170] The memory (1012) constituting the user computing system (1010) according to the present invention may include volatile memory, non-volatile memory, fixed media, removable media, magnetic media, optical media, semiconductor media and / or other types of physically durable storage media.

[0171] For example, memory (1012) may include one or more non-transient / transient computer-readable storage media such as RAM, ROM, HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), EEPROM, EPROM, flash memory device, magnetic disk, and combinations thereof, and may include web storage of a server that performs memory storage functions on the internet. Such memory (1012) may store data and instructions necessary for the at least one processor (1011) to perform the operation of an application that normalizes table data based on B-spline.

[0172] A user computing device (1010) may include one or more user input components (1021) that detect user input. For example, the user input component (1021) may also be referred to as a user interface module. The user input component (1021) may include a touch screen, a computer mouse, a keyboard, a keypad, a touchpad, a trackball, a joystick, a voice recognition module, or other similar devices. However, the present invention does not limit the type of user input component (1021). In this case, the user input component (1021) in the present invention does not necessarily mean a hardware means, but can be understood as a channel for receiving input from a user. Meanwhile, the user of the present invention may refer to an automated agent, script, playback software, etc., that operates on behalf of one or more people.

[0173] A user can interact with a computing system (10000) including at least one computing device through input text, touch, voice, movement, computer vision, gestures and / or other forms of input / output using a user input component (1021). For example, the user input component (1021) may include one or more of a command line interface (CLI), a graphical user interface (GUI), a natural user interface (NUI), a voice command interface and / or other user interface (UI) representations.

[0174] Between the user input component (1021) and the user computing device (1010), one or more application programming interface (API) calls may be made based on user input received from the user interface and / or network.

[0175] Here, the expression "based on" may be interpreted to include cases where it is based on the use of a specific configuration, modified from, derived from, influenced by, dependent on, or otherwise derived from a specific configuration. In some embodiments, an API call may be configured for a specific API, which may be interpreted or converted into an API call configured for another API. Here, an API may refer to a defined interface or connection between computers or between computer programs.

[0176] In one embodiment, the user computing device (1010) may store at least one machine learning model (1020). For example, the user computing device (1010) may be various machine learning models, such as multiple neural networks (e.g., deep neural networks) that perform data normalization based on B-splines for efficient artificial neural network learning, or other types of machine learning models including non-linear models and / or linear models, and may be composed of a combination thereof.

[0177] According to an embodiment of the present invention, a user computing device (1010) may perform a B-spline-based data normalization method for efficient artificial neural network learning by using a local or / and external machine learning model (1020). Alternatively, the user computing device (1010) may perform a B-spline-based data normalization method for efficient artificial neural network learning by using a machine learning model (1040) provided by a server.

[0178] Additionally, according to another embodiment of the present invention, a server computing system (1030) communicating with a user computing device (1010) may provide normalized table data based on a B-spline and a learned artificial neural network to the user computing device (1010) on an application or / and the web in accordance with a user's request received through the user computing device (1010).

[0179] In addition, according to another embodiment of the present invention, at least a part of the user computing device (1010) and the server computing system (1030) are interconnected to perform a B-spline-based data normalization method for efficient artificial neural network learning, thereby providing the user with normalized table data and a learned artificial neural network.

[0180] Additionally, according to various embodiments of the present invention, a user computing device (1010) and / or a server computing system (1030) may learn machine learning models (1020, 1040) performed in a B-spline-based data normalization method for efficient artificial neural network learning through interaction with a training computing system (1050) that is communicatedly connected via a network (1070). In this case, the training computing system (1050) may be a computing system separate from the server computing system (1030). Alternatively, in some embodiments, the training computing system (1050) may be part of the server computing system (1030) or part of the user computing device (1010).

[0181] Meanwhile, the server computing system (1030) may include at least one processor (1031) and memory (1032). Here, the processor (1031) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), an application integrated circuit, an application semiconductor (ASIC), an arithmetic logic unit (ALU), a floating-point arithmetic unit (FPU), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions. For example, at least one processor (1031) may include a circuit and a transistor configured to execute instructions from memory (1032).

[0182] The memory (1032) constituting the server computing system (1030) according to the present invention may include volatile memory, non-volatile memory, fixed media, removable media, magnetic media, optical media, semiconductor media, and / or other types of physically durable storage media. For example, the memory (1032) may include one or more non-transient / transient computer-readable storage media such as RAM, ROM, HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), EEPROM, EPROM, flash memory device, magnetic disk, etc., and combinations thereof, and may include web storage of a server that performs the storage function of memory over the internet. Additionally, the server computing system (1030) may further include a data storage (data store). For example, the data storage may be composed of at least one of a relational database, a NoSQL database, a data warehouse, and a local file system.

[0183] In the memory (1032) constituting the server computing system (1030) according to the present invention, data and instructions necessary for the at least one processor (1031) to perform the operation of an application that normalizes table data based on a B-spline can be stored.

[0184] In one embodiment, the server computing system (1030) may be composed of a single device or a plurality of computing devices, and may be configured to operate according to a sequential or parallel computing architecture. Additionally, a distributed processing system may be configured with a plurality of networked devices.

[0185] Meanwhile, the training computing system (1050) may include at least one processor (1051) and memory (1052). The model trainer (1060) is a logical component that executes the training of at least one machine learning model (1020, 1040) and may be implemented in the form of hardware, firmware, or software. For example, the model trainer (1060) may be executed by the processor (1051) after loading training data (1061) stored in a storage device into memory (1052). For example, the model trainer (1060) may be configured to execute one or more operations (e.g., model training, model reconstruction, model validation, model testing) on ​​at least one machine learning model.

[0186] The machine learning model of the present invention may include at least one of a statistical model, an algorithm, a neural network (NN), a convolutional neural network (CNN), a generative neural network (GNN), a Word2Vec model, a Bag of Words model, a TF-IDF (document frequency-inverse document frequency) model, a GPT (Generative Pre-trained Transformer) model (or other autoregressive models), a PPO (Proximal Policy Optimization) model, a nearest neighbor model (e.g., a k-nearest neighbor model), a linear regression model, a K-means clustering model, a Q-learning model, a TD (Temporal Difference) model, a Deep Adversarial Network model, and all other types of models further described herein.

[0187] Specifically, the model trainer (1060) may execute operations to train a machine learning model, and said operations may include at least one of adding, removing, and modifying model parameters. At this time, the training of the machine learning model may be at least one of supervised learning, semi-supervised learning, and unsupervised learning. In one embodiment, the training of the machine learning model may include the step of repeatedly inputting training data (1061) based on epochs and repeatedly performing the machine learning model training process configured in this way. Here, an epoch may refer to a unit in which the entire set of training data (1061) undergoes forward and backpropagation processing once. In some implementations, different levels of training methods (e.g., supervised learning, semi-supervised learning, unsupervised learning) may be used for different epochs.

[0188] The training data (1061) of the present invention may include input data and / or data previously output from at least one machine learning model (e.g., recursive learning feedback).

[0189] At least one parameter of a machine learning model may include at least one of a seed value, a model node, a model layer, an algorithm, a function, connections between different machine learning models, connections between parameters, machine learning model constraints, and other digital components that influence the output of the machine learning model. In this case, model connections between different machine learning models may include or represent relationships between model parameters and / or models, which may be dependent or interdependent, hierarchical, and / or static or dynamic. The combinations and configurations of model parameters described herein may be too complex to be maintained or utilized by human cognitive abilities.

[0190] In the present invention, the machine learning parameters described according to the embodiments are not limited, and a single machine learning model may further include a plurality of model parameters.

[0191] Meanwhile, FIG. 11 illustrates an example of a block diagram of a computing device (1100) that may be included in a user computing device (1010), a server computing system (1030), and a training computing system (1050), as an embodiment of a computing system (10000) in which the present invention can be implemented.

[0192] As illustrated in FIG. 11, the computing device (1100) may include at least one application (e.g., Application 1 to Application N), and each of the at least one application may include a machine learning library and a model execution environment for performing a machine learning-based data normalization method. The at least one application included in the computing device (1100) may communicate with the sensor, context manager, device state manager, or additional component(s) within the computing device (1100) via an Application Programming Interface (API). In one embodiment, the at least one application may interface with device components, such as receiving sensor data or state data or transmitting prediction results to an output device via a public or private API.

[0193] Meanwhile, FIG. 12 illustrates an example of a block diagram in another aspect of a computing device (1200), which is one of the components of a computing system (10000) that performs a B-spline-based data normalization method according to an embodiment of the present invention.

[0194] A computing device (1200) according to the present invention may include at least one application (e.g., Application 1 to Application N), and at least one application may communicate with a central intelligence layer (1210). Each application may interact with a shared model within the central intelligence layer (1210) through an API (e.g., a common API).

[0195] The central intelligence layer (1210) includes one or more machine learning models and may share them among multiple applications or provide them independently to each. In one embodiment, the central intelligence layer (1210) may be integrated as part of an operating system or implemented as a separate logical layer.

[0196] Additionally, the central intelligence layer (1210) can communicate with the central device data layer (1220). The central device data layer (1220) can store multiple samples containing multiple features included in table data stored within the computing device (1200) and values ​​corresponding to each of the multiple features, and provide these as input data required for efficient artificial neural network learning in the table data. Each device component (e.g., sensor, state manager, etc.) can communicate with the central device data layer (1220) via a private API, etc.

[0197] The technology described herein may be composed of a single or multiple computing devices, and a machine learning model that performs a B-spline-based data normalization method may be executed sequentially or in parallel on one component or multiple distributed components. Data storage, machine learning models, and applications may be distributed and operated locally or over a network, and these configurations can be flexibly applied to various system architectures.

[0198] As described above, according to the data normalization method and system for learning tabular data of the present invention, by identifying the importance of tabular data (i.e., input data) during the learning process of an artificial neural network and applying a normalization function accordingly, it is possible to support the artificial neural network in efficiently allocating capacity to input values ​​of high importance and to support efficient learning of the artificial neural network regarding tabular data.

[0199] In addition, according to the data normalization method and system for learning tabular data of the present invention, tabular data can be normalized using a B-spline-based normalization technique, and an artificial neural network can be trained using a loss function configured to facilitate learning of the normalized tabular data and difficult input sections, while simultaneously compressing already learned sections. Through this, the present invention can improve the performance of the artificial neural network in difficult samples by adjusting the transformation according to the difficulty of each data point. That is, in the case of data points that are difficult to learn, the present invention allocates a wider range in the normalized tabular data, thereby allowing the artificial neural network to efficiently distribute capacity to data points that are difficult to learn, and thus improving learning efficiency.

[0200] Furthermore, according to the data normalization method and system for learning tabular data of the present invention, by parameterizing the tabular data into curves and optimizing them together with the learning of an artificial neural network, adaptive normalization is enabled for each feature of the tabular data, thereby effectively handling the heterogeneity of the tabular data.

[0201] Furthermore, according to the data normalization method and system for learning tabular data of the present invention, by training an artificial neural network using a new loss function that dynamically adjusts B-spline parameters according to the difficulty of each data point, the artificial neural network can efficiently allocate capacity to difficult data points and ensure optimized transformation for both difficult and easy data.

[0202] Thus, the present invention enables an artificial neural network to learn optimal parameters that improve learning efficiency and performance by modeling feature transformations through a B-spline-based normalization technique customized for learning tabular data. Furthermore, the adaptability of the artificial neural network can be enhanced through a new loss function that dynamically stores input transformations according to the learning difficulty of specific data points.

[0203] Meanwhile, the present invention described above can be implemented based on a quantum computer. The present invention implemented based on a quantum computer may include a qubit-based quantum processor and quantum memory, and may include software and hardware interfaces optimized for quantum computation.

[0204] Quantum processors in quantum computers utilize qubits to efficiently process complex operations through parallel computation, quantum entanglement, and quantum superposition, which cannot be performed by the binary bits of classical computers. Quantum processors process data using quantum gates and can provide exponential speed improvements for specific problems.

[0205] Meanwhile, the present invention described above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a computer-readable medium (or recording medium).

[0206] Furthermore, the present invention described above can be implemented as computer-readable code or instructions on a medium on which a program is recorded. That is, the present invention can be provided in the form of a program.

[0207] Meanwhile, computer-readable media include all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0208] Furthermore, the computer-readable medium may be a server or cloud storage that includes a storage and is accessible to an electronic device via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage via wired or wireless communication.

[0209] A computer program may reach the system (100) through various suitable transmission mechanisms. The transmission mechanism may be, for example, a computer-readable storage medium, a computer program product, a memory device, a recording medium such as a CD-ROM or DVD, or a product that tangibly embodies the computer program. The transmission mechanism may be a signal configured to reliably transmit the computer program through air or an electrical connection. The system (100) may propagate or transmit the computer program as a computer data signal.

[0210] Furthermore, references to 'computer-readable storage media,' 'computer program products,' 'computer programs embodied in a tangible form,' etc., or to 'controller,' 'computer,' 'processor,' etc., should be understood to include not only computers with various architectures such as single / multi-processor architectures and sequential (Von Neumann) / parallel architectures, but also specialized circuits such as Field-Programmable Gate Arrays (FPGAs), Application Specific Circuits (ASICs), signal processing units, and other devices. References to computer programs, instructions, code, etc., should be understood to include software for programmable processors or firmware, such as programmable content for hardware devices, whether it is instructions for a processor or configuration settings for a fixed-function device, gate array, or programmable logic device.

[0211] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, namely a CPU (Central Processing Unit), and no special limitations are placed on its type.

[0212] Meanwhile, the above detailed description should not be interpreted restrictively in all respects but should be considered exemplary. The scope of the invention shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

Claims

1. In a computerized method comprising the following, A step of receiving table data configured to include multiple different features and multiple samples having values ​​corresponding to each of the multiple features; A step of obtaining a plurality of normalized features by applying a curve-parameterized normalization function to each of the plurality of features included in the table data; A step of performing training of an artificial neural network using each of the above-mentioned normalized multiple features; and A data normalization method for learning tabular data, characterized by including a step of optimizing the normalization function based on the progress of learning of the artificial neural network.

2. In Paragraph 1, The method further includes the step of applying a min-max transformation to each of the plurality of features included in the table data, In the step of acquiring the above-mentioned normalized multiple features, A data normalization method for learning tabular data, characterized by applying the normalization function to each of the plurality of features to which the above min-max transformation is applied to obtain the above-mentioned normalized plurality of features.

3. In Paragraph 1, The step of training the artificial neural network above is, A step of calculating a learning loss between a learning target value corresponding to the table data and an output value for the normalized plurality of features using a pre-set loss function; and The method includes the step of training the artificial neural network so that the calculated learning loss is reduced, and In the above optimization step, A data normalization method for learning tabular data characterized by optimizing the normalization function based on the calculated learning loss.

4. In Paragraph 3, The step of calculating the above learning loss is, The step of calculating the learning loss between the output values ​​for a plurality of samples, including the learning target value and the value corresponding to each of the normalized plurality of features, using the above loss function, and In the above optimization step, A data normalization method for learning tabular data, characterized by optimizing at least one parameter of the normalization function simultaneously with the learning of the artificial neural network.

5. In Paragraph 4, In the above optimization step, Based on the importance specified for each of the plurality of samples, the parameters of the normalization function are optimized simultaneously with the training of the artificial neural network, and The above importance is, A data normalization method for learning tabular data characterized by being determined based on the learning loss calculated through the above loss function.

6. In Paragraph 5, A step of analyzing the learning difficulty of each of the plurality of samples based on the learning loss calculated through the loss function to determine the importance of each of the plurality of samples; and Based on the analyzed learning difficulty above, the method further includes the step of specifying the importance of each of the plurality of samples, and The above-mentioned optimization step is, A data normalization method for learning tabular data, characterized by a step of adjusting the slope of the normalization function according to the importance of each of the plurality of samples.

7. In Paragraph 6, The above learning difficulty is, The above learning loss includes at least one of a first learning difficulty related to satisfying a first criterion and a second learning difficulty related to satisfying a second criterion, and In the step of specifying the importance mentioned above, To adjust the slope of the above normalization function, the importance of the first sample satisfying the first learning difficulty among the plurality of samples is specified as having the first importance, and A data normalization method for learning tabular data, characterized in that the importance of a second sample satisfying the second learning difficulty among the plurality of samples is specified as having the second importance.

8. In Paragraph 7, In the above optimization step, A data normalization method for learning tabular data, characterized by adjusting the magnitude of the slope of the normalization function to increase in the case of the first sample having the first importance among the plurality of samples.

9. In Paragraph 7, In the above optimization step, A data normalization method for learning tabular data, characterized by adjusting the magnitude of the slope of the normalization function to decrease in the case of the second sample having the second importance among the plurality of samples.

10. In Paragraph 6, The above-mentioned optimization step is, The step of adjusting the parameters of the normalization function according to the learning difficulty of each of the plurality of samples using the above loss function, and The above loss function is, A data normalization method for learning tabular data, characterized by being defined to adjust the slope of the normalization function according to the importance of each of the plurality of samples.

11. In Paragraph 10, The above importance includes at least one of a first importance and a second importance determined according to the learning loss, and In the above optimization step, A data normalization method for learning tabular data, characterized by adjusting the parameters of the normalization function using the loss function so that, in the case of the first sample having the first importance among the plurality of samples, the magnitude of the slope of the normalization function increases.

12. In Paragraph 10, In the above optimization step, A data normalization method for learning tabular data, characterized by adjusting the parameters of the normalization function using the loss function so that, in the case of a second sample having the second importance among the plurality of samples, the magnitude of the slope of the normalization function is reduced.

13. In Paragraph 4, The above-mentioned optimization step is, A data normalization method for learning tabular data, characterized by a step of updating the parameters of the normalization function based on the learning loss for each of the plurality of samples.

14. In Paragraph 13, A data normalization method for learning tabular data, characterized in that when the parameters of the normalization function are updated, the values ​​of each of the normalized multiple features are updated.

15. In Paragraph 4, The above loss function is, A data normalization method for learning tabular data, characterized in that the learning process of an artificial neural network and the parameter optimization process of the normalization function are defined to be performed separately.

16. In Paragraph 1, A data normalization method for learning tabular data, characterized in that the above normalization function is generated for each of the plurality of features included in the above table data.

17. A system comprising memory configured to store executable instructions and one or more processors configured to perform operations by executing one or more instructions, The above system is, Receive table data configured to include multiple different features and multiple samples having values ​​corresponding to each of the multiple features, and For each of the plurality of features included in the table data above, a curve-parameterized normalization function is applied to obtain a plurality of normalized features, and Using each of the above-mentioned normalized multiple features, training of the artificial neural network is performed, and A data normalization system for learning tabular data characterized by optimizing the normalization function based on the progress of learning of the artificial neural network.

18. A program that is executed by one or more processes in an electronic device and stored on a computer-readable recording medium, The above program is, A step of receiving table data configured to include multiple different features and multiple samples having values ​​corresponding to each of the multiple features; A step of obtaining a plurality of normalized features by applying a curve-parameterized normalization function to each of the plurality of features included in the table data; A step of performing training of an artificial neural network using each of the above-mentioned normalized multiple features; and A program stored on a computer-readable recording medium characterized by including instructions for performing a step of optimizing the normalization function based on the progress of learning of the artificial neural network.

Images