Methods for Improved Deep Learning Models

Abstract

Described herein are methods and systems for generating, training, and tuning deep learning models. The methods and systems may use deep learning models to provide a generalized framework for analyzing data records comprising one or more strings (e.g., sequences) of data. Unlike existing deep learning models and frameworks designed to be problem / analysis specific, the generalized framework described herein may be made applicable to a wide range of predictive and / or generative data analyses.

Description

[Technical field]

[0001] The present application relates to methods and systems for improved deep learning models. [Background technology]

[0002] (CROSS REFERENCE TO RELATED PATENT APPLICATIONS) This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 135,265, filed January 8, 2021, the entire contents of which are incorporated herein by reference.

[0003] Most deep learning models, such as artificial neural networks, deep neural networks, deep belief networks, recurrent neural networks, and convolutional neural networks, are designed to be problem / analysis specific. Summary of the Invention [Problem to be solved by the invention]

[0004] As a result, most deep learning models are not generally applicable. Thus, there is a need for a framework for generating, training, and tuning deep learning models that can be made applicable to a variety of predictive and / or generative data analyses. These and other considerations are described herein. [Means for solving the problem]

[0005] It should be understood that both the general description below and the detailed description below are exemplary and explanatory only and not limiting. Methods and systems for improved deep learning models are described herein. In one embodiment, a plurality of data records and a plurality of variables may be used by a computing device to generate and train a deep learning model, such as a predictive model. The computing device may determine a numerical representation for each data record of a first subset of the plurality of data records. Each data record of the first subset of the plurality of data records may include a label, such as a binary label (e.g., yes / no) and / or a percentage value. The computing device may determine a numerical representation for each variable of a first subset of the plurality of variables. Each variable of the first subset of the plurality of variables may include a label (e.g., a binary label and / or a percentage value). A plurality of first encoder modules may generate a vector for each attribute of each data record of the first subset of the plurality of data records. A plurality of second encoder modules may generate a vector for each attribute of each variable of the first subset of the plurality of variables.

[0006] The computing device may determine a number of features for the predictive model. The computing device may generate the concatenated vector. The computing device may train the predictive model. The computing device may train the first encoder modules and / or the second encoder modules. The computing device may output the predictive model, the first encoder modules, and / or the second encoder modules after training. Once trained, the predictive model, the first encoder modules, and / or the second encoder modules may be enabled to provide a variety of predictions and / or generation data analytics.

[0007] As an example, a computing device may receive a previously unseen data record (a first data record) and a plurality of previously unseen variables (a plurality of first variables). The computing device may determine a numeric representation for the first data record. The computing device may determine a numeric representation for each variable of the plurality of first variables. The computing device may determine a vector for the first data record using a plurality of first trained encoder modules. The computing device may determine a vector for the first data record based on the numeric representation for the data record using a plurality of first trained encoder modules.

[0008] The computing device may use a plurality of second trained encoder modules to determine a vector for each attribute of each variable of the plurality of first variables. The computing device may use a plurality of second trained encoder modules to determine a vector for each attribute of each variable of the plurality of first variables based on a numerical representation for each variable of the plurality of variables. The computing device may generate a concatenated vector based on the vector for the first data record and the vector for each attribute of each variable of the plurality of first variables. The computing device may use a trained predictive model to determine one or more of predictions or scores associated with the first data record. The trained predictive model may determine one or more of predictions or scores associated with the first data record based on the concatenated vector.

[0009] The trained predictive models and trained encoder modules as described herein may be enabled to provide a variety of predictions and / or generated data analyses. The trained predictive models and trained encoder modules may be initially trained to provide a first set of predictions and / or generated data analyses, and each may be retrained to provide another set of predictions and / or generated data analyses. Upon retraining, the predictive models and encoder modules described herein may provide another set of predictions and / or generated data analyses. Further advantages of the disclosed methods and systems will be set forth in part in the following description, or in part will be understood from the description, or may be learned by practice of the disclosed methods and systems.

[0010] The accompanying drawings, which are incorporated in and constitute a part of this description, serve to explain the principles of the methods and systems described herein. [Brief description of the drawings]

[0011] [Figure 1] 1 illustrates an exemplary system. [Diagram 2] An exemplary method is shown. [Figure 3A] 1 illustrates components of an exemplary system. [Figure 3B] 1 illustrates components of an exemplary system. [Figure 4A] 1 illustrates components of an exemplary system. [Figure 4B] 1 illustrates components of an exemplary system. [Diagram 5] 1 illustrates an exemplary system. [Figure 6] An exemplary method is shown. [Figure 7] 1 illustrates an exemplary system. [Figure 8] An exemplary method is shown. [Figure 9] An exemplary method is shown. [Figure 10] An exemplary method is shown. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from "about" one particular value, and / or to "about" another particular value. When such ranges are expressed, the alternative configuration includes from the one particular value and / or to the other particular value. Similarly, when values ​​are expressed as approximations, it will be understood that the particular value forms the alternative configuration by use of the antecedent "about." It will be further understood that the endpoints of each of these ranges are significant in relation to the other endpoint, and independently of the other endpoint.

[0013] "Optional" or "optionally" means that the subsequently described event or circumstance may or may not occur, and that the description includes cases where the event or circumstance occurs and cases where the event or circumstance does not occur.

[0014] Throughout this description and claims, the word "comprise" and variations of this word, such as "comprising" and "comprises," mean "including, but not limited to," and are not intended to exclude, for example, other components, integers, or steps. "Exemplary" means "one example of," and is not intended to convey an indication of a preferred or ideal configuration. "Etc." is not used in a limiting sense, but is used for descriptive purposes.

[0015] When combinations, subsets, interactions, groups, etc. of components are disclosed, it is understood that each is specifically contemplated and described herein, although specific reference to each of the various individual and collective combinations and permutations of these components may not be explicitly described. This applies to all parts of this application, including but not limited to steps in the described method. Thus, when there are various additional steps that may be performed, it is understood that each of these additional steps may be performed in any particular configuration or combination of configurations of the described method.

[0016] As will be appreciated by those skilled in the art, hardware, software, or a combination of software and hardware may be implemented. Furthermore, a computer program product on a computer-readable storage medium (e.g., non-transitory) has processor-executable instructions (e.g., computer software) embodied in the storage medium. Any suitable computer-readable storage medium may be utilized, including hard disks, CD-ROMs, optical storage devices, magnetic storage devices, memresistors, non-volatile random access memories (NVRAMs), flash memories, or combinations thereof.

[0017] Throughout this application, reference is made to block diagrams and flowcharts. It will be understood that each block of the block diagrams and flowcharts, and combinations of blocks in the block diagrams and flowcharts, respectively, can be implemented by processor-executable instructions. These processor-executable instructions may be loaded into a general-purpose computer, special-purpose computer, or other programmable data processing device to produce a machine such that the processor-executable instructions, executing on the computer or other programmable data processing device, create a device for performing the functions specified in the flowchart blocks.

[0018] These processor-executable instructions may also be stored in a computer-readable memory that may direct a computer or other programmable data processing apparatus to function in a particular manner, such that the processor-executable instructions stored in the computer-readable memory produce an article of manufacture comprising processor-executable instructions for performing functions identified in the flowchart blocks. The processor-executable instructions may also be loaded into a computer or other programmable data processing apparatus to produce a computer-implemented process by performing a series of operational steps on the computer or other programmable apparatus, such that the processor-executable instructions executing on the computer or other programmable apparatus provide steps for implementing the functions identified in the flowchart blocks.

[0019] The blocks of the block diagrams and flowcharts support combinations of devices for performing the specified functions, combinations of steps for performing the specified functions, and program instruction means for performing the specified functions. It will also be understood that each block of the block diagrams and flowcharts, and combinations of blocks of the block diagrams and flowcharts, can be implemented by special purpose hardware-based computer systems that perform the specified functions or steps, or a combination of special purpose hardware and computer instructions.

[0020] Described herein are methods and systems for improved deep learning models. By way of example, the methods and systems may provide a generalized framework for analyzing data records comprising one or more strings (e.g., sequences) of data using a deep learning model. The framework may generate, train, and tune deep learning models that may be applicable to a variety of predictive and / or generative data analyses. The deep learning model may receive a plurality of data records, each of which may include one or more attributes (e.g., a string of data, a sequence of data, etc.). The deep learning model may use the plurality of data records and the corresponding plurality of variables to output one or more of a binomial prediction, a polynomial prediction, a variational autoencoder, a combination thereof, and / or the like.

[0021] In one embodiment, the plurality of data records and the plurality of variables may be used by a computing device to generate and train a deep learning model, such as a predictive model. Each data record of the plurality of data records may include one or more attributes (e.g., a string of data, a sequence of data, etc.). Each data record of the plurality of data records may be associated with one or more variables of the plurality of variables. The computing device may determine a plurality of features of a model architecture to train the predictive model. The computing device may determine the plurality of features based on a hyper-parameter set comprising, for example, a number of neural network layers / blocks, a number of neural network filters in a neural network layer, etc.

[0022] An element of the hyper-parameter set may include a first subset of a plurality of data records (e.g., data record attributes / variables) for inclusion in the model architecture and for training a predictive model. Another element of the hyper-parameter set may include a first subset of a plurality of variables (e.g., attributes) for inclusion in the model architecture and for training a predictive model. The computing device may determine a numerical representation for each data record of the first subset of a plurality of data records. Each numerical representation for each data record of the first subset of a plurality of data records may be generated based on the corresponding one or more attributes. Each data record of the first subset of a plurality of data records may be associated with a label, such as a binary label (e.g., “yes” / “no”) and / or a percentage value.

[0023] The computing device may determine a numeric representation for each variable of the first subset of variables. Each variable of the first subset of variables may be associated with a label (e.g., a binary label and / or a percentage value). The first encoder modules may generate a vector for each attribute of each data record of the first subset of data records. For example, the first encoder modules may generate a vector for each attribute of each data record of each data record of the first subset of data records based on the numeric representation for each data record of the first subset of data records. The second encoder modules may generate a vector for each attribute of each variable of the first subset of variables. For example, the second encoder modules may generate a vector for each attribute of each variable of the first subset of variables based on the numeric representation for each variable of the first subset of variables.

[0024] The computing device may generate the concatenation vector. For example, the computing device may generate the concatenation vector based on a vector for each attribute of each data record of the first subset of the data records. As another example, the computing device may generate the concatenation vector based on a vector for each attribute of each variable of the first subset of the variables. As discussed above, the plurality of features may include as little as one or all of the corresponding attributes of the data records of the first subset of the data records and the variables of the first subset of the variables. Thus, the concatenation vector may be based on as little as one or all of the corresponding attributes of the data records of the first subset of the data records and the variables of the first subset of the variables. The concatenation vector may indicate a label. For example, the concatenation vector may indicate a label (e.g., a binary label and / or a percentage value) for each attribute of each data record of the first subset of the data records. As another example, the concatenation vector may indicate a label (e.g., a binary label and / or a percentage value) for each variable of the first subset of the variables.

[0025] The computing device may train a predictive model. For example, the computing device may train a predictive model based on the concatenated vector or a portion thereof (e.g., based on attributes of selected particular data records and / or attributes of variables). The computing device may train a plurality of first encoder modules and / or a plurality of second encoder modules. For example, the computing device may train a plurality of first encoder modules and / or a plurality of second encoder modules based on the concatenated vector.

[0026] The computing device may output (e.g., store) the predictive model, the first encoder modules, and / or the second encoder modules after training. Once trained, the predictive model, the first encoder modules, and / or the second encoder modules may be enabled to provide a variety of predictions and / or generative data analyses, such as providing binomial predictions, polynomial predictions, variational autoencoders, combinations thereof, and / or the like.

[0027] As an example, a computing device may receive a previously unseen data record (first data record) and a plurality of previously unseen variables (a plurality of first variables). The plurality of first variables may be associated with the first data record. The computing device may determine a numeric representation for the first data record. For example, the computing device may determine a numeric representation for the first data record in a manner similar to that described above for a first subset of the plurality of data records (e.g., training data records). The computing device may determine a numeric representation for each variable of the plurality of first variables. For example, the computing device may determine a numeric representation for each of the plurality of first variables in a manner similar to that described above for a first subset of the plurality of variables (e.g., training variables). The computing device may determine a vector for the first data record by using a plurality of first trained encoder modules. For example, the computing device may use the plurality of first encoder modules trained with a predictive model when determining the vector for the first data record. The computing device may determine a vector for the first data record based on the numeric representation for the data record by using a plurality of first trained encoder modules.

[0028] The computing device may use the plurality of second trained encoder modules to determine a vector for each attribute of each variable of the plurality of first variables. For example, the computing device may use the plurality of first encoder modules trained with a predictive model when determining a vector for each attribute of each variable of the plurality of first variables. The computing device may use the plurality of second trained encoder modules to determine a vector for each attribute of each variable of the plurality of first variables based on a numerical representation for each variable of the plurality of variables.

[0029] The computing device may generate a concatenated vector based on the vector for the first data record and the vector for each attribute of each variable of the plurality of first variables. The computing device may determine one or more of predictions or scores associated with the first data record by using a trained predictive model. The trained predictive model may include the predictive model described above that is trained together with the plurality of first encoder modules and the plurality of second encoder modules. The trained predictive model may determine one or more of predictions or scores associated with the first data record based on the concatenated vector. The score may indicate the likelihood that the first label applies to the first data record. For example, the first label may include a binary label (e.g., "yes" / "no") and / or a percentage value.

[0030] The trained predictive models and trained encoder modules as described herein may be enabled to provide a variety of predictions and / or generated data analyses. The trained predictive models and trained encoder modules may be initially trained to provide a first set of predictions and / or generated data analyses, and each may be retrained to provide another set of predictions and / or generated data analyses. For example, the first trained encoder modules described herein may be initially trained based on a plurality of training data records associated with a first label and a first hyper-parameter set. The first trained encoder modules may be retrained based on a further plurality of data records associated with a second hyper-parameter set that is at least partially different from the first hyper-parameter set. For example, the second hyper-parameter set and the first hyper-parameter set may include similar data types (e.g., strings, integers, etc.). As another example, the second trained encoder modules described herein may be initially trained based on a plurality of training variables associated with the first label and the first hyper-parameter set. The second trained encoder modules may be retrained based on a further plurality of variables associated with the second hyper-parameter set.

[0031] As a further example, the trained predictive model described herein may be initially trained based on a first concatenation vector. The first concatenation vector may be derived / determined / generated based on a plurality of training data records (e.g., based on a first label and a first hyper-parameter set) and / or based on a plurality of training variables (e.g., based on a first label and a second hyper-parameter set). The trained predictive model may be retrained based on a second concatenation vector. The second concatenation vector may be derived / determined / generated based on a vector for each attribute of each data record of the further plurality of data records. The second concatenation vector may also be derived / determined / generated based on a vector for each attribute of each variable of the further plurality of variables and the associated hyper-parameter set. The second concatenation vector may also be derived / determined / generated based on a further plurality of data records associated with the second hyper-parameter set and / or the further hyper-parameter set. In this manner, the first plurality of encoder modules and / or the second plurality of encoder modules may be retrained based on the second concatenation vector. Once retrained, the predictive model and encoder modules described herein may provide another set of predictions and / or generated data analysis.

[0032] Referring now to FIG. 1, a system 100 is shown. The system 100 may generate, train, and tune deep learning models. The system 100 may include a computing device 106. The computing device 106 may be, for example, a smartphone, a tablet, a laptop computer, a desktop computer, a server computer, etc. The computing device 106 may include one or more groups of servers. The computing device 106 may be configured to generate, store, maintain, and / or update various data structures, including databases for storing data records 104, variables 105, and labels 107.

[0033] The data records 104 may include one or more strings (e.g., sequences) of data and one or more attributes associated with each data record. The variables 105 may include multiple attributes, parameters, etc. associated with the data records 104. The labels 107 may each be associated with one or more of the data records 104 or variables 105. The labels 107 may include multiple binary labels, multiple percentage values, etc. In some embodiments, the labels 107 may include one or more attributes of the data records 104 or variables 105. The computing device 106 may be configured to generate, store, maintain, and / or update various data structures that comprise the database stored in the server 102. The computing device 106 may include a data processing module 106A and a prediction module 106B. The data processing module 106A and the prediction module 106B may be stored and / or configured to operate separately on the computing device 106 or on separate computing devices.

[0034] The computing device 106 may implement a generalized framework for analyzing data records 104, variables 105, and / or labels 107 by using a deep learning model, such as a predictive model. The computing device 106 may receive the data records 104, variables 105, and / or labels 107 from the server 102. Unlike existing deep learning models and frameworks designed to be problem / analysis specific, the framework implemented by the computing device 106 may be made applicable to a broad range of prediction and / or generative data analysis. For example, the framework implemented by the computing device 106 may generate, train, and tune a predictive model that may be made applicable to a variety of prediction and / or generative data analysis. The predictive model may output one or more of binomial prediction, polynomial prediction, variational autoencoder, combinations thereof, and / or the like. The data processing module 106A and the prediction module 106B are highly modularized, allowing for adjustments to the model architecture. The data records 104 may include any type of data records, such as strings (e.g., sequences) of alphanumeric characters, words, phrases, symbols, etc. The data records 104, variables 105, and / or labels 107 may be received as data records in a spreadsheet, such as one or more of a CSV file, a VCF file, a FASTA file, a FASTQ file, or any other suitable data storage format / file known to one of skill in the art.

[0035] As described further herein, the data processing module 106A may process the data records 104 and variables 105 (e.g., strings / sequences of alphanumeric characters, words, phrases, symbols, etc.) into a numerical form in an unlearnable manner via one or more “processors” that convert the data records 104 and variables 105 (e.g., strings / sequences of alphanumeric characters, words, phrases, symbols, etc.) into a numerical representation. These numerical representations, as described further herein, may be further processed in a learnable manner via one or more “encoder modules.” The encoder modules may include blocks of a neural network that are utilized by the computing device 106. The encoder modules may output vector representations of any of the data records 104 and / or any of the variables 105. The vector representation of a given data record and / or a given variable may be based on the corresponding numerical representation of the given data record and / or a given variable. Such vector representations may be referred to herein as “fingerprints.” The fingerprints of the data records may be based on attributes associated with the data records. A fingerprint of a data record may be concatenated with the fingerprints of corresponding variables and other corresponding data records into a single concatenated fingerprint. Such a concatenated fingerprint may be referred to herein as a concatenated vector. A concatenated vector may describe a data record (e.g., an attribute associated with the data record) and its corresponding variables as a single numerical vector.

[0036] As an example, a first data record of the data records 104 may be processed by a processor into a numeric form as described herein. The first data record may include a string (e.g., a sequence) of alphanumeric characters, words, phrases, symbols, etc., where each element of the sequence may be converted into a numeric form. A dictionary mapping between the sequence elements and the respective numeric forms may be generated based on the data type and / or attribute type associated with the data record 104. A dictionary mapping between the sequence elements and the respective numeric forms may also be generated based on a portion of the data record 104 and / or variables 105 used for training. The dictionary may be used to convert the first data record into an integer form and / or into a one-hot representation of the integer form. The data processing module 106A may include a trainable encoder model that may be used to extract features from the numeric representation of the first data record. Such extracted features may include a 1d numeric vector, or a "fingerprint" as described herein. The first variable of the variables 106 may be processed in numeric format by the processor as described herein. The first variable may include strings of alphanumeric characters, words, phrases, symbols, etc., which may be converted to numeric form. A dictionary mapping between the variable input values ​​and their respective numeric forms may be generated based on the data type and / or attribute type associated with the variable 106. The dictionary may be used to convert the first variable to an integer format and / or to a one-hot representation of the integer format. The data processing module 106A and / or the prediction module 106B may include a trainable encoder layer that extracts features (e.g., 1d vectors / fingerprints) from the numeric representation of the first variable. The fingerprint of the first data record and the fingerprint of the first variable may be concatenated into a single concatenated fingerprint / vector.

[0037] The concatenated vector may be passed to a predictive model generated by prediction module 106B. The predictive model may be trained as described herein. The predictive model may process the concatenated vector to provide an output comprising one or more of a prediction, a score, and the like. The predictive model may include one or more final blocks of a neural network as described herein. The predictive models and / or encoders described herein may be trained to perform binomial, polynomial, regression, and / or other tasks, or may be retrained as appropriate. As an example, the predictive models and / or encoders described herein may be used by computing device 106 to provide a prediction of whether a particular data record and / or attribute of a variable (e.g., feature) is indicative of a particular outcome (e.g., a binary prediction, a confidence score, a prediction score, etc.).

[0038] 2 shows a flowchart of an exemplary method 200. The method 200 may be performed by the data processing module 106A and / or the prediction module 106B using a neural network architecture. Some steps of the method 200 may be performed by the data processing module 106A and other steps may be performed by the prediction module 106B.

[0039] The neural network architecture used in the method 200 may include a neural network architecture. For example, the neural network architecture used in the method 200 may include multiple neural network blocks and / or layers that may be used to generate vectors / fingerprints (e.g., based on their attributes) for each of the data records 104 and the variables 105. As described herein, each attribute of each data record of the data records 104 may be associated with a corresponding neural network block, and each attribute of each variable of the variables 105 may be associated with a corresponding neural network block. A subset of the data records 104 and / or a subset of the attributes of each of the data records 104 may be used in place of each and every data record and / or attribute of the data records 104. If a subset of the data records 104 comprises one or more attribute types that do not have a corresponding neural network block, the data records associated with those one or more attribute types may be ignored by the method 200. In this manner, a given predictive model generated by computing device 106 may receive all of the data records 104, but only the subset of the data records 104 having corresponding neural network blocks may be used by method 200. As another example, even if all of the data records 104 have attribute types that each have a corresponding neural network block, the subset of the data records 104 may not be used by method 200. Determining which data records, attribute types, and / or corresponding neural network blocks are used by method 200 may be based on a selected set of hyper-parameters and / or a keyed dictionary / mapping between attribute types and corresponding neural network blocks, for example, as described further herein.

[0040] Method 200 may use multiple processors and / or multiple tokenizers. The multiple processors may convert attribute values, such as strings (e.g., sequences) of alphanumeric characters, words, phrases, symbols, etc., in each of data records 104 to corresponding numeric representations. The multiple tokenizers may convert attribute values, such as strings (e.g., sequences) of alphanumeric characters, words, phrases, symbols, etc., in each of variables 105 to corresponding numeric representations. For ease of explanation, tokenizers may be referred to herein as "processors." In some embodiments, multiple processors may not be used by method 200. For example, multiple processors may not be used for any of data records 104 or variables 105 in numeric format.

[0041] As described herein, each of the plurality of data records 104 may include any type of attribute, such as a string (e.g., a sequence) of alphanumeric characters, words, phrases, symbols, and the like. For illustrative purposes, the method 200 is described herein and illustrated in FIG. 2 as processing two attributes for a data record, namely, attribute "D1" and attribute "DN", and two variable attributes, namely, attribute "V1" and attribute "VN". However, it should be understood that the method 200 may process any number of data record attributes and / or variable attributes. In step 202, the data processing module 106A may receive attributes D1 and DN, and variable attributes V1 and VN. Each of the attributes D1 and DN may be associated with a label, such as a binary label (e.g., "yes" / "no") and / or a percentage value (e.g., the label of label 107). Each of the variable attributes V1 and VN may be associated with a label (e.g., a binary label and / or a percentage value). The data processing module 106A may determine a numeric representation for each of the attributes D1 and DN and each of the variable attributes V1 and VN. The method 200 may use multiple processors and / or multiple tokenizers. The multiple processors may convert the attributes of the data records 104 (e.g., strings / sequences of alphanumeric characters, words, phrases, symbols, etc.) into corresponding numeric representations. The multiple tokenizers may convert the attributes of the variables 105 (e.g., strings / sequences of alphanumeric characters, words, phrases, symbols, etc.) into corresponding numeric representations. For ease of explanation, the tokenizers may be referred to herein as "processors." Method 200 is described herein and shown in Figure 2 as having four processors: a "D1 processor" for attribute D1, a "DN processor" for attribute DN, a "V1 processor" for variable attribute V1, and a "VN processor" for variable attribute VN. However, it should be understood that data processing module 106A may include and method 200 may use any number of processors / tokenizers.

[0042] Each of the processors shown in FIG. 2 may utilize multiple algorithms, such as conversion methods, to convert each of the attributes D1 and DN and each of the variable attributes V1 and VN into a corresponding numerical representation that can be processed by a corresponding neural network block at step 204. The corresponding numerical representation may include a one-dimensional integer representation, a multi-dimensional array representation, a combination thereof, and / or the like. Each of the attributes D1 and DN may be associated with a corresponding neural network block based on a corresponding data type and / or attribute value. As another example, each of the variable attributes V1 and VN may be associated with a corresponding neural network block based on a corresponding data type and / or attribute value.

[0043] FIG. 3A illustrates an exemplary processor for attribute D1 and / or attribute DN. As an example, a data record 104 processed according to method 200 may include grade records for multiple students, each of which may include multiple attributes having a "string" data type for class names and corresponding values ​​having a "string" data type for grades achieved in each class. The processor illustrated in FIG. 3A may convert each of the attributes D1 and DN into a corresponding numeric representation that may be processed by a corresponding neural network block. As illustrated in FIG. 3A, the processor may assign a numeric value of "1" to the "CHEMISTRY" class name for attribute D1. That is, the processor may determine a numeric representation for the string value of "CHEMISTRY" by using the integer value of "1". The processor may determine corresponding integer values ​​for all other class names associated with the data record into corresponding numeric representations. For example, the string value of "MATH" may be assigned an integer value of "2", the string value of "STATISTICS" may be assigned an integer value of "3", and so on. Also, as shown in FIG. 3A, the processor may assign a letter grade (e.g., string value) "A" a numeric value of "1." That is, the processor may determine a numeric representation for the string value of "A" by using the integer value of "1." The processor may determine corresponding integer values ​​of all other letter grades associated with the data record to corresponding numeric representations. For example, the letter grade "B" may be assigned an integer value of "2," and the letter grade "C" may be assigned an integer value of "3."

[0044] As shown in FIG. 3A, the numeric representation for attribute D1 may include a one-dimensional integer representation of "1121314253". The processor may generate the numeric representation for attribute D1 in an ordered manner, with a first position representing a first class (e.g., "CHEMISTRY") listed in attribute D1 and a second position representing a grade (e.g., "A") for the first class listed in attribute D1. The remaining positions may be similarly ordered. Additionally, the processor may generate the numeric representation for attribute D1 in another ordered manner, such as a list of pairs (integer position, integer grade, e.g., "11123"), as will be appreciated by those skilled in the art. As shown in FIG. 3B, the third position (e.g., integer value of "2") in "1121314253" may correspond to the class name "MATH" and the fourth position (e.g., integer value of "1") in "1121314253" may correspond to the letter grade "A". The processor may transform the attribute DN in a manner similar to that described herein with respect to data record attribute D1. For example, the attribute DN may include a one-dimensional integer representation of a grade for a student associated with the data record for another year (e.g., another school year).

[0045] As another example, a variable 105 processed according to the method 200 may be associated with multiple students. The variable 105 may include one or more attributes. For example, and for purposes of illustration, the one or more attributes may include multiple demographic attributes having a "string" data type with corresponding values ​​having a "string" and / or "integer" data type. The multiple demographic attributes may include, for example, AGE, STATE OF RESIDENCE, CITY OF SCHOOL, etc. Each of FIG. 4A illustrates an example processor for a variable attribute, such as variable attribute V1 or variable attribute VN. The processor illustrated in FIG. 4A may convert the variable attributes, which may include the demographic attribute of "STATE," into a corresponding numeric representation that may be processed by a corresponding neural network block. The processor may associate an integer value with each possible string value for the demographic attribute of "STATE." For example, as shown in Figure 4A, a string value of "AL" (e.g., Alabama) may be associated with an integer value of "01", a string value of "GA" (e.g., Georgia) may be associated with an integer value of "10", and a string value of "WY" (e.g., Wyoming) may be associated with an integer value of "50". As shown in Figure 4B, a processor may receive the variable attribute of "STATE:GA" and assign a numeric value of "10" (e.g., indicating Georgia). Each of the one or more attributes associated with the variable 105 may be processed in a similar manner by a processor corresponding to each particular attribute type (e.g., a processor for "CITY", a processor for "AGE", etc.).

[0046] As described herein, the data processing module 106A may include a data record encoder and a variable encoder. For purposes of explanation, the data processing module 106A and the method 200 are described herein and shown in FIG. 2 as having four encoders: a "D1 encoder" for attribute D1, a "DN encoder" for attribute DN, a "V1 encoder" for variable attribute V1, and a "VN encoder" for variable attribute VN. However, it should be understood that the data processing module 106A may include, and the method 200 may utilize, any number of encoders. Each of the encoders shown in FIG. 2 may be an encoder module as described herein, which may include blocks of a neural network utilized by the data processing module 106A and / or the prediction module 100. In step 206, each of the processors may output a corresponding numerical representation of the attribute associated with the data record 104 and the attribute associated with the variable 105. For example, the D1 processor may output a numeric representation for attribute D1 (e.g., "D1 Numeric Input" shown in FIG. 2), and the DN processor may output a numeric representation for attribute DN (e.g., "DN Numeric Input" shown in FIG. 2). The V1 processor may output a numeric representation for variable attribute V1 (e.g., "V1 Numeric Input" shown in FIG. 2), and the VN processor may output a numeric representation for variable attribute VN (e.g., "VN Numeric Input" shown in FIG. 2).

[0047] In step 208, the D1 encoder may receive a numeric representation of the attribute D1, and the DN encoder may receive a numeric representation of the attribute DN. The D1 encoder and the DN encoder shown in FIG. 2 may be configured to encode attributes having a particular data type (e.g., based on the data type of the attribute D1 and / or the attribute DN). Also in step 208, the V1 encoder may receive a numeric representation of the variable attribute V1, and the VN encoder may receive a numeric representation of the variable attribute VN. The V1 encoder and the VN encoder shown in FIG. 2 may be configured to encode variable attributes having a particular data type (e.g., based on the data type of the variable attribute V1 and / or the variable attribute VN).

[0048] In step 210, the D1 encoder may generate a vector for the attribute D1 based on a numeric representation of the attribute D1, and the DN encoder may generate a vector for the attribute DN based on a numeric representation of the attribute DN. Also in step 210, the V1 encoder may generate a vector for the variable attribute V1 based on a numeric representation of the variable attribute V1, and the VN encoder may generate a vector for the variable attribute VN based on a numeric representation of the variable attribute VN. The data processing module 106A may determine a plurality of features for the predictive model. The plurality of features may include one or more attributes of one or more of the data records 104 (e.g., D1 and DN). As another example, the plurality of features may include one or more attributes of one or more of the variables 105 (e.g., V1 and VN).

[0049] In step 212, the data processing module 106A may generate a concatenated vector. For example, the data processing module 106A may generate a concatenated vector based on a plurality of features of the above-mentioned predictive model (e.g., based on a vector for attribute D1, a vector for attribute DN, a vector for variable attribute V1, and / or a vector for variable attribute VN). The concatenated vector may indicate the above-mentioned labels (e.g., binary labels and / or percentage values) for each of D1, DN, V1, and VN.

[0050] In step 214, the data processing module 106A may provide the concatenated vector and / or the encoders D1, DN, V1, and VN to a final machine learning model component of the prediction module 106B. The final machine learning model component of the prediction module 106B may include a final neural network block and / or layer of the neural network architecture used in the method 200. The prediction module 106B may train the final machine learning model component and the encoders D1, DN, V1, and VN. For example, the prediction module 106B may train the final machine learning model component based on the concatenated vector generated in step 212. The prediction module 106B may also train each of the encoders shown in FIG. 2 based on the concatenated vector generated in step 212. For example, the data record may include a data type (e.g., string), and each of the attributes D1 and DN may include a corresponding attribute data type (e.g., string for class / letter grade). The D1 encoder and the DN encoder may be trained based on the data type and the corresponding attribute data type. Once trained, the D1 and DN encoders may be enabled to convert new / unseen data record attributes (e.g., grade records) into a corresponding numeric format and / or a corresponding vector representation (e.g., fingerprints). As another example, each of the variable attributes V1 and VN may include a data type (e.g., string). The V1 and VN encoders may be trained based on the data type. Once trained, the V1 and VN encoders may be enabled to convert new / unseen variable attributes (e.g., demographic attributes) into a corresponding numeric format and / or a corresponding vector representation (e.g., fingerprints).

[0051] In step 216, the prediction module 106B may output (e.g., store) the machine learning model (e.g., neural network architecture) used in the method 200, referred to herein as a "prediction model." Also in step 216, the prediction module 106B may output (e.g., store) the trained encoders D1, DN, V1, and VN. The prediction model and / or the trained encoder may be enabled to provide a variety of predictions and / or generative data analyses, such as providing binomial predictions, polynomial predictions, variational autoencoders, combinations thereof, and / or the like. The prediction model trained by the prediction module 106B may generate outputs, such as predictions, scores, combinations thereof, and / or the like. The output of the prediction model may include data types corresponding to the labels associated with D1, DN, V1, and VN (e.g., binary labels and / or percentage values). When training the prediction model, the prediction module 106B may minimize a loss function, which is described further herein. The output may include, for example, a number of dimensions corresponding to the number of dimensions associated with the labels used during training. As another example, the output may include a keyed dictionary of the output. A loss function may be used when training the predictive model, and one or more parameters of the predictive model may be adjusted to minimize the loss function using a minimization routine. Additionally, a fitting method may be used when training the predictive model. The fitting method may receive a dictionary having keys corresponding to data types associated with D1, DN, V1, and / or VN. The fitting method may also receive labels (e.g., binary labels and / or percentage values) associated with D1, DN, V1, and VN.

[0052] A predictive model trained according to method 200 may provide one or more of predictions or scores associated with the data record and / or associated attributes. As an example, the computing device 106 may receive a previously unseen data record (first data record) and a previously unseen plurality of variables (a plurality of first variables). The data processing module 106A may determine a numerical representation of one or more attributes associated with the first data record. For example, the data processing module 106A may determine a numerical representation for one or more attributes associated with the first data record in a manner similar to that described above for the data record attributes D1 and DN used to train the predictive model. The data processing module 106A may determine a numerical representation for each variable attribute of the plurality of first variables. For example, the data processing module 106A may determine a numerical representation for each variable attribute in a manner similar to that described above for the variable attributes V1 and VN used to train the predictive model. The data processing module 106A may use a plurality of first trained encoder modules to determine a vector for each of the one or more attributes associated with the first data record. For example, the data processing module 106A may use the above-mentioned trained encoders D1 and DN trained with a predictive model when determining vectors for the data record attributes D1 and DN. The data processing module 106A may use multiple first trained encoder modules to determine vectors for one or more attributes associated with a first data record based on a numeric representation for the data record.

[0053] The data processing module 106A may use a plurality of second trained encoder modules to determine a vector for each variable attribute of the plurality of first variables. For example, the data processing module 106A may use the above-mentioned trained encoders V1 and VN trained with a predictive model when determining a vector for each variable attribute of the plurality of first variables. The data processing module 106A may use a plurality of second trained encoder modules to determine a vector for each variable attribute of the plurality of first variables based on a numerical representation for each variable attribute.

[0054] The data processing module 106A may generate a concatenated vector based on the vector for one or more attributes associated with the first data record and the vector for each variable attribute of the plurality of first variables. The prediction module 106B may determine one or more of the predictions or scores associated with the first data record by using a prediction model trained according to the method 200 described above. The prediction module 106B may determine one or more of the predictions or scores associated with the first data record based on the concatenated vector. The score may indicate the likelihood that a first label applies to the first data record based on one or more attributes associated with the first data record and the variable attributes. For example, the first label may be a binary label of the label 107 comprising "Likely to Attend Ivy College" and "Not Likely to Attend an Ivy League College". The prediction may indicate the likelihood (e.g., a percentage) that a student associated with the first data record will attend an Ivy League college (e.g., a percentage indication that the first label "Likely to Attend Ivy College" applies).

[0055] As described herein, the prediction module 106B may determine one or more of the predictions or scores associated with the first data record based on the concatenated vector. The predictions and / or scores may be determined using one or more attributes associated with the first data record and one or more variables associated with the first data record (e.g., using all or less of the known data associated with the first data record). Continuing with the above example of grade records and demographic attributes, the predictions and / or scores may be determined using all of the grade records associated with the data record of a particular student (e.g., all grades) and all of the demographic attributes associated with that particular student. In other examples, the predictions and / or scores may be determined using less than all of the grade records and / or less than the demographic attributes. The prediction module 106B may determine a first prediction and / or a first score based on all of the attributes associated with the first data record and all of the variables associated with the first data record, and the prediction module 106B may determine a second prediction and / or a second score based on a portion of the attributes and / or variables associated with the first data record.

[0056] Although the functionality of the methods and systems is described herein using the example of grade records as data records 104 and demographic attributes as variables 105, it should be understood that the data records 104 and variables 105 are not limited to this example. The methods, systems, and deep learning models described herein, e.g., predictive models, systems 100, methods 200, may be configured to analyze any type of data records and any type of variables that can be expressed (e.g., are represented numerically) in a numerical manner. For example, the data records 104 and variables 105 may include one or more strings (e.g., sequences) of data, one or more integers of data, one or more characters of data, combinations thereof, and / or the like.

[0057] In addition to the grade records described herein, the data records 104 may include and / or relate to sales data, inventory data, genetic data, sports data, stock data, music data, weather data, or any other data that one skilled in the art would recognize as being numerically expressible (e.g., represented numerically). Furthermore, in addition to the demographic attributes described herein, the variables 105 may include and / or relate to product data, company data, biological data, statistical data, market data, equipment data, geological data, or any other data that one skilled in the art would recognize as being numerically expressible (e.g., represented numerically). Furthermore, in addition to the binary labels described above with respect to the grade record examples (e.g., "Likely to Attend Ivy College" vs. "Not Likely to Attend an Ivy League College"), the labels described herein may include percentage values, one or more attributes associated with the corresponding data records and / or variables, one or more values ​​for one or more attributes, or any other label as one skilled in the art would recognize.

[0058] As described further herein, during the training phase, one or more attributes (e.g., values) of the data records 104 and variables 105 may be processed by a deep learning model (e.g., a predictive model) described herein to determine how each correlates, respectively and in combination with other attributes, to a corresponding label. Following the training phase, a deep learning model (e.g., a trained predictive model) described herein may receive new / unseen data records and associated variables and determine whether a label applies to the new / unseen data records and associated variables.

[0059] Referring now to FIG. 5, an exemplary method 500 is illustrated. The method 500 may be performed by the prediction module 106B described herein. The prediction module 106B may be configured to train at least one machine learning ML module 530 configured to provide one or more predictions or scores associated with data records and one or more corresponding variables based on analysis of one or more training datasets 510 by a training module 520 using machine learning (ML) techniques. The prediction module 106B may be configured to train and configure the machine learning ML module 530 using one or more hyperparameters 505 and a model architecture 503. The model architecture 503 may include a predictive model output at step 216 of the method 200 (e.g., a neural network architecture used in the method 200). The hyperparameters 505 may include the number of neural network layers / blocks, the number of neural network filters in a neural network layer, etc. Each set of hyperparameters 505 may be used to construct the model architecture 503, and the elements of each set of hyperparameters 505 may include a number of inputs (e.g., data record attributes / variables) included in the model architecture 503. For example, continuing with the above example of grade records and demographic attributes, the elements of a first set of hyperparameters 505 may include all grade records (e.g., data record attributes) associated with the data records of a particular student (e.g., all grades), and / or all demographic attributes (e.g., variable attributes) associated with that particular student. The elements of a second set of hyperparameters 505 may include grade records (e.g., data record attributes) for only one grade for a particular student, and / or demographic attributes (e.g., variable attributes) associated with that particular student.In other words, an element of each set of hyperparameters 505 may indicate that as little as one or as many as all of the corresponding attributes of the data records and variables are used to construct the model architecture 503 that is used to train the machine learning ML module 530.

[0060] The training data set 510 may include one or more input data records (e.g., data records 104) and one or more input variables (e.g., variables 105) associated with one or more labels 107 (e.g., binary labels ("yes" / "no") and / or percentage values). The label of a given record and / or a given variable may indicate the likelihood that the label applies to the given record. Combining one or more data records 104 and one or more variables 105 may result in the training data set 510. A subset of the data records 104 and / or variables 105 may be randomly assigned to the training data set 510 or the test data set. In some implementations, the assignment of data to the training data set or the test data set may not be completely random. In this case, one or more criteria may be used during the assignment. In general, any suitable method may be used to assign data to the training data set or the test data set. Meanwhile, it may be ensured that the distribution of the labels "yes" and "no" is somewhat similar in the training data set and the test data set.

[0061] The training module 520 may train the machine learning ML module 530 by extracting a feature set from a plurality of data records (e.g., labeled as "yes") in the training dataset 510 by one or more feature selection techniques. The training module 520 may train the machine learning ML module 530 by extracting a feature set from the training dataset 510 that comprises statistically significant features of the positive examples (e.g., labeled as "yes") and statistically significant features of the negative examples (e.g., labeled as "no").

[0062] The training module 520 may extract feature sets from the training dataset 510 in a variety of ways. The training module 520 may perform feature extraction multiple times, using a different feature extraction technique each time. In one embodiment, feature sets generated using different techniques may be used to generate different machine learning based classification models 540A-N, each of which may be used to generate a different machine learning based classification model 540A-N. For example, the feature set with the highest quality metric may be selected for use in training. The training module 520 may use the feature sets to build one or more machine learning based classification models 540A-N that are configured to indicate whether a particular label applies to a new / unseen data record based on its corresponding one or more variables.

[0063] The training dataset 510 may be analyzed to determine any dependencies, associations, and / or correlations between features and “yes” / “no” labels in the training dataset 510. The identified correlations may have the form of a list of features associated with different “yes” / “no” labels. As used herein, the term “feature” may refer to any feature of an item of data that may be used to determine whether an item of data exists within one or more particular categories. The feature selection technique may include one or more feature selection rules. The one or more feature selection rules may include feature generation rules. The feature generation rules may include determining which features in the training dataset 510 occur a threshold number of times and identifying those features that meet the threshold as candidate features.

[0064] A single feature selection rule may be applied to select features, or multiple feature selection rules may be applied to select features. Feature selection rules may be applied in a cascading manner, where feature selection rules are applied in a particular order and are applied to the results of previous rules. For example, feature generation rules may be applied to the training dataset 510 to generate a first list of features. The final list of candidate features may be analyzed by additional feature selection techniques to determine one or more candidate feature sets (e.g., a set of features that can be used to predict whether a label applies or does not apply). Any suitable computing technique may be used to identify the candidate feature sets using any feature selection technique, such as a filter method, a wrapper method, and / or an embedding method. The one or more candidate feature sets may be selected according to a filter method. Filter methods include, for example, Pearson's correlation, linear discriminant analysis, analysis of variance (ANOVA), chi-square, combinations thereof, and the like. The selection of features according to a filter method is independent of any machine learning algorithm. Alternatively, features may be selected based on their scores in various statistical tests for correlation with outcome variables (e.g., "yes" / "no").

[0065] As another example, the one or more candidate feature sets may be selected according to a wrapper method. The wrapper method may use a subset of features and may be configured to train the machine learning model using the subset of features. Features may be added and / or removed from the subset based on inferences drawn from previous models. The wrapper method may comprise, for example, forward feature selection, backward feature reduction, recursive feature reduction, combinations thereof, and the like. As an example, the one or more candidate feature sets may be identified using forward feature selection. Forward feature selection is an iterative method that starts with no features in the machine learning model. In each iteration, the feature that best improves the model is added until the addition of a new variable does not improve the performance of the machine learning model. As an example, the one or more candidate feature sets may be identified using backward reduction. Backward reduction is an iterative method that starts with all features in the machine learning model. In each iteration, the lowest ranking feature is removed until no improvement is observed upon removal of the feature. Recursive feature reduction may be used to identify the one or more candidate feature sets. Recursive feature reduction is a greedy optimization algorithm that aims to find the best performing feature subset. With recursive feature reduction, a model is built iteratively, setting aside the best or worst performing features at each iteration. Recursive feature reduction builds the next model with the remaining features until all features are exhausted. Recursive feature reduction then ranks the features based on their order of reduction.

[0066] As a further example, one or more candidate feature sets may be selected by an embedding method that combines the qualities of the filter and wrapper methods. Embedding methods include, for example, least absolute shrinkage and selection operator (LASSO) and ridge regression, which implement a penalty function to reduce overfitting. For example, LASSO regression performs L1 regularization that adds a penalty equivalent to the absolute value of the coefficient magnitude, and ridge regression performs L2 regularization that adds a penalty equivalent to the square of the coefficient magnitude.

[0067] After the training module 520 generates the feature set, the training module 520 may generate one or more machine learning based classification models 540A-540N based on the feature set. A machine learning based classification model may refer to a complex mathematical model for data classification generated using machine learning techniques. In one example, the machine learning based classification model 740 may include a map of support vectors representing boundary features. In this example, the boundary features may be selected from and / or represent the highest ranked features in a feature set.

[0068] The training module 420 may use the feature set extracted from the training dataset 510 to construct one or more machine learning based classification models 540A-540N for each classification category (e.g., “yes”, “no”). In some embodiments, the machine learning based classification models 540A-540N may be combined into a single machine learning based classification model 740. Similarly, the machine learning ML module 530 may represent a single classifier containing single or multiple machine learning based classification models 740 and / or multiple classifiers containing single or multiple machine learning based classification models 740.

[0069] The extracted features (e.g., one or more candidate features) may be combined in a trained classification model using machine learning approaches, such as discriminant analysis, decision trees, nearest neighbor (NN) algorithms (e.g., k-NN model, replicator NN model, etc.), statistical algorithms (e.g., Bayesian networks, etc.), clustering algorithms (e.g., k-means, mean shift, etc.), neural networks (e.g., reservoir networks, artificial neural networks, etc.), support vector machines (SVMs), logistic regression algorithms, linear regression algorithms, Markov models or chains, principal component analysis (PCA) (e.g., for linear models), multilayer perceptron (MLP) ANN (e.g., for nonlinear models), replicating reservoir networks (e.g., for nonlinear models, typically for time series), random forest classification, combinations thereof, and / or the like. The resulting machine learning ML module 530 may include a decision rule or mapping for each candidate feature.

[0070] In one embodiment, the training module 520 may train the machine learning based classification model 740 as a convolutional neural network (CNN). The CNN may include at least one convolutional feature layer and three fully connected layers leading to a final classification layer (softmax), which may be finally applied to combine the outputs of the fully connected layers using a softmax function as known in the art.

[0071] Using the candidate features and the machine learning ML module 530, the results for each data record in the test dataset may be used to predict whether a label (e.g., attended an Ivy League college) applies to a data record in the test dataset. In one embodiment, the results for each data record in the test dataset include a confidence level corresponding to the likelihood or probability that one or more corresponding variables (e.g., demographic attributes) indicate the label that applies to the data record in the test dataset. The confidence level may be a value between 0 and 1 and may represent the likelihood that a data record in the test dataset belongs to a "yes" / "no" status with respect to one or more corresponding variables (e.g., demographic attributes). In one embodiment, when there are two statuses (e.g., "yes" and "no"), the confidence level may correspond to a value p, which refers to the likelihood that a particular data record in the test dataset belongs to the first status (e.g., "yes"). In this case, the value 1-p may refer to the likelihood that a particular data record in the test dataset belongs to the second status (e.g., "no"). In general, multiple confidence levels may be provided for each data record in the test dataset and for each candidate feature when there are three or more labels. The best performing candidate feature may be determined by comparing the results obtained for each data record with the known "yes" / "no" labels for each data record. In general, the best performing candidate feature will have results that closely match the known "yes" / "no" labels. The best performing candidate feature may be used to predict the "yes" / "no" labels of the data record with respect to one or more corresponding variables. For example, a new data record may be determined / received. The new data record may be provided to the machine learning ML module 530, which may classify a label as either applying to the new data record or not applying to the new data record based on the best performing candidate feature.

[0072] 6, a flow chart illustrating an example training method 600 for generating a machine learning ML module 530 using a training module 520 is shown. The training module 520 can implement supervised, unsupervised, and / or semi-supervised (e.g., reinforcement-based) machine learning-based classification models 540A-740N. The training module 520 may include a data processing module 106A and / or a prediction module 106B. The method 600 illustrated in FIG. 6 is one example of a supervised learning method, and variations of this example of the training method are discussed below, although other training methods may be similarly implemented to train unsupervised and / or semi-supervised machine learning models.

[0073] The training method 600 may determine (e.g., access, receive, retrieve, etc.) a first data record processed by the data processing module 106A in step 610. The first data record may include a set of labeled data records, such as data record 104. The label may correspond to a label (e.g., "yes" or "no") and one or more corresponding variables, such as one or more variables 105. The training method 600 may generate a training data set and a test data set in step 620. The training data set and the test data set may be generated by randomly assigning the labeled data records to either the training data set or the test data set. In some implementations, the assignment of the labeled data records as training or test samples may not be completely random. As an example, a majority of the labeled data records may be used to generate the training data set. For example, 55% of the labeled data records may be used to generate the training data set and 25% may be used to generate the test data set.

[0074] The training method 600 may train one or more machine learning models in step 630. In one embodiment, the machine learning models may be trained using supervised learning. In another embodiment, other machine learning techniques, including unsupervised learning and semi-supervised, may be used. The machine learning models trained in step 630 may be selected based on different criteria depending on the problem to be solved and / or the data available in the training dataset. For example, machine learning classifiers may be subject to different degrees of bias. Thus, more than one machine learning model may be trained in step 630 and optimized, refined, and cross-validated in step 640.

[0075] For example, the loss function may be used in training the machine learning model in step 630. The loss function may take the true label and the predicted output as input, and the loss function may generate a single numerical output. To minimize the loss, one or more minimization techniques may be applied to some or all of the learnable parameters of the machine learning model (e.g., one or more learnable neural network parameters). For example, one or more minimization techniques may not be applied to one or more learnable parameters of the trained encoder module, neural network blocks, neural network layers, etc. This process may be applied continuously until some stopping condition is met, for example, a certain number of training dataset repeats and / or a certain number of iterations, until the loss level of the pull-out validation set stops decreasing. In addition to adjusting these learnable parameters, one or more of the hyperparameters 505 that define the model architecture 503 of the machine learning model may be selected. The one or more hyperparameters 505 may include the number of neural network layers, the number of neural network filters in the neural network layers, etc. For example, as discussed above, each set of hyperparameters 505 may be used to construct the model architecture 503, and the elements of each set of hyperparameters 505 may include the number of inputs (e.g., data record attributes / variables) to include in the model architecture 503. The elements of each set of hyperparameters 505 comprising the number of inputs may be considered "features" as described herein with respect to method 200. That is, the cross-validation and optimization performed in step 640 may be considered a feature selection process. For example, continuing with the above example of grade records and demographic attributes, the elements of the first set of hyperparameters 505 may include all grade records (e.g., data record attributes) associated with the data records for a particular student (e.g., all grades) and / or all demographic attributes (e.g., variable attributes) associated with the particular student.The elements of the second set of hyperparameters 505 may include grade records (e.g., data record attributes) for only one grade for a particular student, and / or demographic attributes (e.g., variable attributes) associated with that particular student. To select the best hyperparameters 505, in step 640, the machine learning model may be optimized by training the machine learning model using some portion of the training data (e.g., based on the elements of each set of hyperparameters 505 with the number of inputs to the model architecture 503). The optimization may be stopped based on a pull-out validation portion of the training data. Cross-validation may be performed using the remaining training data. This process may be repeated a certain number of times, and the machine learning model may be evaluated for a particular level of performance each time and for each set of hyperparameters 505 selected (e.g., based on the number of inputs chosen and the particular inputs).

[0076] The best set of hyperparameters 505 may be selected by picking one or more of the hyperparameters 505 that have the best average evaluation of the "splits" of the training data. The cross-validation object may be used to provide a function that creates a new randomly initialized iteration of the method 200 described herein. This function may be called for each new data split and each new set of hyperparameters 505. The cross-validation routine may determine the type of data present in the input (e.g., attribute type), and a selected amount of the data (e.g., a certain number of attributes) may be split and used as a validation data set. The type of data split may be chosen to partition the data a selected number of times. For each data partition, a set of hyperparameters 505 may be used, and a new machine learning model with a new model architecture 503 may be initialized and trained based on the set of hyperparameters 505. After each training iteration, the machine learning model may be evaluated on the test portion of the data for that particular split. The evaluation may return a single numeric value, which may depend on the output of the machine learning model and the true output label. The evaluation for each split and hyper-parameter set may be stored in a table, and this table may be used to select an optimal set of hyper-parameters 505. The optimal set of hyper-parameters 505 may include one or more of the hyper-parameters 505 that have the highest average evaluation score across all splits.

[0077] The training method 600 may select one or more machine learning models to build a predictive model at step 650. The predictive model may be evaluated using a test dataset. The predictive model may analyze the test dataset and generate one or more of a prediction or score at step 660. The one or more predictions and / or scores may be evaluated at step 670 to determine if a desired accuracy level has been achieved. The performance of a predictive model may be evaluated in a number of ways based on classification of a number of data points represented by the predictive model as true positives, false positives, true negatives, and / or false negatives.

[0078] For example, a false positive of a predictive model may refer to the number of times the predictive model incorrectly classifies a label as applying to a given data record when in reality the label does not apply. Conversely, a false negative of a predictive model may refer to the number of times the machine learning model indicates a label as not applying when in fact the label does apply. True negatives and true positives may refer to the number of times the predictive model correctly classifies one or more labels as applying or not applying. Related to these measurements are the concepts of recall and precision. In general, recall refers to the ratio of true positives to the sum of true positives and false negatives, which quantifies the sensitivity of the predictive model. Similarly, precision refers to the ratio of true positives to the sum of true positives and false positives. When such a desired accuracy level is reached, the training phase ends and the predictive model (e.g., machine learning ML module 530) may be output in step 680. However, when the desired accuracy level is not reached, subsequent iterations of the training method 600 may be initiated and performed in step 610, with variations such as to account for a larger collection of data records.

[0079] FIG. 7 is a block diagram depicting an environment 700 comprising a non-limiting example of a computing device 701 (e.g., computing device 106) and a server 702 connected to each other through a network 704. In one aspect, some or all steps of any method described herein may be performed by the computing device 701 and / or the server 702. The computing device 701 may include one or more computers configured to store one or more of the data records 104, the training data 510 (e.g., labeled data records), the data processing module 106A, the prediction module 106B, etc. The server 702 may comprise one or more computers configured to store the data records 104. The multiple servers 702 may communicate to the computing device 701 through the network 704. In one embodiment, the computing device 701 may include a repository for training data 711 generated by the methods described herein.

[0080] In terms of hardware architecture, the computing device 701 and the server 702 may be digital computers that generally include a processor 708, a memory system 710, an input / output (I / O) interface 712, and a network interface 714. These components (908, 710, 712, and 714) are communicatively coupled via a local interface 716. The local interface 716 may be, for example, but not limited to, one or more buses or other wired or wireless connections known in the art. The local interface 716 may have additional elements to enable communication, such as controllers, buffers (caches), drivers, repeaters, and receivers, which are omitted for simplicity. Additionally, the local interface may include address, control, and / or data connections to enable appropriate communication between the aforementioned components.

[0081] The processor 708 may be a hardware device for executing software, particularly stored in the memory system 710. The processor 708 may be any custom-made or commercially available processor, a central processing unit (CPU), an auxiliary processor among several processors associated with the computing device 701 and the server 702, a semiconductor-based microprocessor (in the form of a microchip or chipset), or generally any device for executing software instructions. When the computing device 701 and / or the server 702 are in operation, the processor 708 may be configured to execute software stored in the memory system 710 to communicate data to and from the memory system 710 and generally control the operation of the computing device 701 and the server 702 in accordance with the software.

[0082] The I / O interface 712 can be used to receive user input from and / or provide system output to one or more devices or components. User input can be provided, for example, via a keyboard and / or a mouse. System output can be provided via a display device and a printer (not shown). The I / O interface 792 can include, for example, a serial port, a parallel port, a small computer system interface (SCSI), an infrared (IR) interface, a radio frequency (RF) interface, and / or a universal serial bus (USB) interface.

[0083] The network interface 714 may be used to transmit and receive from the computing device 701 and / or the server 702 over the network 704. The network interface 714 may include, for example, a 10BaseT Ethernet adapter, a 100BaseT Ethernet adapter, a LAN PHY Ethernet adapter, a Token Ring adapter, a wireless network adapter (e.g., WiFi, cellular, satellite), or any other suitable network interface device. The network interface 714 may include address, control, and / or data connections to enable appropriate communication over the network 704.

[0084] Memory system 710 can include any one or combination of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, etc.)) and non-volatile memory elements (e.g., ROM, hard drives, tape, CD-ROM, DVD-ROM, etc.). Additionally, memory system 710 may incorporate electronic, magnetic, optical, and / or other types of storage media. Note that memory system 710 can have a distributed architecture in which various components are located remotely from one another but can be accessed by processor 708.

[0085] The software in the memory system 710 may include one or more software programs, each of which comprises an ordered list of executable instructions for implementing a logical function. In the example of FIG. 7, the software in the memory system 710 of the computing device 701 may comprise training data 711, a training module 720 (e.g., prediction module 106B), and a suitable operating system (O / S) 718. In the example of FIG. 7, the software in the memory system 710 of the server 702 may comprise data records and variables 724 (e.g., data records 104 and variables 105), and a suitable operating system (O / S) 718. The operating system 718 essentially controls the execution of other computer programs and provides scheduling, input / output control, file and data management, memory management, and communication control, and related services.

[0086] For purposes of illustration, application programs and other executable program components, such as the operating system 718, are illustrated herein as separate blocks, with the understanding that such programs and components may reside at various times in different storage components of the computing device 701 and / or the server 702. An implementation of the training module 520 may be stored on or transmitted to some form of computer-readable medium. Any of the methods of the present disclosure may be performed by computer-readable instructions embodied on a computer-readable medium. A computer-readable medium may be any available medium that can be accessed by a computer. By way of example, and not intended to be limiting, computer-readable media may comprise "computer storage media" and "communications media." A "computer storage medium" may comprise volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Exemplary computer storage media can comprise RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disks (DVD) or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer.

[0087] 8, a flowchart of an exemplary method 800 for generating, training, and outputting an improved deep learning model is shown. Unlike existing deep learning models and frameworks designed to be problem / analysis specific, the framework implemented by method 800 may be made applicable to a wide range of predictive and / or generative data analyses. Method 800 may be performed in whole or in part by a single computing device, multiple computing devices, etc. For example, computing device 106, training module 520, server 702, and / or computing device 704 may be configured to perform method 800.

[0088] In step 810, the computing device may receive a plurality of data records and a plurality of variables. Each of the plurality of data records and each of the plurality of variables may include one or more attributes. Each data record of the plurality of data records may be associated with one or more variables of the plurality of variables. The computing device may determine a plurality of features for the model architecture to train a predictive model as described herein. The computing device may determine a plurality of features based on, for example, a hyperparameter set (e.g., a set of hyperparameters 505). The hyperparameter set may include a number of neural network layers / blocks, a number of neural network filters in a neural network layer, etc. The elements of the hyperparameter set may include a first subset of the plurality of data records (e.g., data record attributes / variables) included in the model architecture and for training a predictive model as described herein. For example, continuing with the example described herein with respect to grade records and demographic attributes, the elements of the hyperparameter set may include all grade records (e.g., data record attributes) associated with the data records for a particular student (e.g., all grades). Other implementations are enabled for the first subset of data records. Another element of the hyper-parameter set may include a first subset of variables (e.g., attributes) for inclusion in the model architecture and for training the predictive model. For example, the first subset of variables may include one or more demographic attributes (e.g., age, state, etc.) described herein. Other implementations are enabled for the first subset of data variables. In step 820, the computing device may determine a numerical representation for each attribute associated with each data record in the first subset of data records.Each attribute associated with each data record of the first subset of data records may be associated with a label, such as a binary label (e.g., yes / no) and / or a percentage value. In step 830, the computing device may determine a numerical representation for each attribute associated with each variable of the first subset of variables. Each attribute associated with each variable of the first subset of variables may be associated with a label (e.g., a binary label and / or a percentage value).

[0089] The computing device may use multiple processors and / or tokenizers when determining a numeric representation for each attribute associated with each variable of the first subset of variables that is not in a numeric format (e.g., a string). For example, determining a numeric representation for each attribute associated with each variable of the first subset of variables may include determining, by the multiple processors and / or tokenizers, a token for each attribute associated with each variable of the first subset of variables. Each respective token may be used to determine a numeric representation for each attribute associated with each variable of the first subset of variables. One or more attributes associated with one or more variables of the first subset of variables may include at least a non-numeric portion, and each may include a numeric representation for at least the non-numeric portion. Thus, in some implementations, a numeric representation for at least the non-numeric portion of each attribute associated with a respective variable may be used to determine a numeric representation for that attribute.

[0090] In step 840, the computing device may generate a vector for each attribute of each data record in the first subset of the data records. For example, the first encoder modules may generate a vector for each attribute of each data record in the first subset of the data records. The first encoder modules may generate a vector for each attribute of each data record in the first subset of the data records based on a numeric representation for each data record in the first subset of the data records.

[0091] In step 850, the computing device may generate a vector for each attribute of each variable in the first subset of variables. For example, the second encoder modules may generate a vector for each attribute of each variable in the first subset of variables. The second encoder modules may generate a vector for each attribute of each variable in the first subset of variables based on a numeric representation for each variable in the first subset of variables.

[0092] At step 860, the computing device may generate a concatenation vector. For example, the computing device may generate the concatenation vector based on a vector for each attribute of each data record of the first subset of data records. As another example, the computing device may generate the concatenation vector based on a vector for each attribute of each variable of the first subset of variables. The concatenation vector may indicate a label. For example, the concatenation vector may indicate a label (e.g., a binary label and / or a percentage value) associated with each attribute of each data record of the first subset of data records. As another example, the concatenation vector may indicate a label (e.g., a binary label and / or a percentage value) for each variable of the first subset of variables. As discussed above, the plurality of features (e.g., based on a hyper-parameter set) may include as little as one or as many as all of the corresponding attributes of the data records of the first subset of data records and the variables of the first subset of variables. Thus, the concatenation vector may be based on as little as one or as many as all of the data records of the first subset of the plurality of data records and corresponding attributes of the variables of the first subset of the plurality of variables.

[0093] In step 870, the computing device may train a model architecture based on the concatenated vector. For example, the computing device may train a predictive model, a plurality of first encoder modules, and / or a plurality of second encoder modules based on the concatenated vector. In step 880, the computing device may output (e.g., store) the model architecture as a trained predictive model, a plurality of first trained encoder modules, and / or a plurality of second trained encoder modules. The plurality of first trained encoder modules may include a plurality of first neural network blocks, and the plurality of second trained encoder modules may include a plurality of second neural network blocks. The plurality of first trained encoder modules may include one or more parameters (e.g., hyperparameters) for the plurality of first neural network blocks based on each attribute of each data record of the first subset of the plurality of data records (e.g., based on attributes of each data record). The plurality of second trained encoder modules may include one or more parameters (e.g., hyperparameters) for the plurality of second neural network blocks based on each variable of the first subset of the plurality of variables (e.g., based on attributes of each variable). The computing device may optimize the predictive model based on the second subset of the data records, the second subset of the variables, and / or a cross-validation technique using a hyper-parameter set as described herein with respect to step 650 of method 600.

[0094] 9, a flowchart of an exemplary method 900 for using deep learning models is shown. Unlike existing deep learning models and frameworks designed to be problem / analysis specific, the framework implemented by method 900 may be made applicable to a wide range of predictive and / or generative data analyses. Method 900 may be performed in whole or in part by a single computing device, multiple electronic devices, and the like. For example, computing device 106, training module 520, server 702, and / or computing device 704 may be configured to perform method 900.

[0095] A model architecture comprising a trained predictive model, a plurality of first encoder modules, and / or a plurality of second encoder modules may be used by a computing device to provide one or more of a score or prediction associated with a previously unseen data record and a plurality of previously unseen variables. The model architecture may have been previously trained based on a number of features, such as a hyperparameter set (e.g., a set of hyperparameters 505). The hyperparameter set may include a number of neural network layers / blocks, a number of neural network filters in a neural network layer, etc. For example, continuing with the example described herein with respect to grade records and demographic attributes, an element of the hyperparameter set may include all grade records (e.g., attributes of the data records) associated with a data record for a particular student (e.g., all grades). Other examples are also possible. Another element of the hyperparameter set may include one or more demographic attributes (e.g., age, state, etc.) described herein. Other examples are also possible.

[0096] At step 910, the computing device may receive a data record and a plurality of variables. Each of the data record and the plurality of variables may each include one or more attributes. The data record may be associated with one or more variables of the plurality of variables. At step 920, the computing device may determine a numerical representation for one or more attributes associated with the data record. For example, the computing device may determine a numerical representation for each of the one or more attributes associated with the data record in a manner similar to that described herein with respect to step 206 of method 200. At step 930, the computing device may determine a numerical representation for each of the one or more attributes associated with each variable of the plurality of variables. For example, the computing device may determine a numerical representation for each of the one or more attributes associated with each of the plurality of variables in a manner similar to that described herein with respect to step 206 of method 200. The computing device may use multiple processors and / or tokenizers when determining a numerical representation for each of the one or more attributes associated with each variable of the plurality of variables. For example, determining a numeric representation for each of the one or more attributes associated with each variable of the plurality of variables may include determining, by a plurality of processors and / or a tokenizer, a token for each of the one or more attributes associated with each variable of the plurality of variables. Each respective token may be used to determine a numeric representation for each of the one or more attributes associated with each variable of the plurality of variables. Each of the one or more attributes associated with each variable of the plurality of variables may include at least a non-numeric portion, and each token may include a numeric representation for at least the non-numeric portion. Thus, in some examples, a numeric representation for at least the non-numeric portion of each attribute associated with a respective variable may be used to determine a numeric representation for that attribute.

[0097] In step 940, the computing device may generate a vector for each of the one or more attributes associated with the data record. For example, the computing device may determine a vector for each of the one or more attributes associated with the data record by using a plurality of first trained encoder modules. The computing device may determine a vector for each of the one or more attributes associated with the data record based on a numerical representation for each of the one or more attributes associated with the data record by using a plurality of first trained encoder modules. In step 950, the computing device may generate a vector for each of the one or more attributes associated with each of the plurality of variables. For example, the computing device may determine a vector for each attribute of each variable of the plurality of variables by using a plurality of second trained encoder modules. The computing device may determine a vector for each attribute of each variable of the plurality of first variables based on a numerical representation for each of the one or more attributes associated with each variable of the plurality of variables by using a plurality of second trained encoder modules. The plurality of first trained encoder modules may include a plurality of first neural network blocks, and the plurality of second trained encoder modules may include a plurality of second neural network blocks. The first plurality of trained encoder modules may include one or more parameters for the first plurality of neural network blocks based on each attribute of each data record of the plurality of data records (e.g., based on attributes of each data record), and the second plurality of trained encoder modules may include one or more parameters for the second plurality of neural network blocks based on each variable of the plurality of variables (e.g., based on attributes of each variable).

[0098] In step 960, the computing device may generate a concatenated vector. For example, the computing device may generate the concatenated vector based on a vector for each of one or more attributes associated with the data record and a vector for each attribute of each variable of the plurality of variables. In step 970, the computing device may determine one or more of the predictions or scores associated with the data record and the plurality of variables. For example, the computing device may determine one or more of the predictions or scores associated with the data record and the plurality of variables by using a trained predictive model of the model architecture. The trained predictive model may include the model architecture described above in method 800. The trained predictive model may determine one or more of the predictions or scores associated with the data record and the plurality of variables based on the concatenated vector. The score may indicate the likelihood that the first label applies to the data record and / or the plurality of variables. For example, the first label may include a binary label (e.g., “yes” / “no”) and / or a percentage value.

[0099] 10, a flowchart of an exemplary method 1000 for retraining a model architecture comprising a trained predictive model (e.g., a trained deep learning model) is shown. Unlike existing deep learning models and frameworks designed to be problem / analysis specific, the framework implemented by the method 1000 may be made applicable to a wide range of predictive and / or generative data analytics. The method 1000 may be performed in whole or in part by a single computing device, multiple electronic devices, and the like. For example, the computing device 106, the training module 520, the server 702, and / or the computing device 704 may be configured to perform the method 1000.

[0100] As described herein, the model architecture comprising the trained predictive model and the trained encoder module may be enabled to provide a variety of predictions and / or generated data analyses. The model architecture comprising the trained predictive model and the trained encoder module may be initially trained to provide a first set of predictions and / or generated data analyses, and may each be retrained according to method 1000 to provide another set of predictions and / or generated data analyses. For example, the model architecture may have been previously trained based on a number of features, such as a hyperparameter set (e.g., a set of hyperparameters 505). The hyperparameter set may include the number of neural network layers / blocks, the number of neural network filters in the neural network layer, etc. For example, continuing with the example described herein with respect to grade records and demographic attributes, an element of the hyperparameter set may include all grade records (e.g., attributes of the data records) associated with the data records of a particular student (e.g., all grades). Other examples are also enabled. Another element of the hyperparameter set may include one or more demographic attributes (e.g., age, state, etc.) described herein. Other examples are also enabled. The model architecture may be retrained according to a different hyper-parameter set and / or different elements of the hyper-parameter set.

[0101] In step 1010, the computing device may receive a plurality of first data records and a plurality of first variables. Each of the plurality of first data records and the plurality of first variables may comprise one or more attributes or be associated with a label. In step 1020, the computing device may determine a numerical representation for each attribute of each data record of the plurality of first data records. In step 1030, the computing device may determine a numerical representation for each attribute of each variable of the plurality of first variables. In step 1040, the computing device may generate a vector for each attribute of each data record of the plurality of first data records. For example, the computing device may generate a vector for each attribute of each data record of the plurality of first data records by using a plurality of first trained encoder modules. Each of the vectors for each attribute of each data record of the plurality of first data records may be based on a corresponding numerical representation for each attribute of each data record of the plurality of first data records. The plurality of first trained encoder modules may have been previously trained based on a plurality of training data records associated with a label and a first hyper-parameter set. The first trained encoder modules may include first parameters (e.g., hyperparameters) for the neural network blocks based on respective attributes of each data record of the training data records. The first data records may be associated with a second hyperparameter set that is at least partially different from the first hyperparameter set. For example, the first hyperparameter set may be a first year grade record of a class, and the second hyperparameter set may be a second year grade record of a class.

[0102] In step 1050, the computing device may generate a vector for each attribute of each variable of the plurality of first variables. For example, the computing device may generate a vector for each attribute of each variable of the plurality of first variables by using a plurality of second trained encoder modules. Each of the vectors for each attribute of each variable of the plurality of first variables may be based on a corresponding numerical representation for each attribute of each variable of the plurality of first variables. The plurality of second trained encoder modules may have been previously trained based on a label and a plurality of training data records associated with the first hyper-parameter set. The plurality of first variables may be associated with a second hyper-parameter set.

[0103] In step 1060, the computing device may generate a concatenation vector. For example, the computing device may generate the concatenation vector based on a vector for each attribute of each data record of the plurality of first data records. As another example, the computing device may generate the concatenation vector based on a vector for each attribute of each variable of the plurality of first variables. In step 1060A, the computing device may retrain the model architecture. For example, the computing device may retrain the model architecture based on the concatenation vector, which may be generated in step 1060 based on another hyper-parameter set and / or another element of the hyper-parameter set. The computing device may also retrain the plurality of first encoder modules and / or the plurality of second encoder modules based on the concatenation vector (e.g., based on another set of hyper-parameters and / or another element of the hyper-parameter set). Once retrained, the plurality of first encoder modules may include a plurality of second parameters (e.g., hyper-parameters) for the plurality of neural network blocks based on each attribute of each data record of the plurality of first data records. The second encoder modules, once retrained, may include a plurality of second parameters (e.g., hyperparameters) for the neural network blocks based on respective attributes of each data record of the first variables. Once retrained, the model architecture may provide another set of predictions and / or generative data analyses. The computing device may output (e.g., save) the retrained model architecture.

[0104] Although specific configurations have been described, the configurations herein are not intended to be limiting and are in all respects possible configurations, and therefore it is not intended to limit the scope to the specific configurations described. Unless otherwise expressly stated, it is in no way intended that any method described herein should be interpreted as requiring its steps to be performed in a specific order. Thus, unless a claim for a method does not actually recite the order in which its steps should be followed, or unless otherwise expressly stated in the claim or specification to be limited to a specific order, no order is intended to be inferred in any respect. This holds true for any possible imprecise criteria for interpretation, including questions of logic regarding the placement of steps or sequence of operations, simple interpretations derived from grammar or punctuation, or the number or type of configurations described herein.

[0105] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope or spirit. Other configurations will be apparent to those skilled in the art from consideration of the specification and practice disclosed herein. It is intended that the specification and described configurations be considered as exemplary only, with a true scope and spirit being indicated by the following claims.

Claims

1. 1. A computer-implemented method, comprising: receiving, by a computing device, a plurality of data records and a plurality of variables; determining a numerical representation for each attribute of each data record of the first subset of data records, each data record of the first subset of data records being associated with a label; determining a numerical representation for each attribute of each variable in the first subset of variables, each variable in the first subset of variables being associated with a label; generating, by a plurality of first encoder modules, a vector for each attribute of each data record of the first subset of the plurality of data records based on the numerical representation for each attribute of each data record of the first subset of the plurality of data records; generating, by a plurality of second encoder modules, one or more vectors for each attribute of each variable in the first subset of variables based on the numerical representation for each attribute of each variable in the first subset of variables; training a machine learning model based on the one or more first vectors, the one or more second vectors, and a first portion of a training dataset, wherein the trained machine learning model comprises a predictive model; generating one or more third vectors representing hyperparameters that define characteristics of the trained machine learning model; concatenating one or more of the first vectors, one or more of the second vectors, and one or more of the third vectors into a single concatenated vector; retraining the trained machine learning model using the single concatenated vector; outputting the retrained machine learning model; The method comprises:

2. each attribute of each data record of the plurality of data records comprises an input sequence; The method of claim 1.

3. each data record of the plurality of data records is associated with one or more variables of the plurality of variables; The method according to claim 1 or 2.

4. The method further comprises optimizing the retrained machine learning model based on a second hyper-parameter set and a cross-validation technique. The method according to any one of claims 1 to 3.

5. Determining the numerical representation for each attribute of each variable in the first subset of variables comprises: determining, by a plurality of tokenizers, a token for at least one attribute of at least one variable of the first subset of the plurality of variables; The method according to any one of claims 1 to 4.

6. the at least one attribute of the at least one variable has at least a non-numeric portion; the token comprises the numerical representation for the at least one attribute of the at least one variable. The method according to claim 5.

7. The method further comprises: receiving an input data record and a plurality of input variables; determining an input numeric representation for each attribute of the input data record; determining an input numeric representation for each attribute of each input variable of the plurality of input variables; generating, by the plurality of first encoder modules, an input vector for each attribute of the input data record based on the input numeric representation for each attribute of the input data record; generating, by the plurality of second encoder modules, an input vector for each attribute of each input variable of the plurality of input variables based on the input numeric representation for each attribute of each input variable of the plurality of input variables; generating an input concatenation vector based on the input vector for each attribute of the input data records and based on the input vector for each attribute of each input variable of the plurality of input variables; determining, by said predictive model, one or more of a prediction or score associated with said input data record based on said input concatenated vector; The method according to any one of claims 1 to 6, comprising:

8. The prediction comprises a binary label. The method according to claim 7.

9. the score indicating the likelihood that the first label applies to the data record.

9. The method according to claim 7 or 8.

10. 10. The method of claim 1, further comprising retraining the predictive model, the first encoder modules and the second encoder modules based on an additional concatenated vector determined based on a vector for each attribute of each additional data record of a plurality of additional data records and based on a vector for each attribute of each additional variable of a plurality of additional variables.

11. The first encoder modules is initially trained based on being associated with the label and based on a first set of hyper-parameters; the plurality of additional data records are associated with a second hyper-parameter set that is at least partially different from the first hyper-parameter set; The method of claim 10.

12. the second encoder modules are initially trained based on the first set of hyper-parameters and based on the plurality of variables associated with the labels; the additional variables are associated with the second hyper-parameter set; The method of claim 11.

13. retraining the plurality of first encoder modules comprises retraining the plurality of first encoder modules based on the second hyper-parameter set; retraining the plurality of second encoder modules comprises retraining the plurality of second encoder modules based on the second hyper-parameter set. The method of claim 12.

14. A method for iteratively generating a machine learning model, the method comprising: receiving data comprising a training data set, a validation data set, and input data records; training a first machine learning model using a first plurality of vectors representing input variables from a first portion of the training dataset; generating a first hyper-parameter set and a second hyper-parameter set, each hyper-parameter defining a characteristic of the first machine learning model; concatenating a plurality of the first vectors and the first set of hyper-parameters to form a first concatenated vector; retraining the first machine learning model using the first concatenated vector to generate a second machine learning model; concatenating a second plurality of vectors representing input variables from a second portion of the training data set to the second set of hyper-parameters to form a second concatenated vector; retraining the first machine learning model using the second concatenated vector to generate a third machine learning model; evaluating the second and third machine learning models with respective portions of the validation dataset to generate respective evaluation scores; outputting the second machine learning model or the third machine learning model based on a comparison of the respective evaluation scores; generating a predicted output by inputting the input data records into an output machine learning model; The method comprises:

15. The method further comprising providing a generated data analysis by the output machine learning model. The method of claim 14.

16. The step of receiving the input data records comprises: processing, by a first processor, the input data records in an unlearnable manner; further processing, by a first encoder module, the input data records in a learnable manner; Equipped with The method of claim 14.

17. The method of claim 17, wherein each of the evaluation scores is generated by minimizing a loss function applied to one or more learnable parameters of the processed input data records.

17. The method of claim 16.

18. A method, comprising: A) training a machine learning model using a first portion of a first training data; B) quantifying performance parameters of the machine learning model based on a loss function; C) retraining the machine learning model using a different portion of the first training data concatenated with a hyperparameter vector characterizing the machine learning model to output an updated machine learning model; D) repeating B) using the updated machine learning model from C); and E) if the quantified performance parameters from D) remain above a threshold, repeating C) and D) with a new portion of the first training data and a new hyper-parameter vector; F) outputting the updated machine learning model from C) if the quantified performance parameter from D) is below the threshold; The method comprises:

19. C) comprises the step of continuously minimizing the loss function related to the performance parameters of the machine learning model until a stopping condition is satisfied.

20. The method of claim 18.

20. The method further comprising providing a generated data analysis by the updated machine learning model.

20. The method of claim 18.

21. The hyper-parameter vector comprises at least one of: a number of neural network layers; or a number of neural network filters within a neural network layer.

20. The method of claim 18.

22. The first training data comprises a numerical representation of each of a plurality of attributes; A) is generating, by a plurality of trained first encoder modules, a vector for each attribute of the plurality of attributes based on a respective numerical representation of each attribute of the plurality of attributes; generating a concatenated vector by concatenating each vector for each attribute of the plurality of attributes; training the machine learning model using the concatenated vector; Equipped with 20. The method of claim 18.

Images