Database-based model training method and related device

Abstract

The application provides a database-based model training method and related equipment, the method comprising: creating a training set data table, a training task table, a node list table, a node relationship table, an array operation function library and a derivative function library of a neural network model in a database, taking training data as input of a forward propagation formula, obtaining an output value of forward propagation of the neural network model and writing the output value into the node list table; calculating a derivative value of a loss function value of the neural network model with respect to the output value of forward propagation of the neural network model; taking the derivative value as input of a backward propagation formula, obtaining an output value of backward propagation of the neural network model and writing the output value into the node list table; and based on the output value of backward propagation of the neural network model, calculating weights of a plurality of neural network nodes and writing the weights of the plurality of neural network nodes into the node list table. The application creates a model training process in the form of a table structure in the database, thereby reducing the technical difficulty of neural network configuration and training.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a model training method based on a database and related equipment. BACKGROUND

[0002] Currently, for training data in a database, the training data needs to be called out of the database to perform corresponding calculations. First, in the calling process, data export and data handling steps need to be performed. For example, data stored in the database is handled to a Matrix Laboratory (Matlab) execution environment for processing, which increases data replication, data cleaning, and data flow steps. Second, after the training data is called out of the database, a neural network model needs to be written using an application programming language, such as a python programming language, a C language (The C Programming Language), a C++ language (The C++ Programming Language), and a Go language (Golang Language), which increases the process and development cost of a deep learning framework development and reduces the efficiency of neural network model training. SUMMARY

[0003] Embodiments of the present application disclose a model training method based on a database and related equipment, which solve the problems of complicated neural network model training steps and low neural network model training efficiency.

[0004] The present application provides a model training method based on a database, which comprises:

[0005] creating, in a database, a training set data table of a neural network model, a training task table, a node list table, a node relationship table, an array operation function library, and a derivative function library, the training set data table storing training data of the neural network model, the training task table storing a training task of the neural network model, and the node list table storing a plurality of neural network nodes of the neural network model and node depths of the plurality of neural network nodes;

[0006] obtaining the training data from the training set data table, the training task from the training task table, and the plurality of neural network nodes and the node depths of the plurality of neural network nodes from the node list table;

[0007] constructing a directed acyclic graph in the node relationship table according to the plurality of neural network nodes and the node depths;

[0008] Based on the directed acyclic graph, a forward propagation formula of the neural network model is established according to an array operation function in the array operation function library, the training data is taken as an input of the forward propagation formula, an output value of the forward propagation of the neural network model is obtained, and the output value of the forward propagation of the neural network model is written into the node list table;

[0009] According to the output value of the forward propagation of the neural network model, a loss function value of the neural network model is calculated, and a derivative value of the loss function value with respect to the output value of the forward propagation of the neural network model is calculated;

[0010] Based on the derivative value and the directed acyclic graph, a backward propagation formula of the neural network model is established according to a derivative function in the derivative function library, the derivative value is taken as an input of the backward propagation formula, an output value of the backward propagation of the neural network model is obtained, and the output value of the backward propagation of the neural network model is written into the node list table;

[0011] Based on the output value of the backward propagation of the neural network model, weights of the plurality of neural network nodes are calculated, and the weights of the plurality of neural network nodes are written into the node list table.

[0012] In some optional embodiments, the array operation function library comprises:

[0013] An array constructor function library for storing a construction matrix function required for creating a two-dimensional array;

[0014] An array elementary operation function library for storing an elementary operation function required in an operation process of creating a two-dimensional array;

[0015] A two-dimensional array transformation function library for storing a transformation function required in the operation process of creating a two-dimensional array;

[0016] A two-dimensional array element granularity aggregation function library for storing a first type of aggregation function for aggregating according to an element granularity in the array operation function library;

[0017] A two-dimensional array array granularity aggregation function library for storing a second type of aggregation function for aggregating according to an array granularity in the array operation function library.

[0018] In some optional embodiments, the method further comprises:

[0019] Creating the array constructor function library in the database, and storing the construction matrix function into the array constructor function library;

[0020] According to the construction matrix function, the elementary operation function is created and the transformation function is created, the elementary operation function is stored to the array elementary operation function library, and the transformation function is stored to the two-dimensional array transformation function library;

[0021] The construction matrix function in the array construction function library, the elementary operation function in the array elementary operation function library and the transformation function in the two-dimensional array transformation function library are element sliced and aggregated, to obtain the first type of aggregation function, and the first type of aggregation function is stored to the two-dimensional array element granularity aggregation function library;

[0022] The construction matrix function in the array construction function library, the elementary operation function in the array elementary operation function library, the transformation function in the two-dimensional array transformation function library and the first type of aggregation function are aggregated, to obtain the second type of aggregation function, and the second type of aggregation function is stored to the two-dimensional array array granularity aggregation function library.

[0023] In some optional embodiments, the method further comprises:

[0024] According to the elementary operation function in the array elementary operation function library or the transformation function in the two-dimensional array transformation function library, a first type of array operator for binary operation and a second type of array operator for monadic operation are created, and the first type of array operator and the second type of array operator are stored to the array operation function library.

[0025] In some optional embodiments, the writing of the output value of the forward propagation of the neural network model into the node list table comprises:

[0026] A neuron attribute value lambda function is obtained from the array operation function library;

[0027] Based on the training task, the node depth and the neuron attribute value lambda function, a neural network node is selected from an initial node in the plurality of neural network nodes, the training data is input into the selected neural network node, attribute value information of the selected neural network node is obtained, and the attribute value information of the selected neural network node is written into the node list table;

[0028] input attribute value information of the selected neural network node into a next neural network node based on a first type array operator or a second type array operator of the next neural network node of the selected neural network node and the neuron derivative lambda function, obtain attribute value information of the next neural network node, and write the attribute value information of the next neural network node into the node list table;

[0029] calculate a derivative matrix of the attribute value information of the next neural network node with respect to the attribute value information of the selected neural network node according to the neuron derivative lambda function, and write the derivative matrix into the node list table.

[0030] In some optional embodiments, the calculating the loss function value of the neural network model according to the output value of the forward propagation of the neural network model and the calculating the derivative value of the loss function value with respect to the output value of the forward propagation of the neural network model include:

[0031] obtaining a real value pre-set in the training set data table and a loss function type in the training task table;

[0032] determining a calculation formula of the loss function according to the loss function type in the training task table;

[0033] substituting the real value and the output value of the forward propagation of the neural network model into the calculation formula of the loss function to obtain the loss function value of the neural network model;

[0034] calculating the derivative value of the loss function value with respect to the output value of the forward propagation of the neural network model based on the loss function value and the output value of the forward propagation of the neural network model.

[0035] In some optional embodiments, after the calculating the weights of the plurality of neural network nodes based on the output value of the backward propagation of the neural network model and the writing the weights of the plurality of neural network nodes into the node list table, the method further includes:

[0036] obtaining a training frequency threshold and a loss function threshold pre-set in the training task table;

[0037] when the training frequency of the neural network model is equal to the training frequency threshold or the loss function value of the neural network model is less than the loss function threshold, ending the training of the neural network model.

[0038] In some optional embodiments, the derivative function library includes an activation, standardization and regularization function library, a convolution and pooling function library, an encoding and decoding function library, and a loss function library.

[0039] The application further provides an electronic device, comprising a processor and a memory, wherein the processor is configured to implement the database-based model training method when executing a computer program stored in the memory.

[0040] The application further provides a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the computer program is configured to implement the database-based model training method when executed by a processor.

[0041] Compared with the prior art, the database-based model training method and related device provided by the application can build a training framework in the form of a table structure in a database, reduce the steps of neural network model training, and reduce the difficulty of neural network model training. The application further creates an array operator in the database, reduces the complexity of mathematical expressions of array operations, reduces the difficulty of code implementation, and improves the efficiency of model deployment and the efficiency of development. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 FIG. 1 is a flowchart of a database-based model training method provided by an embodiment of the application.

[0043] Figure 2 FIG. 2 is a flowchart of writing output values of forward propagation of a neural network model into a node list table.

[0044] Figure 3 FIG. 3 is a schematic diagram of writing output values of forward propagation of a neural network model into a node list table.

[0045] Figure 4 FIG. 4 is a schematic diagram of neural network model training.

[0046] Figure 5 FIG. 5 is a structural schematic diagram of an electronic device provided by an embodiment of the application. DETAILED DESCRIPTION

[0047] For the convenience of understanding, exemplary descriptions of some concepts related to the embodiments of the application are given for reference.

[0048] It should be noted that “at least one” in the application means one or more, and “multiple” means two or more than two. “And / or” describes the association between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The terms “first”, “second”, “third”, “fourth” and the like (if any) in the specification and claims of the application and the drawings are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0049] The database-based model training method provided by the embodiments of the present application is executed by an electronic device, and accordingly, the database-based model training apparatus runs in the electronic device.

[0050] Figure 1 is a flowchart of the database-based model training method. According to different needs, the order of steps in the flowchart shown in Figure 1 may be changed, and some steps can be omitted. In the present embodiment, the database-based model training method includes the following steps:

[0051] 11. Creating, in the database, a training set data table of a neural network model, a training task table, a node list table, a node relationship table, an array operation function library, and a derivative function library, the training set data table storing training data of the neural network model, the training task table storing training tasks of the neural network model, and the node list table storing a plurality of neural network nodes of the neural network model and node depths of the plurality of neural network nodes.

[0052] The database described in the present application can be a relational database, which refers to a database that uses a relational model to organize data. The relational database stores data in the form of rows and columns. A series of rows and columns in the relational database are referred to as a table. The relational database is built in an electronic device, for example, the electronic device can be a database server.

[0053] In an optional embodiment, a structured query language (SQL) is used to create tables in the database. Specifically, a create table syntax in the SQL programming language can be used to create the training set data table, the training task table, the node list table, and the node relationship table in the database. The array operation function library and the derivative function library are also created in the database.

[0054] The training set data table is used to store training data that has been cleaned and processed. The training data can be any of image data, speech waveform data, and natural language text data, or other training data, which is not limited. The table structure of the training set data table includes a training task, a data number of training data, a forward propagation output result, and data input into the training set data table. The training data in the training set data table can be a narrow two-dimensional array composed of pixels of each image data in an image data set, a two-dimensional array aggregated from a one-dimensional array sequence of each speech digital signal in a speech data set, or a two-dimensional array aggregated from a one-dimensional array of each language in a natural language data set after encoding by text context correlation and word contribution probability algorithms.

[0055] The training data is sorted and stored in the training set data table, and the data number of the sorted training data is recorded and stored in the training set data table.

[0056] After receiving the training data stored in the database, the training data is processed, and the training data can be processed using One-Hot Encoding or Huffman Coding. The processed training data is used as the true value of the training data, and the true value is written into the training set data table.

[0057] For example, the create table syntax in the SQL programming language is used to create the training set data table t_nn_train_data table structure, and Table 1 shows the structure information of the training set data table.

[0058] Table 1 Structure information of the training set data table

[0059] Field name Field meaning Field type training_id Training task bigint ord_no Data number bigint output_arr Forward propagation output result decimal[] input_arr Training data input data decimal[]

[0060] The training task table stores the training task of the neural network model, and the training task in the training task table is pre-deployed to provide the training basis of the neural network model. The training task table also stores the training frequency threshold and the loss function threshold, so as to judge whether the training is completed by using the training frequency threshold and the loss function threshold in the training task table in the subsequent neural network model training process.

[0061] The table structure of the training task table includes: training task, training frequency threshold, loss function type, current neural network node loss function value, and loss function threshold.

[0062] For example, the create table syntax in the SQL programming language is used to create the training task table t_nn_train_data table structure, and Table 2 shows the structure information of the training task table.

[0063] Table 2 Structure information of the training task table

[0064] Field name Field meaning Field type training_id Training task bigint training_no Training number threshold bigint loss_fn_type Loss function type varchar loss_delta Loss function value of current neural network node decimal loss_delta_threshold Loss function threshold decimal

[0065] The database receives the training data and stores the training data in the training set data table, and when the training task is received, the training data in the training set data table is extracted as the input of the subsequent training.

[0066] The node list table is configured to store a plurality of neural network nodes of the neural network model and node depths of the plurality of neural network nodes, and provide neural network nodes of a directed acyclic graph. A table structure of the node list table includes a training task, a neural network node, a neural network node type, a neural network node function type, a neural network node hyperparameter, node depths of the plurality of neural network nodes, a training number of forward propagation, a training number of backward propagation, and an output value of a currently trained neural network node.

[0067] Exemplarily, the node list table t_nn_node table structure is created by using a create table syntax in an SQL programming language, and the structure information of the node list table is shown in Table 3.

[0068] Table 3 Structure information of the node list table

[0069] Field name Field meaning Field type training_id Training task bigint node_id Neural network node bigint node_type Neural network node type varchar node_fn_type Neural network node function type varchar node_config Neural network node hyperparameter decimal[] nn_depth_no Node depth of neural network int learn_no_fore Training number of forward propagation int learn_no_back Training number of back propagation int node_output Output value of current training neural network node nn_tensor

[0070] As shown in the structure information of the node list table in Table 3, the neural network node type includes a neural network node and a weight node, the target node includes an input node and an output node, the input node can be an initial neural network node of input training data, and the output node can be a last neural network node of neural network model training. The neural network node hyperparameter is a hyperparameter configuration for storing a neuron function, such as a window width, a sliding step span, an augmented width, an augmented padding, and an alpha value of some activation functions.

[0071] The node relationship table is configured to obtain the neural network nodes and the node depths in the node list table, and create a directed acyclic graph.

[0072] A table structure of the node relationship table includes a training task, a backward node, a forward node, and a sequence number of a propagation path. The backward node is a neural network node of backward propagation, the forward node is a neural network node of forward propagation, the sequence number of the propagation path is a sequence number of connecting any two neural network nodes when performing the backward propagation or the forward propagation, and a node flow direction of the directed acyclic graph is determined according to the sequence number of the propagation path.

[0073] Exemplarily, the node relationship table t_nn_node_path table structure is created by using a create table syntax in an SQL programming language, and the structure information of the node relationship table is shown in Table 4.

[0074] Table 4 Structure information of the node relationship table

[0075] Field name Field meaning Field type training_id Training task bigint back_node_id Backward node bigint fore_node_id Forward node bigint fore_path_no Order number of propagation path int

[0076] The array operation function library and the derivative function library are also created in the database.

[0077] In an optional embodiment, the array operation function library comprises:

[0078] An array constructor function library for storing a construction matrix function required for creating a two-dimensional array;

[0079] An array elementary operation function library for storing an elementary operation function required for creating a two-dimensional array in an operation process;

[0080] A two-dimensional array transformation function library for storing a transformation function required for creating a two-dimensional array in an operation process;

[0081] A two-dimensional array element granularity aggregation function library for storing a first type of aggregation function aggregated according to an element granularity in the array operation function library;

[0082] A two-dimensional array array granularity aggregation function library for storing a second type of aggregation function aggregated according to an array granularity in the array operation function library.

[0083] In an optional embodiment, the method further comprises:

[0084] The array constructor function library is created in the database, and the construction matrix function is stored in the array constructor function library;

[0085] The elementary operation function and the transformation function are created according to the construction matrix function, the elementary operation function is stored in the array elementary operation function library, and the transformation function is stored in the two-dimensional array transformation function library;

[0086] The construction matrix function in the array constructor function library, the elementary operation function in the array elementary operation function library, and the transformation function in the two-dimensional array transformation function library are subjected to element slice granularity aggregation to obtain the first type of aggregation function, and the first type of aggregation function is stored in the two-dimensional array element granularity aggregation function library;

[0087] The construction matrix function in the array constructor function library, the elementary operation function in the array elementary operation function library, the transformation function in the two-dimensional array transformation function library, and the first type of aggregation function are subjected to array element aggregation to obtain the second type of aggregation function, and the second type of aggregation function is stored in the two-dimensional array array granularity aggregation function library.

[0088] In an optional embodiment, the method further comprises:

[0089] According to the elementary operation function in the array elementary operation function library or the transform function in the two-dimensional array transform function library, a first type of array operator for binary operation and a second type of array operator for monadic operation are created, and the first type of array operator and the second type of array operator are stored in the array operation function library.

[0090] The functions in the array operation function library can be written in structured query language, and functions in a function library outside the database can also be directly called.

[0091] The types of the operation objects of the array operation function library include one-dimensional array, two-dimensional array and N (N>2) dimensional array, and the types of the operation objects of the array operation function library also include a combination type constructed according to the one-dimensional array, the two-dimensional array and the N dimensional array as array members, and a database creation type constructed according to the one-dimensional array, the two-dimensional array and the N dimensional array as function library members, and one of the types can be selected according to actual needs.

[0092] The array element types of the operation objects include generic type, polymorphic type, class template type, polymorphic data type (AnyElement), pseudo type, complex number type, real number type, integer type, text type, bit string, Boolean type, time type, enumeration type, range type, geometry type, geographic type, data type in the form of pure string (Java Script Object Notation, Json), eXtensible Markup Language type, encrypted ciphertext type, vector type, tensor type, array type, composite combination type, chemical component type, molecular structure type, topological structure type, tree type, graph relationship type, registration of new data types used in the current database, table record type, table type, iterator type, key-value pair mapping type, hash set type, and one of the types can be selected according to actual needs.

[0093] The array element type of the array type member of the operation object includes: a generic type, a polymorphic type, a class template type, a polymorphic data type, a pseudo type, a complex type, a real type, an integer type, a text type, a bit string, a Boolean type, a time type, an enumeration type, a range type, a geometry type, a geographic type, a pure string form data type (Java Script Object Notation, Json), an eXtensible Markup Language type, an encrypted ciphertext type, a vector type, a tensor type, an array type, a composite combination type, a chemical component type, a molecular structure type, a topological structure type, a tree type, a graph relationship type, a new data type registered in the current database, a table record type, a table type, an iterator type, a key-value pair mapping type, and a hash set type. One of the types can be selected according to actual needs.

[0094] The array constructor library is provided with a matrix construction function f_new, and the matrix construction function f_new is used as a broadcast mechanism of binocular array operation, which refers to content copying and bitwise operation.

[0095] The array constructor library includes: an array creation function with given initial element values, an array creation function with random element values, an array creation function with Gaussian random distribution of element values, a diagonal array creation function, an augmented array creation function, a two-dimensional array dimension reduction to one-dimensional array function, and a one-dimensional array dimension increase to two-dimensional array function.

[0096] Exemplarily, functions in the array constructor library are selected, and the createfunction syntax in the SQL programming language is used to create the input parameters and output parameters of the functions in the array constructor library. Table 5 shows only part of the functions in the array constructor library.

[0097] Table 5 creates functions in the array constructor library in the database

[0098]

[0099]

[0100]

[0101] The array constructor library is created in the database, the matrix construction function is stored in the array constructor library, the elementary operation function and the transformation function are created according to the matrix construction function, the elementary operation function is stored in the array elementary operation function library, and the transformation function is stored in the two-dimensional array transformation function library.

[0102] Exemplarily, a broadcast mechanism for creating the elementary operation functions according to the configuration matrix function f_new is implemented as follows:

[0103] For each element of the matrix arr: = array[[1, 2, 3], [4, 5, 6]] adding 0.5 can be expressed as: arr + `0.5 = arr + `f_new(0.5, array[2, 3]) = array[[1, 2, 3], [4, 5, 6]] + `array[[0.5, 0.5, 0.5], [0.5, 0.5, 0.5]].

[0104] The array elementary operation function library includes: corresponding element addition function, corresponding element subtraction function, each element opposite function, corresponding element multiplication function, corresponding element division function, each element reciprocal function, corresponding element cosine function, corresponding element power function, each element natural exponential e power function, corresponding element logarithm function, each element natural exponential e logarithm function, each element radian sine function, each element radian cosine function, each element radian tangent function, each element radian cotangent function, each element radian secant function, each element radian cosecant function, each element radian arcsine function, each element radian arccosine function, each element radian arctangent function, each element radian arccotangent function, each element radian arcsecant function, each element radian arccosecant function, each element radian arcsecant function, each element radian arccosecant function, each element angle sine function, each element angle cosine function, each element angle tangent function, each element angle cotangent function, each element angle secant function, each element angle cosecant function, each element angle arcsine function, each element angle arccosine function, each element angle arctangent function, each element angle arccotangent function, each element angle arcsecant function, each element angle arccosecant function, each element hyperbolic sine function, each element hyperbolic cosine function, each element hyperbolic tangent function, each element hyperbolic cotangent function, each element hyperbolic secant function, each element hyperbolic cosecant function, each element hyperbolic arcsine function, each element hyperbolic arccosine function, each element hyperbolic arctangent function, each element hyperbolic arccotangent function, each element hyperbolic arcsecant function, each element hyperbolic arccosecant function, radian to angle function, angle to radian function, real number absolute value, complex number modulus function, rounding function, floor function, ceiling function, digit truncation function, corresponding element equality judgment function, corresponding element less than judgment function, corresponding element greater than judgment function, corresponding element less than or equal to judgment function, corresponding element greater than or equal to judgment function, corresponding element size judgment function, corresponding element minimum value function, corresponding element maximum value function, complex number array each element conjugate function, complex number array each element real part function, complex number array each element imaginary part function, complex number array each element imaginary and real part exchange function, complex number array conjugate array function, corresponding element or operation function, corresponding element and operation function, each element not operation function, corresponding element exclusive or function, corresponding element same or function, each element bit left shift function, each element bit right shift function, corresponding element splicing function, corresponding element text regular matching judgment function, corresponding element text regular matching function, corresponding element text regular replacement function, each element text left padding function, each element text right padding function, each element text both sides cleaning function, each element text left side cleaning function, each element text right side cleaning function, and each element text substring interception function. According to actual needs, other array corresponding element elementary operations and broadcast functions can be added in the array elementary operation function library, which is not limited here.

[0105] Exemplarily, functions in the array elementary operation function library are selected, and the create function syntax in the SQL programming language is used to create the input parameters and output parameters of the functions in the array elementary operation function library. Table 6 only shows part of the functions in the array elementary operation function library.

[0106] Table 6 creates functions in the array elementary operation function library in the database

[0107]

[0108]

[0109]

[0110]

[0111]

[0112]

[0113]

[0114]

[0115]

[0116]

[0117]

[0118]

[0119]

[0120]

[0121]

[0122]

[0123]

[0124]

[0125]

[0126]

[0127]

[0128]

[0129] The two-dimensional array transformation function library is configured to store transformation functions required in the operation process of creating a two-dimensional array.

[0130] The transformation functions in the two-dimensional array transformation function library include: up-down flipping function, left-right flipping function, counterclockwise rotation 90° function, clockwise rotation 90° function, rotation 180° function, transposition function, inverse transposition function, up-down concatenation function, left-right concatenation function, inner product function, quantity product function, inner product power, quantity product power function, matrix multiplication function, multiplication function, outer product function, and matrix multiplication power function. These functions are 13 in total. According to actual needs, other two-dimensional array transformation functions can also be added to the two-dimensional array transformation function library, which is not limited herein.

[0131] Exemplarily, functions in the two-dimensional array transformation function library are selected, and the createfunction syntax in the SQL programming language is used to create the input parameters and output parameters of the functions in the two-dimensional array transformation function library. Table 7 shows only part of the functions in the two-dimensional array transformation function library.

[0132] Table 7: Creating functions in the two-dimensional array transformation function library in the database

[0133]

[0134]

[0135]

[0136]

[0137] The derivative function library includes: activation and standardization and regularization function library, convolution and pooling function library, encoding and decoding function library, and loss function library.

[0138] The functions in the activation and standardization and regularization function library, the convolution and pooling function library, the encoding and decoding function library, and the loss function library include: sigmoid function, relu function, tanh function, elu function, selu function, leaky_relu function, gelu function, softplus function, swish function, horizontal softmax function, vertical softmax function, horizontal zscore function, vertical zscore function, L1 regularization function, L2 regularization function, two-dimensional convolution function, maximum pooling function, average pooling function, one-hot encoding function, Huffman encoding function, least squares loss function, and cross-entropy loss function.

[0139] Exemplarily, in the database, there is created a library of activation and standardization and regularization functions, functions in the library of activation and standardization and regularization functions are created using the SQL programming language, and Table 8 shows some function examples in the library of activation and standardization and regularization functions.

[0140] Table 8 shows some function examples in the library of activation and standardization and regularization functions.

[0141]

[0142]

[0143]

[0144]

[0145] Exemplarily, in the database, there is created a library of convolution and pooling functions, functions in the library of convolution and pooling functions are created using the SQL programming language, and Table 9 shows some function examples in the library of convolution and pooling functions.

[0146] Table 9 shows some function examples in the library of convolution and pooling functions.

[0147]

[0148]

[0149]

[0150]

[0151] Exemplarily, in the database, there is created a library of encoding and decoding functions, functions in the library of encoding and decoding functions are created using the SQL programming language, and Table 10 shows some function examples in the library of encoding and decoding functions.

[0152] Table 10 shows some function examples in the library of encoding and decoding functions.

[0153]

[0154]

[0155] Exemplarily, in the database, there is created a library of loss functions, functions in the library of loss functions are created using the SQL programming language, and Table 11 shows some function examples in the library of loss functions.

[0156] Table 11 shows some function examples in the library of loss functions.

[0157]

[0158] Exemplarily, the lambda functions include a neuron attribute value lambda function, a neuron derivative lambda function, a forward propagation function, a backward propagation chain derivative function, and a create function syntax in the SQL programming language is adopted to create functions, and Table 12 is an example of creating lambda functions.

[0159] Table 12 is an example of creating lambda functions.

[0160]

[0161]

[0162] In an optional embodiment, according to the elementary operation functions in the array elementary operation function library or the transformation functions in the two-dimensional array transformation function library, a first type of array operator for binary operation and a second type of array operator for monadic operation are created, and the first type of array operator and the second type of array operator are stored to the array operation function library.

[0163] Exemplarily, the monadic operation includes left monadic operation and right monadic operation, and a natural index e power ^`example is implemented according to the second type of array operator, as follows:

[0164]

[0165] A rounding ~=`example is implemented according to the second type of array operator, as follows:

[0166]

[0167] A matrix multiplication |**| example is implemented according to the second type of array operator, as follows:

[0168]

[0169] The first type of array operator and the second type of array operator both include: corresponding element addition operator, corresponding element subtraction, opposite number operator, corresponding element multiplication, Hadamard product, basic product operator, corresponding element division, reciprocal operator, corresponding element remainder operator, corresponding element power, natural exponential e power operator, corresponding element logarithm, natural exponential e logarithm operator, corresponding element or operation operator, corresponding element and operation operator, corresponding element not operation operator, corresponding element text regular matching judgment operator, complex array element conjugate operator, complex array element real part operator, complex array element imaginary part operator, complex array element real and imaginary part exchange operator, corresponding element XOR operator, corresponding element XNOR operator, corresponding element concatenation operator, each element bit left shift operator, each element bit right shift operator, corresponding element equality judgment operator, corresponding element less than judgment operator, corresponding element greater than judgment operator, corresponding element less than or equal to judgment operator, corresponding element greater than or equal to judgment operator, corresponding element size judgment operator, corresponding element minimum value operator, corresponding element maximum value operator, absolute value (complex modulus) operator, rounding operator, floor operator, ceiling operator, digit truncation operator, up-down flip operator, left-right flip operator, counterclockwise rotation 90° operator, clockwise rotation 90° operator, rotation 180° operator, transpose operator, complex array conjugate operator, inverse transpose operator, up-down concatenation operator, left-right concatenation operator, inner product, quantity product operator, matrix multiplication, multiplication, outer product operator, inner product power, quantity product power operator, matrix multiplication power operator. According to actual needs, other array operators can also be created, which are not limited here.

[0170] Exemplarily, Table 13 is a parameter type description of creating an array operator in a database. Table 13 shown is only exemplary, which is not limited.

[0171] Table 13 Array operator parameter type

[0172]

[0173]

[0174] An array operator in a database is created by using the create function syntax in the SQL programming language. Table 14 is an array operator created in a database.

[0175] Table 14 Array operator created in a database

[0176]

[0177]

[0178]

[0179] The two-dimensional array element granularity aggregation function library is configured to store first type aggregation functions aggregated according to element granularity in the array operation function library.

[0180] The first type aggregation functions stored in the two-dimensional array element granularity aggregation function library include: slice summation function, slice continuous multiplication function, slice average function, slice maximum value function, slice minimum value function, slice population variance function, slice sample variance function, slice population standard deviation function, slice sample standard deviation function, slice mode function, slice median function, slice concatenation function, slice OR function, slice AND function, longitudinal summation function, longitudinal continuous multiplication function, longitudinal average function, longitudinal maximum value function, longitudinal minimum value function, longitudinal population variance function, longitudinal sample variance function, longitudinal population standard deviation function, longitudinal sample standard deviation function, longitudinal mode function, longitudinal median function, longitudinal concatenation function, longitudinal OR function, longitudinal AND function, horizontal summation function, horizontal continuous multiplication function, horizontal average function, horizontal maximum value function, horizontal minimum value function, horizontal population variance function, horizontal sample variance function, horizontal population standard deviation function, horizontal sample standard deviation function, horizontal mode function, horizontal median function, horizontal concatenation function, horizontal OR function, and horizontal AND function. Other two-dimensional array element granularity aggregation functions can also be added in the two-dimensional array element granularity aggregation function library according to actual needs, which are not limited herein.

[0181] Exemplarily, the first type aggregation functions in the two-dimensional array element granularity aggregation function library are selected, the create function syntax in the SQL programming language is used to create the input parameters and output parameters of the first type aggregation functions in the two-dimensional array element granularity aggregation function library. Table 15 shows the creation of the first aggregation functions in the two-dimensional array element granularity aggregation function library in the database, and the functions shown in Table 15 are only part of the functions in the two-dimensional array element granularity aggregation function library.

[0182] Table 15 shows the creation of the first type aggregation functions in the two-dimensional array element granularity aggregation function library in the database

[0183]

[0184]

[0185]

[0186] The two-dimensional array array granularity aggregation function library is configured to store second type aggregation functions aggregated according to array granularity in the array operation function library.

[0187] The second type of aggregation function in the two-dimensional array array granularity aggregation function library includes: corresponding element summation function, corresponding element continuous multiplication function, corresponding element average function, corresponding element maximum value function, corresponding element minimum value function, corresponding element population variance function, corresponding element sample variance function, corresponding element population standard deviation function, corresponding element sample standard deviation function, corresponding element mode function, corresponding element median function, corresponding element splicing function, corresponding element or function, and corresponding element and function. Other two-dimensional array array granularity aggregation functions can be added in the two-dimensional array array granularity aggregation function library according to actual needs, which are not limited herein.

[0188] Exemplarily, the second type of aggregation function in the two-dimensional array array granularity aggregation function library is selected, the create function syntax in the SQL programming language is used to create the input parameters and output parameters of the second type of aggregation function in the two-dimensional array array granularity aggregation function library, and Table 16 is the creation of the second type of aggregation function in the two-dimensional array array granularity aggregation function library in the database.

[0189] Table 16 is the creation of the second type of aggregation function in the two-dimensional array array granularity aggregation function library in the database

[0190]

[0191]

[0192]

[0193]

[0194]

[0195] Exemplarily, the construction matrix function created in the array constructor function library is obtained, the construction matrix function is used as a broadcast mechanism of binocular array operation, functions of different broadcast mechanisms are called by using the SQL programming language, and Table 17 is a broadcast mode description and example of the broadcast mechanism.

[0196] Table 17 is a broadcast mode description and example of the broadcast mechanism

[0197]

[0198]

[0199] The existing array operation is usually implemented by function call, and the bracket nesting is more, which is not easy to distinguish the operation level. For the aggregation operation of multiple arrays, it is usually necessary to write a loop program structure to summarize each item, which increases the complexity of calculation. In order to solve the existing calculation complexity problem, the embodiment of the application creates a first type of array operator for binocular operation and a second type of array operator for monocular operation to simplify the complexity of calculation and reduce the difficulty of code implementation.

[0200] 12, obtaining the training data from the training set data table, obtaining the training task from the training task table, and obtaining the plurality of neural network nodes and the node depth of the plurality of neural network nodes from the node list table.

[0201] 13, constructing a directed acyclic graph in the node relationship table according to the plurality of neural network nodes and the node depth.

[0202] The directed acyclic graph refers to a loop-free directed graph.

[0203] In the embodiment, the plurality of neural network nodes and the node depth of the plurality of neural network nodes are obtained from the node list table, and the neural network nodes are used as the primary key for constructing the directed acyclic graph. The flow direction of the neural network nodes in the directed acyclic graph is determined according to the neural network nodes in the node list table.

[0204] 14, based on the directed acyclic graph, establishing a forward propagation formula of the neural network model according to the array operation function in the array operation function library, taking the training data as the input of the forward propagation formula, obtaining the output value of the forward propagation of the neural network model, and writing the output value of the forward propagation of the neural network model into the node list table.

[0205] The propagation direction of the forward propagation on the directed acyclic graph is taken as the forward flow direction of the neural network nodes. The training data in the training set data table is obtained, and the training data is input into the forward propagation formula.

[0206] Figure 2 The flow chart for writing the output value of the forward propagation of the neural network model into the node list table. As shown in Figure 2 The method for writing the output value of the forward propagation of the neural network model into the node list table includes the following steps:

[0207] 21, obtaining a neuron attribute value lambda function from the array operation function library.

[0208] 22, based on the training task, the node depth and the neuron attribute value lambda function, selecting a neural network node from the initial node in the plurality of neural network nodes, inputting the training data into the selected neural network node, obtaining attribute value information of the selected neural network node, and writing the attribute value information of the selected neural network node into the node list table.

[0209] Obtaining a directed acyclic graph for performing forward propagation, selecting a neural network node in the directed acyclic graph as an initial neural network node, and inputting the training data obtained in the training set data table into the initial neural network node in the directed acyclic graph.

[0210] Calling the neuron attribute value lambda function in the array operation function library to obtain the output value of the initial neural network node, and the output value of the initial neural network node being the attribute value information.

[0211] 23, based on the first type array operator or the second type array operator of the next neural network node of the selected neural network node and the neuron attribute value lambda function, inputting the attribute value information of the selected neural network node into the next neural network node to obtain attribute value information of the next neural network node, and writing the attribute value information of the next neural network node into the node list table.

[0212] Taking the second neural network node as the next neural network node as an example, traversing the second neural network node, and according to the first type array operator or the second type array operator of the second neural network node and the operation anonymous function in the array operation function library, obtaining the output value of the next neural network node, the output value of the next neural network node including the attribute value information of the next neural network node, and writing the attribute value information of the next neural network node into the node list table.

[0213] 24, according to the neuron derivative lambda function, calculating the derivative matrix of the attribute value information of the next neural network node with respect to the attribute value information of the selected neural network node, and writing the derivative matrix into the node list table.

[0214] According to the neuron derivative lambda function, calculating the derivative matrix of the attribute value information of the next neural network node with respect to the attribute value information of the initial neural network node, and writing the derivative matrix into the node list table.

[0215] If the array operator type of the second neural network node is the first type of array operator, i.e. a binary operation, the first derivative information of the attribute value information of the next neural network node with respect to the first variable in the attribute value information of the initial neural network node is calculated, the second derivative information of the attribute value information of the next neural network node with respect to the second variable in the attribute value information of the initial neural network node is calculated, and the first derivative information and the second derivative information are written into the derivative matrix.

[0216] If the array operator type of the second neural network node is the second type of array operator, i.e. a unary operation, the first derivative information of the attribute value information of the next neural network node with respect to the first variable in the attribute value information of the initial neural network node is returned, the second derivative information is empty, and the first derivative information is written into the derivative matrix.

[0217] The above examples only show the calculation process of the first two neural network nodes. Each neural network node is traversed, and the output value of each neural network node in the training process is calculated, which includes the attribute value information of the neural network node and the first derivative information and the second derivative information.

[0218] The attribute value information and the derivative matrix obtained by training each neural network node are written into the node list table.

[0219] Figure 3 An example of writing the output value of the forward propagation of the neural network model into the node list table is shown in FIG. 1. Figure 3 As shown in the figure, the neural network node is simply written as a node in the figure.

[0220] 15. According to the output value of the forward propagation of the neural network model, the loss function value of the neural network model is calculated, and the derivative value of the loss function value with respect to the output value of the forward propagation of the neural network model is calculated.

[0221] In an optional embodiment, the calculation of the loss function of the neural network model according to the output value of the forward propagation of the neural network model comprises:

[0222] The real value set in the training set data table and the loss function type in the training task table are obtained.

[0223] The calculation formula of the loss function is determined according to the loss function type in the training task table.

[0224] The real value and the output value of the forward propagation of the neural network model are substituted into the calculation formula of the loss function to obtain the loss function value of the neural network model.

[0225] Based on the loss function value and the output value of the forward propagation of the neural network model, a derivative value of the loss function value with respect to the output value of the forward propagation of the neural network model is calculated.

[0226] In the node list table, attribute value information of a last neural network node in the forward propagation process or attribute value information of a neural network node output in the training is obtained, and the attribute value information of the last neural network node or the attribute value information of the neural network node output in the training is taken as the output value of the forward propagation of the neural network model.

[0227] In the training set data table, a true value corresponding to the training data is obtained. The true value and the output value of the forward propagation of the neural network model are taken into a calculation formula of the loss function, and a loss function value of the neural network model is calculated. Further, a derivative value of the loss function value with respect to the output value of the forward propagation of the neural network model is calculated.

[0228] 16, based on the loss function and the directed acyclic graph, a back propagation formula of the neural network model is established according to the derivative function in the derivative function library, the derivative value is taken as the input of the back propagation formula, the output value of the back propagation of the neural network model is obtained, and the output value of the back propagation of the neural network model is written into the node list table.

[0229] According to the derivative function in the derivative function library, a back propagation formula of the neural network model is established. The derivative function can be a neuron derivative lambda function. The derivative value of the forward propagation is taken as the input of the back propagation. The derivative value of the forward propagation can be a derivative value calculated by training to a certain neural network node or a derivative value calculated by training to a last neural network node. A back propagation direction opposite to the forward propagation direction is obtained, and a reverse direction of the directed acyclic graph is obtained accordingly. The back propagation is performed.

[0230] According to the node depth, each neural network node is traversed by using the back propagation to obtain an output value of the back propagation. The output value of the back propagation includes derivative information of a first type of array operator with respect to the loss function or derivative information of a second type of array operator with respect to the loss function. The output value of the back propagation is written into the node list table.

[0231] Specifically, a derivative value of a loss function value of the neural network model with respect to an output value of the forward propagation of each neural network node of the neural network model is obtained. The derivative value is taken as the input of the back propagation. A derivative of an input value with respect to an output value of each neural network node is calculated. A derivative of a current loss function value of the neural network model with respect to a current input value of the neural network node and a derivative of the current loss function value of the neural network model with respect to a weight of the current neural network node are obtained.

[0232] 17. based on the output value of the back propagation of the neural network model, calculating the weight of the plurality of neural network nodes, and writing the weight of the plurality of neural network nodes into the node list table.

[0233] After performing one forward propagation and one back propagation, the weight of the plurality of neural network nodes is calculated by using the output value of the back propagation by using the gradient descent method, and the weight of the plurality of neural network nodes is written into the node list table.

[0234] After one round of training is completed, the loss function value of the output of the neural network model is calculated, and the derivative of the loss function value with respect to the weight of each neural network node is calculated.

[0235] In an optional embodiment, after the step of based on the output value of the back propagation of the neural network model, calculating the weight of the plurality of neural network nodes, and writing the weight of the plurality of neural network nodes into the node list table, the method further comprises:

[0236] obtaining a training number threshold and a loss function threshold set in advance in the training task table;

[0237] When the training number of the neural network model is equal to the training number threshold, or the loss function value of the neural network model is less than the loss function threshold, the training of the neural network model is ended.

[0238] The process of performing one forward propagation and one back propagation is regarded as one training number, and the training number threshold is set in advance in the training task table, for example, 20 times, and when the training number of the neural network model being trained is equal to 20 times, the training is stopped. The loss function threshold is also set in advance in the training task table, and when the calculated loss function of the neural network model is less than the loss function threshold, the training is stopped.

[0239] Figure 4 Schematic diagram of neural network model training.

[0240] In a specific embodiment, according to the above neural network model training process, the training data, such as the recognition model of the training handwritten data set, is obtained, the training data is the handwritten data set, the handwritten data set is used for the data set of image processing, and the handwritten data set depicts the numbers of [0, 9]. It can be understood that any neural network model can be trained by using the training method described in the present application, and the following examples are only exemplary.

[0241] After the training data of the handwritten dataset is input into the database and the recognition model of the handwritten dataset is trained in the database based on the above method, test data is input into the database, the test data can be a handwritten digital picture in the handwritten dataset, the handwritten digital picture is stored in the training set data table, and according to the training task, the training task can be to identify the number in the input handwritten digital picture. According to the depth of the neural network node, the dependent variable of each neural network node is calculated, the attribute value information of the last neural network node is output, and the attribute value information output by the last neural network node is taken as the prediction value.

[0242] The prediction value is a one-dimensional vector, such as [0, 0, 0, 0, 0, 0, 0.3, 0, 0.8, 0], the probability of outputting the number 6 is 0.3, the probability of outputting the number 8 is 0.8, and the probability of outputting other numbers is 0, and it is judged that the number in the handwritten digital picture is 8. The recognition model created in the database has the expected recognition effect.

[0243] In addition to the training of the recognition model described in the above specific embodiments, the method proposed in the present application can also train a face recognition model, a speech recognition model or a natural language recognition model.

[0244] Specifically, for the face recognition model, image data is input into the face recognition model trained based on the scheme described in the present application, the image data is preprocessed to obtain a two-dimensional array aggregated by one-dimensional arrays of each image pixel after flattening, the two-dimensional array is input into the face recognition model, and the probability value of the recognition result is output, and the final recognition result is obtained based on the probability value.

[0245] For the speech recognition model, speech data is input into the speech recognition model trained based on the scheme described in the present application, the speech data is preprocessed to obtain a two-dimensional array aggregated by one-dimensional array sequences of each speech digital signal, the two-dimensional array is input into the trained speech recognition model, and the probability value of speech recognition is output, and the final speech recognition result is obtained based on the probability value.

[0246] For the natural language recognition model, similar to the above method, a two-dimensional array aggregated by one-dimensional arrays encoded by text context correlation, word contribution probability and other algorithms for each language is obtained, the two-dimensional array is input into the natural language recognition model, the probability value of natural language recognition is output, and the final natural language recognition result is obtained based on the probability value. It can also be a natural language recognition model that needs to be translated between Chinese and English, by inputting unknown text, through the natural language recognition model for translation between Chinese and English, the translated text and the probability value corresponding to the translated text are output, and the final translation result is obtained.

[0247] The method provided in the application is applied in a relational database, creates a neural network training process, reduces data carrying and copying execution steps, does not need to be written by means of other structured programming codes, reduces code writing complexity, reduces development cost, and improves model training accuracy.

[0248] Figure 5 A structural schematic diagram of an electronic device is provided for an embodiment of the application. The electronic device 5 includes, but is not limited to, a memory 53 and at least one processor 51. The memory 53 and the processor 51 can be connected through a communication bus 52, or can be directly connected. In the embodiment, the memory 53 can be an internal memory of the electronic device 5, that is, a memory built in the electronic device 5. In other embodiments, the memory 53 can also be an external memory of the electronic device 5, that is, a memory connected to the electronic device 5.

[0249] In some embodiments, the memory 53 is used to store program codes and various data, and to realize high-speed and automatic access of programs or data in the running process of the electronic device 5.

[0250] The memory 53 can include a random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0251] In an embodiment, the processor 51 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor.

[0252] The program code and various data in the memory 53, if implemented in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods, such as the database-based model training method, can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), etc.

[0253] It can be understood that the above-described module division is a logical function division, and actual implementation can have another division manner. In addition, each functional module in each embodiment of the present application can be integrated in the same processing unit, or each module can be physically present alone, or two or more modules can be integrated in the same unit. The integrated module can be realized in the form of hardware or hardware plus software function module.

[0254] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A database-based model training method, characterized in that, The method includes: A structured query language is used to create a training set data table, a training task table, a node list table, a node relationship table, an array operation function library, and a derivative function library for a neural network model in a database. The training set data table stores the training data of the neural network model, the training task table stores the training tasks of the neural network model, and the node list table stores multiple neural network nodes of the neural network model and the node depth of the multiple neural network nodes. The database is a relational database. The array operation function library includes an array constructor library, in which a constructor f_new is created. The constructor f_new is used as a broadcast mechanism for binary array operations, and the broadcast mechanism includes content copying and bitwise operations. The training data is obtained from the training set data table, the training task is obtained from the training task table, and the plurality of neural network nodes and the node depth of the plurality of neural network nodes are obtained from the node list table. Based on the plurality of neural network nodes and the node depth, a directed acyclic graph is constructed in the node relationship table; Based on the directed acyclic graph, the forward propagation formula of the neural network model is established according to the array operation functions in the array operation function library. The training data is used as the input of the forward propagation formula to obtain the output value of the forward propagation of the neural network model. The output value of the forward propagation of the neural network model is written into the node list table. Based on the output value of the forward propagation of the neural network model, calculate the loss function value of the neural network model, and calculate the derivative of the loss function value with respect to the output value of the forward propagation of the neural network model; Based on the derivative value and the directed acyclic graph, the backpropagation formula of the neural network model is established according to the derivative function in the derivative function library. The derivative value is used as the input of the backpropagation formula to obtain the output value of the backpropagation of the neural network model. The output value of the backpropagation of the neural network model is written into the node list table. Based on the output value of the backpropagation of the neural network model, the weights of the multiple neural network nodes are calculated, and the weights of the multiple neural network nodes are written into the node list table.

2. The database-based model training method according to claim 1, characterized in that, The array operation function library includes: An array constructor library that stores the matrix construction functions needed to create two-dimensional arrays; This is a library of elementary array operations functions, used to store the elementary operation functions needed during the creation of two-dimensional arrays. A library of transformation functions for two-dimensional arrays, used to store the transformation functions needed during the creation and operation of two-dimensional arrays; A library of two-dimensional array element-level aggregation functions, used to store first-class aggregation functions that perform aggregation based on the element-level granularity in the array operation function library; A two-dimensional array granularity aggregation function library is used to store a second type of aggregation functions that aggregate based on the array granularity in the array operation function library.

3. The database-based model training method according to claim 2, characterized in that, The method further includes: Create the array constructor library in the database, and store the matrix construction function in the array constructor library; Based on the constructed matrix function, the elementary operation function and the transformation function are created, the elementary operation function is stored in the array elementary operation function library, and the transformation function is stored in the two-dimensional array transformation function library; Element-slice granular aggregation is performed on the matrix construction function in the array constructor library, the elementary operation function in the array elementary operation function library, and the transformation function in the two-dimensional array transformation function library to obtain the first type of aggregation function, and the first type of aggregation function is stored in the two-dimensional array element-granular aggregation function library; The array elements are aggregated using the matrix construction function in the array constructor library, the elementary operation function in the array elementary operation function library, the transformation function in the two-dimensional array transformation function library, and the first type of aggregation function to obtain the second type of aggregation function. The second type of aggregation function is then stored in the two-dimensional array granularity aggregation function library.

4. The database-based model training method according to claim 2, characterized in that, The method further includes: Based on the elementary operation functions in the array elementary operation function library or the transformation functions in the two-dimensional array transformation function library, create a first type of array operator for binary operations and a second type of array operator for unary operations, and store the first type of array operator and the second type of array operator in the array operation function library.

5. The database-based model training method according to claim 4, characterized in that, The step of using the training data as input to the forward propagation formula to obtain the output value of the forward propagation of the neural network model, and writing the output value of the forward propagation of the neural network model into the node list table, includes: The lambda function retrieves neuron attribute values ​​from the array operation function library. Based on the training task, the node depth, and the lambda function of the neuron attribute values, a neural network node is selected from the initial node among the plurality of neural network nodes. The training data is input into the selected neural network node to obtain the attribute value information of the selected neural network node. The attribute value information of the selected neural network node is written into the node list table. Based on the first or second type of array operator of the next neural network node of the selected neural network node, and the lambda function of the neuron attribute value, the attribute value information of the selected neural network node is input into the next neural network node to obtain the attribute value information of the next neural network node, and the attribute value information of the next neural network node is written into the node list table. Based on the derivative of the neuron's lambda function, calculate the derivative matrix of the attribute value information of the next neural network node with respect to the attribute value information of the selected neural network node, and write the derivative matrix into the node list table.

6. The database-based model training method according to claim 1, characterized in that, The step of calculating the loss function value of the neural network model based on the output value of the forward propagation of the neural network model, and calculating the derivative of the loss function value with respect to the output value of the forward propagation of the neural network model, includes: Obtain the preset true values ​​from the training set data table and the loss function type from the training task table; The calculation formula for the loss function is determined based on the loss function type in the training task table; Substituting the true value and the output value of the forward propagation of the neural network model into the calculation formula of the loss function, the loss function value of the neural network model is obtained; Based on the loss function value and the output value of the forward propagation of the neural network model, calculate the derivative of the loss function value with respect to the output value of the forward propagation of the neural network model.

7. The database-based model training method according to claim 1, characterized in that, After calculating the weights of the plurality of neural network nodes based on the output values ​​of the backpropagation of the neural network model, and writing the weights of the plurality of neural network nodes into the node list table, the method further includes: Obtain the pre-set training count threshold and loss function threshold from the training task table; Training of the neural network model ends when the number of training iterations equals the training iteration threshold, or when the loss function value of the neural network model is less than the loss function threshold.

8. The database-based model training method according to claim 1, characterized in that, The derivative function library includes: activation and normalization and regularization function libraries, convolution and pooling function libraries, encoding and decoding function libraries, and loss function libraries.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the database-based model training method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which, when executed by a processor, implements the database-based model training method as described in any one of claims 1 to 8.

Images