Calculation method and computing device thereof

Abstract

To provide a calculation method for a computing device.SOLUTION: A computation method for a computing device includes steps of converting input data into augmented data according to a hyperparameter combination, and inputting the augmented data into a primary network. A hypernetwork is configured to output a plurality of primary-network parameters of the primary network in accordance with the hyperparameter combination using a plurality of hypernetwork parameters. The primary network is configured to generate output data in accordance with the augmented data using the primary-network parameters. The hypernetwork parameters are trained or being trained, and the primary-network parameters are not trained.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a calculation method and its computing device, and particularly to a calculation method and its computing device that can improve model performance and shorten calculation time.

Background Art

[0002] In the development of deep learning models (such as image deep learning models), various data augmentation methods can be used to provide a larger amount of training data, enabling the deep learning model to train or learn using more diverse training data. However, selecting an inappropriate data augmentation method or an inappropriate combination of hyperparameters can lead to unnecessary calculation time and waste of resources, and even reduce the performance of the deep learning model. Existing technologies manually select a data augmentation method and its corresponding hyperparameter combination, train deep learning models one by one for the hyperparameter combination, and determine which deep learning model of a specific hyperparameter combination brings the best performance. However, this existing technology requires manually selecting hyperparameter combinations and training multiple deep learning models, which consumes a significant amount of labor and computing resources. Therefore, selecting an appropriate data augmentation method and its corresponding hyperparameter combination remains a major issue in the development of existing deep learning models.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Therefore, the main object of the present application is to provide a calculation method and its computing device for improving the drawbacks of the prior art.

Means for Solving the Problems

[0004] Embodiments of this application disclose a computation method for a computing device, the method comprising the steps of: transforming input data into augmented data according to a hyperparameter combination; and inputting the augmented data into a first-order network, the hypernetwork being configured to output a plurality of first-order network parameters of the first-order network according to a hyperparameter combination using a plurality of hypernetwork parameters, the first-order network being configured to output output data according to the augmented data using the first-order network parameters, the hypernetwork parameters being trained or in the process of being trained, and the plurality of first-order network parameters being untrained.

[0005] Another embodiment of the present invention discloses a computing device comprising a processing circuit configured to execute a primary network and a hypernetwork, and a storage circuit coupled to the processing circuit and configured to store instructions. The processing circuit is configured to execute instructions, which include converting input data into augmented data according to a hyperparameter combination, and inputting the augmented data to a primary network, the hypernetwork is configured to output a plurality of primary network parameters of the primary network according to a hyperparameter combination using a plurality of hypernetwork parameters, the primary network is configured to output output data according to the augmented data using the primary network parameters, the hypernetwork parameters are trained or in the process of being trained, and the plurality of primary network parameters are not trained.

[0006] Embodiments of the present invention disclose that multiple hypernetwork parameters are optimized during the training phase.

[0007] Embodiments of the present invention disclose that a plurality of hyperparameter combinations comprises a plurality of first hyperparameter combinations and at least one second hyperparameter combination, and in the test phase, at least one second hyperparameter combination is selected from the plurality of hyperparameter combinations according to a plurality of first model metrics corresponding to the plurality of first hyperparameter combinations, and the best hyperparameter combination is selected from the plurality of hyperparameter combinations according to at least a plurality of first model metrics and at least one second model metric corresponding to at least one second hyperparameter combination.

[0008] Embodiments of the present invention disclose that after the training phase is completed, the multiple hypernetwork parameters remain unchanged regardless of the hyperparameter combination.

[0009] Embodiments of the present invention disclose that in the training phase, a hypernetwork is trained using at least one third hyperparameter combination, and in the test phase, the hypernetwork outputs a plurality of fourth primary network parameters of a plurality of primary networks corresponding to a plurality of fourth hyperparameter combinations using the trained plurality of hypernetwork parameters so that the best hyperparameter combination is selected from a plurality of fourth hyperparameter combinations, wherein at least one of the plurality of fourth hyperparameter combinations is different from at least one of the at least one third hyperparameter combination.

[0010] Embodiments of the present invention disclose that in the training phase, a hypernetwork is trained using at least one third hyperparameter combination, and in the test phase, the hypernetwork outputs multiple fourth primary network parameters of a primary network corresponding to a plurality of fourth hyperparameter combinations using the trained plurality of hypernetwork parameters so that the best hyperparameter combination is selected from a plurality of fourth hyperparameter combinations, wherein the upper limit of the plurality of fourth hyperparameter combinations is less than or equal to the upper limit of at least one third hyperparameter combination, and the lower limit of the plurality of fourth hyperparameter combinations is greater than or equal to the lower limit of at least one third hyperparameter combination.

[0011] Embodiments of the present invention disclose that in the training phase, a hypernetwork is trained using a plurality of third hyperparameter combinations, and in the test phase, the hypernetwork outputs a plurality of fourth primary network parameters of a primary network corresponding to a plurality of fourth hyperparameter combinations using the trained plurality of hypernetwork parameters so that the best hyperparameter combination is selected from a plurality of fourth hyperparameter combinations, wherein the difference between any two of the plurality of fourth hyperparameter combinations is smaller than the difference between any two of the third hyperparameter combinations.

[0012] These and other objects of the present invention will be undoubtedly apparent to those skilled in the art by reading the following detailed description of the various drawings and preferred embodiments illustrated in the drawings. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram of a computing device according to an embodiment of the present invention. [Figure 2] This is a schematic diagram of a calculation method according to an embodiment of the present invention. [Figure 3] This is a schematic diagram of a computing device according to an embodiment of the present invention. [Figure 4] This is a schematic diagram of a computing device according to an embodiment of the present invention. [Figure 5] This is a schematic diagram of AUROC for different hyperparameter combinations. [Figure 6] This is a schematic diagram of AUROC for different hyperparameter combinations. [Figure 7] This is a schematic diagram of the computing architecture of an existing neural network architecture. [Figure 8] This is a schematic diagram of the computing architecture of a hyper-primary network architecture according to an embodiment of the present invention. [Figure 9] This is a schematic diagram of a computing device according to an embodiment of the present invention. [Figure 10] This is a schematic diagram of input data and augmentation data according to embodiments of the present invention. [Modes for carrying out the invention]

[0014] Figure 1 is a schematic diagram of a computing device 10 according to an embodiment of the present invention. The computing device 10 (e.g., a chip, computer, or host) includes a memory circuit 110 and a processing circuit 120. The computing device 10 may be deployed in industrial production lines, drones, sensors, etc. The computing device 10 has at least one optimal hyperparameter

number

number

number

[0015] For example, the computing device 10 may automatically select a rotation angle of 175° or an image brightness of 0.8, resulting in a hyperparameter combination σ ∨ This includes 175° and 0.8. Furthermore, after rotating the input data 10IN by 175° and adjusting the image brightness of the input data 10IN to 0.8 to convert it into augmented data 10UT, the computing device 10 can automatically obtain the output data 10PD corresponding to the input data 10IN. Also, in the case of a classification task (or segmentation task), the output data 10PD is a more accurate class (or a more accurate segmented image). In other words, the computing device 10 can automatically and efficiently select an appropriate data augmentation method or an appropriate hyperparameter combination and automatically and efficiently optimize the corresponding deep learning model in the inference phase, thereby saving labor, computation time, or resources.

[0016] In one embodiment, the training dataset and validation dataset may be used in the training phase. The test dataset may be used in the test phase. The inference dataset, which may be used in the inference phase, includes unlabeled data.

[0017] Figure 2 is a schematic diagram of a calculation method 20 according to an embodiment of the present invention. The calculation method 20 may be used on a computing device (e.g., 10). At least a portion of the calculation method 20 may be compiled into program code. The calculation method 20 may include the following steps:

[0018] Step S202: The computing device or user defines the data augmentation method to employ. For example, the data augmentation method may include, but is not limited to, image inversion, image rotation, image shift, image scaling, image brightness or contrast adjustment, or a combination thereof.

[0019] Step S204: The computing device or user defines the possible range of hyperparameters for each data augmentation method. For example, the hyperparameter range for image rotation may be from 0 to 360 degrees, and the hyperparameters used in the training phase may be integers or floating-point numbers between 0 and 360 degrees within the hyperparameter range. For example, the hyperparameter range for image brightness adjustment may be from 0 to 1, and the hyperparameters used in the training phase may be floating-point numbers between 0 and 1 degree within the hyperparameter range.

[0020] Step S206: The computing device samples a hyperparameter combination (e.g., σ in Figure 3). For example, through random sampling, the hyperparameter combination σ is randomly sampled within a hyperparameter range having a random distribution p(σ), where p(σ) may be any random distribution (e.g., a uniform distribution). In one embodiment, sampling a particular hyperparameter combination means determining a particular data augmentation method.

[0021] Step S208: The computing device performs data augmentation based on the sampled hyperparameter combinations. For example, the computing device applies a hyperparameter combination σ (e.g., rotation angle 45° or image brightness 0.5) to one or more input data in the training dataset so that each input data (e.g., 30IN in Figure 3) is individually transformed into augmented data (e.g., x in Figure 3).

[0022] Step S210: The computing device generates first-order network parameters based on the sampled hyperparameter combination. For example, the computing device inputs the hyperparameter combination σ into the hypernetwork. The hypernetwork uses the hypernetwork parameters (e.g., ω in Figure 3) to generate the corresponding first-order network parameters (e.g., in Figure 3) according to the hyperparameter combination σ.

number

[0023] Step S212: The computing device calculates the output data. For example, the computing device inputs the augmented data created by the transformation in step S208 into the primary network. The primary network uses the primary network parameters to calculate the corresponding output data (for example, in Figure 3) according to each augmented data (for example, x).

number

[0024] Step S214: The computing device updates the hypernetwork parameters (e.g., ω). For example, the computing device calculates a loss function or model metric and uses backpropagation to optimize or tune the hypernetwork parameters.

[0025] Step S216: The computing device determines whether an epoch has been completed. For example, the computing device determines whether all input data (e.g., 30 IN) of the training dataset has been processed once. If the computing device determines that there is input data that has not yet been computed, it proceeds with training using the remaining input data, for example by re-executing step S208 or S206, and converts at least one of the remaining input data into at least one augmented data. If the computing device determines that an epoch has been completed, it executes step S218, for example.

[0026] Step S218: The computing device determines whether the training phase is complete. For example, if the loss function converges or the model metrics meet the target, the computing device determines that the training phase is complete and then performs step S220. If the training phase is not complete, the computing device performs step S206 again, for example, using the same or a different hyperparameter combination for training.

[0027] Step S220: The computing device determines the hyperparameter range to search for each data augmentation method. The hyperparameter range in step S220 may be the same as or different from the hyperparameter range in step S204 (for example, the same or smaller). For example, the upper limit of the hyperparameter range in step S220 may be smaller than the upper limit of the hyperparameter range in step S204, and the lower limit of the hyperparameter range in step S220 may be larger than the lower limit of the hyperparameter range in step S204. In one embodiment, during the training phase, the rotation angle defined in step S204 may be between 90 and 180 degrees. During the test or inference phase, the rotation angle may be between 90 and 180 degrees. However, in another embodiment, the rotation angle may be set to 240 degrees in step S220, and the computing device can still perform calculations.

[0028] Step S222: The computing device selects a hyperparameter combination (e.g., σ1 in Figure 4). In one embodiment, selecting a particular hyperparameter combination means determining a particular data augmentation method. For example, a rotation angle of 0° means no image rotation. In one embodiment, the data augmentation method used in the training phase may be selected in step S222, and a different hyperparameter combination than that used in the training phase may be selected in step S222. For example, image scaling is not used in the training phase and is not used in the test or inference phases.

[0029] Step S224: The computing device performs data augmentation according to the selected hyperparameter combination. For example, the computing device applies the selected hyperparameter combination σ1 (e.g., a rotation angle of 60° or an image brightness of 0.8) to one or more input data in the test dataset so that each input data (e.g., 40IN in Figure 4) is transformed into augmented data (e.g., x1 in Figure 4).

[0030] Step S226: The computing device generates first-order network parameters according to the selected hyperparameter combination. For example, the computing device inputs the selected hyperparameter combination σ1 into the hypernetwork. The hypernetwork then generates the trained hypernetwork parameters (for example, in Figure 4).

number

number

[0031] Step S228: The computing device calculates the output data. For example, the computing device inputs the augmented data created by the transformation in step S224 into the primary network. The primary network uses the primary network parameters to calculate the corresponding output data (for example, x1 in Figure 4) for each augmented data (for example, in Figure 3).

number

[0032] Step S230: The computing device determines whether further calculations are needed for other hyperparameter combinations. For example, the computing device determines whether all hyperparameters within the hyperparameter range of step S220 have been calculated once (e.g., Figure 5). Alternatively, the computing device directly performs step S222 to select the hyperparameter combinations to be calculated (e.g., Figure 6). The computing device determines which hyperparameter combinations require calculation (e.g., σ2~σ in Figure 8). n If it is determined that ) still exists, for example, step S222 or S224 is re-executed; otherwise, the process proceeds to step S232.

[0033] Step S232: The computing device selects the best hyperparameter combination. For example, based on the model metrics corresponding to the hyperparameter combination, the computing device selects all calculated hyperparameter combinations (e.g., σ1~σ in Figure 8). n Select the best hyperparameter combination (e.g., σ² in Figure 8) from the above.

[0034] Step S234: The computing device performs data augmentation according to the best hyperparameter combination. For example, the computing device applies the best hyperparameter combination (e.g., σ in FIG. 1 ∨ ) to the input data of the inference dataset (e.g., 10IN in FIG. 1) to convert the input data into augmented data (e.g., 10UT in FIG. 1).

[0035] Step S236: The computing device determines the primary network parameters based on the best hyperparameter combination. For example, the computing device inputs the best hyperparameter combination σ ∨ into the hypernetwork. The hypernetwork uses the trained hypernetwork parameters (e.g., w in FIG. 4 ∨ ) and correspondingly outputs the primary network parameters of the primary network according to the hyperparameter combination σ ∨ . Alternatively, the computing device looks up a table to determine the primary network parameters. The hyperparameter combination may be represented in vector form.

[0036] Step S238: The computing device calculates the output data. For example, the computing device inputs the augmented data created by the transformation in Step S234 into the primary network. The primary network uses the primary network parameters to output the corresponding output data (e.g., 10PD in FIG. 1) for the augmented data.

[0037] One or more of steps S202 to S238 may be omitted or their order may be changed as needed. For example, in one embodiment, only at least one of S206 to S214 (e.g., step S208) may be executed to perform or implement the training phase. In one embodiment, the iteration of the training phase may include at least one of steps S206 to S214 (e.g., step S208 or S214). In one embodiment, the epoch of the training phase may include at least one of steps S206 to S214. In one embodiment, for a complete batch, step S216 may be omitted. In one embodiment, the order of steps S208 and S210 may be changed or they may be performed in parallel. In one embodiment, only at least one of steps S222 to S232 (e.g., step S224) may be executed to perform or implement the test phase. In one embodiment, the order of steps S224 and S226 may be changed or they may be performed in parallel. In one embodiment, step S220 may be omitted. In one embodiment, only at least one of steps S234 to S238 (e.g., step S234) may be performed to execute or carry out the inference phase. In one embodiment, the order of steps S234 and S236 may be reversed or performed in parallel.

[0038] Figure 3 is a schematic diagram of a computing device 30 according to one embodiment of the present invention. Computing device 10, input data 10IN, extended data 10UT, output data 10PD and hyperparameter combination σ ∨ These may be implemented using a computing device 30, input data 30IN, extended data x, output data y^, and hyperparameter combination σ, respectively, or vice versa. The computing device 30 may include a primary network 30P and a hypernetwork 30H. Primary network parameters of primary network 30P

number

number

number

[0039] The first-order network 30P includes multiple layers. Each layer includes multiple neurons. The output of any given layer is, for example, its input and at least one first-order network parameter (e.g., θ^ l1 It is a linear combination or function of ). In step S212, after the augmented data x is input to the first-order network 30P, the first-order network 30P generates output data y^ according to the first-order network parameter θ^. The first-order network 30P is the model architecture

number

number

[0040] The hypernetwork 30H consists of multiple layers, each containing multiple neurons. The output of any given layer is its input and at least one hypernetwork parameter (e.g., the weight ω of a particular layer). h,li Or the bias ω of a specific layer b,liIt is a linear combination or function of ). For example, the output, its input and hypernetwork parameters are:

number

[0041] From step S214, the hypernetwork parameter ω of hypernetwork 30H is trainable. The training phase is the loss function

number

number

[0042] For example, the computing device 30 sets the partial derivative of the loss function L(y^,y) to 0 with respect to the hypernetwork parameter ω (for example

number

[0043] Alternatively, the computing device 30 can determine the optimal hypernetwork parameters w ∨ You can iteratively find the optimal hypernetwork parameters w, or use methods such as gradient descent to find them. ∨ It can get closer to it. Hypernetwork parameter ω h,li Let's take this as an example. In one iteration, the updated hypernetwork parameters ω are used to decrease the loss function L(y^,y). ∨ h,li This is the original hypernetwork parameter ω h,li minus

number

number

number

[0044] From step S210, the primary network parameter θ^ of the primary network 30P is untrainable. In step S210, after the hyperparameter combination σ is input to the hypernetwork 30H, the hypernetwork 30H outputs the primary network parameter θ^ according to the hypernetwork parameter ω. In one embodiment, the hyperparameter combinations of any two iterations may be different or the same. In other words, a hyperparameter combination (referred to as the fifth hyperparameter combination) may be sampled in one iteration of step S206, and a different hyperparameter combination (referred to as the sixth hyperparameter combination) may be sampled in another iteration of step S206. However, even if the same hyperparameter combination σ is sampled in two iterations (for example, the fifth hyperparameter combination is the same as the sixth hyperparameter combination), the primary network parameters of the two iterations will be different. Specifically, in step S210 of a given iteration, the hypernetwork 30H outputs multiple primary network parameters (each referred to as the fifth primary network parameter). After the hypernetwork parameter ω is updated in this iteration, in step S210 of the next iteration, the hypernetwork 30H outputs several primary network parameters (each referred to as the sixth primary network parameter) that are different from the fifth primary network parameter. In other words, after each iteration, the hypernetwork parameter ω changes, and the primary network parameter θ^ output from the hypernetwork 30H also changes.

[0045] In short, the hypernetwork parameter ω or the hypernetwork is trained in the training phase of this application. However, the primary network parameter θ^ cannot be trained (for example, the primary network parameter θ^ is not trained or will not be trained). Instead, the primary network parameter θ^ is passively provided to the primary network 30P by the hypernetwork 30H. In other words, after the training phase is completed, the hypernetwork parameter ω does not change with respect to the hyperparameter combination σ, but the primary network parameter θ^ changes with respect to the hyperparameter combination σ or based on calculations by the hypernetwork 30H.

[0046] Figure 4 is a schematic diagram of a computing device 40 according to an embodiment of the present invention. Computing device 10, input data 10IN, extended data 10UT, output data 10PD and hyperparameter combination σ ∨ Each of these can be implemented using a computing device 40, input data 40IN, extended data x1, output data y^1, and hyperparameter combination σ1, or vice versa. The computing device 40 may include a primary network 40P and a hypernetwork 40H, which are structurally and functionally the same as or similar to the primary network 30P and the hypernetwork 30H, respectively. Primary network parameters

number

number

[0047] In another embodiment, Figures 3 and 4 may be used to illustrate the training phase and test phase (or inference phase) of a computing device, respectively. For example, hypernetwork parameter ω is hypernetwork parameter ω ∨ It is updated to . Therefore, even if the hyperparameter combination σ is the same as the hyperparameter combination σ1, the first-order network parameter θ^ is updated to the first-order network parameter θ^ σ1 This may differ from the above.

[0048] In one embodiment, before the test phase, the hypernetwork parameter ω is the optimal hypernetwork parameter ω ∨ It has been updated to this state. Taking step S226, the hypernetwork 40H has different hyperparameter combinations (e.g., σ1 or σ in Figure 8). n Corresponding to this, the primary network parameters of the primary network (for example, θ^ in Figure 8) σ1 or θ^ σn ) outputs. In step S228, the first-order network 40P calculates output data corresponding to the augmented data using the first-order network parameters. In step S232, the computing device 40 calculates a model metric for each hyperparameter combination. After comparing all acquired model metrics, the computing device 40 may select the best model metric. Corresponding to the best model metric, the computing device 40 outputs all calculated hyperparameter combinations (σ1~σ n ) to the best hyperparameter combination (e.g., σ ∨ ) can be selected.

[0049] From another perspective, hyperparameter combinations (e.g., σ) ∨) may be considered as input to the hypernetwork 40H. Therefore, multiple hyperparameters in a hyperparameter combination are neither updated nor trained. Instead, the best hyperparameter combination is selected from multiple calculated hyperparameter combinations.

[0050] In one embodiment, before the inference phase, the hypernetwork parameter ω is the optimal hypernetwork parameter ω ∨ The computing device 40 has been updated and has determined the best hyperparameter combination. Thus, the primary network parameters are determined. Therefore, step S236 or hypernetwork 40H may be omitted. Also, the primary network 40P in step S238 can directly use the known primary network parameters to infer output data corresponding to any extended data in step S234.

[0051] In step S222, the method by which the computing device 40 selects a hyperparameter combination may be adaptively adjusted. For example, Figures 5 and 6 are schematic diagrams of receiver operating characteristic curves (AUROC) for different hyperparameter combinations according to embodiments of the present invention.

[0052] In one embodiment, the best hyperparameter combination can be found using a one-by-one search method. For example, Figure 5 shows AUROC for each hyperparameter combination. In step S220, for image rotation, the computing device 40 may define the hyperparameter range to be from 0 to 360 degrees, define the hyperparameters within the hyperparameter range as integers between 0 and 360 degrees, and set the common difference between hyperparameters to 5 degrees. For image brightness, the computing device 40 may define the hyperparameter range to be from 0 to 1, define the hyperparameters within the hyperparameter range as floating-point numbers between 0 and 1, and set the common difference between hyperparameters to 0.1. In step S222, the computing device 40 may sequentially select one of 720 hyperparameter combinations (i.e., (360 ÷ 5) × (1 ÷ 0.1) = 720) to perform step S224 or S226. After searching these 720 hyperparameter combinations, in step S228, the computing device 40 can calculate the corresponding model metric (e.g., AUROC). As shown in Figure 5, since the AUROC for a rotation angle of 175° and an image brightness of 0.8 is closest to 1, in step S232, the hyperparameter combination of 175° and 0.8 is the best hyperparameter combination (e.g., σ ∨ ) may be selected. The computing device 40 may use a rotation angle of 175° and an image brightness of 0.8 as hyperparameters for subsequent data augmentation (e.g., step S234) to improve estimation accuracy.

[0053] In one embodiment, to reduce the number of hyperparameter combinations to search, an optimization algorithm (e.g., Bayesian optimization or a tree-structured Tarzen estimator algorithm) may be used to find the best hyperparameter combination. For example, in step S222, the computing device 40 may randomly select several hyperparameter combinations (each called a first hyperparameter combination) and compute their corresponding model metrics in step S228. Thus, the computing device 40 may select hyperparameter combinations from the first hyperparameter combinations that correspond to better model metrics (e.g., σ2, σ2 in Figure 8). n ) can then find a better hyperparameter combination (e.g., σ², σ²). Next, returning to step S222, the computing device 40 can find a better hyperparameter combination (e.g., σ², σ²). n ) Hyperparameter combinations that are close to (or far from) each of these (for example, σ3, σ in Figure 8) n-1 The computing device selects (each referred to as the second hyperparameter combination). Thus, in step S228, the computing device 40 can calculate the model metric corresponding to the second hyperparameter combination and find a hyperparameter combination (e.g., σ3) that corresponds to a better model metric from the second hyperparameter combination. The computing device 40 then iterates back to step S222 and can select a hyperparameter combination (e.g., σ4 in Figure 8) that is close to the hyperparameter combination (e.g., σ3) selected in the previous iteration and calculate the corresponding model metric. In this way, after searching a certain number of hyperparameter combinations, the computing device 40 can find the best hyperparameter combination (e.g., σ in Figure 1). ∨ ) is the searched hyperparameter combination (e.g., σ2~σ4, σ n-1 ~σ n ) can be found from.

[0054] For example, Figure 6(a) shows the AUROC corresponding to 200 hyperparameter combinations. The computing device 40 may set the maximum number of hyperparameter combinations to search to 200 and stop the search after a certain period of time. Similarly, Figure 6(b) shows the AUROC corresponding to 100 hyperparameter combinations. Figure 6(a) or (b) shows that the AUROC for a rotation angle of 175° and an image brightness of 0.8 is closest to 1, and therefore, in step S232, the hyperparameter combination of 175° and 0.8 is the best hyperparameter combination (e.g., σ ∨ ) can be selected as. However, compared to Figures 5 and 6(a), Figure 6(b) uses only 100 hyperparameter combinations, saving computation time or resources.

[0055] In one embodiment, the computing device selects the best hyperparameter combination based on the type of input data (e.g., input images). For example, if the input image in the inference phase relates to a screw, the input image in the training phase also relates to a screw, and the best hyperparameter combination is selected for the screw. For example, if the input image in the inference phase relates to a defect in an embedded wire, the best hyperparameter combination is selected for the defect in the embedded wire. In one embodiment, the best hyperparameter combination selected may be related to the type of input data (e.g., input images), but independent of the size or ratio of the input data.

[0056] Table 1 lists the differences between existing neural network architectures and the hyper-first-order network architecture proposed by the present invention. Figure 7 is a schematic diagram of the computing architecture of an existing neural network architecture. Figure 8 is a schematic diagram of the computing architecture of a hyper-first-order network architecture according to an embodiment of the present invention. [Table 1]

[0057] In existing neural network architectures, multiple deep learning models 70M1 to 70Mn use different hyperparameter combinations σ1 to σ n Each model needs to be trained individually. Furthermore, existing neural network architectures employ grid search. Due to the diverse and numerous data augmentation methods and corresponding hyperparameter combinations, the number of deep learning models (70M1 to 70Mn) that need to be trained can exponentially double with each addition of a new data augmentation method. Therefore, existing neural network architectures or existing hyperparameter search methods are not ideal.

[0058] In the hyper-first-order network architecture of the present invention, the hyper-first-order network includes a first-order network (e.g., 30P or 40P) and a hypernetwork (e.g., 30H or 40H). Regardless of the number of data augmentation methods or hyperparameter combinations, only one deep learning model 80M (e.g., hypernetwork) needs to be trained. After training the deep learning model 80M is complete, different hyperparameter combinations σ1~σ n However, this can be input into a trained deep learning model 80M. Furthermore, the hypernetwork uses different hyperparameter combinations σ1~σ n The optimal primary network parameter θ^ corresponding to each of these values. σ1 ~θ^ σn This provides the ability to build different primary networks in the test phase. Therefore, different hyperparameter combinations σ1~σ n When dynamically adjusted, the present invention provides the primary network parameter θ^ to the primary network. σ1 ~θ^ σn This ensures that it is optimal and improves the model performance of the first-order network. In other words, after training the deep learning model 80M is complete, different hyperparameter combinations σ1~σ nThis makes it possible to quickly search for the model, and the present invention also guarantees that the corresponding primary network has optimal model performance.

[0059] In one embodiment, as shown in Table 1 or Figure 8, the hyperparameter combinations used in the test phase (e.g., step S222) or the inference phase (e.g., step S234 or S236) are either sampled or unsampled in the training phase (step S206). In other words, the hyperparameter combinations used in the test phase (each referred to as the fourth hyperparameter combination) may be different from the hyperparameter combinations used to train the hypernetwork (each referred to as the third hyperparameter combination). However, the present invention does not require retraining with respect to the fourth hyperparameter combination. Instead, the present invention generates primary network parameters for the primary network corresponding to the fourth hyperparameter combination, enabling direct inference on the input data.

[0060] In one embodiment, as shown in Table 1 or Figure 8, the continuity of hyperparameter combinations used in the test phase (e.g., step S222) or inference phase (e.g., step S234 or S236) may be higher than the continuity of hyperparameter combinations sampled in the training phase (step S206). In other words, the difference between hyperparameters of any two fourth hyperparameter combinations (e.g., the difference between 174.99° and 175°) may be smaller than the difference between hyperparameters of any two third hyperparameter combinations (e.g., the difference between 170° and 175°). In other words, the hyperparameter search space in the test phase of existing neural network architectures is discrete. In contrast, the hyperparameter search space in the test phase of the present invention may be continuous.

[0061] Therefore, the present invention can rapidly and automatically generate various augmented data (e.g., automated optical detection images), reduce the training time of the deep learning model 80M, and further reduce the time spent searching for the best data augmentation method or best hyperparameter combination using an optimization algorithm. Furthermore, the model performance of the present invention is better compared to existing neural network architectures.

[0062] The present invention incorporates a hypernetwork, which is used to provide the primary network with optimal primary network parameters. Therefore, the primary network of the present invention can be applied to different model architectures or different image tasks. In another embodiment, when using different types of data, the primary network may be modified and replaced with a different primary network corresponding to the data type. For example, a first-order network may use, but is not limited to, image classification models (e.g., Residual Neural Network (ResNet), Densely Connected Convolutional Network (DesNet), MobileNet, EfficientNet, etc.), image segmentation models (e.g., UNet, Pyramid Scene Parsing Network (PSPNet), Feature Pyramid Network (FPN), LinkNet, etc.), or object detection models (e.g., You Only Look Once (YOLO) algorithm, Single Shot Detector (SSD), Region-based Convolutional Neural Network (R-CNN), Mask R-CNN, etc.).

[0063] In one embodiment, the primary network may be a CNN-based deep learning network. The primary network and the hypernetwork may be coupled to a hyper-primary network deep learning network architecture. For example, Figure 9 is a schematic diagram of a computing device 90 according to an embodiment of the present invention. Computing device 10, input data 10IN, augmented data 10UT, output data 10PD and hyperparameter combination σ ∨ These may be implemented using a computing device 90, input data 90IN, extended data x9, output data y^9, and hyperparameter combination σ, respectively, and vice versa. The computing device 90 may include a primary network 90P and a hypernetwork 90H. The primary network 30P (or 40P) and the hypernetwork 30H (or 40H) may be implemented using a primary network 90P and a hypernetwork 90H, respectively, and vice versa.

[0064] The primary network 90P includes multiple layers (e.g., 90C1, 90C2, 90N1, 90N2, 90R1, 90R2, 90P1). Layers 90C1 and 90C2 may be convolutional layers. Layers 90N1 and 90N2 may be batch normalization layers. Layers 90R1 and 90R2 may be rectified linear unit (ReLU) layers. Layer 90P1 may be a dense ReLU layer. The present invention is not limited thereto.

[0065] The hypernetwork 90H includes multiple layers (e.g., 90H1, 90H2, 90D1-90Dj). Layers 90H1 and 90H2 may be dense ReLU layers, and layers 90D1-90Dj may be dense layers. Alternatively, layer 90H1 or 90H2 may be

number

[0066] In one embodiment, the hypernetwork parameter (e.g., ω) h,l1 ~ω b,li or ω ∨ h,l1 ~ω ∨ b,li ) or first-order network parameters (e.g., θ^ l1 ~θ^ lj or θ^ σ1,l1 ~θ^ σ1,lj ) may be a scalar, vector, or matrix. For example, the last layer of hypernetwork 90H is used to output the first-order network parameter θ^ to first-order network 90P. Therefore, the number of values ​​output by hypernetwork 90H depends on the number of first-order network parameters θ^ required by first-order network 90P. For example, layer 90C1 may be a 3x3 convolutional layer, and the first-order network parameter θ^ of layer 90C1 may be l1 teeth,

number

number

[0067] In one embodiment, the input data (e.g., 10IN) or extended data (e.g., 10UT) of the present invention may be of various data types. In one embodiment, the input data or extended data of the present invention is image data. For example, Figure 10 is a schematic diagram of input data 11IN and extended data 11UT1 and 11UT2 according to an embodiment of the present invention. Extended data 11UT1 is implemented using input data 11IN rotated by 45°. Extended data 11UT2 is implemented using input data 11IN rotated by 45° with image brightness adjusted to 0.5. However, the present invention is not limited thereto. The input data or extended data may be numerical data (e.g., moving average, time adjustment, bootstrap, etc.), text data (e.g., word replacement, word insertion, word deletion, etc.), audio or video data (e.g., video speed adjustment, segment shift, pitch adjustment, etc.), or signal data (e.g., signal mixing, signal amplification / reduction, sampling frequency, etc.).

[0068] Since training deep learning models requires data augmentation methods to reduce overfitting, the computational method of the present invention can be adopted in any technical field. In one embodiment, the present invention belongs to computer vision technology and can be applied to various fields (e.g., medical image processing, general everyday image processing, advanced driver-assistance systems (ADAS), automated inspection, etc.). However, the present invention is not limited thereto and can be applied to other fields as well.

[0069] In one embodiment, the model metric may be, but is not limited to, AUROC or precision. In one embodiment, the optimization method may be, but is not limited to, backpropagation or the Adam Optimizer. In one embodiment, the loss function may be, but is not limited to, calculated using binary cross-entropy. In one embodiment, the test dataset may contain one or more training data from the training dataset, or the intersection of the test dataset and the training dataset may be an empty set. In one embodiment, i, j, k, m, or n are positive integers. In one embodiment, a particular data augmentation method may not be used. For example, if image rotation is not used, the rotation angle is set to 0°. If image brightness adjustment is not used, the image brightness is set to 1. If image mirroring is not used, the image mirroring is set to the identity matrix. Details or modifications of data augmentation methods are disclosed in Taiwan Patent Application No. 113116850, which is incorporated herein by reference in its entirety and becomes part of this specification. The technical features described in the embodiments above may be mixed or combined in various ways, as long as they do not conflict with each other.

[0070] In summary, the present invention enables rapid and efficient verification or testing of model performance under different data augmentation methods and hyperparameter combinations by training only a single deep learning model. Furthermore, the present invention introduces an optimization algorithm to further reduce the time required to search for the best data augmentation method and best hyperparameter combination. Moreover, the present invention enables the construction of an automated machine learning (AutoML) system, allowing users without prior knowledge of machine learning to apply different data augmentation methods and automatically generate various augmented images in batches.

[0071] Those skilled in the art will readily understand that many modifications and changes to the devices and methods can be made while maintaining the teachings of the present invention. Accordingly, the above disclosure should be construed as being limited only by the boundaries of the appended claims.

Claims

1. A method of computation performed by a computing device, The steps include: converting input data into augmented data according to a combination of hyperparameters, A step of inputting the extended data into a primary network, wherein the primary network is configured to output output data according to the extended data using a plurality of primary network parameters. Includes, The plurality of primary network parameters of the primary network are output by the hypernetwork using the plurality of hypernetwork parameters according to the hyperparameter combination. During the training phase, the multiple hypernetwork parameters are either being trained or in the process of being trained. The aforementioned multiple primary network parameters are not trained during the training phase. In the test phase following the training phase, the labeled input data is transformed into multiple augmented data according to multiple hyperparameter combinations, and the best model metric is selected from the multiple hyperparameter combinations by comparing multiple model metrics corresponding to the multiple hyperparameter combinations, thereby selecting the best hyperparameter combination corresponding to the best model metric. Calculation method.

2. In the training phase, at least one hyperparameter among the hyperparameter combination is randomly sampled from a plurality of hyperparameters to convert the labeled input data into the augmented data according to the hyperparameter combination. The calculation method according to claim 1.

3. In the inference phase following the training phase and the test phase, the unlabeled input data is converted into augmented data based on the selected best hyperparameter combination, and the plurality of primary network parameters are generated by the hypernetwork based on the selected best hyperparameter combination and according to the trained plurality of hypernetwork parameters. The calculation method according to claim 1.

4. In computing devices, A processing circuit configured to execute a primary network and a hypernetwork, wherein the processing circuit includes a processing circuit configured to execute instructions, The system comprises a storage circuit coupled to the processing circuit and configured to store the instruction, The aforementioned instruction is, The process involves transforming input data into augmented data according to a combination of hyperparameters, The extended data is input to a primary network, and the primary network is configured to output output data according to the extended data using a plurality of primary network parameters. Includes, The plurality of primary network parameters of the primary network are output by the hypernetwork using the plurality of hypernetwork parameters according to the hyperparameter combination. During the training phase, the multiple hypernetwork parameters are being trained, and during the test and inference phases following the training phase, the multiple hypernetwork parameters are pre-trained. The aforementioned plurality of primary network parameters are not trained in any of the training phase, the test phase, or the inference phase. In the test phase, the labeled input data is transformed into multiple augmented data according to multiple hyperparameter combinations, and the best model metric is selected from the multiple hyperparameter combinations by comparing multiple model metrics corresponding to the multiple hyperparameter combinations, thereby selecting the best hyperparameter combination corresponding to the best model metric. Computing device.

Images