Artificial Intelligence Training Parameter Iteration Method

Abstract

The present invention relates to an artificial intelligence training parameter iteration method, which comprises: a setting step, providing a training set and setting a numerical range of the training parameters; an initialization step, selecting three initial setting values ​​from the numerical range of the training parameters, and calculating the accuracy of each initial setting value according to the training set, the initial setting value with the highest accuracy is a first core value, and a first parameter range is set; a parameter optimization step, selecting three first iteration values ​​within the first parameter range, calculating the accuracy of each first iteration value, comparing the accuracy of each first iteration value, and setting a second parameter range with the highest first iteration value as the second core value; and a judgment step, if the accuracy of the second core value is higher than 0.9, the second core value is used as the training parameter standard value, if not higher than 0.9, the parameter optimization step is repeated until the accuracy of the test core value is higher than 0.9, and the test core value is used as the training parameter standard value.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an artificial intelligence training parameter iteration method. Background Art

[0002] With the advancement of science and technology, the application of artificial intelligence is becoming more and more extensive. For example, defect detection, face recognition, medical judgment, etc. can gradually see the application of artificial intelligence. Before general artificial intelligence actually enters the application field, it usually needs to be trained with data. The data training method can be algorithms such as neural network and convolutional neural network (CNN).

[0003] In the prior art, CNN deep learning is the most common image recognition learning and training method. In the prior art, since the training parameters are usually set as random numbers and the amount of input data is huge, the calculation process may generate a large amount of calculation data, which puts a heavy burden on memory and computing resources, resulting in long training time and poor efficiency. Summary of the invention

[0004] Here, an artificial intelligence training parameter iteration method is provided. The artificial intelligence training parameter iteration method includes a setting step, an initialization step, a parameter optimization step, and a judgment step. The setting step is to provide a training set and set the numerical range of at least two training parameters. The initialization step is to randomly select at least three initial setting values ​​from the numerical range of the training parameters, and calculate the accuracy of each of the three initial setting values ​​according to the training set. The initial setting value with the highest accuracy among the at least three initial setting values ​​is the first core value, and the first parameter range is set with the parameter coordinate value of the first core value as the physical center. The parameter optimization step is to select at least three first iteration values ​​in the first parameter range, calculate the accuracy of each first iteration value according to the training set, and compare the accuracy of at least three first iteration values, and take the first iteration value with the highest accuracy as the second core value, and set the second parameter range according to the parameter coordinate value of the second core value. The judgment step is to determine whether the accuracy of the second core value is higher than 0.9. If it is higher than 0.9, stop and use the second core value as the standard value of the training parameter. If it is not higher than 0.9, the second core value and the second parameter range are used to replace the first core value and the first parameter range, and the parameter optimization step is repeated until the accuracy of the test core value is higher than 0.9, and the parameter coordinates of the test core value are used as the standard values ​​of the training parameters.

[0005] In some embodiments, at least two parameters include batch learning number and learning rate, the range of batch learning number is 0.5 to 1.5, the range of learning rate is 0.5 to 1.5, and the first parameter range is a circle with the first core value as the center on a coordinate system with batch learning number and learning rate as the horizontal axis and vertical axis respectively. In more detail, in some embodiments, the range of learning number is 0.7 to 1.3, and the range of learning rate is 0.7 to 1.3.

[0006] More specifically, in some embodiments, in the initialization step, two graphics processors respectively select the batch learning quantity and learning rate of the initial setting values, and calculate the accuracy rate.

[0007] In some embodiments, the at least two parameters further include momentum, and the range of momentum is 0 to 1. Here, the first parameter range is a sphere with the first core value as the center on a coordinate system where the batch learning number, the learning rate and the momentum are the x-axis, the y-axis and the z-axis respectively. In more detail, the range of momentum is 0.3 to 0.8.

[0008] In some embodiments, at least two parameters further include normalization, and the range of normalization is 0.00001 to 0.001, wherein the first parameter range is a physical quantity range with the first core value as the physical center on the coordinate system of the batch learning number, learning rate, momentum, and normalization, respectively, on the x-axis, y-axis, z-axis, and w-axis. In more detail, in some embodiments, the range of normalization is 0.0001 to 0.0005.

[0009] In some embodiments, in the initialization step, any two of the batch learning number, learning rate, momentum, and regularization are selected by the first graphics processor, and the other two of the batch learning number, learning rate, momentum, and regularization are selected by the second graphics processor, and the accuracy is calculated by the third graphics processor.

[0010] In some embodiments, the artificial intelligence training parameter iteration method further includes a verification step. The verification step is to provide a test set and calculate the accuracy using the second core value or the test core value with an accuracy higher than 0.9 in the judgment step, and if the accuracy calculated based on the test set is lower than 0.9, the initialization step is performed again.

[0011] In summary, the artificial intelligence training parameter iteration method can accelerate the selection of parameter values ​​and speed up the overall artificial intelligence training process, so that the effect of rapid gradient descent in artificial training can be achieved with fewer files and shorter speeds, which can shorten the training time and greatly improve the training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 Flowchart of an iterative method for training parameters for artificial intelligence.

[0013] Figure 2 Schematic diagram of the initialization step.

[0014] Figure 3 and Figure 4 Schematic diagram of the parameter optimization steps.

[0015] [List of Reference Numerals]

[0016] S1 AI training parameter iteration method

[0017] S10 Setup Steps

[0018] S20 initialization steps

[0019] S30 parameter optimization steps

[0020] S40 Judgment Steps

[0021] S45 Select training parameters

[0022] S50 Inspection Steps

[0023] A1, A2, A3 initial setting value

[0024] B1, B2, B3 first iteration value

[0025] C1, C2, C3 second iteration value

[0026] R1 First parameter range

[0027] R2 Second parameter range

[0028] R3 The third parameter range DETAILED DESCRIPTION

[0029] Figure 1 The flowchart of the iterative method for artificial intelligence training parameters is shown in Figure 2. Figure 1 As shown, the artificial intelligence training parameter iteration method S1 includes a setting step S10, an initialization step S20, a parameter optimization step S30, and a judgment step S40.

[0030] The setting step S10 is to provide a training set and set the numerical ranges of at least two training parameters. Here, a convolutional neural network (CNN) is used for artificial intelligence training, and a model of a stochastic gradient descent method is used as a training set, which can be shown in the following program (1). However, this is only an example and not intended to be limiting. The training set here also includes calculation methods such as loss calculation.

[0031] Equation (1): Where w represents the weight, n is the batch size, and η is the learning rate.

[0032] As shown in the above equation (1), the batch learning amount and the learning rate will actually affect the convergence of the weights. When the batch learning amount is large, the number of batches can be reduced, and the training time can be reduced. However, relatively speaking, a larger learning amount will require a larger number of iterations, resulting in poor performance of the model and a larger amount of data generated. If the learning rate is too small, it means that the update range for the weights is very small, which will make the training very slow; and if the learning rate is too large, it may lead to failure to converge. Therefore, in addition to providing an appropriate training set, setting step S10 also requires setting the range of parameters. For example, the set parameters include the batch learning number and the learning rate. The range of the batch learning number is 0.5 to 1.5, and the range of the learning rate is 0.5 to 1.5. Preferably, the range of the learning number is 0.7 to 1.3, and the range of the learning rate is 0.7 to 1.3. However, the above is only an example and not intended to be limiting. The relevant parameters are not only the batch learning number and the learning rate.

[0033] Figure 2 is a schematic diagram of the initialization step, such as Figure 2 As shown, the initialization step S20 is to randomly select at least three initial setting values ​​A1, A2, and A3 from the numerical range of the training parameters. Here, the following uses the batch learning amount as the horizontal axis and the learning rate as an example, and randomly selects the coordinate values ​​of the three initial setting values ​​A1, A2, and A3 on the coordinate system as (x1, y1), (x2, y2), and (x3, y3). This is only for the convenience of presentation on the plane, and the actual parameters are not limited to this.

[0034] Next, the accuracy of each of the three initial setting values ​​is calculated based on the training set, and the calculation method of the accuracy rate (ACC) is 1-loss. After calculation, the accuracy rates of the three are compared, and the initial setting value with the highest accuracy among at least three initial setting values ​​A1, A2, and A3 (assuming it is A1) is used as the first core value, and the first parameter range R1 is set with the parameter coordinate value (x1, y1) of the first core value as the physical center. Here, the first parameter range R1 is a circle with the first core value (x1, y1) as the center on a coordinate system with the batch learning number and the learning rate as the horizontal axis and the vertical axis respectively. Here, the radius of the circle can be However, this is only an example, and a specific radius value may be preselected as the first parameter range R1.

[0035] Figure 3 is a schematic diagram of the parameter optimization steps. Figure 3As shown, the parameter optimization step S30 is to select at least three first iteration values ​​B1, B2, and B3 in the first parameter range R1, and calculate the accuracy of each first iteration value B1, B2, and B3 according to the training set, and compare the accuracy of at least three first iteration values ​​B1, B2, and B3, and take the first iteration value with the highest accuracy (assuming it is B3) as the second core value, and set the second parameter range R2 according to the parameter coordinate value (x6, y6) of the second core value as the physical center. Here, the radius of the circle can be However, this is only an example, and a specific radius value may be preselected as the first parameter range R1.

[0036] The judgment step S40 is to judge whether the accuracy of the second core value is higher than 0.9. If it is higher than 0.9, stop and enter step S45 to select the second core value (x6, y6) as the standard value of the training parameter for training.

[0037] Figure 4 is a schematic diagram of the parameter optimization steps. Figure 4 As shown, if the accuracy of the second core value is not higher than 0.9 in the judgment step S40, the first core value (x1, y1) and the first parameter range R1 are replaced by the second core value (x6, y6) and the second parameter range R2, and the parameter optimization step S30 is repeated to select at least three second iteration values ​​C1, C2, and C3, and calculate the accuracy of each second iteration value C1, C2, and C3 according to the training set, and compare the accuracy of at least three second training settings C1, C2, and C3 values, and take the second iteration value with the highest accuracy (assuming it is C1) as the third core value, and set the third parameter range R3 based on the parameter coordinate value (x7, y7) of the third core value C1 as the physical center. Here, the radius of the circle can be The determination step S40 and the parameter optimization step S30 may be repeated in this manner until the accuracy of a certain test core value is higher than 0.9, and the parameter coordinates of the test core value are used as the standard values ​​of the training parameters.

[0038] See again Figure 1 , the artificial intelligence training parameter iteration method S1 also includes a test step S50. The test step S50 is to provide a test set, and the values ​​in the test set are different from those in the training set. In the test step S50, the second core value with an accuracy higher than 0.9 in the judgment step S40, or the test core value obtained by the subsequent repeated parameter optimization step S30 is used to calculate the accuracy based on the test set. If the accuracy calculated based on the test set is higher than 0.9, the second core value or the test core value is used as the standard value of the training parameter. If the accuracy is lower than 0.9, the parameter is discarded and the initialization step S20 is performed again.

[0039] In the aforementioned embodiment, in the initialization step S20, two graphics processors respectively select the batch learning quantity and learning rate of the initial setting value, and calculate the accuracy rate. That is, Figure 2 The selection of the initial setting values ​​A1, A2, and A3, and the calculation of the accuracy of the initial setting values ​​A1, A2, and A3 are calculated by two graphics processors, respectively. In this way, the computing resources of the graphics processors can be dispersed to achieve faster computing efficiency.

[0040] However, Figures 2 to 4 It is just an example. The parameters that actually affect the training efficiency include momentum and regularization. If three parameters are set, for example, the number of batches, the learning rate, and the momentum, it can be understood that the first parameter range is a sphere with the first core value as the center in a coordinate system where the number of batches, the learning rate, and the momentum are the x-axis, y-axis, and z-axis respectively. If there are four parameters, the number of batches, the learning rate, the momentum, and the regularization, the first parameter range is the range of physical quantities with the first core value as the physical center in a coordinate system where the number of batches, the learning rate, the momentum, and the regularization are the x-axis, y-axis, z-axis, and w-axis respectively. However, it is not easy to show the iterative effect when drawing three-axis or four-axis spaces on a plane, so it is not drawn here. Those skilled in the art can refer to Figures 2 to 4 , considering the transformation in three-axis and four-axis space.

[0041] In more detail, if momentum and normalization are taken into consideration, the momentum ranges from 0 to 1, preferably from 0.3 to 0.8, and the normalization ranges from 0.00001 to 0.001, preferably from 0.0001 to 0.0005.

[0042] Furthermore, in the initialization step S20, considering momentum and regularization, any two of the batch learning number, learning rate, momentum, and regularization can be selected by the first GPU, and the other two can be selected by the second GPU, and the accuracy rate can be calculated by the third GPU. In this way, the computing resources are distributed by three GPUs to achieve faster computing effects, so that the gradient decreases faster.

[0043] Here, the comparative example is compared with the embodiment. The training sets actually used are all standard training sets provided by Google. The comparative example uses a standard AI framework, a Tesla 40m (2880CUDA, 12GB) GPU, and the Google tensorflow mode for training; while the embodiment uses four GTX 1050i (768CUDA, 4GB) GPUs in series, adopts the aforementioned embodiment, and takes the batch learning number, learning rate, momentum, and regularization in the range of 0.7 to 1.3, 0.7 to 1.3, 0.3 to 0.8, and 0.001 to 0.005, respectively, for training. The result of the comparative example is 86% accuracy in 14400 seconds; while the result of the embodiment is 98% accuracy in 900 seconds.

[0044] In summary, through the artificial intelligence training parameter iteration method of the present application, the parameter selection can be quickly optimized and the gradient descent convergence effect can be quickly achieved, which helps to improve the efficiency of training, reduce computing resources, and complete the training at a cheaper hardware cost.

[0045] Although the technical contents of the present invention have been disclosed as above in the form of preferred embodiments, they are not intended to limit the present invention. Any slight changes and modifications made by any person skilled in the art without departing from the spirit of the present invention should be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be based on the scope defined in the appended claims.

Claims

1. An artificial intelligence training parameter iteration method for defect detection, face recognition or medical judgment, comprising: The setting step provides a training set and sets the value range of at least two training parameters; wherein, The training set is image data; An initialization step, randomly selecting at least three initial setting values ​​from the numerical range of the training parameter, and calculating the accuracy of each of the initial setting values ​​according to the training set, wherein the initial setting value with the highest accuracy among the at least three initial setting values ​​is a first core value, and a first parameter range is set with the parameter coordinate value of the first core value as the physical center, wherein the accuracy is 1 minus the loss, and the first parameter range is a sphere with the first core value as the center and the reciprocal of twice the distance from the parameter coordinate value of the first core value to the origin of the parameter coordinate value as the radius; a parameter optimization step, selecting at least three first iteration values ​​in the first parameter range, calculating the accuracy of each of the first iteration values ​​according to the training set, comparing the accuracy of the at least three first iteration values, taking the first iteration value with the highest accuracy as the second core value, and setting a second parameter range as the physical center according to the parameter coordinate value of the second core value, wherein the second parameter range is a sphere with the second core value as the center and the reciprocal of twice the distance from the parameter coordinate value of the second core value to the origin of the parameter coordinate value as the radius; and A judging step, judging whether the accuracy of the second core value is higher than 0.9, if higher than 0.9, stopping, and taking the second core value as the standard value of the training parameter, if not higher than 0.9, replacing the first core value and the first parameter range with the second core value and the second parameter range, repeating the parameter optimization step until the accuracy of the test core value is higher than 0.9, and taking the parameter coordinates of the test core value as the standard value of the training parameter; wherein the at least two parameters include a batch learning number and a learning rate, the batch learning number ranges from 0.5 to 1.5, the learning rate ranges from 0.5 to 1.5, and the first parameter range is a circle with the first core value as the center on a coordinate system with the batch learning number and the learning rate as the horizontal axis and the vertical axis respectively.

2. The artificial intelligence training parameter iteration method as described in claim 1, wherein the range of the batch learning number is 0.7 to 1.3, and the range of the learning rate is 0.7 to 1.

3.

3. The artificial intelligence training parameter iteration method as described in claim 2, wherein in the initialization step, two graphics processors respectively perform the selection of the batch learning quantity and the learning rate of the initial setting value and the calculation of the accuracy rate.

4. The artificial intelligence training parameter iteration method as described in claim 1, wherein the at least two parameters also include momentum, the range of the momentum is 0 to 1, wherein the first parameter range is a sphere with the first core value as the center on the coordinate system where the batch learning number, the learning rate and the momentum are respectively the x-axis, the y-axis and the z-axis.

5. The artificial intelligence training parameter iteration method as claimed in claim 4, wherein the momentum ranges from 0.3 to 0.

8.

6. The artificial intelligence training parameter iteration method as described in claim 4, wherein the at least two parameters also include normalization, the range of the normalization is 0.00001 to 0.001, wherein the first parameter range is a physical quantity range with the first core value as the physical center on the coordinate system of the batch learning number, the learning rate, the momentum, and the normalization, respectively, on the x-axis, y-axis, z-axis, and w-axis.

7. The artificial intelligence training parameter iteration method as claimed in claim 6, wherein the normalized range is 0.0001 to 0.0005.

8. The artificial intelligence training parameter iteration method as described in claim 6, wherein in the initialization step, any two of the batch learning number, the learning rate, the momentum, and the regularization are selected by the first graphics processor, and the other two of the batch learning number, the learning rate, the momentum, and the regularization are selected by the second graphics processor, and the accuracy rate is calculated by the third graphics processor.

9. The artificial intelligence training parameter iteration method as described in claim 1 further includes a verification step, providing a test set, and calculating the accuracy based on the test set using the second core value or the test core value with an accuracy higher than 0.9 in the judgment step; if the accuracy calculated based on the test set is lower than 0.9, the initialization step is repeated.

Images