A method and device for setting learning rate of target detection model
By calculating the step size and initialization learning rate, and dynamically adjusting the learning rate according to the average accuracy mean of the verification set, the problem of low efficiency in setting the learning rate of the object detection model is solved, and a more efficient model training effect is achieved.
Patent Information
- Application Number
- CN201910919042.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-09-26
- Publication Date
- 2025-02-25
- Estimated Expiration
- 2039-09-26
AI Technical Summary
In the prior art, the initial value of the learning rate and adjustment strategy of the object detection model are difficult to effectively set, resulting in low training efficiency and multiple attempts to adjust are required to obtain a suitable learning rate adjustment plan.
By setting the total number of pictures of the training set and the number of pictures of each iteration training round, calculate the step size, and initialize the learning rate. During the training process, each step iteration increases the learning rate by a preset value, and the average accuracy mean of the verification set is calculated to obtain the minimum and maximum learning rates. Based on these learning rates, the learning rate of each iteration is calculated based on the number of current iterations within the step size, and the object detection model is trained in a cycle with a preset number of steps as one cycle.
This method improves the efficiency of learning rate setting, avoids the process of manual attempts to adjust, and can obtain the optimal learning rate that adapts to the current data set faster, thereby improving the model training effect.
Smart Images

Figure CN112560883B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and device for setting a learning rate of a target detection model. Background Art
[0002] In the process of training the target detection model, the loss function value is continuously reduced through stochastic gradient descent and back propagation mechanism, so that the position of the predicted box is close to the position of the real box, while improving the category confidence and continuously enhancing the effect of the model in detecting the target. Finally, an optimal target detection model is obtained. The learning rate is an important hyperparameter for training the target detection model. When setting the learning rate, the initial value must be determined first, and it must be adjusted continuously during the training process. Both steps affect the final model detection effect.
[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:
[0004] At present, the optimal initial value of the learning rate is related to the data set. It takes multiple attempts to find the optimal initial value of the learning rate. In addition, the learning rate adjustment strategy during training will affect the final model effect, and the timing and degree of adjustment are not easy to grasp. In reality, it often takes multiple invalid training attempts to set up a suitable learning rate adjustment plan. Summary of the invention
[0005] In view of this, an embodiment of the present invention provides a method and device for setting the learning rate of a target detection model, which can solve the problem of low efficiency in setting the learning rate of training a target detection model in the prior art.
[0006] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present invention, a method for setting the learning rate of a target detection model is provided, including setting the total number of images in a training set and the number of images in each round of iterative training to calculate the step size; initializing the learning rate to train the target detection model on the training set, and increasing the learning rate by a preset value for each iteration, and calculating the average precision mean of the validation set once, thereby obtaining the minimum learning rate and the maximum learning rate; based on the minimum learning rate and the maximum learning rate, calculating the learning rate for each iteration according to the number of current iterations within the step size, and then cyclically training the target detection model with a preset number of steps as a cycle until the preset maximum number of iterations is completed.
[0007] Optionally, get the minimum and maximum learning rates, including:
[0008] The minimum learning rate is obtained when the average accuracy of the validation set starts to increase from the area close to 0;
[0009] The maximum learning rate is obtained when the mean average accuracy on the validation set starts to decrease.
[0010] Optionally, the target detection model is cyclically trained with a preset number of steps as a cycle, including:
[0011] The target detection model is trained cyclically with 2 steps as a cycle; during the first step, the learning rate gradually increases from the minimum learning rate to the maximum learning rate; during the second step, the learning rate gradually decreases from the maximum learning rate to the minimum learning rate.
[0012] Optionally, based on the minimum learning rate and the maximum learning rate, the learning rate for each iteration is calculated according to the number of current iterations within the step size, including:
[0013] During the first step, the learning rate for each iteration is the difference between the maximum learning rate and the minimum learning rate multiplied by the quotient of the number of current iterations in the step and the step length, plus the minimum learning rate;
[0014] During the second step, the learning rate for each iteration is the difference between the maximum learning rate and the minimum learning rate multiplied by the difference between the step size and the number of current iterations within the step size, divided by the step size, and finally added with the minimum learning rate
[0015] In addition, according to one aspect of an embodiment of the present invention, a device for setting the learning rate of a target detection model is provided, including a setting module for setting the total number of images in a training set and the number of images in each round of iterative training to calculate the step size; a processing module for initializing the learning rate to train the target detection model on the training set, and increasing the learning rate by a preset value for each iteration step, and calculating the average accuracy mean of the verification set once, thereby obtaining a minimum learning rate and a maximum learning rate; based on the minimum learning rate and the maximum learning rate, the learning rate for each iteration is calculated according to the number of current iterations within the step size, and then the target detection model is cyclically trained with a preset number of steps as a cycle until the preset maximum number of iterations is completed.
[0016] Optionally, the processing module obtains the minimum learning rate and the maximum learning rate, including:
[0017] The minimum learning rate is obtained when the average accuracy of the validation set starts to increase from the area close to 0;
[0018] The maximum learning rate is obtained when the mean average accuracy on the validation set starts to decrease.
[0019] Optionally, the processing module cyclically trains the target detection model with a preset number of steps as a cycle, including:
[0020] The target detection model is trained cyclically with 2 steps as a cycle; during the first step, the learning rate gradually increases from the minimum learning rate to the maximum learning rate; during the second step, the learning rate gradually decreases from the maximum learning rate to the minimum learning rate.
[0021] Optionally, the processing module calculates the learning rate of each iteration based on the minimum learning rate and the maximum learning rate according to the number of current iterations within the step size, including:
[0022] During the first step, the learning rate for each iteration is the difference between the maximum learning rate and the minimum learning rate multiplied by the quotient of the number of current iterations in the step and the step length, plus the minimum learning rate;
[0023] During the second step, the learning rate for each iteration is the difference between the maximum learning rate and the minimum learning rate multiplied by the step size and the number of current iterations within the step size, divided by the step size, and finally added with the minimum learning rate.
[0024] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, including:
[0025] one or more processors;
[0026] a storage device for storing one or more programs,
[0027] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above-mentioned embodiments for setting the learning rate of the target detection model.
[0028] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in any of the above-mentioned embodiments for setting the learning rate of the target detection model is implemented.
[0029] An embodiment of the above invention has the following advantages or beneficial effects: the present invention calculates the step length by setting the total number of images in the training set and the number of images in each round of iterative training; initializes the learning rate to train the target detection model on the training set, and increases the learning rate by a preset value for each iteration, and calculates the average precision mean of the verification set at the same time, thereby obtaining the minimum learning rate and the maximum learning rate; based on the minimum learning rate and the maximum learning rate, calculates the learning rate for each iteration according to the number of current iterations within the step length, and then cyclically trains the target detection model with a preset number of steps as a cycle until the preset maximum number of iterations is completed. . Thus, the present invention proposes a universal learning rate setting method, and enables the initialization and adjustment of the learning rate to meet the characteristics of the current data set, thereby obtaining a better training effect while avoiding manual attempts to adjust.
[0030] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.
[0032] Figure 1 Schematic diagram of the main process of the method for setting the learning rate of the target detection model according to the first embodiment of the present invention
[0033] Figure 2 is a schematic diagram of the main process of a method for setting a learning rate of a target detection model according to a second embodiment of the present invention;
[0034] Figure 3 is a schematic diagram of main modules of a device for setting a learning rate of a target detection model according to an embodiment of the present invention;
[0035] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;
[0036] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION
[0037] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0038] Figure 1 is a schematic diagram of the main process of a method for setting a learning rate of a target detection model according to a first embodiment of the present invention. The method for setting a learning rate of a target detection model may include:
[0039] Step S101, setting the total number of images in the training set and the number of images in each round of iterative training to calculate the step size.
[0040] Preferably, the step size is the quotient of the total number of images in the training set and the number of images in each round of iterative training multiplied by a preset multiple.
[0041] Step S102, initialize the learning rate to train the target detection model on the training set, and increase the learning rate by a preset value for each iteration, and calculate the average accuracy of the validation set once, so as to obtain the minimum learning rate and the maximum learning rate.
[0042] Preferably, the minimum learning rate is obtained when the average precision of the validation set starts to increase from the region close to 0. The maximum learning rate is obtained when the average precision of the validation set starts to decrease.
[0043] Step S103, based on the minimum learning rate and the maximum learning rate, according to the number of current iterations within the step length, the learning rate of each iteration is calculated, and then the target detection model is cyclically trained with a preset number of steps as a cycle until the preset maximum number of iterations is completed.
[0044] Preferably, the target detection model is trained cyclically with 2 steps as a cycle; wherein, during the first step, the learning rate gradually increases from the minimum learning rate to the maximum learning rate; during the second step, the learning rate gradually decreases from the maximum learning rate to the minimum learning rate.
[0045] Furthermore, during the first step, the learning rate for each iteration is the difference between the maximum and minimum learning rates multiplied by the quotient of the number of current iterations in the step and the step, plus the minimum learning rate. During the second step, the learning rate for each iteration is the difference between the maximum and minimum learning rates multiplied by the difference between the step and the number of current iterations in the step, divided by the step, and finally the minimum learning rate.
[0046] In addition, it is preferred that the maximum number of iterations is an even multiple of the step size.
[0047] Therefore, the present invention proposes a more general learning rate setting method, which can obtain the minimum learning rate and maximum learning rate of the adaptation data set. During iterative training, two step lengths are used as cycles, the first step length learning rate changes from small to large, and the second step length learning rate changes from large to small, and the cycle continues until the training is completed. At the same time, the learning rate setting method of the present invention can avoid manual attempts to adjust and save time. When consuming the same training time and resources, a better model training effect can be obtained.
[0048] Figure 2 is a schematic diagram of the main process of a method for setting a learning rate of a target detection model according to a second embodiment of the present invention. The method for setting a learning rate of a target detection model may include:
[0049] Step S201, setting the total number of images in the training set and the number of images in each round of iterative training to calculate the step size.
[0050] In an embodiment, a training set and a validation set of a target detection model need to be prepared in advance. The target detection model may be Faster R-CNN, R-FCN, SSD, etc. Then, the total number of images in the training set is set to nums, and the number of images in each round of iterative training is set to batchsize (for example, batchsize is set to 16).
[0051] Preferably, the step size stepsize is the quotient of the total number of images in the training set nums and the number of images in each round of iterative training batchsize multiplied by a preset multiple.
[0052] Further, stepsize=A*(nums / batchsize), where A can be between 2 and 10. Further, A is 8.
[0053] Step S202, initialize the learning rate to train the target detection model on the training set, and increase the learning rate by a preset value for each iteration, and calculate the average accuracy of the validation set.
[0054] Preferably, the learning rate lr is initialized to 0.000001, the target detection model is trained on the training set, and the learning rate lr is increased by 10% for each iteration of a step size stepsize, and the average precision mean map of the validation set is calculated once. Among them, map refers to the average precision mean, a performance measurement indicator of the target detection algorithm.
[0055] Step S203, when the average precision of the validation set starts to increase from the region close to 0, the minimum learning rate is obtained. When the average precision of the validation set starts to decrease, the maximum learning rate is obtained.
[0056] In the embodiment, according to the trend of the validation set map, the validation set map will maintain a small value close to 0 (for example, less than 0.001) at the beginning. When the learning rate lr increases to a certain value, the validation set map leaves the area close to 0 and begins to increase. At this time, the value to which the learning rate lr increases is the minimum learning rate min_lr. As the iteration proceeds, the learning rate lr gradually increases, and the validation set map also gradually increases. When the learning rate lr is greater than a certain value, the validation set map begins to decrease, and the certain value at this time is the maximum learning rate max_lr.
[0057] Step S204, training the target detection model in a loop with a step length of 2 as a cycle until a preset maximum number of iterations is completed.
[0058] In an embodiment, during the first step size stepsize, the learning rate lr gradually increases from the minimum learning rate to the maximum learning rate max_lr. During the second step size stepsize, the learning rate lr gradually decreases from the maximum learning rate to the minimum learning rate max_lr.
[0059] Among them, during the first step size stepsize, the learning rate lr is the difference between the maximum learning rate max_lr and the minimum learning rate min_lr multiplied by the quotient of the number of current iterations in the step size and the step size stepsize, plus the minimum learning rate min_lr.
[0060] That is, lr = min_lr + ((max_lr-min_lr)*itr / stepsize), itr is the number of current iterations in the step size (for example, the current iteration is the second time), itr gradually increases from 0 to stepsize during a step size, and becomes 0 after each step size. It can be seen that the learning rate lr gradually increases during this period according to the ratio of itr to the step size.
[0061] During the second step size stepsize, the learning rate lr is the difference between the maximum learning rate max_lr and the minimum learning rate min_lr multiplied by the difference between the step size stepsize and the number of current iterations itr in the step size divided by the step size stepsize, and finally the minimum learning rate min_lr is added.
[0062] That is, lr = min_lr + ((max_lr-min_lr)*(stepsize-itr) / stepsize), itr is the number of current iterations in the step size (for example, the current iteration is the second time), itr gradually increases from 0 to stepsize during a step size, and becomes 0 after each step size. It can be seen that the learning rate lr gradually decreases during this period.
[0063] In addition, the maximum number of iterations max_iters is an even multiple of the step size stepsize, for example, the maximum number of iterations max_iters is 120000.
[0064] It is also worth noting that the training process of the present invention can be implemented using the caffe deep learning framework. Experiments have shown that the maximum map of the target detection model validation set obtained by the prior art after the maximum number of iterations max_iters is 120,000 times is 0.689. When other hyperparameters are kept unchanged and only the learning rate setting is changed to the present invention, the validation set map reaches 0.695 when it is iterated to 104,000 times, and the validation set map reaches 0.710 after 120,000 iterations, which greatly exceeds the prior art.
[0065] Figure 3 is a schematic diagram of the main modules of the device for setting the learning rate of the target detection model according to an embodiment of the present invention, such as Figure 3As shown, the device 300 for setting the learning rate of the target detection model includes a setting module 301 and a processing module 302. The setting module 301 is used to set the total number of images in the training set and the number of images in each round of iterative training to calculate the step size; the processing module 302 is used to initialize the learning rate to train the target detection model on the training set, and increase the learning rate by a preset value for each iteration of one step size, and calculate the average precision mean of the verification set once, thereby obtaining the minimum learning rate and the maximum learning rate; based on the minimum learning rate and the maximum learning rate, the learning rate of each iteration is calculated according to the number of current iterations within the step size, and then the target detection model is cyclically trained with a preset number of steps as a cycle until the preset maximum number of iterations is completed.
[0066] Preferably, the maximum number of iterations is an even multiple of the step length; wherein the step length is the quotient of the total number of images in the training set and the number of images in each round of iterative training multiplied by a preset multiple.
[0067] In a further embodiment, the processing module 302 obtains the minimum learning rate and the maximum learning rate, including:
[0068] The minimum learning rate is obtained when the mean average precision of the validation set starts to increase from the area close to 0. The maximum learning rate is obtained when the mean average precision of the validation set starts to decrease.
[0069] In addition, the processing module 302 cyclically trains the target detection model with 2 steps as a cycle; wherein, during the first step, the learning rate gradually increases from the minimum learning rate to the maximum learning rate; during the second step, the learning rate gradually decreases from the maximum learning rate to the minimum learning rate.
[0070] Furthermore, the processing module 302 calculates the learning rate of each iteration based on the minimum learning rate and the maximum learning rate according to the number of current iterations within the step size, including:
[0071] During the first step, the learning rate for each iteration is the difference between the maximum learning rate and the minimum learning rate multiplied by the quotient of the number of current iterations in the step and the step length, plus the minimum learning rate;
[0072] During the second step, the learning rate for each iteration is the difference between the maximum learning rate and the minimum learning rate multiplied by the step size and the number of current iterations within the step size, divided by the step size, and finally added with the minimum learning rate.
[0073] It should be noted that the method for setting the learning rate of the target detection model and the device for setting the learning rate of the target detection model described in the present invention have a corresponding relationship in terms of specific implementation contents, so the repeated contents will not be described again.
[0074] Figure 4An exemplary system architecture 400 is shown to which a method for setting a learning rate of a target detection model or an apparatus for setting a learning rate of a target detection model according to an embodiment of the present invention can be applied.
[0075] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, 403, network 404 and server 405. Network 404 is used to provide a medium for communication links between terminal devices 401, 402, 403 and server 405. Network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0076] Users can use terminal devices 401, 402, 403 to interact with server 405 through network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).
[0077] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.
[0078] The server 405 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 401, 402, and 403. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.
[0079] It should be noted that the method for setting the learning rate of the target detection model provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the device for setting the learning rate of the target detection model is generally set in the server 405 .
[0080] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0081] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of a terminal device suitable for implementing an embodiment of the present invention. Figure 5 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0082] like Figure 5As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0083] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.
[0084] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.
[0085] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0086] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0087] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a setting module and a processing module. The names of these modules do not, in some cases, constitute limitations on the modules themselves.
[0088] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: setting the total number of images in the training set and the number of images in each round of iterative training to calculate the step size; initializing the learning rate to train the target detection model on the training set, and increasing the learning rate by a preset value for each iteration of the step size, and calculating the average precision mean of the validation set once, thereby obtaining the minimum learning rate and the maximum learning rate; based on the minimum learning rate and the maximum learning rate, according to the number of current iterations within the step size, calculating the learning rate of each iteration, and then cyclically training the target detection model with a preset number of steps as a cycle until the preset maximum number of iterations is completed. .
[0089] The technical solution according to the embodiment of the present invention can solve the problem of low efficiency in setting the learning rate of the target detection model training in the prior art.
[0090] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for setting the learning rate of a target detection model, characterized in that: include: Set the total number of images in the training set and the number of images in each round of iterative training to calculate the step length; where the step length is the quotient of the total number of images in the training set and the number of images in each round of iterative training multiplied by a preset multiple; Initialize the learning rate to train the target detection model on the training set, and increase the learning rate by a preset value for each iteration step, and calculate the average precision of the validation set once, so as to obtain the minimum learning rate and the maximum learning rate, including: obtaining the minimum learning rate when the average precision of the validation set starts to increase from the area close to 0; obtaining the maximum learning rate when the average precision of the validation set starts to decrease; Based on the minimum learning rate and the maximum learning rate, the learning rate of each iteration is calculated according to the number of current iterations within the step size, and then the target detection model is trained in a cycle with a preset number of steps as a cycle until the preset maximum number of iterations is completed.
2. The method according to claim 1, characterized in that The target detection model is trained in a cycle with a preset number of steps, including: The target detection model is trained cyclically with 2 steps as a cycle; during the first step, the learning rate gradually increases from the minimum learning rate to the maximum learning rate; during the second step, the learning rate gradually decreases from the maximum learning rate to the minimum learning rate.
3. The method according to claim 2, characterized in that Based on the minimum learning rate and the maximum learning rate, the learning rate for each iteration is calculated according to the number of current iterations within the step size, including: During the first step, the learning rate for each iteration is the difference between the maximum learning rate and the minimum learning rate multiplied by the quotient of the number of current iterations in the step and the step length, plus the minimum learning rate; During the second step, the learning rate for each iteration is the difference between the maximum learning rate and the minimum learning rate multiplied by the step size and the number of current iterations within the step size, divided by the step size, and finally added with the minimum learning rate.
4. A device for setting a learning rate of a target detection model, characterized in that: include: A setting module, used to set the total number of images in the training set and the number of images in each round of iterative training to calculate the step length; wherein the step length is the quotient of the total number of images in the training set and the number of images in each round of iterative training multiplied by a preset multiple; The processing module is used to initialize the learning rate to train the target detection model on the training set, and increase the learning rate by a preset value for each step of iteration, and calculate the average precision mean of the verification set once, so as to obtain the minimum learning rate and the maximum learning rate, including: obtaining the minimum learning rate when the average precision mean of the verification set starts to increase from the area approaching 0; obtaining the maximum learning rate when the average precision mean of the verification set starts to decrease; based on the minimum learning rate and the maximum learning rate, calculate the learning rate of each iteration according to the number of current iterations within the step, and then cyclically train the target detection model with a preset number of steps as a cycle until the preset maximum number of iterations is completed.
5. The device according to claim 4, characterized in that The processing module cyclically trains the target detection model with a preset number of steps as one cycle, including: The target detection model is trained cyclically with 2 steps as a cycle; during the first step, the learning rate gradually increases from the minimum learning rate to the maximum learning rate; during the second step, the learning rate gradually decreases from the maximum learning rate to the minimum learning rate.
6. The device according to claim 5, characterized in that The processing module calculates the learning rate of each iteration based on the minimum learning rate and the maximum learning rate according to the number of current iterations within the step size, including: During the first step, the learning rate for each iteration is the difference between the maximum learning rate and the minimum learning rate multiplied by the quotient of the number of current iterations in the step and the step length, plus the minimum learning rate; During the second step, the learning rate for each iteration is the difference between the maximum learning rate and the minimum learning rate multiplied by the step size and the number of current iterations within the step size, divided by the step size, and finally added with the minimum learning rate.
7. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 3.
8. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.
Citation Information
Patent Citations
Learning rate determination method and device based on deep learning
CN108921293A