Pruning training method, device and equipment based on perception model of unmanned aerial vehicle end
By using pruning training methods in the perception model on the drone side, training and pruning models in stages, the problems of limited resources and high performance requirements in the perception deployment tasks of the drone side are solved, and model scale compression and performance improvement are achieved.
Patent Information
- Application Number
- CN202510142321.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-08
- Publication Date
- 2025-06-27
AI Technical Summary
The perceptual deployment tasks on the drone side face challenges such as limited computing and memory resources and real-time response requirements, and it is difficult to meet the requirements of high throughput and high inference accuracy.
The pruning training method based on the drone-side perception model is adopted. By configuring the pruning training parameters, it is divided into preheating training stage, sparse training stage, intensive training stage and sparse fine-tuning stage. The pruning perception model is gradually trained to reduce the amount of model calculation and improve training efficiency.
Through pruning technology, the model scale is compressed, the calculation amount is reduced, the model training efficiency and performance are improved, and the resource limitations on the drone side are adapted.
Smart Images
Figure CN120218159A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of UAV perception, and in particular, to a pruning training method, device and equipment for a perception model based on the UAV side. Background Art
[0002] Currently, UAVs integrated with perception technology can achieve environmental perception, path planning and target recognition, which can significantly improve the intelligent level of the equipment. Among them, for the perception deployment technology on the UAV side, it is necessary to meet requirements such as real-time response, high throughput and high inference accuracy under limited computing and memory resources. However, due to limitations in aspects such as the volume and weight of UAVs, energy efficiency, and real-time response requirements, the perception deployment task on the UAV side still faces certain challenges. Summary of the Invention
[0003] According to one aspect of the present application, there is provided a pruning training method for a perception model based on the UAV side, including: configuring pruning training parameters for a preset perception model; training the perception model according to a preset pruning training stage based on the pruning training parameters; and determining the parameters of the trained perception model.
[0004] According to some embodiments, before configuring pruning training parameters for a preset perception model, a preset data set is obtained; and the preset data set is preprocessed according to the perception model.
[0005] According to some embodiments, the pruning training stage includes a warm-up training stage, a sparse training stage, a dense training stage and / or a sparse fine-tuning stage; configuring pruning training parameters for a preset perception model includes: setting the number of training rounds in the warm-up training stage; setting the alternating training cycle of the sparse training stage and the dense training stage, and the number of training rounds corresponding to the alternating training cycle; setting the number of training rounds in the sparse fine-tuning stage; and / or setting the pruning rate of the perception model.
[0006] According to some embodiments, training the perception model in the warm-up training stage through the preprocessed preset data set according to the set number of training rounds in the warm-up training stage; training the perception model according to a preset pruning training stage based on the pruning training parameters includes: training all network layers of the perception model in the warm-up training stage.
[0007] According to some embodiments, in accordance with the training rounds corresponding to the sparse training phase in the set alternating training cycle, the perception model is trained in the sparse training phase through the preprocessed preset data set; based on the pruning training parameters, the perception model is trained in the preset pruning training phase, including: in the starting round of the sparse training phase, obtaining the set of importance scores of the output channels of the convolutional layer of the perception model; generating a pruning mask matrix for the output channels of the convolutional layer according to the set of importance scores of the output channels of the convolutional layer; determining the output channels masked by the perception model in the sparse training phase according to the pruning mask matrix to perform the pruning operation on the perception model; calculating the parameters of the perception model after the pruning operation according to the pruning mask matrix; and training the perception model in the sparse training phase according to the parameters of the perception model after the pruning operation.
[0008] According to some embodiments, based on the pruning training parameters, the perception model is trained in the preset pruning training phase, including: retaining the parameters of the perception model before the pruning operation; after the training in the sparse training phase is completed, training the perception model in the dense training phase through the preprocessed preset data set in accordance with the training rounds corresponding to the dense training phase in the set alternating training cycle according to the parameters of the perception model before the pruning operation; training the perception model alternately in the sparse training phase and the dense training phase for a preset number of alternating training cycles, and retaining the parameters after the training of the preset number of alternating training cycles is completed, wherein the last alternating training cycle in the preset number of alternating training cycles only includes the sparse training phase; and training the perception model in the sparse fine-tuning phase according to the parameters after the training of the preset number of alternating training cycles is completed.
[0009] According to some embodiments, determining the parameters of the trained perception model includes: obtaining the parameters of the perception model after the training in the sparse fine-tuning phase is completed; screening the output channels of the current layer of the perception model and the corresponding input channels of the next layer according to the parameters after the training in the sparse fine-tuning phase is completed; and obtaining the perception model after the pruning operation based on the screened channels and their initial parameters in the perception model.
[0010] According to one aspect of the present application, there is provided a pruning training device for a perception model based on a drone side, including: a configuration module for configuring pruning training parameters for a preset perception model; a training module for training the perception model in a preset pruning training phase based on the pruning training parameters; and a confirmation module for determining the parameters of the trained perception model.
[0011] According to one aspect of the present application, there is provided an electronic device, including: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.
[0012] According to one aspect of the present application, there is provided a computer-readable storage medium storing a computer program or instructions, which, when executed by a processor, implement the method as described above.
[0013] According to an embodiment of the present application, the perception model applied to the drone side can be trained by a structured pruning technique to compress the model scale, reduce the model calculation amount, and improve the model training efficiency and model performance.
[0014] It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application.
[0016] Figure 1 The flowchart showing the pruning training method of the perception model based on the drone side according to an exemplary embodiment of the present application.
[0017] Figure 2 The schematic diagram showing the change of the model calculation amount and file size of the trained perception model according to an exemplary embodiment of the present application.
[0018] Figure 3 The block diagram showing the pruning training device of the perception model based on the drone side according to an exemplary embodiment of the present application.
[0019] Figure 4 The block diagram showing the electronic device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0020] Now, the exemplary embodiments will be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the figures denote the same or similar parts, and thus their repeated description will be omitted.
[0021] The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will recognize that the technical solutions of the present application may be practiced without one or more of these specific details, or other methods, components, materials, devices, or operations, etc. may be employed. In such cases, well-known structures, methods, devices, implementations, materials, or operations will not be shown or described in detail.
[0022] The flowcharts shown in the accompanying drawings are merely illustrative and not necessarily include all the contents and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps may be decomposed, while some operations / steps may be combined or partially combined. Therefore, the actual execution order may be changed according to the actual situation.
[0023] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products, or devices.
[0024] The present application provides a pruning training method, device, and equipment for a perception model based on an unmanned aerial vehicle (UAV) end, which can improve the training efficiency and model performance of the perception model.
[0025] Next, a pruning training method, device, and equipment for a perception model based on an unmanned aerial vehicle (UAV) end according to an embodiment of the present application will be described in detail with reference to the accompanying drawings.
[0026] Figure 1 A flowchart showing a pruning training method for a perception model based on an unmanned aerial vehicle (UAV) end according to an exemplary embodiment of the present application is shown.
[0027] As Figure 1 shown, in step S100, pruning training parameters are configured for a preset perception model.
[0028] For example, in step S100, a pruning training device configures pruning training parameters such as a training phase, number of training rounds, and pruning rate for a preset perception model.
[0029] Before configuring pruning training parameters for a preset perception model, the pruning training device obtains a preset data set and preprocesses the preset data set based on the perception model for training the perception model.
[0030] According to some embodiments, the preset perception model may adopt the pre-trained YoloV5 object detection model. The preset dataset may select the VisDrone dataset from the perspective of a drone.
[0031] According to some embodiments, the pruning training device may combine the target upper left corner coordinate information and the absolute position in the bounding box labels of the VisDrone dataset with the corresponding image width and height, and convert them into the target center coordinate information and relative position percentage in the YoloV5 output format to complete the preprocessing of the preset dataset.
[0032] The pruning training device divides the pruning training stage into multiple different training stages and configures the number of training rounds for each training stage.
[0033] According to some embodiments, the pruning training device may divide the pruning training stage into a warm-up training stage, a sparse training stage, a dense training stage, and / or a sparse fine-tuning stage through the AC / DC training paradigm, and set the alternating training cycle of the sparse training stage and the dense training stage.
[0034] Furthermore, the pruning training device configures the alternating training cycle of the warm-up training stage, the sparse training stage, and the dense training stage and the number of training rounds of the sparse fine-tuning stage.
[0035] For example, the pruning training device sets the total number of rounds of the pruning training stage to N, the number of rounds of the warm-up training stage to N w , the number of alternating training cycles of the sparse training stage and the dense training stage to T, and the number of training rounds of the sparse training stage and the dense training stage within the alternating training cycle to N s and N d respectively, and the number of training rounds of the sparse fine-tuning stage to N sf . Among them, only the sparse training stage is included in the last alternating training cycle to ensure the correct operation of the sparse fine-tuning stage.
[0036] The relationship between the total number of rounds N of the pruning training stage and the number of training rounds of each training stage therein can be expressed by the following formula.
[0037] N = N w +(T - 1)*(N s +N d )+N s +N sf , (1)
[0038] Furthermore, the starting training round of the sparse training stage in the i-th alternating training cycle can be expressed by the following formula.
[0039]
[0040] Further, the starting training round of the intensive training phase in the i-th alternating training cycle can be expressed by the following formula.
[0041]
[0042] The pruning training device also configures the pruning rate of the perception model. For example, the pruning training device sets the initial pruning rate of the perception model to P s , and the target pruning rate to P t , and adjusts the pruning rate of the perception model through a polynomial strategy within the alternating training cycles of T sparse training phases and intensive training phases. Among them, the pruning rate P i in the i-th alternating training cycle can be expressed by the following formula.
[0043]
[0044] where d is the speed that controls the change of the pruning rate over time, represents the number of rounds of the sparse training phase after completing the last alternating training cycle.
[0045] According to some embodiments, the pruning training device does not prune the detection head module in the perception model to alleviate the problem of the decrease in the accuracy rate of the perception model caused by the pruning operation. And, the pruning training device does not prune the modules with residual connections in the perception model to ensure the normal operation of the residual connections.
[0046] In step S200, based on the pruning training parameters, the perception model is trained according to the preset pruning training phases.
[0047] For example, in step S200, the pruning training device trains each phase in the pruning training phase of the perception model through the preprocessed preset dataset according to the set pruning training parameters.
[0048] According to the set number of training rounds, the pruning training device first trains the perception model through the preprocessed preset dataset in the warm-up training phase.
[0049] According to some embodiments, all network layers of the perception model are trained in the warm-up training phase so that the perception model maintains a relatively stable state before entering the training of the sparse training phase. Among them, the network layers in the perception model include multiple convolutional layers, which can be represented by C = {C1, C2,..., C p}.
[0050] After being trained in the preheating training stage, the perception model enters an alternating training cycle of the sparse training stage and the dense training stage. The pruning training device determines the pruning training stage in which the perception model is located by comparing the current training round of the perception model with the starting training round of the sparse training stage or the dense training stage.
[0051] For example, the current training round of the perception model is N c , the starting training round of the sparse training stage the starting training round of the dense training stage In case, the perception model enters the sparse training stage.
[0052] According to the set training rounds, the pruning training device trains the perception model in the sparse training stage through a pre-processed preset data set.
[0053] In the starting round of the sparse training stage, the pruning training device obtains the set of importance scores of the output channels of the convolutional layer of the perception model.
[0054] According to some embodiments, for the convolutional layer C in the perception model j , set its parameter weight to O j is the number of output channels, I j is the number of input channels, K j is the width (height) of the convolutional kernel. For the k-th output channel of the convolutional layer C j , set its parameter weight to
[0055] Furthermore, the pruning training device counts the gradient j obtained by the k-th output channel of the convolutional layer C in the most recent backpropagation and calculates the Hessian matrix according to the following formula
[0056]
[0057] According to the gradient and the Hessian matrix the pruning training device calculates the importance score of the second-order Taylor expansion of the gradient of the k-th output channel of the convolutional layer C j through the following formula
[0058] where abs(x) refers to taking the absolute value of x, is to change the shape of 1 is to Sum the data in the second dimension.
[0059] Furthermore, the pruning training device obtains the current convolutional layer C j The set of importance scores for the output channels is
[0060] The pruning training device generates a pruning mask matrix for the output channels of the convolutional layer according to the set of importance scores of the convolutional layer output channels.
[0061] According to some embodiments, the pruning training device calculates the importance scores of the second-order Taylor expansion of the gradients of the output channels of all convolutional layers in the perception model, and obtains the global output channel importance array of the perception model
[0062] Furthermore, the pruning training device sorts the global output channel importance array of the perception model in ascending order to obtain the array S a , and selects the round(L*P i )-th value as the importance threshold T for the current sparse training stage i . The importance threshold T i Can be expressed by the following formula.
[0063] T i = S a [round(L*P i )], (7)
[0064] Where round(·) converts a floating-point number to an integer index.
[0065] The pruning training device filters the set of importance scores S i of the output channels of the current convolutional layer C j according to the importance threshold T j to generate a pruning mask matrix for the output channels of the convolutional layer C j Among them, The value range of can be expressed by the following formula.
[0066]
[0067] And, the set of pruning mask matrices generated in the current sparse training stage
[0068] According to the pruning mask matrix, the pruning training device determines the output channels masked in the sparse training stage of the perception model to perform the pruning operation of the perception model.
[0069] According to some embodiments, based on the above formula (8), if the pruning training device determines that the output channels of its corresponding convolutional layer C j are masked. It can be understood that the input channels of the next layer corresponding to the masked output channels of convolutional layer C j are also masked.
[0070] According to some embodiments, if the pruning mask matrix is all 0, the pruning training device determines that the output channels of convolutional layer C j are all masked. The pruning training device restores the output channels of convolutional layer C j with the highest importance scores in the set S j of the output channels, so that convolutional layer C j can continue training.
[0071] According to the pruning mask matrix, the pruning training device calculates the parameters of the perception model after the pruning operation.
[0072] According to some embodiments, in each forward propagation process of the current sparse training phase, the pruning training device calculates the weight matrix WP of the perception model after the pruning operation in the current sparse training phase through the following formula according to the pruning mask matrix j and the Bias vector of the BN layer
[0073]
[0074] where, is the Bias vector of the BN layer after convolutional layer C i , and
[0075] According to the parameters of the perception model after the pruning operation, the pruning training device conducts training on the perception model in the sparse training phase.
[0076] According to some embodiments, the pruning training device only performs the pruning operation on the perception model at the start round of the sparse training phase in the alternating training cycle of the current sparse training phase and the dense training phase, and uses the parameters after the pruning operation for training in the remaining training rounds of the sparse training phase.
[0077] According to some embodiments, the channel parameters masked in the pruning operation of the perception model are retained in the perception model but do not participate in the training of the sparse training phase. Moreover, the pruning training device does not update the masked channel parameters.
[0078] According to some embodiments, the start training round of the dense training phase is in the current training round of the perception model In this case, the perception model enters the intensive training phase.
[0079] According to some embodiments, the pruning training device retains the parameters of the perception model before the pruning operation. After the training in the sparse training phase of the current alternating cycle is completed, the pruning training device sets all the sets M of pruning mask matrices i to 1 and restores the parameters of the perception model before the pruning operation to perform the training in the intensive training phase of the current alternating cycle. Among them, the parameters of the perception model before the pruning operation include the parameters of the convolutional layer and the BN layer corresponding to the channels masked in the pruning operation.
[0080] According to the parameters of the perception model before the pruning operation and the set number of training rounds, the pruning training device trains the perception model in the intensive training phase through a pre-processed preset data set.
[0081] Within the preset number of alternating training cycles, the pruning training device alternately trains the perception model in the sparse training phase and the intensive training phase according to the set number of training rounds and retains the parameters after the training is completed. Among them, the preset number of alternating training cycles of the sparse training phase and the intensive training phase is T, and the parameters of the perception model after the training in T alternating training cycles include the set M of pruning mask matrices in T alternating training cycles T .
[0082] According to some embodiments, only the sparse training phase of the perception model is performed in the last alternating training cycle of the T alternating training cycles to ensure the correct operation of the subsequent sparse fine-tuning phase.
[0083] According to the parameters after the training in the preset number of alternating training cycles and the set number of training rounds, the pruning training device trains the perception model in the sparse fine-tuning phase.
[0084] In step S300, the parameters of the trained perception model are determined.
[0085] For example, in step S300, the pruning training device determines the parameters of the perception model that has undergone the pruning training phase and obtains the perception model after the pruning operation.
[0086] After the training in the sparse fine-tuning phase is completed, the pruning training device obtains the parameters of the perception model and filters the output channels of the current layer of the perception model and the corresponding input channels of the next layer based on this.
[0087] According to some embodiments, according to the set M of pruning mask matrices in T alternating training cycles T, the pruning training device screens the output channels of the current layer of the perception model and the corresponding input channels of the next layer, deletes the output channels with a value of 0 in the pruning mask matrix of the current layer of the perception model and their corresponding input channels in the next layer, and retains the remaining channels and their initial parameters.
[0088] Based on the screened channels and their initial parameters in the perception model, the pruning training device obtains the perception model after the pruning operation.
[0089] According to some embodiments, such as Figure 2 shown, the perception models after pruning operations according to different target pruning rates (such as 50%, 70%, and 90%) can achieve a significant reduction in model volume and computational load.
[0090] According to the embodiments of the present application, the model scale can be compressed and the model computational load can be reduced, improving the model training efficiency and model performance.
[0091] Figure 3 Shows a block diagram of a pruning training device for a perception model based on a drone side according to an exemplary embodiment of the present application.
[0092] As Figure 3 shown, the pruning training device 100 includes a configuration module 110, a training module 120, and a confirmation module 130.
[0093] Before configuring the pruning training parameters for a preset perception model, the configuration module 110 obtains a preset data set and preprocesses the preset data set based on the perception model for training the perception model.
[0094] The configuration module 110 divides the pruning training phase into multiple different training phases and configures the number of training rounds for each training phase.
[0095] The configuration module 110 configures the pruning rate of the perception model.
[0096] According to the set number of training rounds, the training module 120 trains the perception model in the warm-up training phase through the preprocessed preset data set.
[0097] After the training in the warm-up training phase, the perception model enters an alternating training cycle of the sparse training phase and the dense training phase. The training module 120 determines the pruning training phase in which the perception model is located by comparing the current training round of the perception model with the start training round of the sparse training phase or the dense training phase.
[0098] According to the set number of training rounds, the training module 120 trains the perception model in the sparse training phase through the preprocessed preset data set.
[0099] At the beginning round of the sparse training phase, the training module 120 obtains a set of importance scores for the output channels of the convolutional layer of the perception model.
[0100] The training module 120 generates a pruning mask matrix for the output channels of the convolutional layer according to the set of importance scores for the output channels of the convolutional layer.
[0101] According to the pruning mask matrix, the training module 120 determines the output channels masked in the sparse training phase of the perception model to perform the pruning operation on the perception model.
[0102] According to the pruning mask matrix, the training module 120 calculates the parameters of the perception model after the pruning operation.
[0103] According to the parameters of the perception model after the pruning operation, the training module 120 performs training in the sparse training phase on the perception model.
[0104] According to the parameters of the perception model before the pruning operation, and according to the set number of training rounds, the training module 120 performs training in the dense training phase on the perception model through a pre-processed preset data set.
[0105] Within a preset number of alternating training cycles, the training module 120 alternately performs training in the sparse training phase and the dense training phase on the perception model according to the set number of training rounds, and retains the parameters after the training is completed.
[0106] According to the parameters after the training in the preset number of alternating training cycles is completed, and according to the set number of training rounds, the training module 120 performs training in the sparse fine-tuning phase on the perception model.
[0107] After the training in the sparse fine-tuning phase is completed, the confirmation module 130 obtains the parameters of the perception model, and filters the output channels of the current layer of the perception model and the corresponding input channels of the next layer based on this.
[0108] Based on the filtered channels and their initial parameters in the perception model, the confirmation module 130 obtains the perception model after the pruning operation.
[0109] Figure 4 A block diagram of an electronic device according to an exemplary embodiment of the present application is shown.
[0110] As Figure 4 shown, the electronic device 600 is merely an example and should not impose any limitation on the functions and scope of use of the embodiments of the present application.
[0111] As Figure 4As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc. Among them, the storage unit stores program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 executes the methods according to various exemplary embodiments of the present application described in this specification. For example, the processing unit 610 can execute as Figure 1 shown in the method.
[0112] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.
[0113] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205. Such program modules 6205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.
[0114] The bus 630 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.
[0115] The electronic device 600 can also communicate with one or more external devices 700 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface 650. And, the electronic device 600 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 660. The network adapter 660 can communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0116] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software or by a combination of software and necessary hardware. The technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal or a network device, etc.) to execute the method according to the embodiments of the present application.
[0117] The software product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0118] The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.
[0119] The program code for performing the operations of this application can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).
[0120] The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the computer-readable medium implements the foregoing functions.
[0121] Those skilled in the art can understand that the above-mentioned modules can be distributed in the device according to the description of the embodiments, or can be correspondingly changed and distributed in one or more devices that are different from the present embodiment only. The modules of the above embodiments can be combined into one module, or can be further split into multiple sub-modules.
[0122] The above has introduced the embodiments of the present application in detail. The description of the above embodiments is only used to help understand the method and its core idea of the present application. At the same time, any changes or deformations made by those skilled in the art based on the idea of the present application in terms of the specific implementation manner and application scope of the present application all belong to the protection scope of the present application. In summary, the content of this specification should not be construed as a limitation on the present application.
Claims
1. A pruning training method based on a perception model of a drone, characterized in that: include: Configure pruning training parameters for the preset perception model; Based on the pruning training parameters, training the perception model according to a preset pruning training stage; Determine the parameters of the trained perception model.
2. The method according to claim 1, characterized in that Before configuring pruning training parameters for the preset perception model, the method further includes: Get the preset data set; The preset data set is preprocessed according to the perception model.
3. The method according to claim 2, characterized in that The pruning training phase includes a warm-up training phase, a sparse training phase, a dense training phase and / or a sparse fine-tuning phase; Configure pruning training parameters for the preset perception model, including: Setting the training rounds of the warm-up training phase; Setting an alternating training cycle between the sparse training phase and the intensive training phase, and a training round corresponding to the alternating training cycle; Setting the number of training rounds in the sparse fine-tuning phase; and / or Set the pruning rate of the perception model.
4. The method according to claim 3, characterized in that According to the set training rounds of the warm-up training phase, the perception model is trained in the warm-up training phase using the preprocessed preset data set; Based on the pruning training parameters, the perception model is trained according to a preset pruning training stage, including: The warm-up training phase is performed on all network layers of the perception model.
5. The method according to claim 3, characterized in that: According to the training rounds corresponding to the sparse training phase in the set alternating training cycle, the perception model is trained in the sparse training phase using the preprocessed preset data set; Based on the pruning training parameters, the perception model is trained according to a preset pruning training stage, including: In the initial round of the sparse training phase, obtaining a set of importance scores of the convolutional layer output channels of the perception model; Generate a pruning mask matrix of the output channel of the convolutional layer according to the importance score set of the output channel of the convolutional layer; Determining, according to the pruning mask matrix, output channels of the perceptual model that are masked in the sparse training phase, so as to perform a pruning operation on the perceptual model; Calculating parameters of the perception model after pruning according to the pruning mask matrix; The sparse training phase is performed on the perception model according to the parameters of the perception model after the pruning operation.
6. The method according to claim 5, characterized in that Based on the pruning training parameters, the perception model is trained according to a preset pruning training stage, including: Retaining the parameters of the perception model before the pruning operation; After the training in the sparse training phase is completed, according to the parameters of the perception model before the pruning operation, the perception model is trained in the intensive training phase through the preprocessed preset data set according to the training rounds corresponding to the intensive training phase in the set alternating training cycle; According to a preset number of alternating training cycles, the perception model is trained alternately in the sparse training phase and the intensive training phase, and the parameters after the training of the preset number of alternating training cycles are retained, wherein the last alternating training cycle in the preset number of alternating training cycles only includes the sparse training phase; The perception model is trained in the sparse fine-tuning phase according to the parameters after the preset number of alternating training cycles are completed.
7. The method according to claim 6, characterized in that Determine the parameters of the trained perception model, including: Obtaining parameters of the perception model after training in the sparse fine-tuning phase is completed; According to the parameters after the training of the sparse fine-tuning stage, the output channel of the current layer of the perception model and the corresponding input channel of the next layer are selected; Based on the filtered channels and their initial parameters in the perception model, a perception model after pruning operation is obtained.
8. A pruning training device based on a perception model of a drone, characterized in that: include: A configuration module, used to configure pruning training parameters for a preset perception model; A training module, used for training the perception model according to a preset pruning training stage based on the pruning training parameters; The validation module is used to determine the parameters of the trained perception model.
9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the method according to any one of claims 1 to 7 is implemented.