Random depth-based model training method and system, medium and equipment
By dynamically adjusting the depth value of the auxiliary feature extraction block in the training of deep learning models, the problem of high training costs in the existing technology is solved, and resource saving and training efficiency are improved.
Patent Information
- Application Number
- CN202411715119.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-27
- Publication Date
- 2025-03-04
AI Technical Summary
The training cost of existing deep learning models is high, especially on resource-constrained devices. How to reduce training costs has become an urgent problem.
By setting auxiliary feature extraction blocks in model training and dynamically adjusting their depth values according to the number of training iterations, the computational amount and resource requirements during the training process are reduced.
It effectively reduces the training cost of deep learning models, improves training efficiency, and reduces the pressure on resources.
Smart Images
Figure CN119559465A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of deep learning, and in particular to a random depth-based model training method, system, storage medium and device. Background Art
[0002] In recent years, deep learning has achieved remarkable results in various fields, but as the model size continues to increase, its computing cost and storage requirements have also soared. This has brought challenges to the application of deep learning, especially on resource-constrained devices such as mobile devices and embedded systems.
[0003] In order to solve the efficiency problem of deep learning models, a large number of resource-saving methods have emerged. Model-assisted training is one of the simple and efficient solutions. Its main principle is to assist the main branch in back-propagation operations during model training to improve the speed of deep learning model training convergence, and automatically remove it during the inference stage, so as not to affect the inference efficiency of the deep learning model. For example, in the semantic segmentation algorithm BiSeNetV1, researchers introduced deep supervision branches in the last few layers of the model to assist segmentation model training; for example, in the target detection algorithm YOLOV9, a programmable gradient branch of the same magnitude is added to the original main branch to improve the efficiency of gradient calculation of back-propagation during training.
[0004] Although existing auxiliary training methods can enable deep learning models to achieve good inference results on resource-constrained devices, these results are based on high training server resource costs, transferring the costs originally used on inference devices to training devices, which does not actually reduce the burden. Therefore, how to reduce the training cost of deep learning models is a technical problem that technicians in this field need to solve urgently. Summary of the invention
[0005] The purpose of this application is to provide a model training method, system, storage medium and electronic device based on random depth. By setting the depth value of the auxiliary feature extraction block, different depth values can be used in different training stages, saving training resources and improving training efficiency.
[0006] In order to solve the above technical problems, this application provides a model training method based on random depth, and the specific technical solution is as follows:
[0007] In each iterative training, the image data to be trained is obtained, and the current number of training iterations is determined;
[0008] If the current training iteration number is less than the maximum training iteration number, the image data to be trained is input into the feature extraction branch and the auxiliary feature extraction branch respectively; wherein the auxiliary feature extraction branch includes a plurality of auxiliary feature extraction blocks;
[0009] Determine the depth value of the auxiliary feature extraction block applied during this training according to the current training iteration number; the depth value is the number of auxiliary feature extraction blocks actually used when applying the auxiliary feature extraction branch, and the current training iteration number is negatively correlated with the depth value;
[0010] The current iterative training is performed based on the depth value until the current training iteration number is not less than the maximum training iteration number, and then the training is terminated and the model is output.
[0011] Optionally, a downsampling unit is provided between adjacent auxiliary feature extraction blocks in the auxiliary feature extraction branch, and the downsampling unit is used to reduce the resolution of the image data to be trained.
[0012] Optionally, performing this iterative training based on the depth value includes:
[0013] Determining the random skipping number of the auxiliary feature extraction block during this iterative training based on the depth value;
[0014] Abandoning the auxiliary feature extraction blocks corresponding to the random skipping number and retaining the downsampling unit to obtain a modified auxiliary feature extraction branch;
[0015] This feature extraction training is performed according to the feature extraction branch and the corrected auxiliary feature extraction branch.
[0016] Optionally, a feature fusion branch is further included between the feature extraction branch and the auxiliary feature extraction branch, and the feature fusion branch includes at least one feature fusion block, and the feature fusion block is used to fuse feature extraction data corresponding to different feature extraction blocks in the feature extraction branch.
[0017] Optionally, when performing this iterative training based on the depth value, it also includes:
[0018] The auxiliary feature extraction block in the auxiliary feature extraction branch is used to perform auxiliary feature extraction on the feature fusion result of the feature fusion block and the auxiliary feature extraction result of the auxiliary feature extraction block at the previous level.
[0019] Optionally, determining the depth value of the auxiliary feature extraction block to be applied during the current training according to the current training iteration number includes:
[0020] Determine a training number interval corresponding to the current training iteration number;
[0021] The depth value of the auxiliary feature extraction block applied during the current training is determined according to the training number interval.
[0022] Optionally, determining the depth value of the auxiliary feature extraction block to be applied during the current training according to the current training iteration number includes:
[0023] Obtaining a functional relationship between the current training iteration number and the depth value;
[0024] The current training iteration number is substituted into the functional relationship to determine the depth value of the auxiliary feature extraction block applied during this training.
[0025] The present application also provides a model training system based on random depth, comprising:
[0026] The iteration number determination module is used to obtain the image data to be trained and determine the current training iteration number during each iteration training;
[0027] A data input module, used for inputting the image data to be trained into the feature extraction branch and the auxiliary feature extraction branch respectively if the current training iteration number is less than the maximum training iteration number; wherein the auxiliary feature extraction branch includes a plurality of auxiliary feature extraction blocks;
[0028] A depth determination module, used to determine the depth value of the auxiliary feature extraction block applied during this training according to the current training iteration number; the depth value is the number of auxiliary feature extraction blocks actually used when applying the auxiliary feature extraction branch, and the current training iteration number is negatively correlated with the depth value;
[0029] The training module is used to perform the iterative training based on the depth value until the current training iteration number is not less than the maximum training iteration number, and then terminate the training and output the model.
[0030] The present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above-mentioned method when executed by a processor.
[0031] The present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned method when calling the computer program in the memory.
[0032] The present application provides a model training method based on random depth, comprising: obtaining image data to be trained and determining the current training iteration number during each iterative training; if the current training iteration number is less than the maximum training iteration number, inputting the image data to be trained into a feature extraction branch and an auxiliary feature extraction branch respectively; wherein the auxiliary feature extraction branch includes a plurality of auxiliary feature extraction blocks; determining the depth value of the auxiliary feature extraction block applied during this training according to the current training iteration number; the depth value is the number of auxiliary feature extraction blocks actually used when applying the auxiliary feature extraction branch, and the current training iteration number is negatively correlated with the depth value; performing this iterative training based on the depth value until the current training iteration number is not less than the maximum training iteration number, then terminating the training and outputting the model.
[0033] This application not only updates the model architecture and adds an auxiliary feature extraction branch, but also uses adaptive depth values for training for different iterative training, so that the auxiliary feature extraction branch can select different numbers of auxiliary feature extraction blocks at different training stages, and the closer to the later stage of training, the fewer auxiliary feature extraction blocks participate in the training, thereby reducing the amount of calculation in the training process, thereby saving resource usage and reducing the resource pressure of the server.
[0034] The present application also provides a random depth-based model training system, storage medium and electronic device, which have the above-mentioned beneficial effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0036] Figure 1 A flowchart of a random depth-based model training method provided in an embodiment of the present application;
[0037] Figure 2 A schematic diagram of the model training process provided in the embodiment of the present application;
[0038] Figure 3 A schematic diagram of the structure of a model training system based on random depth provided in an embodiment of the present application;
[0039] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0041] Please refer to Figure 1 , Figure 1 A flowchart of a model training method based on random depth provided in an embodiment of the present application, the method comprising:
[0042] S101: In each iterative training, image data to be trained is obtained, and the current number of training iterations is determined;
[0043] S102: If the current training iteration number is less than the maximum training iteration number, input the image data to be trained into the feature extraction branch and the auxiliary feature extraction branch respectively; wherein the auxiliary feature extraction branch includes a plurality of auxiliary feature extraction blocks;
[0044] S103: Determine, according to the current training iteration number, a depth value of the auxiliary feature extraction block to be applied during this training;
[0045] S104: Execute the iterative training based on the depth value until the current training iteration number is not less than the maximum training iteration number, then terminate the training and output the model.
[0046] The embodiments of the present application can be applied to model training in the deep learning process, especially target detection type training and semantic segmentation type training. Target detection type training aims to identify targets in images and determine their locations. Target detection algorithms based on deep learning can be divided into one-stage and two-stage methods. One-stage methods, such as the YOLO (You Only LookOnce) series, go directly from image pixels to bounding boxes and category probabilities, are fast and suitable for real-time detection. Two-stage methods, such as Faster R-CNN, first generate candidate regions, and then classify and regress bounding boxes for each candidate region, with high accuracy but slow speed.
[0047] Semantic segmentation type training requires classifying each pixel in the image to understand the image content. The application of deep learning in semantic segmentation includes convolutional neural networks (CNN) and various encoder-decoder architectures such as SegNet and DeepLab. DeepLab proposed Atrous Convolution (hole convolution) and Atrous Spatial PyramidPooling (ASPP) to enhance the network's recognition ability for objects of different scales, and combined with fully connected conditional random fields (CRF) to improve boundary positioning. In addition, there are studies that propose a weakly supervised semantic segmentation network guided by salient background, which trains the semantic segmentation network by generating high-quality pseudo-pixel category labels, effectively improving the segmentation accuracy.
[0048] In each training iteration, it is necessary to obtain the image data to be trained and determine the current number of training iterations. When the current number of training iterations does not meet the maximum number of training iterations, the feature extraction branch and the auxiliary feature extraction branch are used to extract the features of the image data to be trained. The feature extraction branch is usually responsible for extracting useful features from the input data. In object detection and semantic segmentation tasks, this branch may contain multiple convolutional layers to capture local features and texture information of the image. For example, in the YOLOv3 model, the feature extraction branch performs object detection through feature maps of multiple scales, which can capture objects of different sizes. The auxiliary feature extraction branch is to enhance the feature representation ability of the feature extraction branch, focusing more on extracting global features or specific types of features to supplement the features extracted by the main branch. For example, in a binocular stereo matching algorithm based on three-branch hybrid feature extraction, the auxiliary branch may include a Swin Transformer branch to extract global features and combine them with the local feature expression ability of the CNN branch.
[0049] There is no limitation on the deep learning algorithms used in the feature extraction branch and the auxiliary feature extraction branch. The feature extraction branch may use algorithms including but not limited to VGGNet, ResNet, and ConvNeXt.
[0050] VGGNet is a very deep convolutional neural network, which is composed of multiple convolutional layers and pooling layers stacked together, without the complex structure of region proposal networks or fully connected layers. A key feature of VGGNet is its uniform network structure, that is, there are only 3 different convolution kernel sizes (3x3) in the entire network, which makes the network have good consistency between different layers. VGGNet achieves feature extraction by gradually reducing the dimension of the feature map and increasing the depth of the feature by continuously stacking 3x3 convolutional layers and 2x2 pooling layers.
[0051] ResNet (residual network) solves the degradation problem in deep network training by introducing the "residual learning" framework, that is, the problem that the accuracy decreases as the network depth increases. The core idea of ResNet is to use identity mappings as skip connections, allowing signals in the network to pass directly from one block to another, which makes it easier for the network to train deeper models. ResNet builds very deep networks by stacking multiple residual units (each unit contains several convolutional layers and a skip connection), which can learn more complex feature representations.
[0052] ConvNeXt is a pure convolutional neural network that draws on advanced methods in the Transformer architecture and makes improvements in training methods, macro-design, and micro-design. ConvNeXt introduces deep separable convolution to improve the model's computing speed, uses an inverse bottleneck layer structure to effectively avoid information loss, and replaces the batch normalization layer with a layer normalization layer to solve the problem of normalized statistical deviation. ConvNeXt forms a multi-stream ConvNeXt network through multi-scale convolution parallel processing, which enhances the network's ability to adapt to changes in local feature information and enables the network to extract richer feature types.
[0053] The feature extraction branch may include several feature extraction blocks, and the specific number of feature extraction blocks can be set by those skilled in the art according to the training requirements. Similarly, the auxiliary feature extraction branch may also include several auxiliary feature extraction blocks, and the number of auxiliary feature extraction blocks is not related to the number of feature extraction blocks, and the two numbers may be the same or different.
[0054] In a feasible implementation, a downsampling unit is provided between adjacent auxiliary feature extraction blocks in the auxiliary feature extraction branch, and the downsampling unit is used to reduce the resolution of the image data to be trained. The downsampling unit is used to perform downsampling to reduce the image resolution. Since the maximum downsampling of the current target detection and semantic segmentation framework is 32 times, a maximum of 5 downsampling units can be set. As the maximum downsampling multiple increases, the number of downsampling units can be increased accordingly.
[0055] The specific method of downsampling is not limited here, and may include but is not limited to pooling downsampling, convolution downsampling and interpolation downsampling.
[0056] This application does not make specific requirements on the content of the auxiliary feature extraction block. You can choose lightweight methods such as inverted residual blocks, depthwise separable convolutions, etc. The specific settings can be selected according to the actual task.
[0057] The inverted residual block is a structure in the residual network (ResNet), which is an improvement on the standard residual block. In the standard residual block, the convolution layer usually first extracts spatial features (using a 1x1 convolution kernel) and then adjusts the number of channels (using a 3x3 convolution kernel). In the inverted residual block, this order is reversed, that is, the number of channels is adjusted by depthwise separable convolution, and then spatial features are extracted by 1x1 convolution. This structure is widely used in the MobileNet series of models because it can reduce the number of parameters and computation of the model while maintaining or improving the performance of the model.
[0058] Depthwise separable convolution is an efficient convolution operation that decomposes the standard convolution operation into two smaller operations: depthwise convolution and pointwise convolution. In depthwise convolution, a convolution kernel is applied independently to each input channel; in pointwise convolution, a 1x1 convolution kernel is used to perform feature fusion on the result of depthwise convolution.
[0059] In S103, it is necessary to determine the depth value of the auxiliary feature extraction block applied during this training according to the current training iteration number. The depth value is the number of auxiliary feature extraction blocks actually used when applying the auxiliary feature extraction branch, and the current training iteration number is negatively correlated with the depth value. That is, the larger the current training iteration number, the smaller the depth value, that is, the fewer the number of auxiliary feature extraction blocks applied during this training process. When the current training iteration number is not less than the maximum training iteration number, the training is terminated and the model is output.
[0060] There is no limitation on how to determine the depth value. This application provides two determination methods:
[0061] In the first method, the training number interval corresponding to the current training iteration number can be determined, so as to determine the depth value of the auxiliary feature extraction block applied in this training according to the training number interval.
[0062] In the second method, the functional relationship between the current training iteration number and the depth value can be obtained, and the current training iteration number is substituted into the functional relationship to determine the depth value of the auxiliary feature extraction block used in this training.
[0063] When performing local iterative training, you can follow the following steps:
[0064] The first step is to determine the random skipping number of the auxiliary feature extraction block during this iterative training based on the depth value;
[0065] Step 2: discard the auxiliary feature extraction block corresponding to the random skip number, and retain the downsampling unit to obtain a modified auxiliary feature extraction branch;
[0066] The third step is to perform feature extraction training according to the feature extraction branch and the corrected auxiliary feature extraction branch.
[0067] For example, at a certain moment, the auxiliary feature extraction block 2 is skipped, and auxiliary training is performed only using auxiliary feature extraction blocks 1, 3, 4, 5, and 6. Since there is an auxiliary feature extraction block 2 between the data of auxiliary feature extraction block 1 and auxiliary feature extraction block 3, there is no auxiliary feature extraction block 2 and no further downsampling can be performed. However, the input of 3 requires 4 times downsampling of the original image. Therefore, the result of auxiliary feature extraction block 1 is subjected to traditional downsampling again on the basis of pooling 2 times downsampling, and the result is then transmitted to auxiliary feature extraction block 3.
[0068] It can be seen that the more the auxiliary feature extraction blocks are randomly skipped in the embodiment of the present application at the end of the training, the more the model gradually reduces its dependence on the auxiliary feature extraction branch, so that the model can switch losslessly between the training and reasoning stages. For example, set the training to 400 rounds, the auxiliary feature extraction branch contains 6 auxiliary feature extraction blocks, and only 1 auxiliary feature extraction block is randomly skipped in the first 100 rounds; 2 auxiliary feature extraction blocks are randomly skipped starting from the 200th round; 3 auxiliary feature extraction blocks are randomly skipped starting from the 300th round; 4 auxiliary feature extraction blocks are randomly skipped starting from the 350th round; 5 auxiliary feature extraction blocks are randomly skipped starting from the 390th round, until the end of the training.
[0069] Compared with most existing auxiliary training methods that pile up a large number of feature extraction blocks during the training phase, this application adopts a method of randomly discarding auxiliary feature extraction blocks, which not only increases the disturbance during training to improve the robustness of the model, but also reduces the demand for training resources. At the same time, this application gradually reduces the model's dependence on the auxiliary feature extraction branch during the training phase, so that it can smoothly switch to the inference state and retain the target detection capability during training.
[0070] The model trained in this application can be used to perform target detection or image content recognition in an image. And the image data to be trained described in this application can be data such as pictures, videos, video frames, etc.
[0071] This application not only updates the model architecture and adds an auxiliary feature extraction branch, but also uses adaptive depth values for training for different iterative training, so that the auxiliary feature extraction branch can select different numbers of auxiliary feature extraction blocks at different training stages, and the closer to the later stage of training, the fewer auxiliary feature extraction blocks participate in the training, thereby reducing the amount of calculation in the training process, thereby saving resource usage and reducing the resource pressure of the server.
[0072] See also Figure 2 , Figure 2 A schematic diagram of the model training process provided in the embodiment of the present application, Figure 2 The middle feature extraction branch and the auxiliary feature extraction branch each contain 6 extraction blocks, namely feature extraction block_1, feature extraction block_2, feature extraction block_3, feature extraction block_4, feature extraction block_5 and feature extraction block_6, while the auxiliary feature extraction branch contains auxiliary feature extraction block_1, auxiliary feature extraction block_2, auxiliary feature extraction block_3, auxiliary feature extraction block_4, auxiliary feature extraction block_5 and auxiliary feature extraction block_6.
[0073] There is also a feature fusion branch between the feature extraction branch and the auxiliary feature extraction branch. The feature fusion branch includes at least one feature fusion block for fusing feature extraction data corresponding to different feature extraction blocks in the feature extraction branch. Figure 2 There are three feature fusion blocks, namely feature fusion block_1, feature fusion block_2, and feature fusion block_3. Specifically, the auxiliary feature extraction block in the auxiliary feature extraction branch can be used to perform auxiliary feature extraction on the feature fusion result of the feature fusion block and the auxiliary feature extraction result of the auxiliary feature extraction block at the previous level.
[0074] The feature fusion block_2 fuses the feature data transmitted by the feature fusion block_1 and the feature extraction block 5. It should be noted that the data of the feature fusion block_1 and the feature extraction block_6 are actually consistent.
[0075] In addition, not every feature extraction block is followed by a corresponding feature fusion block. This is because the features extracted by the previous feature extraction blocks are relatively rough and have large noise. Therefore, feature fusion blocks are usually added after several extraction blocks at the end of the feature extraction branch.
[0076] Figure 2 The result prediction branch is also included. The result prediction branch is provided with three result prediction blocks, namely, result prediction block_1, result prediction block_2 and result prediction block_3, which correspond to output_1, output_2 and output_3 respectively.
[0077] This embodiment introduces a random depth method on the auxiliary feature extraction branch, so that the branch selects different numbers of auxiliary feature extraction blocks in different training stages (the later the training, the fewer auxiliary training blocks involved in the training). This not only saves the amount of calculation and memory usage during the training period and reduces the resource pressure of the training server, but also enables the deep learning model to adapt in advance to the state where the auxiliary training branch does not participate in the calculation.
[0078] A random depth-based model training system provided in an embodiment of the present application is introduced below. The random depth-based model training system described below and the random depth-based model training method described above can refer to each other.
[0079] See also Figure 3 , Figure 3 A schematic diagram of a random depth-based model training system provided in an embodiment of the present application. The present application also provides a random depth-based model training system, including:
[0080] The iteration number determination module is used to obtain the image data to be trained and determine the current training iteration number during each iteration training;
[0081] A data input module, used for inputting the image data to be trained into the feature extraction branch and the auxiliary feature extraction branch respectively if the current training iteration number is less than the maximum training iteration number; wherein the auxiliary feature extraction branch includes a plurality of auxiliary feature extraction blocks;
[0082] A depth determination module, used to determine the depth value of the auxiliary feature extraction block applied during this training according to the current training iteration number; the depth value is the number of auxiliary feature extraction blocks actually used when applying the auxiliary feature extraction branch, and the current training iteration number is negatively correlated with the depth value;
[0083] The training module is used to perform the iterative training based on the depth value until the current training iteration number is not less than the maximum training iteration number, and then terminate the training and output the model.
[0084] Based on the above embodiment, as a preferred embodiment, a downsampling unit is provided between adjacent auxiliary feature extraction blocks in the auxiliary feature extraction branch, and the downsampling unit is used to reduce the resolution of the image data to be trained.
[0085] Based on the above embodiment, as a preferred embodiment, the training module includes:
[0086] A quantity determination unit, used to determine the random skipping quantity of the auxiliary feature extraction block during this iterative training based on the depth value;
[0087] A correction unit, used to discard the auxiliary feature extraction block corresponding to the random skip number and retain the downsampling unit to obtain a corrected auxiliary feature extraction branch;
[0088] The training unit is used to perform the feature extraction training according to the feature extraction branch and the corrected auxiliary feature extraction branch.
[0089] Based on the above embodiments, as a preferred embodiment, a feature fusion branch is also included between the feature extraction branch and the auxiliary feature extraction branch, and the feature fusion branch includes at least one feature fusion block, and the feature fusion block is used to fuse the feature extraction data corresponding to different feature extraction blocks in the feature extraction branch.
[0090] Based on the above embodiments, as a preferred embodiment, it also includes:
[0091] The auxiliary feature extraction module is used to apply the auxiliary feature extraction block in the auxiliary feature extraction branch to perform auxiliary feature extraction on the feature fusion result of the feature fusion block and the auxiliary feature extraction result of the auxiliary feature extraction block at the previous level.
[0092] Based on the above embodiment, as a preferred embodiment, the depth determination module includes:
[0093] The first depth determination unit is used to determine a training number interval corresponding to the current training iteration number; and determine a depth value of the auxiliary feature extraction block applied during this training according to the training number interval.
[0094] Based on the above embodiment, as a preferred embodiment, the depth determination module includes:
[0095] The first depth determination unit is used to obtain a functional relationship between the current training iteration number and the depth value; substitute the current training iteration number into the functional relationship to determine the depth value of the auxiliary feature extraction block used in this training.
[0096] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0097] The present application also provides an electronic device, see Figure 4 , a structural diagram of an electronic device provided in an embodiment of the present application, such as Figure 4 As shown, a processor 1410 and a memory 1420 may be included.
[0098] Among them, the processor 1410 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1410 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1410 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1410 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1410 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0099] The memory 1420 may include one or more computer-readable storage media, which may be non-transitory. The memory 1420 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 1420 is at least used to store the following computer program 1421, wherein, after the computer program is loaded and executed by the processor 1410, it can implement the relevant steps in the method performed by the electronic device side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 1420 may also include an operating system 1422 and data 1423, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 1422 may include Windows, Linux, Android, etc.
[0100] In some embodiments, the electronic device may further include a display screen 1430 , an input / output interface 1440 , a communication interface 1450 , a sensor 1460 , a power source 1470 , and a communication bus 1480 .
[0101] certainly, Figure 4 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of the present application. In actual applications, the electronic device may include Figure 4 More or fewer components than shown, or combinations of certain components.
[0102] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system provided in the embodiment, since it corresponds to the method provided in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.
[0103] Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.
[0104] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
Claims
1. A model training method based on random depth, characterized in that: include: In each iterative training, the image data to be trained is obtained, and the current number of training iterations is determined; If the current training iteration number is less than the maximum training iteration number, the image data to be trained is input into the feature extraction branch and the auxiliary feature extraction branch respectively; wherein the auxiliary feature extraction branch includes a plurality of auxiliary feature extraction blocks; Determine the depth value of the auxiliary feature extraction block applied during this training according to the current training iteration number; the depth value is the number of auxiliary feature extraction blocks actually used when applying the auxiliary feature extraction branch, and the current training iteration number is negatively correlated with the depth value; The iterative training is performed based on the depth value until the current training iteration number is not less than the maximum training iteration number, and then the training is terminated and the model is output; the model is used to perform image processing.
2. The random depth-based model training method according to claim 1, characterized in that: A downsampling unit is provided between adjacent auxiliary feature extraction blocks in the auxiliary feature extraction branch, and the downsampling unit is used to reduce the resolution of the image data to be trained.
3. The random depth-based model training method according to claim 2, characterized in that: Performing this iterative training based on the depth value includes: Determining the random skipping number of the auxiliary feature extraction block during this iterative training based on the depth value; Abandoning the auxiliary feature extraction blocks corresponding to the random skipping number and retaining the downsampling unit to obtain a modified auxiliary feature extraction branch; This feature extraction training is performed according to the feature extraction branch and the corrected auxiliary feature extraction branch.
4. The random depth-based model training method according to claim 1, characterized in that: A feature fusion branch is further included between the feature extraction branch and the auxiliary feature extraction branch. The feature fusion branch includes at least one feature fusion block. The feature fusion block is used to fuse feature extraction data corresponding to different feature extraction blocks in the feature extraction branch.
5. The random depth-based model training method according to claim 4, characterized in that: When performing this iterative training based on the depth value, it also includes: The auxiliary feature extraction block in the auxiliary feature extraction branch is used to perform auxiliary feature extraction on the feature fusion result of the feature fusion block and the auxiliary feature extraction result of the auxiliary feature extraction block at the previous level.
6. The random depth-based model training method according to claim 1, characterized in that: Determining the depth value of the auxiliary feature extraction block used in this training according to the current training iteration number includes: Determine a training number interval corresponding to the current training iteration number; The depth value of the auxiliary feature extraction block applied during the current training is determined according to the training number interval.
7. The random depth-based model training method according to claim 1, characterized in that: Determining the depth value of the auxiliary feature extraction block used in this training according to the current training iteration number includes: Obtaining a functional relationship between the current training iteration number and the depth value; The current training iteration number is substituted into the functional relationship to determine the depth value of the auxiliary feature extraction block applied during this training.
8. A model training system based on random depth, characterized in that: include: The iteration number determination module is used to obtain the image data to be trained and determine the current training iteration number during each iteration training; A data input module, used for inputting the image data to be trained into the feature extraction branch and the auxiliary feature extraction branch respectively if the current training iteration number is less than the maximum training iteration number; wherein the auxiliary feature extraction branch includes a plurality of auxiliary feature extraction blocks; A depth determination module, used to determine the depth value of the auxiliary feature extraction block applied during the current training according to the current training iteration number; the depth value is the number of auxiliary feature extraction blocks actually used when applying the auxiliary feature extraction branch, and the current training iteration number is negatively correlated with the depth value; A training module is used to perform the iterative training based on the depth value until the current training iteration number is not less than the maximum training iteration number, then the training is terminated and the model is output; the model is used to perform image processing.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the random depth-based model training method as described in any one of claims 1 to 7 are implemented.
10. An electronic device, characterized in that: It includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the random depth-based model training method as described in any one of claims 1 to 7 are implemented.