Lightweight method and device for small target assisted real-time recognition model of unmanned equipment

Abstract

The application discloses a lightweight method and device for real-time identification of small targets of unmanned equipment, and is suitable for a single-stage target identification algorithm structure mainly containing backbone, neck and head structures, and small target identification below a specified pixel, adopts a structured pruning scheme, and performs a model lightweight strategy; the method comprises the following steps: step one: starting, constructing an original network, and training the constructed original network; step two: network pruning of 10%, training for one round, and parameter adjustment; step three: determining whether pruning is completed, if not, returning to network pruning of 10%, and if yes, ending and saving model information; the application has the beneficial effects that the structured pruning scheme is adopted for small target identification below a specified pixel in the single-stage target identification algorithm structure mainly containing backbone, neck and head structures, the deep learning model is lightweighted, the model parameter quantity and inference calculation quantity can be reduced, and the application is suitable for the scene of edge offline real-time inference.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a lightweight method and device for real-time identification of small targets of unmanned equipment. BACKGROUND

[0002] Limited environment: scenarios that require lightweight models, generally edge or mobile end.

[0003] The deployment structure of general intelligent devices is divided into cloud, edge and end; generally, unmanned equipment such as unmanned aerial vehicles and airborne heterogeneous processing chips NPU belong to edge devices; when intelligent devices perform tasks, many intelligent modules cannot be processed in the cloud, such as image recognition and target tracking, because the network environment does not allow it, or the round-trip transmission delay is too long, and the task has real-time requirements, and the task needs to be processed in the edge.

[0004] The general load of unmanned equipment is limited, the processing capability is limited, and the device power consumption is also limited, unlike cloud devices, which have strong processing capability and high power consumption; if a deep learning model is too complex, has many layers and parameters, and has large computation, it cannot meet the requirements of the edge; at this time, the model needs to be lightweight, and under the premise of maintaining accuracy or controlling the impact on accuracy within an acceptable range, the network structure is simplified, the parameters are reduced, and the computation is reduced, which is an important work; mobile devices have limited computing power, and if the computing task is completed under the condition of limited computing power, it is the work that needs to be done for model lightweight.

[0005] Applicable scenarios: scenarios of unmanned equipment assisted control, in the presence of human operation, the driver's field of vision is prompted to key targets, and the driver is warned.

[0006] The scene needs real-time performance, the recognition speed of the model needs to follow the conversion of the field of vision, and the calculation can only be performed in the edge, not in the cloud, because the remote network connection is not guaranteed, and the intelligent agent needs to work independently off the network; the processing capability of the general edge chip is limited, and the deep learning model used for identification needs to be lightweight to adapt to the hardware computing power limit.

[0007] The mainstream of the existing industrial target recognition model is the YOLO series, especially YOLOv3 and the like, these models are used for general scene target detection, and there are generally two problems, one is that the default resolution of the input picture is low, the existing default configuration is generally 416X416, and the larger one is 640X640 and the like, these models are public, and several reference sources can be attached to explain; and the resolution of general assisted driving is relatively high, the commonly used original image is 1080X1920 and the like, if the picture is directly compressed to 416X416, most of the features of the small target on the original image will be lost, and it cannot be recognized, if the picture is not compressed, the input picture is divided into 416X416 blocks for recognition, and the general edge chip computing power is not enough, and the real-time output detection result cannot be achieved. SUMMARY

[0008] The purpose of the present application is to provide a lightweight method and device for small target assisted real-time recognition model of unmanned equipment, which adjusts and lightens the model, reduces the model parameter quantity and inference calculation quantity.

[0009] To achieve the above purpose, the present application provides the following technical scheme: a lightweight method for small target assisted real-time recognition model of unmanned equipment, for single-stage, containing backbone, neck, head structure of target recognition algorithm structure, specifying small target recognition below pixel, adopting structured pruning scheme, and performing model lightweight strategy; the method is as follows:

[0010] Step one: start, build the original network, and train the built original network;

[0011] Step two: network pruning 10%, train a round, and adjust the parameters;

[0012] Step three: judge whether the pruning is completed, if not, return to network pruning 10% again, if completed, end and save the model information.

[0013] As a preferred technical scheme of the present application, the backbone is a plurality of convolution operations, which extracts the features of the input image.

[0014] As a preferred technical scheme of the present application, the neck adopts a feature pyramid, which connects features of different levels to adapt to large, medium and small targets.

[0015] As a preferred technical scheme of the present application, the head is responsible for target detection and result output.

[0016] As a preferred technical scheme of the present application, different algorithms can adopt different strategies, including anchor (anchor) and anchor-free.

[0017] As a preferred technical solution of the present application, the layer adopted by the neck is determined by the specific down-sampling position and the input resolution and the target resolution.

[0018] As a preferred technical solution of the present application, during the training, the size of the training sample is first judged, and the target with a size greater than 1.5 times the maximum size is ignored.

[0019] The application also discloses a lightweight device for small target auxiliary real-time identification model of unmanned equipment, which comprises

[0020] A construction module is configured to construct an original network.

[0021] A training module is configured to train the constructed original network.

[0022] A pruning and adjusting module is configured to perform network pruning, training and parameter adjustment.

[0023] A judging module is configured to judge whether the pruning is completed.

[0024] A saving module is configured to save model information.

[0025] Compared with the prior art, the application has the following beneficial effects:

[0026] For a single-stage target recognition algorithm structure mainly comprising a backbone, a neck and a head structure, the application adopts a structured pruning scheme to perform lightweight processing on a deep learning model, so that the model parameter quantity and inference calculation quantity can be reduced, and the application is suitable for edge offline real-time inference scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 Fig. 1 is a network pruning structure diagram of the application;

[0028] Figure 2 Fig. 2 is a training method diagram of the application;

[0029] Figure 3 Fig. 3 is a device diagram of the application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application. Embodiment 1

[0031] Please refer to Figure 1, Figure 2 This is the first embodiment of the present invention, which provides a lightweight method for a real-time target recognition model for unmanned equipment. It targets a single-stage target recognition algorithm structure including a backbone, neck, and head, and identifies small targets below a specified pixel size. A structured pruning scheme is employed for model lightweighting. The backbone involves numerous convolutional operations to extract features from the input image, using Darknet or Mobilenet. The neck uses a feature pyramid to connect features at different levels to adapt to targets of varying sizes, employing the YOLO series. The head is responsible for target detection and result output; different algorithms can use different strategies, including anchor-based and anchor-free approaches.

[0032] Determine the minimum target size range to be detected in the original resolution, with a maximum of 30x30. Targets exceeding this range are considered large targets and do not need to be detected. This description uses 30x30 as an example for ease of explanation, but in actual writing, it can be abstracted to a description of the maximum M×M.

[0033] Based on the target's maximum resolution, the backbone network is pruned. Taking Darknet53 as an example, the backbone network mainly consists of a series of convolutional layers. Large-span convolutions can be broken down into 3x3 convolutions. For example, a 5x5 convolution can be completely replaced by two 3x3 convolutions. After each convolution, the spatial scale features of the target are reduced. The larger the stride, the greater the impact on the spatial scale. Based on the target's maximum size, the number of convolutional layers is pruned. When the spatial scale features decrease to 1x1, no further convolutions are performed. For example, for a target with a maximum size of 5x5 and a stride of 1, without considering padding, after one 3x3 convolution, the feature map becomes 3x3. After two convolutions, it becomes 1x1, and the spatial scale features cannot be further extracted. If the stride is greater than 1, the reduction is even faster.

[0034] For the neck layer, YOLO typically uses three layers. In order to reduce the computational load, this application removes the layer that identifies large targets. The actual layer from which to start cropping the backbone is determined based on the specific cropping of the backbone. Generally, the last layer is removed, and the top two layers are retained because the last layer is the deepest and corresponds to the features of large targets. This is determined by the specific downsampling position and the input and target resolutions.

[0035] In training, first determine the size of the training sample, ignore the target whose size is greater than 1.5 times the maximum size, and this size is represented by the parameter S, to prevent overfitting of features, because large target samples may have too many details; this is reflected in the loss function, by adding a regularization term or directly ignoring in training:

[0036] = ;

[0037] The loss function of the Kth sample is represented by L(x, y | w), which is the basic loss function, where The size (W*H) of the kth sample is not greater than S, if it is greater than S, it is 0, and if it is less than S, it is 1; the loss is 0, and the parameter is not updated, or directly ignored;

[0038] The loss function is added to the original basis, and the L1 regularization term is defined as:

[0039] = ;

[0040] Therefore, during the training process, the adjustment direction of the parameters is

[0041] W = , L1 regularization can make the parameters sparse;

[0042] The lightweight method includes the following steps:

[0043] Step one: start, build the original network, and train the built original network;

[0044] Step two: network pruning 10%, train a round, and adjust the parameters;

[0045] Step three: judge whether the pruning is completed, if not, return to network pruning 10%, if completed, end and save the model information.

[0046] The present application is only suitable for pure small target detection, and the model does not have the ability to detect large targets. The structured pruning method is used to lightweight the deep learning model to adapt to special application scenarios; structured pruning does not affect heterogeneous hardware acceleration and heterogeneous model deployment.

[0047] Embodiment 2

[0048] Please refer to Figures 1-3 , which is the second embodiment of the present application, which is based on the previous embodiment, except that:

[0049] The lightweight device of the unmanned equipment small target auxiliary real-time identification model comprises

[0050] Modules are used to build the original network;

[0051] The training module is used to train the original network that has been constructed.

[0052] The cropping and adjustment module is used for network cropping, training, and parameter adjustment;

[0053] The judgment module is used to determine whether the cropping is complete;

[0054] The save module is used to save model information.

[0055] Although embodiments of the invention have been shown and described in detail above, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A lightweight method for a real-time small target recognition model for unmanned equipment, characterized by: For single-stage target recognition algorithms that include a backbone, neck, and head structure, a structured pruning approach is adopted to lightweight the model for small target recognition below a specified pixel size. The method is as follows: Step 1: Begin by building the original network and training it. Step 2: Prune the network by 10%, train it once, and adjust the parameters; Step 3: Determine if the cropping is complete. If not, return to the network and crop by 10%. If cropping is complete, end the process and save the model information. During training, the size of the training samples is first determined, and targets larger than 1.5 times the maximum size are ignored. This size is represented by the parameter S, which is reflected in the loss function by adding a regularization term. ， Let L(x, y | w) represent the loss function for the Kth sample, where L(x, y | w) is the base loss function. This indicates whether the size (W*H) of the k-th sample is greater than S. If it is greater than S, the value is 0; if it is less than S, the value is 1. The loss is 0, and no parameter update is performed. The loss function is modified by adding an L1 regularization term. The definition of L1 regularization is: ， During training, the direction of parameter adjustment is as follows: L1 regularization can make parameters sparse.

2. The lightweight method for the real-time recognition model of small targets assisted by unmanned equipment according to claim 1, characterized in that: The backbone consists of many convolutional operations that extract features from the input image.

3. The lightweight method for the real-time recognition model of small targets assisted by unmanned equipment according to claim 1, characterized in that: The neck uses a feature pyramid to connect features at different levels to adapt to targets of different sizes.

4. The lightweight method for the real-time recognition model of small targets assisted by unmanned equipment according to claim 1, characterized in that: The head is responsible for target detection and result output.

5. The lightweight method for the real-time recognition model of small targets assisted by unmanned equipment according to claim 1, characterized in that: Different algorithms can employ different strategies, including anchor-based and anchor-free approaches.

6. The lightweight method for the real-time recognition model of small targets assisted by unmanned equipment according to claim 1, characterized in that: The layer used in the neck is determined by the specific downsampling position, input resolution, and target resolution.

7. A lightweight device for a real-time recognition model for small targets assisted by unmanned equipment, used to implement the lightweight method as described in any one of claims 1-6, characterized in that: include Modules are used to build the original network; The training module is used to train the original network that has been constructed. The cropping and adjustment module is used for network cropping, training, and parameter adjustment; The judgment module is used to determine whether the cropping is complete; The save module is used to save model information.

Images