Image 2D object detection network construction method based on backbone layered recursive cascade architecture

Abstract

The present invention provides a method for constructing an image two-dimensional target detection network based on a backbone network hierarchical recursive cascade architecture, comprising: selecting VAN‑B1 as the backbone network, replacing the backbone network in a two-dimensional target detection network model YOLO, splitting multiple existing convolutional cascade units in a feature pyramid network in the two-dimensional target detection network model YOLO, modifying the connection topology between three layers of the backbone network and the feature pyramid network, and realizing low-level reasoning reuse of the backbone network based on residual connections. The method can improve the two-dimensional target detection performance without increasing the computational complexity, and is more suitable for embedded AI chip deployment.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method for constructing an image two-dimensional target detection network based on a backbone network hierarchical recursive cascade architecture. Background Art

[0002] Image two-dimensional object detection is an extremely important application of artificial intelligence. It has the ability to simultaneously identify multiple target categories and their two-dimensional positions in an image. It is widely used in many fields such as autonomous driving, intelligent security, and industrial manufacturing detection. The single-stage target detection network model based on the convolutional neural network (CNN) has the advantages of low computational complexity (low computing power requirements), standard operators, and easy deployment on embedded AI chips. It has become the mainstream image two-dimensional object detection method in the industry.

[0003] The Transformer algorithm is based on the self-attention mechanism. Compared with the convolutional neural network (CNN), it has long-range dependencies and the ability to learn global context, and has become a research hotspot in the field of deep learning. However, the Transformer requires new operators and is difficult to deploy directly on existing embedded AI chips.

[0004] The classic single-stage two-dimensional object detection network model can be divided into three parts: the backbone network, the feature pyramid network FPN (Feature Pyramid Network) and the detection head. An important type of optimization scheme to improve the performance of the existing two-dimensional object detection network model focuses on the optimization of the feature pyramid network model, but these optimization schemes such as PAN (Path Aggregation Network), Bi-FPN, and DetectoRS RFP (Recursive Feature Pyramid) mostly adopt optimization strategies such as deepening the network depth, that is, increasing the number of network layers or optimizing the inter-layer connection structure through search. On the basis of improving performance, the algorithm complexity is increased or the AI ​​chip resource utilization rate is reduced due to the inability to fully utilize the embedded AI chip architecture.

[0005] Some of the terms are as follows:

[0006] FPN, Feature Pyramid Networks, feature pyramid network;

[0007] Bi-FPN, bidirectional feature pyramid network;

[0008] PAN, Path Aggregation Network, path aggregation network;

[0009] RFP, Recursive Feature Pyramid Network. Summary of the invention

[0010] In order to solve at least one technical problem in the prior art, an embodiment of the present invention provides a method for constructing an image two-dimensional object detection network based on a hierarchical recursive cascade architecture of a backbone network, which can improve the two-dimensional object detection performance without increasing the computational complexity and is more suitable for embedded AI chip deployment. To achieve the above technical objectives, the technical solution adopted by the embodiment of the present invention is:

[0011] An embodiment of the present invention provides a method for constructing an image two-dimensional target detection network based on a backbone network hierarchical recursive cascade architecture, comprising: selecting VAN-B1 as the backbone network, replacing the backbone network in a two-dimensional target detection network model YOLO, splitting the existing multiple convolutional cascade units in a feature pyramid network in the two-dimensional target detection network model YOLO, modifying the connection topology between three layers of the backbone network and the feature pyramid network, and realizing low-level reasoning reuse of the backbone network based on residual connections.

[0012] Furthermore, the method for constructing an image two-dimensional object detection network based on a backbone network hierarchical recursive cascade architecture specifically includes the following steps:

[0013] Step S10, select VAN-B1 as the backbone network to replace the backbone network in the two-dimensional object detection network model YOLO;

[0014] Step S20, splitting the original three cascade units consisting of 1×1 convolutions and 3×3 convolutions in the feature pyramid networks FPN-Neck2 and FPN-Neck1 of the two-dimensional object detection network model YOLO respectively connected to layer 3 and layer 4 of the backbone network into two cascade units consisting of 1×1 convolutions and 3×3 convolutions and one cascade unit consisting of 1×1 convolutions and 3×3 convolutions;

[0015] Step S30, connecting the output of backbone network layer 3 with the output of the cascade unit consisting of two 1×1 convolutions with 128 channels and 3×3 convolutions with 256 channels in feature pyramid network FPN-Neck2 through residual connection, and then using it as the input of backbone network layer 4;

[0016] Step S40, connecting the output of backbone network layer 4 with the output of the cascade unit consisting of two 1×1 convolutions with 256 channels and 3×3 convolutions with 512 channels in feature pyramid network FPN-Neck1 through residual connection, and then using it as the input of backbone network layer 5;

[0017] Step S50, the output of the backbone network layer 5 passes through the cascade unit consisting of three 1×1 convolutions with 512 channels and 3×3 convolutions with 1024 channels of the feature pyramid network FPN-Neck0, and the obtained output is input to the detection head of the feature layer corresponding to the backbone network layer 5. At the same time, the obtained output is upsampled by 2 times, channel-concatenated with the output of step S40, and then input to the cascade unit consisting of the last 1×1 convolution with 256 channels and 3×3 convolution with 512 channels in the feature pyramid network FPN-Neck1;

[0018] Step S60, the output of the cascade unit consisting of the last 1×1 convolution with a channel number of 256 and the 3×3 convolution with a channel number of 512 in the feature pyramid network FPN-Neck1 is input to the detection head of the feature layer corresponding to the backbone network layer 4, and at the same time, the output is upsampled by 2 times, channel-concatenated with the output of step S30, and then input to the cascade unit consisting of the last 1×1 convolution with a channel number of 128 and the 3×3 convolution with a channel number of 256 in the feature pyramid network FPN-Neck2, and then input to the detection head of the feature layer corresponding to the backbone network layer 3;

[0019] Step S70: the detection heads and non-maximum suppression (NMS) of each feature layer are consistent with the two-dimensional target detection network model YOLO.

[0020] Furthermore, only the features with resolutions of 1 / 8, 1 / 16, and 1 / 32 of the original image, i.e., the outputs of layers 3, 4, and 5 of the backbone network, are connected to the feature pyramid network.

[0021] Furthermore, layer 3 of the backbone network is lower than layer 4, and layer 4 is lower than layer 5.

[0022] The beneficial effect of the technical solution provided by the embodiment of the present invention is: compared with the standard FPN, the present application does not have any new model parameters and inference operations, but through the recursive cascading method of the FPN network and the backbone network, the depth of the backbone network and the depth of the FPN network are deepened at the same time, so the target detection performance is greatly improved, but the algorithm complexity is not increased at all. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Schematic diagram of a method for constructing a two-dimensional object detection network for images in an embodiment of the present invention.

[0024] Figure 2 It is a comparison diagram of the network model in the embodiment of the present invention and the existing network model. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0026] In the description of the embodiments of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0027] In the description of the embodiments of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0028] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0029] The embodiment of the present application proposes a method for constructing an image two-dimensional object detection network based on a backbone network hierarchical recursive cascade architecture, including: selecting VAN-B1 as the backbone network, replacing the backbone network in the two-dimensional object detection network model YOLO, splitting the existing multiple convolutional cascade units in the feature pyramid network in the two-dimensional object detection network model YOLO, modifying the connection topology of the three layers of the backbone network and the feature pyramid network, and realizing the reuse of low-level reasoning of the backbone network based on residual connections.

[0030] This application can improve the two-dimensional object detection performance without increasing the computational complexity and model parameters.

[0031] Specifically, the embodiment of the present application proposes a method for constructing an image two-dimensional object detection network based on a backbone network hierarchical recursive cascade architecture, comprising the following steps:

[0032] Step S10, select VAN-B1 as the backbone network to replace the backbone network in the two-dimensional object detection network model YOLO;

[0033] VAN-B1 is a type of VAN (Visual attention network), -B1 indicates its size; VAN network can be found in the literature: Meng-Hao Guo, Cheng-Ze Lu, et.al. "Visual attention network", arXiv, preprint arXiv:2202.09741, 2022;

[0034] In order to be compatible with the feature pyramid network of the YOLO series, only the features with a resolution of 1 / 8, 1 / 16, and 1 / 32 of the original image, that is, the outputs of layers 3, 4, and 5 of the backbone network, are connected to the feature pyramid network; layers 3, 4, and 5 of the backbone network are connected to the feature pyramid network. Figure 1 and Figure 2 In the figure, L3, L4 and L5 are used to represent layer 3, where layer 3 is lower than layer 4 and layer 4 is lower than layer 5.

[0035] Step S20, splitting the original three cascade units consisting of 1×1 convolutions and 3×3 convolutions in the feature pyramid networks FPN-Neck2 and FPN-Neck1 of the two-dimensional object detection network model YOLO respectively connected to layer 3 and layer 4 of the backbone network into two cascade units consisting of 1×1 convolutions and 3×3 convolutions and one cascade unit consisting of 1×1 convolutions and 3×3 convolutions;

[0036] Figure 1 and Figure 2 In the middle, backbone network layer 3 is connected to YOLO’s feature pyramid network FPN-Neck2, and backbone network layer 4 is connected to YOLO’s feature pyramid network FPN-Neck1; the convolutional cascade unit of YOLO’s feature pyramid network FPN-Neck0 is not split;

[0037] Step S30, connecting the output of backbone network layer 3 with the output of the cascade unit consisting of two 1×1 convolutions with 128 channels and 3×3 convolutions with 256 channels in feature pyramid network FPN-Neck2 through residual connection, and then using it as the input of backbone network layer 4;

[0038] The residual connection retains the original backbone network connection topology while realizing the recursive cascading of the feature pyramid network and the backbone network.

[0039] Step S40, connecting the output of backbone network layer 4 with the output of the cascade unit consisting of two 1×1 convolutions with 256 channels and 3×3 convolutions with 512 channels in feature pyramid network FPN-Neck1 through residual connection, and then using it as the input of backbone network layer 5;

[0040] Step S50, the output of the backbone network layer 5 passes through the cascade unit consisting of three 1×1 convolutions with 512 channels and 3×3 convolutions with 1024 channels of the feature pyramid network FPN-Neck0, and the obtained output is input to the detection head of the feature layer corresponding to the backbone network layer 5. At the same time, the obtained output is upsampled by 2 times, channel-concatenated with the output of step S40, and then input to the cascade unit consisting of the last 1×1 convolution with 256 channels and 3×3 convolution with 512 channels in the feature pyramid network FPN-Neck1;

[0041] Step S60, the output of the cascade unit consisting of the last 1×1 convolution with a channel number of 256 and the 3×3 convolution with a channel number of 512 in the feature pyramid network FPN-Neck1 is input to the detection head of the feature layer corresponding to the backbone network layer 4, and at the same time, the output is upsampled by 2 times, channel-concatenated with the output of step S30, and then input to the cascade unit consisting of the last 1×1 convolution with a channel number of 128 and the 3×3 convolution with a channel number of 256 in the feature pyramid network FPN-Neck2, and then input to the detection head of the feature layer corresponding to the backbone network layer 3;

[0042] In step S70, the detection heads and non-maximum suppression NMS (Non-Maximum Suppression) of each feature layer are consistent with the two-dimensional target detection network model YOLO and are not adjusted.

[0043] To further illustrate the advantages of this application, Figure 2 The simplified standard FPN, PAN, DetectoRS RFP and the network model connection topology proposed in this application are given. In the figure, L3, L4 and L5 represent the backbone network layer 3, layer 4 and layer 5 respectively; it can be seen that PAN has added three feature pyramid networks (also called neck modules) FPN-Neck10, FPN-Neck11 and FPN-Neck12 relative to the standard FPN. Although the performance is improved by increasing the network depth, the algorithm complexity is also increased; although the RFP network of DetectoRS does not add a new neck module and will not increase the number of model parameters, the recursive method introduces 2 or more times of backbone network and FPN network reasoning, which greatly increases the algorithm complexity.

[0044] The network model proposed in this application splits the original standard FPN feature pyramid network (also called neck module) FPN-Neck1 and FPN-Neck2, one part of which is embedded in the reasoning process of the backbone network, and the other part continues to execute the reasoning process of the standard FPN. Therefore, compared with the standard FPN, there are no new model parameters and reasoning operations, but through the recursive cascading method of the FPN network and the backbone network, the depth of the backbone network and the depth of the FPN network are deepened at the same time, so the target detection performance is greatly improved, but the algorithm complexity is not increased at all.

[0045] Finally, it should be noted that the above specific implementation methods are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for constructing an image two-dimensional object detection network based on a backbone network hierarchical recursive cascade architecture, characterized in that: include: Select VAN-B1 as the backbone network, replace the backbone network in the two-dimensional object detection network model YOLO, split the existing multiple convolutional cascade units in the feature pyramid network in the two-dimensional object detection network model YOLO, modify the connection topology between the three layers of the backbone network and the feature pyramid network, and realize the reuse of low-level reasoning of the backbone network based on residual connections; The specific steps include: Step S10, select VAN-B1 as the backbone network to replace the backbone network in the two-dimensional object detection network model YOLO; Step S20, splitting the original three cascade units consisting of 1×1 convolution and 3×3 convolution in the feature pyramid networks FPN-Neck2 and FPN-Neck1 of the two-dimensional object detection network model YOLO respectively connected to layer 3 and layer 4 of the backbone network into two cascade units consisting of 1×1 convolution and 3×3 convolution and one cascade unit consisting of 1×1 convolution and 3×3 convolution; Step S30, connecting the output of backbone network layer 3 with the output of two cascade units consisting of 1×1 convolution with 128 channels and 3×3 convolution with 256 channels in feature pyramid network FPN-Neck2 through residual connection, and then using them as the input of backbone network layer 4; Step S40, connecting the output of backbone network layer 4 with the output of two cascade units consisting of 1×1 convolution with 256 channels and 3×3 convolution with 512 channels in feature pyramid network FPN-Neck1 through residual connection, and then using them as the input of backbone network layer 5; Step S50, the output of the backbone network layer 5 passes through three cascade units of the feature pyramid network FPN-Neck0, which are composed of 1×1 convolution with a channel number of 512 and 3×3 convolution with a channel number of 1024, and the obtained output is input to the detection head of the feature layer corresponding to the backbone network layer 5. At the same time, the obtained output is upsampled by 2 times and channel-concatenated with the output of step S40, and then input to the last cascade unit of the feature pyramid network FPN-Neck1, which is composed of 1×1 convolution with a channel number of 256 and 3×3 convolution with a channel number of 512; Step S60, the output of the last cascade unit consisting of a 1×1 convolution with a channel number of 256 and a 3×3 convolution with a channel number of 512 in the feature pyramid network FPN-Neck1 is input to the detection head of the feature layer corresponding to the backbone network layer 4, and at the same time, the output is upsampled by 2 times, channel-concatenated with the output of step S30, and then input to the last cascade unit consisting of a 1×1 convolution with a channel number of 128 and a 3×3 convolution with a channel number of 256 in the feature pyramid network FPN-Neck2, and then input to the detection head of the feature layer corresponding to the backbone network layer 3; In step S70, the detection heads and non-maximum suppression (NMS) of each feature layer are consistent with the two-dimensional target detection network model YOLO.

2. The method for constructing an image two-dimensional object detection network based on a backbone network hierarchical recursive cascade architecture as claimed in claim 1, characterized in that: Only features with resolutions of 1 / 8, 1 / 16, and 1 / 32 of the original image, corresponding to the outputs of layers 3, 4, and 5 of the backbone network, are connected to the feature pyramid network.

3. The method for constructing an image two-dimensional object detection network based on a backbone network hierarchical recursive cascade architecture as claimed in claim 1, characterized in that: The layer 3 of the backbone network is lower than the layer 4, and the layer 4 is lower than the layer 5.

Images