Target detection method and system based on features and path search

Abstract

The application provides a target detection method and system based on features and path search, comprising: constructing a feature fusion network comprising a fusion feature search unit and a fusion path search unit; the fusion feature search unit searches for optimal features at each scale from multi-scale features of a picture; channels of the optimal features at each scale are grouped, the fusion path search unit searches for candidate features from each group, and a plurality of candidate feature pairs each comprising two candidate features are constructed; the candidate feature pairs are fused based on a preset fusion strategy, and fusion features of each candidate feature pair are obtained to determine a predicted position category of a detection target in the picture.

Description

Technical Field

[0001] This invention relates to the fields of image processing and deep learning object detection technology, and particularly to an object detection method and system based on feature and path search. Background Technology

[0002] Current deep learning object detection frameworks mainly consist of three parts: a backbone network, a feature fusion network, and a detection head. The backbone network, typically a network used in image recognition such as ResNet, VGG, and MobileNet, is used to extract features from images. The feature fusion network improves object detection accuracy by fusing the multi-scale features extracted by the backbone network. The detection head then outputs the location and size of the detection box based on the fused features.

[0003] The feature fusion network is a key part of this framework. Existing techniques often improve detection performance by designing fusion paths for multi-scale features.

[0004] Existing feature fusion networks often fuse features by adding two features together. However, features at different levels should be associated with objects of different sizes, so the commonly used addition operation as a fusion operation may lead to conflicts between features at different levels and degrade the performance of the fusion network. Furthermore, existing fusion networks fix the input features to specific layers in the backbone network, such as the last layer of each stage of the backbone network. However, this ignores the possibility that selecting features from other layers might yield better performance. Summary of the Invention

[0005] The purpose of this invention is to improve the detection accuracy of the prior art by proposing a target detection method based on feature and path search, which includes:

[0006] Step 1: Obtain training images with labeled location category tags and construct a feature fusion network including a fusion feature search unit and a fusion path search unit;

[0007] Step 2: The fusion feature search unit searches for the optimal features at each scale from the multi-scale features of the training image; the channels of the optimal features at each scale are grouped, and the fusion path search unit searches for candidate features from each group to construct multiple candidate feature pairs consisting of two candidate features; based on a preset fusion strategy, the candidate feature pairs are fused to obtain the fused features of each candidate feature pair, so as to determine the predicted location category of the detected target in the training image.

[0008] Step 3: Construct a loss function using the predicted location category and the location category label, train the feature fusion network to obtain the final model; input the multi-scale features of the image to be detected into the final model to obtain its fused features, so as to obtain the target location category of the image to be detected.

[0009] The target detection method based on feature and path search, wherein the feature fusion network further includes a fusion pattern search unit;

[0010] Step 2 includes: based on the candidate feature pair, the fusion mode search unit searches for a fusion strategy from an operation set including multiple candidate fusion operations, and performs feature fusion on the candidate feature pair using the fusion strategy to obtain the fusion feature of each candidate feature pair;

[0011] The set of operations includes spatial self-attention operations, channel self-attention operations, spatial attention operations, channel attention operations, and addition operations.

[0012] The target detection method based on feature and path search described herein extracts multi-scale features from the training image through a backbone network. Layers that output features of the same scale in the backbone network are considered as the same network stage. The fusion feature search unit searches for the optimal feature among the output features of the first layer, the output features of the intermediate layers, and the output features of the top layer in each network stage.

[0013] The aforementioned target detection method based on feature and path search, wherein the fusion path search unit divides the channels of each optimal feature into K groups, and the feature with scale s in the k (k=1,…,K) groups can be represented as F s,k The fusion path search unit uses an n-to-1 search operation to select features at different scales. t≠s, by and This constitutes the candidate feature pair.

[0014] This invention also proposes a target detection system based on feature and path search, which includes:

[0015] The initial module is used to acquire training images with labeled location category tags and construct a feature fusion network including a fusion feature search unit and a fusion path search unit;

[0016] The search module enables the fusion feature search unit to search for the optimal features at each scale from the multi-scale features of the training image; the channels of the optimal features at each scale are grouped, and the fusion path search unit searches for candidate features from each group to construct multiple candidate feature pairs consisting of two candidate features; based on a preset fusion strategy, the candidate feature pairs are fused to obtain the fused features of each candidate feature pair, so as to determine the predicted location category of the detected target in the training image;

[0017] The detection module is used to construct a loss function with the predicted location category and the location category label, train the feature fusion network, and obtain the final model; the multi-scale features of the image to be detected are input into the final model to obtain its fused features, so as to obtain the target location category of the image to be detected.

[0018] The target detection system based on feature and path search further includes a fusion pattern search unit in the feature fusion network.

[0019] The search module is used to: based on the candidate feature pair, the fusion mode search unit searches for a fusion strategy from an operation set including multiple candidate fusion operations, and performs feature fusion on the candidate feature pair using the fusion strategy to obtain the fusion feature of each candidate feature pair;

[0020] The set of operations includes spatial self-attention operations, channel self-attention operations, spatial attention operations, channel attention operations, and addition operations.

[0021] The target detection system based on feature and path search extracts multi-scale features from the training image through a backbone network. Layers that output features of the same scale in the backbone network are considered as the same network stage. The fusion feature search unit searches for the optimal feature among the output features of the first layer, the middle layer, and the top layer of each network stage.

[0022] The aforementioned target detection system based on feature and path search, wherein the fusion path search unit divides the channels of each optimal feature into K groups, and the feature with scale s in the k (k = 1, ..., K) groups can be represented as F s,k The fusion path search unit uses an n-to-1 search operation to select features at different scales. t≠s, by and This constitutes the candidate feature pair.

[0023] The present invention also proposes a storage medium for storing a program that executes any of the feature- and path-search-based target detection methods.

[0024] The present invention also proposes a client for any of the aforementioned feature- and path-search-based target detection systems.

[0025] As can be seen from the above solutions, the advantages of the present invention are:

[0026] When the backbone network is ResNet50, compared with the state-of-the-art search network NAS-FPN, the proposed method reduces the number of parameters and computation by 29.6% and 22.3%, respectively, while achieving similarity detection accuracy. Compared with the existing manually designed network FPN, the proposed method improves the average detection accuracy by 2.1%. When the backbone network is VGG, compared with the state-of-the-art search network Auto-FPN, the proposed method improves the average detection accuracy by 1.7%. Attached Figure Description

[0027] Figure 1 This is the overall structure of the integrated network search framework of the present invention;

[0028] Figure 2 This is a structural diagram of the fusion mode search unit;

[0029] Figure 3 A diagram illustrating the fusion process when the feature dimensions are divided into two groups;

[0030] Figure 4 This is a structural diagram of the fusion mode search unit;

[0031] Figure 5 Network structure for predicting time cost networks. Detailed Implementation

[0032] The inventors, in their research on feature fusion networks within deep learning detection frameworks, discovered that most existing feature fusion network designs only consider the fusion path, neglecting two crucial factors: the fusion method and the selection of fusion features. This invention focuses on designing the architecture of the fusion network from a higher perspective. To improve the ability to fuse multi-scale features, unlike existing works that only consider the fusion path, this invention considers three aspects: input fusion features, fusion path, and fusion operation.

[0033] This invention considers the above design factors while also taking into account the hardware overhead of deploying the algorithm on the target platform. Currently, object detection algorithms are widely used in various scenarios such as security monitoring, autonomous driving, medical image processing, and robotics. To ensure the algorithm can run efficiently on the target hardware, this invention needs to consider the hardware characteristics of the target platform when designing the algorithm. However, achieving a good trade-off between accuracy, computational efficiency, and parameter quantity within this vast design space is undoubtedly very difficult. Therefore, this invention uses a neural architecture search algorithm, which has significant effects in automatically and effectively discovering the optimal network in the search space. Based on the above analysis, this invention proposes a new network architecture search framework for designing hardware-aware attention-aware fusion networks. Specifically, this invention includes the following key technical points:

[0034] Key Point 1: Proposes three key factors for feature fusion network design; Technical Effect: Designs feature fusion networks from a more comprehensive perspective, improving detection accuracy;

[0035] Key point 2: A neural network search algorithm was used, and three search units were designed based on the aforementioned key factors: a fusion feature search unit, a fusion path search unit, and a fusion pattern search unit. Technical effects: The design of the search units implemented weight reuse and group fusion, reducing computational and memory overhead during the search and effectively finding good structures.

[0036] Key point 3: Take the hardware overhead of the feature fusion network on the target hardware as the search target; Technical effect: A good balance between accuracy and computational overhead can be achieved on the target hardware.

[0037] To make the above features and effects of the present invention clearer and easier to understand, specific embodiments are described below, and detailed descriptions are provided in conjunction with the accompanying drawings.

[0038] This invention uses an evolutionary algorithm-based neural architecture search method to solve the network structure design problem. The following sections first introduce the structure and search space of AFN (attention-aware fusion network), and then explain how to search for hardware-aware feature fusion networks based on the characteristics of the target hardware platform.

[0039] Attention-based feature fusion networks:

[0040] The AFN proposed in this invention aims to fuse multi-scale features and improve the classification and localization capabilities of the target detection framework. Figure 1 The overall structure of the network search framework proposed in this invention is shown. AFN takes the features of the future backbone network as input to generate fused features. AFN is scalable and can be stacked between any backbone network and detection head network.

[0041] To search for network structure, AFN consists of three types of units: fused feature search units, fused path search units, and fused pattern search units. These three types of units constitute the layers in AFN. The entire AFN structure is denoted as S. AFN Assume AFN contains T+1 layers: S AFN ={L0,L1,…,L T Each layer generates N different scales of features:

[0042] The first layer, L0, takes M candidate features from the future autonomous backbone network as input and outputs the most suitable N features for different scales. This invention searches the structure of L0 by fusing feature search units.

[0043] In addition, the next T layer L i (i = 1, ..., T) merges from the previous layer L i-1 The system generates N features while maintaining the same number of features. The final features input to the detection head are the N features generated in the last layer.

[0044] To improve the ability to fuse multi-scale features, L i The structure is determined by the fusion path search unit and the fusion pattern search unit. The fusion path search unit selects the optimal cross-scale connection, while the fusion pattern search unit determines the optimal fusion operation among multiple attention mechanisms. Through these three basic units, this invention divides the entire search space into a series of individually changeable choices. This allows for weight sharing among different network structures within the search space, thereby reducing computational and storage overhead during the search. The following section will describe these three basic units in detail.

[0045] Fusion Feature Search Unit:

[0046] The fusion feature search unit aims to select optimal multi-scale features from different layers of the backbone network. Typically, this invention assumes that layers generating feature maps with the same spatial size are in the same network stage. To provide more options, this invention selects features as candidate features from the first, middle, and top layers of each stage, not just the top layer. These three layers represent the features with the strongest spatial information, the most semantic information, the most balanced spatial information, and the most semantic information, respectively, for each stage. The features from the i-th backbone stage are represented as... Where n i Let i be the layer number of the i-th stage, and the superscript indicates the layer index.

[0047] This invention uses ResNet-50 as an example, selecting features from stages 3-5 as input; therefore, candidate features can be represented as... Assume P candidateThere are a total of M candidate features, which are the inputs to the first layer L0 of AFN. L0 produces N features. Corresponding to different scales of N. Spatial size from arrive Downsampling is performed progressively by a factor of 2. Each output feature at scale s... Using fusion feature search unit Select, this unit applies the selection operation. For candidate feature P candidate , represented as

[0048] To generate features with richer multi-scale information, the selection operation either selects a candidate feature or selects the sum of two of the features (the features will first be downsampled or upsampled to the target size).

[0049] For M candidate features, the possible number of operations is: Finally, the selected feature is always followed by a 3x3 convolution, a batch normalization, and a ReLU activation function.

[0050] Fusion Pattern Search Unit:

[0051] The fusion pattern search unit determines the operation for fusing multi-scale features. Besides the commonly used addition operation, many different attention mechanisms exhibit strong fusion capabilities in various aspects. Therefore, different types of attention mechanisms are considered together with the addition operation as candidate fusion operations, denoted as A = {A1, A2, ..., A...}. Q}, where Q is the number of candidate fusion operations.

[0052] This invention constructs the following five fusion operations. These fusion operations cover all dimensional choices for applying the attention mechanism, as well as all feature choices for generating attention weights. Details of these fusion operations are as follows: Figure 4 As shown. The feature fusion method used in the fusion pattern search unit. and Candidate fusion operations. It is a feature that has the same scale as the output feature. Features obtained by upsampling or downsampling at other scales. For the third and fourth operations, the positions of the input features can be swapped.

[0053] 1. Spatial-wise self-attention: This operation uses a 1×1 convolution to compute attention weights for two features, and applies a SoftMax function between these two weights. These attention weights have the same spatial size as the features, but they both have 1 channel. The features are then multiplied by their respective attention weights and summed.

[0054] 2. Channel-wise self-attention: This operation is similar to spatial-wise self-attention. The only difference is that the attention weights are generated through global pooling, so they have the same number of channels as the features, but their spatial size is 1.

[0055] 3. Spatial-wise attention: In this operation, a feature is multiplied by an attention map, which is calculated using another feature through a 1×1 convolution. The attention map has the same spatial size as the feature, but has only one channel. Finally, the two features are added together.

[0056] 4. Channel-wise attention: This operation is similar to spatial-wise attention. The difference is that the attention weights are generated through global pooling and have the same number of channels as the features, but the spatial size of the attention weights is 1.

[0057] 5. Add: This operation will directly add the two features together.

[0058] In layer L i Fusion operation A with output scale s s Required features and t≠s is used as input. The scale s is related to the scale of the backbone network. If the backbone network is a ResNet50 with five output scales, then s represents the 3rd, 4th, and 5th stage output scales, and the 5th stage is downsampled once (ratio=0.5) and downsampled twice.

[0059] Fusion Operation A s It involves performing an n-to-1 selection operation in set A using a neural network search. We obtained it. Therefore, layer L... i The fusion pattern search unit corresponding to the mesoscale s It can be represented as

[0060] Fusion path search unit:

[0061] There are two key points in the design of the fusion path search unit.

[0062] 1) Based on the various structures of existing feature fusion networks, this invention follows a design principle when constructing the search space: the input to the fusion operation always includes features from the previous layer that have the same scale as the output features. This principle improves fusion efficiency by preserving information from the current scale and aggregating cross-scale information from features at another scale.

[0063] 2) As described in the fusion pattern search unit, the fusion operations used in this invention are all dual-input functions. Therefore, in order to fuse more scales while reducing computational overhead, this invention divides the feature channels into several groups before fusion. For example... Figure 3 As shown, based on the above points, this invention first divides the channels of each scale feature into K groups. Then, for L... i The features of scale s in the k (k = 1, ..., K) group of layers can be represented as: Fusion Path Search Unit Using an n-choose-1 operation To select features at different scales t≠s. Then features and characteristics The features are fused by the fusion pattern search unit. To fuse output features at different scales, this invention adds a fusion convolution.

[0064] Search algorithm:

[0065] This invention employs a neural architecture search method based on evolutionary algorithms to find the optimal structure in the aforementioned search space. As described above, the three types of units constitute the entire search space. Each unit contains an N-to-1 selection operation based on neural network search, and all these operations form a path, i.e., a network structure.

[0066] Since all these paths share parameters and form a supernetwork, this invention can train the supernetwork and use it as a performance estimator for all network structures (paths) in the search space of this invention. Because the search structure is divided into multiple units, when a unit changes, not all parameters on the network are changed; only the parameters corresponding to the changed unit are altered, meaning parameters are shared across different structures. The search space becomes very large as the number of search units increases, but obviously the number of parameters that need to be stored is not large.

[0067] Inspired by GreedyNAS, this invention employs a multi-path sampling strategy and greedily filters out poorer paths to train the supernetwork. Once the supernetwork is trained, its performance is used as part of a scoring function, and an evolutionary algorithm is executed to obtain the optimal network structure. The search process will be described in detail below.

[0068] Referring to the DARTS algorithm, this invention divides the original training data into two disjoint sets: training set D.train and verification set D var In the search process of this invention, the training set D train Parameters used to train the supernetwork, verifying D val This invention is used to verify the performance of hypernetworks. For each object category, 20 images are randomly selected to form a validation set D. val The remaining images constitute the training set D. train .

[0069] The search process of this invention is divided into three stages: pre-training stage, greedy training stage, and evolutionary algorithm stage.

[0070] During the warm-up phase, all operations in the search unit are uniformly sampled (randomly selected) and used with the training set D. train Training. The pre-training phase is crucial to the entire search process. At the start of the search, the weights of all operations are not fully trained. If greedy training begins at this point, operations that converge faster rather than perform better will have an advantage in greedy training, causing the search to deviate from the optimal value. After the pre-training phase, all operations should be fully trained.

[0071] During the greedy training phase, this invention employs the GreedyNAS method. During training, this invention maintains a buffer of size [size missing]. candidate pool And the initial candidate pool The paths in the search space are randomly selected. In each training iteration, this invention samples m paths from the candidate pool with probability ∈ , and samples from the entire search space with probability 1-∈ . These m paths will be selected in D val The test will be conducted on the target function F. greedy Sort the data. Then use the top k paths to update the candidate pool. When the candidate pool is full, F will be discarded. greedy The path with the lowest value.

[0072] After the greedy training phase, this invention obtains a supernetwork as a performance estimator for different network structures. Then, this invention executes an evolutionary algorithm using the top n paths from the candidate pool as the initial population. This invention uses the objective function F... evolution The network structure is evaluated. In these evaluated architectures, the top-n networks are selected as parents to generate sub-networks. Each sub-network is generated by mutation and crossover, with half being the parent network. This process is repeated iteratively, and the neural architecture search result is the network structure with the highest objective function score in the last population.

[0073] For all experiments, this invention will adjust the size of the candidate pool. The parameters are set to 500, the number of sample paths m is 10, and the number of selected paths k is 5. During the greedy training process, this invention linearly increases the pooling probability from 0 to 0.8. The pseudocode for the warm-up and greedy training phases is shown in Algorithm 1 below. The pseudocode for obtaining the network structure using an evolutionary algorithm is shown in Algorithm 2 below.

[0074] Hardware-aware search targets

[0075] To achieve a good trade-off between hardware overhead and detection accuracy in the searched network structure, this invention incorporates hardware overhead into its search objective. During the search process, this invention uses an objective function to represent the search objective and rank the network structures. Therefore, the objective function designed in this invention considers both accuracy and the storage and computational overhead of the network on the target hardware. The objective function during greedy training of the supernetwork is:

[0076] F greedy =-L val -α t *T(Arch)

[0077] -α M *M(Arch)

[0078] Where L val For the network in D val The average loss function is given by T(Arch), where T(Arch) is the computation time cost of the network on the target hardware, and M(Arch) is the storage cost of the network parameters.

[0079] The objective function when executing the evolutionary algorithm is:

[0080] F evolution =mAP val -α t *T(Arch)

[0081] -α M *M(Arch)

[0082] Where mAP val For functions in the network in D val The average detection accuracy.

[0083] Time cost T(Arch) prediction:

[0084] In the above objective function, the average loss function L val Average detection accuracy mAP valThe parameter storage overhead can be obtained during the search process, but the computational time overhead T(Arch) is difficult to obtain directly. Although this invention can directly obtain the computational cost corresponding to the network structure, computational cost and computation time are often not linearly related on GPUs or neural processing chips. Therefore, this invention trains a predictive neural network P to estimate the network's computational time overhead T(Arch). The network structure of neural network P is as follows: Figure 5 As shown.

[0085] This invention randomly samples N network structures from the search space before the search, runs each of these network structures M times on the target hardware platform, records the running time, and records the average of the M runs as training data. Then, the collected data is used to train the neural network P. The loss function is:

[0086]

[0087] During the search process, this invention can directly use the output of the trained and converged network P to estimate the computation time cost T(Arch).

[0088] Algorithm 1. Hypernetwork Training

[0089]

[0090] Algorithm 2. Evolutionary Algorithm

[0091]

[0092] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0093] This invention also proposes a target detection system based on feature and path search, which includes:

[0094] The initial module is used to acquire training images with labeled location category tags and construct a feature fusion network including a fusion feature search unit and a fusion path search unit;

[0095] The search module enables the fusion feature search unit to search for the optimal features at each scale from the multi-scale features of the training image; the channels of the optimal features at each scale are grouped, and the fusion path search unit searches for candidate features from each group to construct multiple candidate feature pairs consisting of two candidate features; based on a preset fusion strategy, the candidate feature pairs are fused to obtain the fused features of each candidate feature pair, so as to determine the predicted location category of the detected target in the training image;

[0096] The detection module is used to construct a loss function with the predicted location category and the location category label, train the feature fusion network, and obtain the final model; the multi-scale features of the image to be detected are input into the final model to obtain its fused features, so as to obtain the target location category of the image to be detected.

[0097] The target detection system based on feature and path search further includes a fusion pattern search unit in the feature fusion network.

[0098] The search module is used to: based on the candidate feature pair, the fusion mode search unit searches for a fusion strategy from an operation set including multiple candidate fusion operations, and performs feature fusion on the candidate feature pair using the fusion strategy to obtain the fusion feature of each candidate feature pair;

[0099] The set of operations includes spatial self-attention operations, channel self-attention operations, spatial attention operations, channel attention operations, and addition operations.

[0100] The target detection system based on feature and path search extracts multi-scale features from the training image through a backbone network. Layers that output features of the same scale in the backbone network are considered as the same network stage. The fusion feature search unit searches for the optimal feature among the output features of the first layer, the middle layer, and the top layer of each network stage.

[0101] The aforementioned target detection system based on feature and path search, wherein the fusion path search unit divides the channels of each optimal feature into K groups, and the feature with scale s in the k (k = 1, ..., K) groups can be represented as F s,k The fusion path search unit uses an n-to-1 search operation to select features at different scales. t≠s, by and This constitutes the candidate feature pair.

[0102] The present invention also proposes a storage medium for storing a program that executes any of the feature- and path-search-based target detection methods.

[0103] The present invention also proposes a client for any of the aforementioned feature- and path-search-based target detection systems.

[0104] This invention also proposes a hardware-aware, high-efficiency feature fusion network search method, characterized by comprising:

[0105] The initial module is used to acquire training images with labeled location category tags and construct a feature fusion network including a fusion feature search unit, a fusion path search unit, and a fusion pattern search unit.

[0106] The fusion module enables the fusion feature search unit to search for the optimal features at each scale from the multi-scale features of the training image; group the channels of the optimal features at each scale, and the fusion path search unit searches for candidate features from each group to construct multiple candidate feature pairs; based on the candidate feature pairs, the fusion pattern search unit searches for a fusion strategy from an operation set including multiple candidate fusion operations, and performs feature fusion on the candidate feature pairs using the fusion strategy to obtain the fusion feature of each candidate feature pair; based on the fusion feature, the predicted location category of the detected target in the training image is determined;

[0107] The training module is used to construct a loss function based on the predicted location category and the location category label; the search operations of the fusion feature search unit, the fusion path search unit, and the fusion pattern search unit are used as paths, all paths share the same parameters and form a super network, and the network structure parameters of the super network are determined based on the objective function constructed based on the hardware overhead of the path execution and the loss function, so as to obtain the final model;

[0108] The detection module is used to input the multi-scale features of the image to be detected into the final model to obtain its fused features, so as to obtain the target location category of the image to be detected.

[0109] The hardware-aware, high-efficiency feature fusion network search method is characterized in that the training module includes:

[0110] In the pre-trained module, all search operations in the fusion feature search unit, the fusion path search unit, and the fusion pattern search unit are uniformly sampled and used with the training set D. train Train the feature fusion network;

[0111] Greedy training module, retaining a size of Path candidate pool In each training iteration, m paths are sampled from the candidate pool with probability ∈ , and samples are also sampled from the search space consisting of multiple paths with probability 1-∈ ; these m paths will be used in the test set D. val The test was conducted on the above, and the test results were calculated according to the objective function F.greedy Sort the candidate pool and use the top k paths to update the candidate pool. When the candidate pool When it is full, China F greedy The path with the lowest value is discarded;

[0112] The evolutionary training module uses the supernetwork obtained in this greedy training step as the performance estimator; with the candidate pool... The first n paths in the algorithm are used as the initial population to perform an evolutionary algorithm, which is then applied to the objective function F. evolution Evaluate the network structure of the top n paths; in the evaluated network structure, select the top-n scorers as parents to generate sub-networks, where each sub-network is generated by mutation and crossover; using F... evolution The network with the highest score is used as the final network structure of the supernetwork.

[0113] The hardware-aware, high-efficiency feature fusion network search method is characterized by the objective function F greedy for:

[0114] F greedy =-L val -α t *T(Arch)

[0115] -α M *M(Arch)

[0116] Where L val For the network in D val The average loss function is given by T(Arch), where T(Arch) is the computation time cost of the network on the target hardware, and M(Arch) is the storage cost of the network parameters.

[0117] Objective function F evolution for:

[0118] F evolution =mAP val -α t *T(Arch)

[0119] -α M *M(Arch)

[0120] Where mAP val For functions in the network in D val The average detection accuracy.

[0121] The hardware-aware, high-efficiency feature fusion network search method is characterized by obtaining the time cost T(Arch) by inputting the network structure into the prediction model; randomly sampling multiple network structures from the search space, running them multiple times on the target hardware platform and recording the running time, taking the average of the multiple running times as training data, and training the neural network to obtain the prediction model.

[0122] This invention proposes a target detection system based on feature and fusion pattern search, which includes:

[0123] The initial module is used to acquire training images with labeled location category tags and construct a feature fusion network including a fusion feature search unit and a fusion pattern search unit;

[0124] The search module is used to enable the fusion feature search unit to search for the optimal features at each scale from the multi-scale features of the training image; select two features as candidate feature pairs from all the optimal features; the fusion pattern search unit searches for a fusion strategy from the operation set including multiple candidate fusion operations based on the candidate feature pairs; performs feature fusion on the candidate feature pairs to obtain fused features; and determines the predicted location category of the detected target in the training image based on the fused features.

[0125] The detection module is used to construct a loss function with the predicted location category and the location category label, train the feature fusion network, and obtain the final model; the multi-scale features of the image to be detected are input into the final model to obtain its fused features, so as to obtain the target location category of the image to be detected.

[0126] The target detection system based on feature and fusion pattern search further includes a fusion path search unit in the feature fusion network.

[0127] The search module is used for: the fusion path search unit to group the channels of the optimal feature at each scale; the fusion path search unit divides the channels of each optimal feature into K groups, where the feature at scale s in the k (k = 1, ..., K) groups can be represented as F. s,k The fusion path search unit uses an n-to-1 search operation to select features at different scales. t≠s, by and This constitutes the candidate feature pair.

[0128] The target detection system based on feature and fusion pattern search includes a set of operations such as spatial self-attention operation, channel self-attention operation, spatial attention operation, channel attention operation, and addition operation.

[0129] The target detection system based on feature and fusion pattern search extracts multi-scale features from the training image through a backbone network. Layers that output features of the same scale in the backbone network are considered as the same network stage. The fusion feature search unit searches for the optimal feature among the output features of the first layer, the output features of the middle layer, and the output features of the top layer in each network stage.

Claims

1. A target detection method based on feature and path search, characterized in that, include: Step 1: Obtain training images with labeled location category tags and construct a feature fusion network including a fusion feature search unit and a fusion path search unit; Step 2: The fusion feature search unit searches for the optimal features at each scale from the multi-scale features of the training image; the channels of the optimal features at each scale are grouped, and the fusion path search unit searches for candidate features from each group to construct multiple candidate feature pairs consisting of two candidate features; based on a preset fusion strategy, the candidate feature pairs are fused to obtain the fused features of each candidate feature pair, so as to determine the predicted location category of the detected target in the training image. Step 3: Construct a loss function using the predicted location category and the location category label, train the feature fusion network to obtain the final model; input the multi-scale features of the image to be detected into the final model to obtain its fused features, so as to obtain the target location category of the image to be detected; Step 2 includes: extracting multi-scale features from the training image through the backbone network, wherein layers of the backbone network that output features of the same scale are considered as the same network stage; the fusion feature search unit searches for the optimal feature among the output features of the first layer, the output features of the intermediate layers, and the output features of the top layer in each network stage; and the fusion path search unit divides the channels of each optimal feature into K groups. The feature with scale s in the group can be represented as The fusion path search unit uses an n-to-1 search operation to select features at different scales. ,Depend on and This constitutes the candidate feature pair.

2. The target detection method based on feature and path search as described in claim 1, characterized in that, This feature fusion network also includes a fusion pattern search unit; Step 2 includes: based on the candidate feature pair, the fusion mode search unit searches for a fusion strategy from an operation set including multiple candidate fusion operations, and performs feature fusion on the candidate feature pair using the fusion strategy to obtain the fusion feature of each candidate feature pair; The set of operations includes spatial self-attention operations, channel self-attention operations, spatial attention operations, channel attention operations, and addition operations.

3. A target detection system based on feature and path search, characterized in that, include: The initial module is used to acquire training images with labeled location category tags and construct a feature fusion network including a fusion feature search unit and a fusion path search unit; The search module is used to enable the fusion feature search unit to search for the optimal features at each scale from the multi-scale features of the training image. The channels of the optimal feature at each scale are grouped, and the fusion path search unit searches for candidate features from each group to construct multiple candidate feature pairs consisting of two candidate features. Based on a preset fusion strategy, the candidate feature pairs are fused to obtain the fused features of each candidate feature pair, so as to determine the predicted location category of the detected target in the training image. The detection module is used to construct a loss function with the predicted location category and the location category label, train the feature fusion network, and obtain the final model; the multi-scale features of the image to be detected are input into the final model to obtain its fused features, so as to obtain the target location category of the image to be detected; The search module includes: extracting multi-scale features from the training image through the backbone network, wherein layers that output features of the same scale from the backbone network are considered as the same network stage; the fusion feature search unit searches for the optimal feature among the output features of the first layer, the output features of the intermediate layers, and the output features of the top layer in each network stage; and the fusion path search unit divides the channels of each optimal feature into K groups. The feature with scale s in the group can be represented as The fusion path search unit uses an n-to-1 search operation to select features at different scales. ,Depend on and This constitutes the candidate feature pair.

4. The target detection system based on feature and path search as described in claim 3, characterized in that, This feature fusion network also includes a fusion pattern search unit; The search module is used to: based on the candidate feature pair, the fusion mode search unit searches for a fusion strategy from an operation set including multiple candidate fusion operations, and performs feature fusion on the candidate feature pair using the fusion strategy to obtain the fusion feature of each candidate feature pair; The set of operations includes spatial self-attention operations, channel self-attention operations, spatial attention operations, channel attention operations, and addition operations.

5. A storage medium for storing a program that performs the feature- and path-search-based target detection method as described in claim 1 or 2.

6. A client for the feature- and path-search-based target detection system of claim 3 or 4.

Images