Multi-scale YOLOv8 neck network architecture searching method and system

Abstract

The invention discloses a multi-scale YOLOv8 neck network architecture searching method and system, and belongs to the technical field of computer vision and deep neural networks. The method comprises the steps that common convolution used for feature extraction in a YOLOv8 neck network is replaced with multi-scale convolution, a channel attention mechanism is added behind a feature splicing module, and the quality of multi-scale feature extraction and fusion is preliminarily improved through dynamic channel weighting; designing a search space for the neck network of the YOLOv8 and constructing a full-connection super network; training the hypernet by using a differentiable architecture search method until a target function is converged, and obtaining an optimal neck network architecture; and carrying out retraining on the optimal YOLOv8 neck network architecture obtained by searching so as to verify the high efficiency of the optimal YOLOv8 neck network architecture. According to the method, the detection precision of the model is remarkably improved under the condition that the parameter quantity is slightly increased.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and deep neural networks, and in particular to a multi-scale YOLOv8 neck network architecture search method and system. Background Art

[0002] Object detection, a core task in computer vision, relies heavily on the effectiveness of feature extraction and fusion. The neck network, a key component connecting the backbone network and the detection head, performs the crucial functions of multi-scale feature fusion and enhancement. In recent years, with the advancement of deep learning technology, neck network architectures have evolved from simple concatenation to complex interactions. The neck network of the YOLOv8 model combines the features of a feature pyramid network and a path aggregation network to improve the model's detection capabilities for objects of varying scales. The feature pyramid network is a top-down feature pyramid structure that upsamples strong semantic features from higher layers to lower layers. It then laterally connects these features with feature maps of the same size in the backbone network for feature fusion, thereby enhancing the semantic information of the entire pyramid network. Unlike the feature pyramid network, the path aggregation network is a bottom-up feature pyramid structure that downsamples localization features from lower layers to higher layers. It then laterally connects these features with feature maps of the same size in the feature pyramid network for feature fusion, thereby enhancing the localization information of the entire pyramid network. However, both types of pyramid networks only perform lateral connections between feature maps of the same size, lacking sufficient exploration of cross-scale feature interactions. Secondly, the fixed manually designed patterns in the current architecture are difficult to adapt to the needs of different scenarios. As the complexity of detection tasks increases, the manually designed neck network often finds it difficult to achieve an ideal balance between computational efficiency and detection accuracy.

[0003] To address these challenges, a growing number of research efforts have recently introduced neural network architecture search methods to automate the design of neck structures in object detection networks. Compared to traditional manual design methods, neural network architecture search has the ability to automatically discover optimal structures within a vast search space, effectively improving the network's adaptability to diverse scenarios and tasks while achieving a better balance between computational efficiency and detection accuracy. However, currently mainstream neural network architecture search methods, mostly based on reinforcement learning or evolutionary algorithms, still suffer from certain limitations in practical applications. For one thing, reinforcement learning-based methods typically require a large number of training samples and repeated architecture evaluation, resulting in lengthy search times and significant computational resource consumption, making them difficult to meet the efficiency requirements of practical deployments. Furthermore, while evolutionary algorithms offer some global exploration capabilities during the search process, they converge relatively slowly, are prone to getting stuck in local optima, and similarly suffer from high computational costs. Furthermore, most of these methods employ discrete search strategies, typically requiring independent training for each candidate architecture, further increasing the resource burden of the search process. Summary of the Invention

[0004] Purpose of the invention: In response to the above problems, the purpose of the present invention is to provide a multi-scale YOLOv8 neck network architecture search method and system, which significantly improves the model detection accuracy while slightly increasing the number of parameters.

[0005] Technical solution: One aspect of the present invention provides a multi-scale YOLOv8 neck network architecture search method, comprising the following steps: Step 1: Improve the YOLOv8 neck network to obtain a new neck network; Step 2: construct a search space in the new neck network to form a fully connected supernet; Step 3: Construct the objective function and use the differentiable architecture search method to train the supernet until the objective function converges. The YOLOv8 neck network architecture obtained after training is used as the optimal neck network architecture, and a new YOLOv8 model is formed using the optimal neck network architecture. Step 4: In the image target recognition scenario, the image to be recognized is input into the new YOLOv8 model, image features are extracted, and the corresponding category labels are output.

[0006] Furthermore, step 1 includes: Step 101: Replace the convolution used for feature extraction in the YOLOv8 neck network with multi-scale convolution, which can be expressed as follows: , in, represents the output item of the multi-scale convolution module, Represents different convolution kernel sizes, The convolution modules with different convolution kernel sizes are expressed as follows: , in, represents the activation function, represents the batch normalization operation, Indicates that the convolution kernel size is Convolution operation; Step 102: Add a channel attention mechanism after each feature concatenation module of the YOLOv8 neck network, which can be expressed as: , in, and Represent the input features and output features of the channel attention mechanism respectively, represents global average pooling, represents a one-dimensional convolution operation, express activation function, Represents element-wise multiplication with the original input features.

[0007] Furthermore, step 2 includes: The feature map of each layer in the new neck network in YOLOv8 is regarded as a node, and the connection between each node is regarded as an edge. The candidate operation set contained on the edge for{ Ordinary convolution, Ordinary convolution, Depthwise Separable Convolution, Dilated convolution}, the search space of the supernet is composed of nodes, edges and candidate operations; During the search process, the candidate operations on each edge are Weighted combination, that is, from the node To Node Each edge Introducing architecture parameters , continuously relax the search space and transform the discrete operation selection into the weighted sum of all candidate operations, which can be expressed as: , in, Representation node To Node The mixing operation on this edge, that is The weighted combination, represents the number of candidate operations, Indicates the candidate operations, Indicates the The architectural parameters of candidate operations, expressed as fractions Normalization operation; Then, for each intermediate node, all its predecessor nodes are passed through The output after weighted combination is then weighted accumulated, which can be expressed as follows: , in, Indicates the The output items of the predecessor nodes.

[0008] Furthermore, step 3 includes: Step 301: Jointly learn architecture parameters through a two-layer optimization method and network weights , the constructed objective function is divided into two levels, one is the inner layer optimization, that is, the fixed architecture parameters , update the network weights with training data , then the objective function is expressed as: , in, is a trainable parameter; Represents the loss on the training set, used to optimize the model weights , Represents the optimal network weight under the current architecture; The other is outer layer optimization, which is to fix the network weights , update the schema parameters with validation data , then the objective function is expressed as: , in, Represents the loss on the validation set, used to optimize the architecture parameters ; During the training process, the two-layer optimization steps are performed alternately until the training is completed; Step 302: Optimize the architecture parameters learned through training. , the architectural parameters of all candidate operations on the edges between nodes Sorting is expressed as follows: , in, Represents an edge The architectural parameter with the largest value among all candidate operations above, Represents a slave node To Node The first The architectural parameters of the candidate operations; Step 303: for all predecessor nodes, Sort the values ​​and keep the two with the highest importance, which can be expressed as: , in, Indicates the current node, Representation node All predecessor nodes, that is, all pointing nodes Node A collection of Indicates return The two predecessor nodes with the largest values ; Step 304: retain the two predecessor nodes , select each edge The specific operation above is expressed as follows: , in, Represents an edge The candidate operation with the largest architectural parameter; Step 305: construct an optimal YOLOv8 neck network architecture based on the retained edges and the convolution operations on the edges.

[0009] Furthermore, during the training of the supernet, a loss function is constructed, which is expressed as: , Where, , , Represent the weight coefficients of the three loss functions respectively, represents the bounding box regression loss, which is used to accurately adjust the position and size of the prediction box. Represents classification loss, which is used to optimize the category prediction probability, Represents the confidence loss, which is used to determine whether there is a target in the grid. The calculation formulas are expressed as follows: , , , in, represents the intersection-over-union ratio of the predicted box and the true box, Represents the Euclidean distance between the center point of the predicted box and the center point of the real box, Indicates the diagonal length of the minimum circumscribed rectangle of the predicted box and the true box, Indicates the consistency of the aspect ratio between the predicted box and the real box. Represents the weight coefficient, used to balance the aspect ratio loss term The impact of represents the total number of categories in the dataset, Indicates the true label 1 or 0, 1 means the target belongs to class, 0 means the target does not belong to kind, Indicates that the target belongs to The probability of the class, Indicates the number of grids into which the input image is divided, Indicates the true label 1 or 0, 1 means the current grid has a target, 0 means the current grid has no target, Indicates the confidence level of the target in the current grid.

[0010] Furthermore, the connection mode between each node includes vertical connection, horizontal connection or diagonal connection.

[0011] Another aspect of the present invention provides a multi-scale YOLOv8 neck network architecture search system, comprising: Multi-scale convolution and channel attention modules are used to initially improve the feature extraction and fusion quality between multi-scale feature maps in the YOLOv8 neck network; The construction module is used to construct a set of candidate operations and build a fully connected supernet for the YOLOv8 neck network architecture after adding the channel attention module; The training module is used to train the YOLOv8 neck network architecture supernet until the objective function reaches convergence; The retraining module is used to retrain the optimal YOLOv8 neck network architecture found to verify its efficiency; The image recognition module inputs the image to be recognized into the image recognition module, extracts image features and outputs the corresponding category label.

[0012] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: 1. This paper replaces the ordinary convolution used for feature extraction in the YOLOv8 neck network with multi-scale convolution to preliminarily improve the multi-scale feature extraction quality of the model, and adds a channel attention mechanism after each layer of feature splicing module, so that the model can dynamically adjust the weights of different channels and enhance the ability of multi-scale feature fusion; 2. In addition, traditional manually designed network structures usually rely on expert experience and find it difficult to achieve the optimal balance between efficiency and accuracy. However, by constructing a fully connected supernet, this invention can establish a more flexible and comprehensive feature fusion path between nodes, allowing feature information at different scales and semantic levels to fully flow and interact. 3. In this invention, based on the differentiable architecture search method, the gradient optimization mechanism is further utilized to transform the discrete architecture selection problem into an optimization problem in a continuous space, and the optimal sub-architecture is efficiently mined in a large-scale architecture candidate space. Compared with traditional architecture search methods based on reinforcement learning or evolutionary algorithms, differentiable architecture search has higher search efficiency and lower computational cost. Applying it to the neck network architecture design of YOLOv8 can not only improve the degree of automation of network architecture design, but also provide new ideas and means for the construction of high-performance target detection models. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is the overall framework diagram of the present invention; Figure 2 This is a structural diagram of the multi-scale convolution in the present invention; Figure 3 This is the structure diagram of the channel attention mechanism in the present invention; Figure 4 This is a diagram of the neck network supernet architecture in the present invention; Figure 5 This is the optimal neck network architecture diagram found in the present invention; Figure 6 This is a comparison chart of accuracy; Figure 7 The following is a comparison chart of the detection effects. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solutions and advantages of this application more clear, this application is further described in detail below with reference to the accompanying drawings and embodiments.

[0015] Example 1 In a specific embodiment of the present invention, a multi-scale YOLOv8 neck network architecture search method is proposed. Figure 1 This is a framework diagram of the entire method, starting from the initial optimization of the neck network architecture, to the construction of a supernet to search for the optimal neck network architecture and retraining to verify its efficiency. It will detail the implementation process and details of the multi-scale YOLOv8 neck network architecture search method proposed in this invention. The method of the present invention specifically includes the following steps: Step 1: Improve the YOLOv8 neck network to obtain a new neck network.

[0016] Furthermore, step 1 includes: Step 101, replace the convolution for feature extraction in the YOLOv8 neck network with Figure 2 The multi-scale convolution shown is expressed as: , in, represents the output item of the multi-scale convolution module, Represents different convolution kernel sizes, The convolution modules with different convolution kernel sizes are expressed as follows: , in, represents the activation function, represents the batch normalization operation, Indicates that the convolution kernel size is Convolution operation; Although ordinary convolution performs well in extracting local features, its receptive field is limited by the fixed size of the convolution kernel. It can often only focus on information at a certain scale in the image, making it difficult to capture the details of the target and the global context at the same time. This leads to insufficient feature expression when facing targets with large scale variations or complex background interference. In addition, ordinary convolution usually requires stacking multiple layers of convolution to expand the receptive field when processing multi-scale targets, which not only increases the computational overhead of the model, but may also introduce redundant information, affecting detection efficiency and accuracy. In contrast, multi-scale convolution can fuse features from different receptive fields by introducing multiple convolution kernels of different sizes in parallel in the same layer, which not only retains fine-grained information but also enhances the ability to understand large-scale structures, thereby effectively improving the model's recognition ability and overall robustness for multi-scale targets. As Figure 2 As shown, in this example, convolution, Convolution and Convolution, as a component of multi-scale convolution, not only enhances the multi-scale feature extraction capability of the model, but also does not increase the number of model parameters too much. In each convolution block, a convolution layer, a batch normalization layer, and an activation function layer are set. The three channels are then fused to obtain the final output item of the feature splicing module.

[0017] Step 102: Add a channel attention mechanism after each feature concatenation module of the YOLOv8 neck network, which can be expressed as: , in, and Represent the input features and output features of the channel attention mechanism respectively, represents global average pooling, represents a one-dimensional convolution operation, express activation function, Represents element-wise multiplication with the original input features.

[0018] The feature concatenation module in the YOLOv8 neck network is used to concatenate feature information extracted from different layers. For example, a high-level feature containing coarse-grained semantic information is combined with a low-level feature containing detailed information. The result is a feature map with expanded channels containing multi-scale and multi-semantic information. These channels originate from different layers and have greater information redundancy. The channel-by-channel attention mechanism introduced in this invention can adaptively assign weights to channels, emphasizing useful channels and suppressing useless ones.

[0019] like Figure 3 As shown in the figure, first, global average pooling is performed on the input feature map to obtain the global feature representation of each channel. Then, the vector obtained by global average pooling is input into a one-dimensional convolution layer to capture the local interaction information between channels. Unlike the fully connected operation of the traditional channel attention mechanism, the channel attention mechanism in this invention uses a one-dimensional convolution, and its convolution kernel size is adaptively determined according to the number of channels of the one-dimensional vector, thereby balancing computational efficiency and capture range. The calculation formula is expressed as: , in, represents the convolution kernel size of one-dimensional convolution, represents the number of channels of a one-dimensional vector, Indicates taking The nearest odd number, and It is a hyperparameter used to control the size of the convolution kernel; Next, use The activation function generates channel weights to represent the importance of each channel. Finally, the generated channel weights are multiplied element-wise with the original input feature map to adjust the channel weights and enhance the network's attention to important channels. Figure 3 Different color blocks in the output feature map represent the importance of different channel layers in the output channel.

[0020] In this example, the YOLOv8 model used includes the backbone network, the neck network, and the detection head. The backbone network and the detection head are not improved. In this example, only the neck network is improved. Figure 4 As shown in the figure, the improved neck network includes a fusion module, each fusion module includes a multi-scale feature extraction module and a feature splicing module, and a channel attention module is set after each feature splicing module. In the channel attention module, weights are adaptively assigned to channels through the channel attention mechanism, useful channels are strengthened, and useless channels are suppressed.

[0021] Step 2: Construct a search space in the new neck network to form a fully connected supernet.

[0022] In order to overcome the problem of non-differentiable and high computational cost in the discrete structure search process, differentiable architecture search is performed by performing Weighted combination transforms discrete operation selection into weight optimization in continuous space. (From the node To Node ), introduce architectural parameters , continuously relax the search space so that the architecture parameters It can learn directly through gradient descent, which significantly improves search efficiency and supports end-to-end training, while also retaining the diversity and scalability of structural choices.

[0023] Furthermore, step 2 includes: The feature map of each layer in the new neck network in YOLOv8 is regarded as a node, and the connection between each node is regarded as an edge. The candidate operation set contained on the edge for{ Ordinary convolution, Ordinary convolution, Depthwise Separable Convolution, Dilated convolution}, the search space consists of nodes, edges and candidate operations; For each candidate operation on the edge Weighted combination, that is, from the node To Node Each edge Introducing architecture parameters , Use a small random number at the beginning, continuously relax the search space, and transform the discrete operation selection into the weighted sum of all candidate operations, which can be expressed as: , in, Representation node To Node The mixing operation on this edge, that is The weighted combination, represents the number of candidate operations, Indicates the candidate operations, Indicates the The architectural parameters of candidate operations, expressed as fractions Normalization operation; Then, for each intermediate node, all its predecessor nodes are passed through The output after weighted combination is then weighted accumulated, which can be expressed as follows: , in, Indicates the The output items of the predecessor nodes.

[0024] Furthermore, the connection mode between each node includes vertical connection, horizontal connection or diagonal connection.

[0025] After adding multi-scale convolution and channel attention mechanism, a super network is constructed for the neck network, so that each sub-architecture contains multi-scale convolution and channel attention mechanism during training, which helps to improve the consistency of the model's feature expression ability and design a search space for the neck network of YOLOv8. Figure 4 As shown in Figure 1, the green fusion module in the neck network includes a multi-scale feature extraction module, a feature splicing module, and the channel attention module added in step 1. The present invention searches for the connections between fusion modules in the YOLOv8 neck network and the convolution operations on the connections, so each fusion module is regarded as a node, and the vertical connection, horizontal connection or diagonal connection between each node is regarded as an edge, that is, Figure 4 The red arrow in . The set of candidate operations contained on the edge for{ Ordinary convolution, Ordinary convolution, Depthwise Separable Convolution, Dilated convolution}. Among them, Ordinary convolution is used to promote information exchange between channels and enhance the nonlinear representation ability of the model. Depthwise separable convolution reduces computational complexity while still effectively extracting information from feature maps. Dilated convolution increases the receptive field of the convolution kernel without increasing the number of parameters, thereby capturing a wider range of contextual information.

[0026] Three different scales obtained from the backbone network ( , , ) feature map as the three input nodes of the neck network supernet, and the three fusion modules in the middle top-down feature pyramid structure as intermediate nodes. Each intermediate node passes all its predecessor nodes through The output of the weighted combination is then weighted accumulated. Then, the output of the three fusion modules in the middle is used as the input of the bottom-up feature pyramid structure on the right. The three fusion modules on the right are used as new intermediate nodes. Each intermediate node combines all its predecessor nodes through The weighted combined output is then weighted accumulated. The two sub-supernets are combined to form the overall YOLOv8 neck network supernet architecture. Finally, the neck network obtains 3 different scales ( , , ) feature maps are input into the corresponding detection heads respectively.

[0027] Step 3: Construct the objective function and use the differentiable architecture search method to train the supernet until the objective function converges. The YOLOv8 neck network architecture obtained after training is used as the optimal neck network architecture, and a new YOLOv8 model is formed using the optimal neck network architecture.

[0028] Using the differentiable architecture search method as the search strategy, the improved YOLOv8 neck network supernet is trained to obtain the optimal YOLOv8 neck network architecture. Unlike the high computational cost of traditional neural network architecture search, the differentiable architecture search jointly learns the architecture parameters through a two-layer optimization method. and network weights , using continuous relaxation to transform discrete architecture search into a differentiable problem, so that the model can optimize performance on the training set while guiding the architecture search direction through the validation set, ultimately achieving efficient and automated neural network structure design.

[0029] Furthermore, step 3 includes: Step 301: Jointly learn architecture parameters through a two-layer optimization method and network weights , the constructed objective function is divided into two levels, one is the inner layer optimization, that is, the fixed architecture parameters , update the network weights with training data , then the objective function is expressed as: , in, Represents the convolution kernel weight, which is a trainable parameter; Represents the loss on the training set, used to optimize the model weights , Represents the optimal network weight under the current architecture; The other is outer layer optimization, which is to fix the network weights , update the schema parameters with validation data , then the objective function is expressed as: , in, Represents the loss on the validation set, used to optimize the architecture parameters ; During the training process, the two-layer optimization steps are performed alternately until the training is completed; In one example, an experiment was conducted on an RTX 4090 GPU. The dataset used was the MS-COCO 2017 standard dataset, which contains 80 different object categories, 118,287 training set images, 5,000 validation set images, and 40,670 test set images. During the training process, the training set was divided into two parts in a 1:1 ratio, used to train the architecture parameters and network weights respectively. At the same time, the training cycle was set to 50 rounds, and the architecture parameters and network weights were alternately optimized using the SGD optimizer until the end of training. According to the architecture parameters learned during training, , determine the optimal YOLOv8 neck network architecture. After the search phase, in order to build the final discrete network structure, each intermediate node will retain two connections with the highest operation weights from all predecessor nodes; at the same time, on each retained edge, only the single operation with the largest weight is selected, and the rest of the candidate operations are discarded. In order to compare the importance of different predecessor nodes, the architectural parameters of all candidate operations between nodes need to be Sort by.

[0030] Step 302: Optimize the architecture parameters learned through training. , the architectural parameters of all candidate operations on the edges between nodes Sorting is expressed as follows: , in, Represents an edge The architectural parameter with the largest value among all candidate operations above, Represents a slave node To Node The first The architectural parameters of the candidate operations; Step 303: for all predecessor nodes, Sort the values ​​and keep the two with the highest importance, which can be expressed as: , in, Indicates the current node, Representation node All predecessor nodes, that is, all pointing nodes Node A collection of Indicates return The two predecessor nodes with the largest values ; Step 304: retain the two predecessor nodes , select each edge The specific operation above is expressed as follows: , in, Represents an edge The candidate operation with the largest architectural parameter; Step 305 constructs the optimal YOLOv8 neck network architecture based on the retained edges and the convolution operations on the edges.

[0031] For example, the first intermediate node has 3 predecessor nodes, according to the candidate operation set for{ Ordinary convolution, Ordinary convolution, Depthwise Separable Convolution, The order of the extended convolution}, the architecture parameter values ​​on the three edges are expressed as follows: the first edge , the second edge , the third side The maximum architectural parameter values ​​on each edge are , , , so the two predecessor nodes retained by the intermediate node are node 1 and node 2, that is, the two retained edges are and , the optimal operations retained on each edge are Dilated convolution and Depthwise Separable Convolution.

[0032] like Figure 5 As shown, Figure 5 (a) is the initial YOLOv8 neck network architecture, where only horizontal connections of the same scale are retained between the input feature map and the intermediate feature map. Figure 5 (b) is the optimal neck network architecture obtained by searching, and the specific edges retained are: [[ , ], [ , ], [ , ], [ , ], [ , ], [ , ]]. The specific network architecture is: [[ Dilated convolution, Depthwise Separable Convolution], [ Depthwise Separable Convolution, Depthwise Separable Convolution], [ Depthwise Separable Convolution, Depthwise Separable Convolution], [ Dilated convolution, Depthwise Separable Convolution], [ Dilated convolution, Depthwise Separable Convolution], [ Dilated convolution, Depthwise Separable Convolution].

[0033] Furthermore, during the training of the supernet, a loss function is constructed, which is expressed as: , Where, , , Represent the weight coefficients of the three loss functions respectively, represents the bounding box regression loss, which is used to accurately adjust the position and size of the prediction box. Represents classification loss, which is used to optimize the category prediction probability, Represents the confidence loss, which is used to determine whether there is a target in the grid. The calculation formulas are expressed as follows: , , , in, represents the intersection-over-union ratio of the predicted box and the true box, Represents the Euclidean distance between the center point of the predicted box and the center point of the real box, Indicates the diagonal length of the minimum circumscribed rectangle of the predicted box and the true box, Indicates the consistency of the aspect ratio between the predicted box and the real box. Represents the weight coefficient, used to balance the aspect ratio loss term The impact of represents the total number of categories in the dataset, Indicates the true label 1 or 0, 1 means the target belongs to class, 0 means the target does not belong to kind, Indicates that the target belongs to The probability of the class, Indicates the number of grids into which the input image is divided, Indicates the true label 1 or 0, 1 means the current grid has a target, 0 means the current grid has no target, Indicates the confidence level of the target in the current grid.

[0034] The resulting neck network architecture was used to replace the original YOLOv8 neck network architecture, resulting in an improved YOLOv8 model. The training cycle was set to 300 epochs, and the model was retrained on the MS-COCO 2017 dataset using the SGD optimizer and the hyperparameters provided by YOLOv8.

[0035] Step 4: In the image target recognition scenario, the image to be recognized is input into the new YOLOv8 model, image features are extracted, and the corresponding category labels are output.

[0036] like Figure 6 As shown in the figure, the YOLOv8 model finally searched achieved a mAP@0.5:0.95 accuracy of 46.5% on the MS-COCO dataset, which is 1.8% higher than the mAP@0.5:0.95 accuracy of 44.7% of the original YOLOv8 model, verifying the efficiency of the method of the present invention.

[0037] Figure 7 The comparison of the detection effect of the improved YOLOv8 model and the original YOLOv8 model on the MS-COCO 2017 test set is shown. Figure 7 (a) shows several test images from the MS-COCO 2017 test set, including a variety of detection targets such as cars, trucks, trains, benches, apples, bananas, people, frisbees, handbags, laptops, keyboards, chairs, books, and cats, as well as the detection probabilities of the corresponding detection targets. Figure 7 (b) is the detection result of the original YOLOv8 model; Figure 7 Figure (c) shows the detection results of the improved YOLOv8 model. It can be seen that the improved YOLOv8 model can detect more objects, including some that the original YOLOv8 model cannot detect, such as a bench, a smaller car, a handbag, a book, a keyboard, and so on. The only drawback is that it mistakenly identified a frisbee as a bowl. However, the improved YOLOv8 model generally outperforms the original YOLOv8 model, further validating the efficiency of the proposed method.

[0038] Example 2 The multi-scale YOLOv8 neck network architecture search system described in this embodiment includes: Multi-scale convolution and channel attention modules are used to initially improve the feature extraction and fusion quality between multi-scale feature maps in the YOLOv8 neck network; The construction module is used to construct a set of candidate operations and build a fully connected supernet for the YOLOv8 neck network architecture after adding the channel attention module; The training module is used to train the YOLOv8 neck network architecture supernet until the objective function reaches convergence; The retraining module is used to retrain the optimal YOLOv8 neck network architecture found to verify its efficiency; The image recognition module inputs the image to be recognized into the image recognition module, extracts image features and outputs the corresponding category label.

Claims

1. A multi-scale YOLOv8 neck network architecture search method, characterized in that: The following steps are involved: Step 1: Improve the YOLOv8 neck network to obtain a new neck network; Step 2: construct a search space in the new neck network to form a fully connected supernet; Step 3: Construct the objective function and use the differentiable architecture search method to train the supernet until the objective function converges. The YOLOv8 neck network architecture obtained after training is used as the optimal neck network architecture, and a new YOLOv8 model is formed using the optimal neck network architecture. Step 4: In the image target recognition scenario, the image to be recognized is input into the new YOLOv8 model, image features are extracted, and the corresponding category labels are output.

2. A multi-scale YOLOv8 neck network architecture search method according to claim 1, characterized in that Step 1 includes: Step 101: Replace the convolution used for feature extraction in the YOLOv8 neck network with multi-scale convolution, which can be expressed as follows: ， in, represents the output item of the multi-scale convolution module, Represents different convolution kernel sizes, The convolution modules with different convolution kernel sizes are expressed as follows: ， in, represents the activation function, represents the batch normalization operation, Indicates that the convolution kernel size is Convolution operation; Step 102: Add a channel attention mechanism after each feature concatenation module of the YOLOv8 neck network, which can be expressed as: ， in, and Represent the input features and output features of the channel attention mechanism respectively, represents global average pooling, represents a one-dimensional convolution operation, express activation function, Represents element-wise multiplication with the original input features.

3. A multi-scale YOLOv8 neck network architecture search method according to claim 2, characterized in that: Step 2 includes: The feature map of each layer in the new neck network in YOLOv8 is regarded as a node, and the connection between each node is regarded as an edge. The candidate operation set contained on the edge for{ Ordinary convolution, Ordinary convolution, Depthwise Separable Convolution, Dilated convolution}, the search space of the supernet is composed of nodes, edges and candidate operations; During the search process, the candidate operations on each edge are Weighted combination, that is, from the node To Node Each edge Introducing architecture parameters , continuously relax the search space and transform the discrete operation selection into the weighted sum of all candidate operations, which can be expressed as: ， in, Representation node To Node The mixing operation on this edge, that is The weighted combination, represents the number of candidate operations, Indicates the candidate operations, Indicates the The architectural parameters of candidate operations, expressed as fractions Normalization operation; Then, for each intermediate node, all its predecessor nodes are passed through The output after weighted combination is then weighted accumulated, which can be expressed as follows: ， in, Indicates the The output items of the predecessor nodes.

4. A multi-scale YOLOv8 neck network architecture search method according to claim 3, characterized in that: Step 3 includes: Step 301: Jointly learn architecture parameters through a two-layer optimization method and network weights , the constructed objective function is divided into two levels, one is the inner layer optimization, that is, the fixed architecture parameters , update the network weights with training data , then the objective function is expressed as: ， in, is a trainable parameter; Represents the loss on the training set, used to optimize the model weights , Represents the optimal network weight under the current architecture; The other is outer layer optimization, which is to fix the network weights , update the schema parameters with validation data , then the objective function is expressed as: ， in, Represents the loss on the validation set, used to optimize the architecture parameters ; During the training process, the two-layer optimization steps are performed alternately until the training is completed; Step 302: Optimize the architecture parameters learned through training. , the architectural parameters of all candidate operations on the edges between nodes Sorting is expressed as follows: ， in, Represents an edge The architectural parameter with the largest value among all candidate operations above, Represents a slave node To Node The first The architectural parameters of the candidate operations; Step 303: for all predecessor nodes, Sort the values ​​and keep the two with the highest importance, which can be expressed as: ， in, Indicates the current node, Representation node All predecessor nodes, that is, all pointing nodes Node A collection of Indicates return The two predecessor nodes with the largest values ; Step 304: retain the two predecessor nodes , select each edge The specific operation above is expressed as follows: ， in, Represents an edge The candidate operation with the largest architectural parameter; Step 305: construct an optimal YOLOv8 neck network architecture based on the retained edges and the convolution operations on the edges.

5. A multi-scale YOLOv8 neck network architecture search method according to claim 4, characterized in that: During the training of the supernet, a loss function is constructed, which is expressed as: ， Where, , , Represent the weight coefficients of the three loss functions respectively, represents the bounding box regression loss, which is used to accurately adjust the position and size of the prediction box. Represents classification loss, which is used to optimize the category prediction probability, Represents the confidence loss, which is used to determine whether there is a target in the grid. The calculation formulas are expressed as follows: ， ， ， in, represents the intersection-over-union ratio of the predicted box and the true box, Represents the Euclidean distance between the center point of the predicted box and the center point of the real box, Indicates the diagonal length of the minimum circumscribed rectangle of the predicted box and the true box, Indicates the consistency of the aspect ratio between the predicted box and the real box. Represents the weight coefficient, used to balance the aspect ratio loss term The impact of represents the total number of categories in the dataset, Indicates the true label 1 or 0, 1 means the target belongs to class, 0 means the target does not belong to kind, Indicates that the target belongs to The probability of the class, Indicates the number of grids into which the input image is divided, Indicates the true label 1 or 0, 1 means the current grid has a target, 0 means the current grid has no target, Indicates the confidence level of the target in the current grid.

6. A multi-scale YOLOv8 neck network architecture search method according to any one of claims 3 to 5, characterized in that: The connection modes between the nodes include vertical connection, horizontal connection or diagonal connection.

7. A multi-scale YOLOv8 neck network architecture search system, characterized by: include: Multi-scale convolution and channel attention modules are used to initially improve the feature extraction and fusion quality between multi-scale feature maps in the YOLOv8 neck network; The construction module is used to construct a set of candidate operations and build a fully connected supernet for the YOLOv8 neck network architecture after adding the channel attention module; The training module is used to train the YOLOv8 neck network architecture supernet until the objective function reaches convergence; The retraining module is used to retrain the optimal YOLOv8 neck network architecture found to verify its efficiency; The image recognition module inputs the image to be recognized into the image recognition module, extracts image features and outputs the corresponding category label.

Images