A YOLO target detection method and system based on differentiable architecture search

Abstract

The present invention provides a YOLO target detection method and system based on differentiable architecture search, wherein the method first constructs a candidate operation set. Then, based on the network architecture of YOLO, a supernet is designed. Subsequently, the supernet is pre-trained using a differentiable architecture search method, and network weights and architecture parameters are alternately optimized. Finally, the searched subnet is obtained according to the architecture parameters, and a coordinate attention mechanism is introduced to further improve the model performance. Unlike the manual design method, the present invention greatly reduces the experiments and experience required for manual work by automatically searching for neural network architectures, has higher search efficiency and performance, and can more quickly find an efficient YOLO improved model suitable for target detection.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and deep neural network technology, and in particular to a YOLO target detection method and system based on differentiable architecture search. Background Art

[0002] Object detection is a core technology in computer vision, which aims to identify and locate multiple objects in images or videos and provide category and location information. This technology has a wide range of applications in many fields, such as security monitoring, autonomous driving, medical imaging, agricultural and environmental monitoring, entertainment and media. With the rapid development of deep learning, object detection technology has undergone significant evolution, from early methods based on sliding windows and hand-designed features, to modern methods using deep convolutional neural networks for feature extraction, to the continuous improvement of one-stage detectors and two-stage detectors. Each generation of methods has made significant progress in detection speed, accuracy, and computational efficiency. However, hand-designed neural network architectures often rely on the designer's experience and prior knowledge, which may introduce bias and lead to suboptimal models.

[0003] In recent years, neural architecture search has achieved great success in image classification tasks. When we tried to migrate the automatically generated architectures in these classification tasks to the target detection task, we found that the effect was not very good. This is because compared with the classification task, the target detection task is more complicated. Not only does it need to further figure out the location of the object, but it also faces challenges such as processing multiple tasks, multiple scales, and real-time performance. After the early migration exploration, people began to study neural architecture search specifically for target detection and proposed a series of search methods, including search strategies based on reinforcement learning and search strategies based on evolutionary computation. Although these methods significantly improve the performance of the searched network, their high computational cost, low search efficiency, and insufficient architecture generalization are still the biggest problems these methods face in target detection applications.

[0004] In order to solve the above problems, in recent years, it has been proposed to use differentiable architecture search to search for target detection models. This method uses shared parameters in a supernet to search for architectures, which not only avoids the high computational cost of independent training of each candidate architecture in traditional neural architecture search methods, but also significantly speeds up the speed of architecture search through differentiable optimization methods. At the same time, target detection tasks usually require processing multi-scale features. Differentiable architecture search can flexibly select and combine operations at different scales during the search process to meet the detection needs of targets of different sizes. As a target detection method, YOLO has become a popular tool in this field due to its high-speed detection, high precision, easy deployment, and stable performance. However, research on the application of neural search architectures to YOLO is still very scarce, which highlights the potential of combining neural architecture search with YOLO. Summary of the invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a YOLO target detection method and system based on differentiable architecture search in view of the shortcomings of the prior art.

[0006] The method comprises the following steps:

[0007] Step 1, construct a candidate operation set;

[0008] Step 2: Based on the architecture of the YOLO network, a set of candidate operations is used to build a supernet.

[0009] Step 3: Use the differentiable architecture search method to pre-train the supernet on the training data set and search for the best subnet;

[0010] Step 4: Introduce the coordinate attention mechanism into the best subnet to further improve the network model of the best subnet;

[0011] Step 5: Retrain the network model and finally obtain an efficient YOLO improved model suitable for target detection.

[0012] Step 1 includes: selecting and defining a set of basic operations, combining the basic operations into a candidate operation set, wherein the candidate operation set O contains N candidate operations, including 1*1 convolution, 3*3 convolution, 3*3 dilated convolution, 5*5 convolution and 5*5 dilated convolution. The reason for selecting only convolution operations and discarding other operations such as pooling and jump connection operations is that the present invention retains the connection mode within some modules in YOLO and only optimizes the convolution operations used therein, which greatly reduces the consumption of computing resources in the search process while improving the overall performance.

[0013] In step 1, the candidate operation set Treat it as a whole and build a multi-branch candidate operation module, each branch corresponds to a candidate operation, and assign architecture parameters to each branch , Indicates candidate operations The corresponding architecture parameters are used to indicate the importance of the branch. ;

[0014] During the architecture search process, the architecture parameters are optimized and applied to the architecture parameters in the entire candidate operation module. The function performs weighted selection and finally determines the optimal combination of convolution operations. During the architecture search process, the candidate operation modules of multiple branches are expressed as follows:

[0015] ,

[0016] in, Indicates input, represents the output of the candidate operation module, Indicates candidate operations The normalized architecture parameters on the corresponding branch, Indicates candidate operations Applies to the output within the brackets.

[0017] Step 2 includes:

[0018] Step 2.1, based on the backbone network architecture of the YOLO network, construct a backbone supernet architecture: the backbone network of the YOLO network includes indivual Module, each The module contains convolution operations; supernet is a technique used in neural architecture search to uniformly represent and explore large search spaces. By embedding all possible neural network architectures into a collective structure, supernet allows different candidate operations and paths to be jointly optimized under the framework of shared parameters. During the training process, supernet effectively evaluates and selects the optimal subnet architecture through dynamic activation and weight adjustment. Ultimately, by extracting the subnet with the best performance from the supernet, it greatly improves the automation and efficiency of neural network design while reducing the consumption of computing resources.

[0019] based on Modules, build new Module, that is, retaining the original While connecting the modules, the candidate operation modules with multi-branch structure are replaced All convolution operations in the module;

[0020] By adding all Module replaced by Modules are used to construct a new backbone supernet architecture. The search space of the entire backbone supernet is expressed as follows:

[0021] ,

[0022] in, represents the size of the backbone supernet search space, represents the total number of candidate operations, Indicates each The total number of convolution operations contained in the module;

[0023] Step 2.2, based on the neck network architecture of the YOLO network, build the neck super network architecture: In the neck network of the YOLO network, it contains feature fusion layers, spanning A fully connected structure is added to the neck network of the YOLO network, and a candidate operation module with a multi-branch structure is used to replace the convolution operations between all feature layers and feature fusion layers in the neck network, as well as the convolution operations between feature fusion layers, so as to construct a neck super network architecture.

[0024] The search space of the entire neck supernet is expressed as:

[0025] ,

[0026] in, represents the size of the neck supernet search space, represents the total product of the search space of each feature fusion layer, Represents each edge The total number of candidate operations, represents the total number of edges finally retained in each feature fusion layer, represents the total number of input edges of each feature fusion layer in the supernet, Indicates the size of the search space for input edge selection at each feature fusion layer;

[0027] During the search process, the calculation formula of the feature fusion layer is expressed as:

[0028] ,

[0029] in, represents the feature fusion layer after multi-scale fusion, express The front input node of express The feature map is obtained by adding the processed results of all the previous nodes. represents the architecture parameters on the connection edge after normalization, Represents edge The candidate operation set, Represents edge For each candidate operation, Represents edge Candidate operations on The normalized architecture parameters on the branches of Represents edge Candidate operations on Applies to the output within the brackets;

[0030] The backbone supernet architecture and the neck supernet architecture are spliced ​​together to construct the final supernet architecture, in which the detection head still retains the structure of the original YOLO network. The search space of the entire supernet is expressed as follows:

[0031] ,

[0032] Among them, S represents the total search space size of the supernet;

[0033] In the present invention, specifically, a supernet is constructed based on the network architecture of YOLOv7. The backbone network of YOLOv7 is mainly composed of The backbone high-efficiency layer aggregation module consists of convolution operations. Based on the backbone efficient layer aggregation module, a new search module is constructed. That is, while retaining the original backbone efficient layer aggregation module connection structure, a multi-branch structure candidate operation module is used to replace all convolution operations in the backbone efficient layer aggregation module. By replacing all backbone efficient layer aggregation modules in the YOLO backbone network with search modules, a new backbone supernet architecture is constructed. The search space size of the entire backbone supernet is In the neck network of YOLOv7, it contains feature fusion layers, spanning A fully connected structure is added to the original structure, and a multi-branch structure candidate operation module is used to replace all feature layers and feature fusion layers in the neck network, and the convolution operations between feature fusion layers and feature fusion layers to construct the neck supernet architecture. The search space size of the entire neck supernet is The constructed backbone supernet and neck supernet are used to replace the backbone network and neck network in the original YOLOv7 to obtain the constructed overall supernet structure.

[0034] Step 3 includes:

[0035] Step 3.1, use the differentiable architecture search method to pre-train the supernet, alternately optimize the network weights and architecture parameters of the supernet until the set number of iterations is exhausted, expressed as:

[0036] , subject to ,

[0037] The training was conducted on the PASCAL VOC 2012 object detection standard dataset, and the dataset was divided into a training set and a validation set, and the supernet architecture parameters and network weights were optimized respectively; Represents the architecture parameters Minimize represents the verification loss; Indicated in the architecture parameter Next, the optimal weights obtained through training; Indicates that the minimum value in the brackets is returned ; represents the training loss; represents the supernet weight; the above formulas are performed alternately to continuously optimize the supernet weight and architecture parameters until the pre-training rounds are exhausted;

[0038] Step 3.2: Search for the best subnet based on the pre-trained supernet.

[0039] Step 3.2 includes:

[0040] Step 3.2.1: After the supernet pre-training is completed, in the backbone supernet, the largest architecture parameter after normalization in each candidate operation module of the multi-branch structure is The corresponding candidate operation replaces the candidate operation module of the current multi-branch structure to form the best backbone subnetwork obtained by the search;

[0041] Step 3.2.2, in the supernet pre-training stage, in the neck supernet, the feature fusion layer of each scale contains or Input, introduce edge architecture parameters Representative Input Edges After the supernet pre-training, each feature fusion layer is based on the normalized edge architecture parameters , sort all input edges of the feature fusion layer in descending order, and keep the top The input edge corresponding to the name, the choice of edge convolution is the same as the backbone network, to form the best neck subnet obtained by search, (In the present invention, the value );

[0042] In the present invention, specifically, in the supernet pre-training stage, in the neck supernet, the feature fusion layer of each scale includes or input, after the supernet pre-training is completed, each feature fusion layer is normalized according to the edge architecture parameters The size of the input edges is sorted in descending order, and the first The input edge corresponding to the name and the choice of convolution on the edge are the same as the backbone network to form the best neck subnet obtained by search.

[0043] In step 3.2.3, the best backbone subnet and the best neck subnet obtained by searching are used to replace the backbone network and the neck network of the original YOLO network respectively to obtain the best subnet.

[0044] In step 4, the coordinate attention mechanism is introduced into the optimal subnet. The coordinate attention mechanism processes the features in the horizontal and vertical directions respectively, so that the model can capture the aspect ratio, shape complexity, and relative position of the target more accurately. This enhanced spatial information perception ability enables the model to more accurately locate and identify objects in target detection tasks, especially in complex scenes, and the detection effect of small targets and dense objects is particularly significant. At the same time, it can also reduce the false detection rate and missed detection rate, and improve the overall performance. It can be expressed as:

[0045] ,

[0046] in, represents the feature map obtained by applying coordinate attention, represents application-level attention, Indicates the application of vertical attention, and the calculation formula is:

[0047] ,

[0048] ,

[0049] in, represents the original feature map, represents the horizontal attention weight, Represents the vertical attention weight, which is calculated as:

[0050] ,

[0051] ,

[0052] in, represents a normalization function, which converts a real value of a vector into a probability distribution so that the sum of the probability of each element and the probability of other elements is 1. represents the learned weight matrix used to calculate the horizontal attention weights, represents the learning weight matrix used to calculate the vertical attention weight, Indicates flattening the coordinate information into a vector. represents the horizontal coordinate, Represents the vertical coordinate.

[0053] In step 5, the best subnet after introducing the coordinate attention mechanism is retrained on the standard dataset PASCAL VOC 2012, thereby obtaining an efficient YOLO improved model suitable for target detection. The loss function of the retraining is expressed as follows:

[0054] ,

[0055] in, represents the total loss function, represents the target existence loss, represents the category loss, represents the bounding box regression loss, which is calculated as:

[0056] ,

[0057] ,

[0058] ,

[0059] in, represents the weight coefficient of the target existence loss, Indicates the accumulation of loss values ​​of all grid cells. Indicates The ground-truth target existence label of each grid cell (1 for existence, 0 for non-existence), represents the probability of target existence predicted by the model, Represents the weight coefficient of category loss, Represents all grid cells and all categories The loss is double-summed, Indicates The true category label of the object in the grid cells, The model predicts the In the grid cell The probability of the class, represents the weight coefficient of the bounding box regression loss, For all grid cells The loss values ​​are accumulated. Indicates The intersection-over-combination ratio of grid cells, Indicates The true bounding box coordinates of the grid cells, The model predicts the The bounding box coordinates of the grid cells.

[0060] The present invention also provides a YOLO target detection system based on differentiable architecture search, comprising:

[0061] The supernet building module is used to build a set of candidate operations and a supernet based on the YOLO network architecture.

[0062] The search module is used to pre-train the supernet and search for the best subnet.

[0063] The optimization module is used to further optimize the subnetwork using the coordinate attention mechanism and retrain the subnetwork to finally obtain an efficient YOLO improved model suitable for target detection.

[0064] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the described method.

[0065] The present invention applies differentiable architecture search to the YOLO model, so that it can automatically optimize the network architecture, overcome the limitations of manual design, and significantly improve the model performance. Through continuous gradient optimization, differentiable architecture search realizes efficient architecture exploration, and can find the optimal operation and hierarchy in a short time, thereby enhancing the effect of feature extraction and multi-scale fusion. In order to further optimize the model, a coordinate attention mechanism is also introduced to enhance the spatial information perception ability of the model. The method of the present invention not only improves the detection accuracy of the YOLO model, but also can flexibly adapt to different application scenarios and hardware environments, while maintaining real-time detection, optimizes the processing of complex targets, and improves the robustness and reliability of the model.

[0066] The beneficial effect of the present invention is that the present invention proposes a method for applying a differentiable architecture search in the field of YOLO target detection. The method constructs a supernet based on the YOLO model, and uses a differentiable architecture search algorithm to find the best subnet. Compared with other search algorithms, the algorithm no longer relies on discrete search and independent training of each candidate architecture. By using continuous gradient optimization during the training process, the architecture weight is directly adjusted, thereby achieving efficient architecture optimization, and significantly reducing the computational cost and time consumption in the supernet training process. As an important tool in the field of real-time target detection, YOLO has excellent characteristics such as high-speed detection, high accuracy, and easy deployment. By performing efficient architecture search on the structure of YOLO, it not only accelerates the design process of the supernet, but also improves the detection accuracy of the model while maintaining the efficient real-time detection performance of YOLO. It allows the network to be dynamically adjusted to adapt to different data and task requirements, significantly enhancing the robustness and adaptability of the model. Introducing the coordinate attention mechanism in the searched model further enhances the model's perception of spatial information, making the model more sensitive to the detection of complex targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a candidate operation module for multiple branches in the present invention.

[0068] Figure 2 This is a diagram of the overall supernet architecture in the present invention.

[0069] Figure 3It is the best backbone subnet module and the best neck subnet graph obtained by the supernet search in the present invention.

[0070] Figure 4 It is a schematic diagram of the results of the method of the present invention on the data set. DETAILED DESCRIPTION

[0071] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0072] Figure 1 Take the multi-branch candidate operation module containing 5 candidate operations as an example. Figure 2 Taking the YOLOv7 network architecture as an example, the implementation process and details of a YOLO target detection method based on differentiable architecture search proposed in this embodiment will be introduced in detail.

[0073] Figure 1 The specific process of building a multi-branch candidate operation module is shown. Then, based on the YOLOv7 network architecture, Figure 1 The multi-branch candidate operation module constructed in Figure 2 The entire supernet architecture shown in Figure 2 Pre-train the supernet in and search for the best subnet. Figure 3 The best backbone subnetwork is shown. The main modules and the best neck subnet are constructed. Finally, the coordinate attention mechanism is added to the best subnet searched and the model is retrained to obtain the efficient improved YOLOv7 model suitable for target detection.

[0074] like Figure 1 As shown, the multi-branch candidate operation module in this embodiment performs the following steps:

[0075] Step 1: First, build a set of candidate operations , which contains all possible operations in the search space, including convolution, convolution, Atrous convolution, convolution, Atrous convolution. Then, Figure 1 As shown, the candidate operation set As a whole, a multi-branch candidate operation module is constructed on this basis. Each branch corresponds to a candidate operation. By default, a batch normalization operation and an activation function are added after each candidate operation. The architecture parameters are allocated on each branch. To indicate the importance of this branch. When there is a feature map input, the specific calculation method is: ,in, Indicates input, express The output after the candidate operation module (i.e. the result after weighted summation in the figure), Represents the set of all candidate operations in the candidate operation module of multiple branches, Indicates candidate operations The normalized architecture parameters on the branches, Indicates candidate operations Applies to the output within the brackets.

[0076] like Figure 2 As shown in FIG. 1 , it is a diagram of the overall supernet architecture proposed in this embodiment. The overall supernet performs the following steps:

[0077] Step 2: Based on the YOLOv7 network architecture, use the multi-branch candidate operation module in step 1 to build the entire supernet. It includes the following steps:

[0078] Step 2.1: Based on the YOLOv7 backbone network architecture, build a backbone supernet architecture. The backbone network of YOLOv7 is mainly composed of The backbone high-efficiency layer aggregation module consists of convolution operation. Figure 2 As shown in Figure 2, we build a new search module based on the backbone efficient layer aggregation module, such as Figure 2 The structure in the red box in the middle, when the feature map is input into the module, its channel number is split There are two branches on the left and right respectively. candidate operation modules, and on the right branch there are candidate operation modules, and finally The results of the branches are combined and then The search module retains the original backbone efficient layer aggregation module connection structure while adopting a multi-branch structure candidate operation module ( Figure 1 ) replaces all convolution operations in the backbone efficient layer aggregation module. By replacing all backbone efficient layer aggregation modules in the YOLO backbone network with search modules, a new backbone supernet architecture is constructed ( Figure 2 Backbone supernet part). Figure 2 The pre-processing module in the backbone super network (including A series of Convolution operation) and pooling module (green arrow, the module contains branches, the left branch contains the maximum pooling operation and Convolution operation, the right branch contains Convolution and Convolution operations) are all part of the original YOLOv7 backbone network and remain unchanged during the construction of the backbone supernet.

[0079] Step 2.2: Based on the YOLOv7 neck network architecture, build the neck super network architecture. Figure 2 As shown, the neck network of YOLOv7 contains feature fusion layers, spanning Scales ( , , ). A fully connected structure is added to its original connection structure, and a candidate operation module with a multi-branch structure is used ( Figure 2 The red arrow in the figure replaces all feature layers and feature fusion layers in the neck network, as well as the convolution operations between feature fusion layers and feature fusion layers, to build the neck supernet architecture ( Figure 2 Neck super mesh part). Figure 2 The pyramid cross-linking module in the middle neck super network and the efficient layer aggregation module (similar to the search module structure, both modules contain The pooling modules (green arrows) are all parts of the original YOLOv7 neck network and remain unchanged during the construction of the neck supernet. The detection head retains the original YOLOv7 structure, including reparameterized convolutional modules, and Output results of different scales are used to predict large, medium and small targets respectively.

[0080] Step 3: Use differentiable architecture search method to Figure 2 The supernet is pre-trained to search for the best subnet. The steps are as follows:

[0081] Step 3.1, use the differentiable architecture search method to pre-train the supernet, alternately optimize the network weights and architecture parameters of the supernet until the set number of iterations is exhausted, which can be expressed as, , subject to The algorithm divides the data set into a training set and a validation set, and optimizes the supernet architecture parameters and network weights respectively. Express To minimize, Represents the architecture parameters, represents the validation loss, Indicates that the architecture parameters are specific The optimal weights obtained through training. Indicates that the minimum value in the brackets is returned , represents the training loss, The above formulas are repeated alternately to continuously optimize the supernet weights and architecture parameters until the pre-training rounds are exhausted.

[0082] Step 3.2: Search for the best subnet based on the architecture parameters in the pre-trained supernet. This includes the following steps:

[0083] Step 3.2.1: After the supernet pre-training is completed, in the backbone supernet, the largest normalized architecture parameter of each candidate operation module of the multi-branch structure is ( , The candidate operation corresponding to the candidate operation set replaces the candidate operation module of the current multi-branch structure. For example, the normalized architecture parameters in a candidate operation module of a multi-branch structure =0.12, =0.21, =0.15, =0.16, =0.36, corresponding to convolution, convolution, Atrous convolution, convolution, Atrous convolution replaces the candidate operation module of the multi-branch structure with Finally, the search yielded four main modules that constitute the best backbone subnetwork, such as Figure 3 The best subnet module after the backbone supernet search is shown in the module structure. The module structure is consistent with the search module, where the first module contains indivual convolution, indivual convolution, indivual Atrous convolution, the second module contains indivual convolution, indivual convolution, indivual Atrous convolution, the third module contains indivual convolution, indivual convolution, indivual Atrous convolution, indivual Atrous convolution, the fourth module contains indivual convolution, indivual Atrous convolution, indivual convolution, indivual Atrous convolution. The four best subnet modules obtained after the search replace the four search modules in the backbone supernet to obtain the best backbone subnet.

[0084] Step 3.2.2: In the supernet pre-training stage, the feature fusion layer of each scale in the neck supernet contains or Input, introduce edge architecture parameters Representative Input Edges After the supernet pre-training is completed, each feature fusion layer is normalized according to the architecture parameters behind The size of the feature fusion layer is sorted in descending order, and the top The input edge corresponding to the name, for example, the input of a feature fusion layer includes The normalized architecture parameters on the edges are =0.14, =0.11, =0.23, =0.52, will save and The corresponding input edge is deleted and The convolution operation on the corresponding input edge is the same as that in the backbone network. The neck network obtained after processing is as follows: Figure 3 The best subnet after the mid-necked network search is shown in the figure, where the leftmost column The circles represent feature layers, and the middle Column and rightmost Column The circles represent the feature fusion layer, the black arrows represent the saved input edges, and the annotations on the arrows represent the convolution operations on the input edges. convolution, convolution, Atrous convolution, convolution, The hole convolutions correspond to the following figures: , , , , . represents the convolution kernel size, Indicates whether it is a dilated convolution ( For No, for yes).

[0085] Step 3.2.3: Replace the original YOLO backbone network and neck network with the best backbone subnet and neck subnet respectively to obtain the best subnet.

[0086] Step 4: Introduce the coordinate attention mechanism into the best subnet obtained by searching, and add it to the end of the backbone network with the best subnet as the position. The coordinate attention mechanism can enhance the spatial information perception ability, enabling the model to more accurately locate and identify objects in target detection tasks, especially in complex scenes, and the detection effect of small targets and dense objects is particularly significant. It can also reduce the false detection rate and missed detection rate, and improve the overall performance. It can be expressed as, ,in, represents the feature map obtained by applying coordinate attention, represents application-level attention, indicates the application of vertical attention, and The corresponding calculation formula is: , ,in, represents the original feature map, represents the horizontal attention weight, represents the vertical attention weight, and The corresponding calculation formula is: , ,in, represents a normalization function, which converts a real value of a vector into a probability distribution so that the sum of the probability of each element and the probability of other elements is 1. represents the learned weight matrix used to calculate the horizontal attention weights, represents the learning weight matrix used to calculate the vertical attention weight, Indicates flattening the coordinate information into a vector. represents the horizontal coordinate, Represents the vertical coordinate.

[0087] Step 5: Retrain the best subnet after introducing the coordinate attention mechanism on the standard dataset to obtain the final efficient YOLO improved model suitable for target detection. The loss function of the retraining is expressed as, ,in, represents the total loss function, represents the target existence loss, represents the category loss, represents the bounding box regression loss, , and The corresponding calculation formula is: ,in, represents the weight coefficient of the target existence loss, Indicates the accumulation of loss values ​​of all grid cells. Indicates The ground-truth target existence label of each grid cell (1 for existence, 0 for non-existence), Represents the probability of target existence predicted by the model. ,in, Represents the weight coefficient of category loss, For all grid cells and all categories The loss is double-summed, Indicates The true category label of the object in the grid cells, The model predicts the In the grid cell The probability of the class. ,in, represents the weight coefficient of the bounding box regression loss, For all grid cells The loss values ​​are accumulated. Indicates The intersection-over-combination ratio of grid cells, Indicates The true bounding box coordinates of the grid cells, The model predicts the The bounding box coordinates of the grid cells.

[0088] Compared with the prior art, the present invention greatly reduces the experiments and experience required by manual work by automatically searching for neural network architectures, has higher search efficiency and performance, and can more quickly find an efficient YOLO improved model suitable for target detection. In addition, the present invention also introduces a coordinate attention mechanism to further enhance the performance of the model. The present invention tested the performance on the PASCAL VOC 2012 target detection standard dataset, which contains 20 categories, of which 13,500 images are used for training and 3,595 images are used for verification and testing. The present invention conducted experiments on an Nvidia RTX 4090 GPU and eventually obtained a network architecture with a mAP@0.5:0.95 (accuracy index) of up to 61.4%, which is 3.4% higher than the accuracy index of the original YOLOv7 model. Specifically, it includes the following steps: Step 1, construct a set of candidate operations, such as Figure 1 Step 2: construct a supernet, as shown in Figure 2 Step 3: Use the differentiable architecture search method to search for the best subnet, as shown in Figure 3In step 4 and step 5, the coordinate attention mechanism is added to retrain the model. The results are as follows Figure 4 shown. Figure 4 In the figure, the horizontal axis is the number of retraining rounds, and the vertical axis is the accuracy index. Compared with the original model, the searched model is superior to the original model in terms of convergence speed and final accuracy, which proves the effectiveness of the method of the present invention.

[0089] The present invention provides a YOLO target detection method and system based on differentiable architecture search. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A YOLO target detection method based on differentiable architecture search, characterized in that: The following steps are involved: Step 1, construct a candidate operation set; Step 2: Based on the architecture of the YOLO network, a set of candidate operations is used to build a supernet. Step 3: Use the differentiable architecture search method to pre-train the supernet on the training data set and search for the best subnet; Step 4: Introduce the coordinate attention mechanism into the best subnet to further improve the network model of the best subnet; Step 5: Retrain the network model to finally obtain an efficient YOLO improved model suitable for target detection; Step 1 includes: selecting and defining a set of basic operations, combining the basic operations into a candidate operation set, wherein the candidate operation set O contains N candidate operations, including 1*1 convolution, 3*3 convolution, 3*3 dilated convolution, 5*5 convolution and 5*5 dilated convolution; In step 1, the candidate operation set O is considered as a whole, and a multi-branch candidate operation module is constructed. Each branch corresponds to a candidate operation, and the architecture parameter α is assigned to each branch. o , α o Represents the architectural parameters corresponding to the candidate operation o, which is used to indicate the importance of the branch, o∈O; In the process of architecture search, the optimal convolution operation combination is finally determined by optimizing the architecture parameters and applying the Softmax function to the architecture parameters in the entire candidate operation module for weighted selection. In the process of architecture search, the candidate operation module of multiple branches is expressed by the formula: Among them, f1 represents the input, f2 represents the output of the candidate operation module, represents the normalized architecture parameters on the branch corresponding to the candidate operation o, and o(·) represents the candidate operation o applied to the output in the brackets; Step 2 includes: Step 2.1, based on the backbone network architecture of the YOLO network, construct a backbone supernet architecture: the backbone network of the YOLO network includes n m modules, each of which contains c convolution operations; Based on the m-module, a new sm-module is constructed, that is, while retaining the original m-module connection structure, a candidate operation module with a multi-branch structure is used to replace all convolution operations in the m-module; By replacing all m modules in the backbone network of the YOLO network with sm modules, a new backbone supernet architecture is constructed. The search space of the entire backbone supernet is expressed as follows: Among them, S b represents the size of the backbone supernet search space, represents the total number of candidate operations, and c represents the total number of convolution operations contained in each m module; Step 2.2, based on the neck network architecture of the YOLO network, a neck supernet architecture is constructed: in the neck network of the YOLO network, i feature fusion layers are included, spanning x scales, a fully connected structure is added on the basis of the neck network of the YOLO network, and a candidate operation module with a multi-branch structure is used to replace the convolution operations between all feature layers and feature fusion layers in the neck network, as well as the convolution operations between feature fusion layers and feature fusion layers, so as to construct a neck supernet architecture; The search space of the entire neck supernet is expressed as: Among them, S f represents the size of the neck supernet search space, ∏ i (·) represents the total product of the search space of each feature fusion layer, represents the total number of candidate operations on each edge e, k represents the total number of edges finally retained by each feature fusion layer, p represents the total number of input edges of each feature fusion layer in the supernet, Indicates the size of the search space for input edge selection at each feature fusion layer; During the search process, the calculation formula of the feature fusion layer is expressed as: Among them, x j represents the feature fusion layer after multi-scale fusion, x i Represents x j The front input node of i<j [·] represents x j The feature map obtained by adding the processed results of all the previous nodes is represents the architecture parameters on the connection edge after normalization, O e represents the set of candidate operations on edge e, o e represents each candidate operation on edge e, represents the candidate operation o on edge e e The normalized architecture parameters on the branch of e (·) represents the candidate operation o on edge e e Applies to the output within the brackets; The backbone supernet architecture and the neck supernet architecture are spliced ​​together to construct the final supernet architecture, in which the detection head still retains the structure of the original YOLO network. The search space of the entire supernet is expressed as follows: S=S b ·S f , Among them, S represents the total search space size of the supernet; Step 3 includes: Step 3.1, use the differentiable architecture search method to pre-train the supernet, alternately optimize the network weights and architecture parameters of the supernet until the set number of iterations is exhausted, expressed as: min α L val (w * (α),α), subject to w * (α) = argmin w L train (w,α), The training is performed on the data set, and the data set is divided into a training set and a validation set, and the supernet architecture parameters and network weights are optimized respectively; among them, min α (·) represents minimization of the architecture parameter α; L val represents the verification loss; w * (α) represents the optimal weight obtained through training under the architecture parameter α; argmin w (·) indicates that w in the brackets returns to the minimum value; L train represents the training loss; w represents the supernet weight; Step 3.2, search for the best subnet based on the pre-trained supernet; Step 3.2 includes: Step 3.2.1: After the supernet pre-training is completed, in the backbone supernet, the largest architecture parameter after normalization in each candidate operation module of the multi-branch structure is The corresponding candidate operation replaces the candidate operation module of the current multi-branch structure to form the best backbone subnetwork obtained by the search; Step 3.2.2, in the supernet pre-training stage, in the neck supernet, the feature fusion layer of each scale contains a or b inputs, and the edge architecture parameter α is introduced ei Represents the importance of the i-th input edge e; after the supernet pre-training is completed, each feature fusion layer is normalized according to the edge architecture parameters The size of , sort all the input edges of the feature fusion layer in descending order, retain the input edges corresponding to the top k, and the selection of the convolution on the edge is the same as the backbone network to form the best neck subnet obtained by the search, a ≥ b > k; Step 3.2.3, replace the backbone subnet and the neck subnet of the original YOLO network with the best backbone subnet and the best neck subnet respectively to obtain the best subnet; In step 4, the coordinate attention mechanism is introduced into the best subnet, which is expressed as: Among them, X ca represents the feature map obtained by applying coordinate attention, represents application-level attention, Indicates the application of vertical attention, and the calculation formula is: Among them, X sp represents the original feature map, A h represents the horizontal attention weight, A w Represents the vertical attention weight, which is calculated as: A h =Softmax(W h ·Flatten(pos h )), A w =Softmax(W w ·Flatten(pos w )), Among them, Softmax(·) represents the normalization function, W h represents the learning weight matrix used to calculate the horizontal attention weight, W W represents the learning weight matrix used to calculate the vertical attention weight, Flatten(·) represents flattening the coordinate information into a vector, pos h Indicates the horizontal coordinate, pos W Indicates the vertical coordinate; In step 5, the best subnet after introducing the coordinate attention mechanism is retrained on the standard dataset to obtain an efficient YOLO improved model suitable for target detection. The loss function of the retraining is expressed as: L=L obj +L cls +L box , Among them, L represents the total loss function, L obj represents the target existence loss, L cls represents the category loss, L box represents the bounding box regression loss, which is calculated as: Among them, λ obj Represents the weight coefficient of the target existence loss, ∑ i (·) indicates the accumulation of the loss values ​​of all grid cells, y i represents the true target existence label of the i-th grid cell, represents the probability of target existence predicted by the model, λ cls Represents the weight coefficient of category loss, ∑ i ∑ c (·) denotes the double summation of the losses over all grid cells and all categories c, y i,c represents the true category label of the target in the i-th grid cell, represents the probability of the cth class in the ith grid cell predicted by the model, λ box represents the weight coefficient of bounding box regression loss, ∑ i (·) indicates the accumulation of the loss values ​​of all grid cells i, IoU i represents the intersection-over-union ratio of the i-th grid cell, bbox i represents the true bounding box coordinates of the ith grid cell, represents the bounding box coordinates of the i-th grid cell predicted by the model.

2. A YOLO target detection system based on differentiable architecture search implemented according to the method of claim 1, characterized in that include: The supernet building module is used to build a set of candidate operations and a supernet based on the YOLO network architecture. Search module, used to pre-train the supernet and search for the best subnet; The optimization module is used to further optimize the subnetwork using the coordinate attention mechanism and retrain the subnetwork to finally obtain an efficient YOLO improved model suitable for target detection.

3. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to claim 1.

Images