Cross-scale adaptive object detection method, device and medium based on neural architecture search

Abstract

The application discloses a cross-scale adaptive target detection method and device based on neural architecture search and a medium, obtains a full-scene data set, and divides the full-scene data set into eight sub-class data sets; embeds a functional module unit for realizing each detection task into an existing target detection model, and trains the embedded target detection model by using a corresponding sub-class data set and a loss function, to obtain eight optimal target detection models corresponding to the sub-class data sets one by one; judges a newly input image, classifies the newly input image into a certain sub-class data set, selects a corresponding optimal target detection model according to the sub-class data set, and outputs a target detection result. The application can avoid performance decline caused by a single inherent detection network due to changes in scene data distribution, solve the deficiency that a neural architecture search method is difficult to extend to a target detection field, and thus achieve multi-scene, full-scale and high-precision target detection tasks.

Description

Technical Field

[0001] This invention relates to a cross-scale adaptive target detection method, device, and medium based on neural architecture search, belonging to the field of computer vision technology. Background Technology

[0002] Neural Architecture Search (NAS) is a technique that uses machines to automatically search and construct neural network architectures. NAS was initially applied primarily to classification problems, but in recent years, its application in image object detection has increased significantly. Given that typical detection models consist of four parts (input, backbone, neck, and head), researchers mainly explore NAS searches for the latter three parts of object detection to obtain rich and effective image features and determine accurate spatial locations.

[0003] In published literature, DeNAS searches the backbone of a detection model by decoupling weight training and structure search. NAS-FPN discusses how the neck of a detection model automatically performs hierarchical feature connections to achieve a trade-off between accuracy and speed. AutoFPN's innovation focuses on the neck and head networks; the neck performs an auto-fusion operation on backbone features, while the auto-head uses NAS search to obtain a network for classification and regression. NAS-FCOS is similar to Auto-FPN, searching the neck and head parts, but it uses reinforcement learning for the search, and the search space is slightly different. It treats FCOS (a fully convolutional one-stage object detector) as an encoder-decoder architecture, with the backbone representing the encoder and FPN and head representing the decoder. SpineNet proposes a meta-structure model called scale-permuted, which allows the spatial resolution of intermediate feature maps to increase or decrease at any time, so that the model can retain spatial information as the depth increases. Furthermore, the connections between feature maps can cross feature scales to promote multi-scale feature fusion. Hit-Detector is the first NAS that can simultaneously search the backbone, neck, and head of a detection model, and it designs a hierarchical method to search for different structures in different sub-search spaces.

[0004] However, existing NAS technologies all construct neural networks for specific objects. For example, the general object detection dataset MS COCO uses a level-based image acquisition method, while the UAV aerial object detection dataset uses a top-down image acquisition method. Different acquisition methods cause changes in the scale distribution and density of objects. In the former, the target scale variation is small, and individual objects are easy to distinguish, while in the latter, targets exhibit multi-scale characteristics and severe edge overlap. Furthermore, factors such as weather and lighting conditions affect the foreground and background contrast in images acquired in different environmental scenarios. The level of contrast determines the ease with which foreground targets can be identified from a complex background.

[0005] Therefore, how to address the impact of significant differences in data distribution across different datasets on object detection is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] Objective: To overcome the shortcomings of existing technologies, this invention provides a cross-scale adaptive object detection method, device, and medium based on neural architecture search. This method addresses the deficiencies of existing neural architecture search methods, adapts to various object detection tasks across different scenarios, and automatically embeds different functional modules into the basic detection model to complete object detection tasks across all scenarios. To achieve the above objective, this invention employs the following technical solution:

[0007] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0008] Firstly, a cross-scale adaptive target detection method based on neural architecture search includes the following steps:

[0009] Step 1: Obtain the full-scene dataset and divide it into eight sub-datasets.

[0010] Step 2: Use each of the eight subclass datasets as the training set, and use neural architecture search to obtain the functional module units that implement each detection task.

[0011] Step 3: Embed the functional modules that implement each detection task into the existing target detection model to obtain the embedded target detection model. Then, train the embedded target detection model using the corresponding subclass datasets and loss functions to obtain eight optimal target detection models that correspond one-to-one with the subclass datasets.

[0012] Step 4: Analyze the new input image and assign it to a specific subclass dataset. Select the best object detection model for the subclass dataset, input the new input image into the best object detection model, and output the object detection result.

[0013] Furthermore, the full-scene dataset includes:

[0014] Multiple open-source target detection datasets were selected.

[0015] The images were captured and labeled using various camera devices at different times, angles, and weather conditions.

[0016] It was obtained by searching and downloading from the Internet using a search engine and then annotating the results.

[0017] Furthermore, the full-scene dataset is divided into eight sub-datasets, including:

[0018] Define three dimensions of the target in the image: sparsity, scale, and contrast with the background.

[0019] Each dimension has two standards: density (dense) and sparseness (sparse); scale (large) and small; and contrast with the background (bright) and dark.

[0020] Based on three dimensions and two criteria for each dimension, the full-scene dataset is divided into eight sub-datasets: dark images of dense, small-scale objects; bright images of dense, small-scale objects; dark images of dense, large-scale objects; bright images of dense, large-scale objects; dark images of sparse, small-scale objects; bright images of sparse, small-scale objects; dark images of sparse, large-scale objects; and bright images of sparse, large-scale objects.

[0021] Furthermore, the neural architecture search employs DARTS.

[0022] Furthermore, the step of embedding the functional module units that implement each detection task into the existing target detection model to obtain the embedded target detection model includes:

[0023] The YOLO series of one-stage detection models were selected as the existing target detection models.

[0024] With the goal of optimizing network performance, the functional modules for each detection task are embedded into the backbone and neck structures of the existing target detection model to obtain the embedded target detection model.

[0025] Furthermore, the loss function L CE ,include:

[0026]

[0027] In the formula, λ represents the weight between loss and lat(θ), loss represents the loss function between the true label and the search prediction, y represents the true label value, and x represents the true label value. l p(x) represents the feature input of the l-th layer. l ;W,θ) represents the output predicted by the l-th layer, the function lat(θ) represents the network delay function, n represents the number of feature extraction layers, W represents the set of weight parameters, and θ represents the optimal training parameters.

[0028] Furthermore, the weight parameters W and the optimal training parameters θ of the trained network are trained using the batch stochastic gradient descent algorithm, wherein the parameter update rules are as follows:

[0029]

[0030]

[0031] In the formula, w t+1 Let w be the parameter value at the (t+1)th iteration. t The parameter value at the t-th iteration. For the target loss with respect to parameter w t The partial derivatives of θ t+1 Let θ be the parameter value at the (t+1)th iteration. t The parameter value at the t-th iteration. For the target loss with respect to parameter θ t The partial derivatives of η t This is the learning rate.

[0032] Furthermore, the learning rate η t The calculation formula is as follows:

[0033]

[0034] In the formula, η0 is the maximum learning rate, t is the current iteration number, t0 is the warm-up iteration number, and t n Let π represent the total number of iterations, and π be the value of pi.

[0035] Furthermore, the step of judging the newly input image and classifying it into a certain subclass dataset includes:

[0036] The newly input image is decomposed using a quadtree to find the correspondence between the pixel value at each position and the pixel value of its neighbors. The image type label is then used to determine the degree of sparseness, scale, and contrast with the background of the target in the image.

[0037] Based on the sparsity of the objects in the image, their scale, and their contrast with the background, newly input images are assigned to a specific subclass dataset.

[0038] In a second aspect, a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a cross-scale adaptive target detection method based on neural architecture search as described in any of the first aspects.

[0039] Thirdly, a computer device comprising:

[0040] Memory is used to store instructions.

[0041] A processor for executing the instructions, causing the computer device to perform operations of a cross-scale adaptive object detection method based on neural architecture search as described in any of the first aspects.

[0042] Beneficial effects: The cross-scale adaptive target detection method, device and medium based on neural architecture search provided by this invention mainly consider the differences in data sample distribution caused by the different density, scale and brightness of targets reflected in the dataset images of various real-world individuals. The aim is to find a general target detection neural network architecture that can automatically adapt to different data distributions without manual design.

[0043] This invention avoids the performance degradation caused by relying on a single inherent detection network due to changes in scene data distribution, and solves the shortcomings of neural architecture search methods that are difficult to extend to the field of target detection, thereby achieving multi-scene, full-scale, and high-precision target detection tasks. Attached Figure Description

[0044] Figure 1 This is a flowchart of a cross-scale adaptive target detection method based on neural architecture search according to the present invention. Detailed Implementation

[0045] The present invention will be further described below with reference to specific embodiments.

[0046] like Figure 1 As shown, the first embodiment of a cross-scale adaptive target detection method based on neural architecture search includes the following steps:

[0047] Step 1: Obtain the full-scene dataset and divide it into eight sub-datasets.

[0048] Step 2: Use each of the eight subclass datasets as the training set, and use neural architecture search to obtain the functional module units that implement each detection task.

[0049] Step 3: Embed the functional modules that implement each detection task into the existing target detection model to obtain the embedded target detection model. Then, train the embedded target detection model using the corresponding subclass datasets and loss functions to obtain eight optimal target detection models that correspond one-to-one with the subclass datasets.

[0050] Step 4: Analyze the new input image and assign it to a specific subclass dataset. Select the best object detection model for the subclass dataset, input the new input image into the best object detection model, and output the object detection result.

[0051] Furthermore, the full-scene dataset includes:

[0052] Multiple open-source target detection datasets were selected.

[0053] The images were captured and labeled using various camera devices at different times, angles, and weather conditions.

[0054] It was obtained by searching and downloading from the Internet using a search engine and then annotating the results.

[0055] Furthermore, the full-scene dataset is divided into eight sub-datasets, including:

[0056] Define three dimensions of the target in the image: sparsity, scale, and contrast with the background.

[0057] Each dimension has two standards: density (dense) and sparseness (sparse); scale (large) and small; and contrast with the background (bright) and dark.

[0058] Based on three dimensions and two criteria for each dimension, the full-scene dataset is divided into eight sub-datasets: dark images of dense, small-scale objects; bright images of dense, small-scale objects; dark images of dense, large-scale objects; bright images of dense, large-scale objects; dark images of sparse, small-scale objects; bright images of sparse, small-scale objects; dark images of sparse, large-scale objects; and bright images of sparse, large-scale objects.

[0059] Furthermore, the neural architecture search employs DARTS.

[0060] Furthermore, the step of embedding the functional module units that implement each detection task into the existing target detection model to obtain the embedded target detection model includes:

[0061] The YOLO series of one-stage detection models were selected as the existing target detection models.

[0062] With the goal of optimizing network performance, the functional modules for each detection task are embedded into the backbone and neck structures of the existing target detection model to obtain the embedded target detection model.

[0063] Furthermore, the loss function includes:

[0064]

[0065]

[0066] (1) In the formula, λ represents the weight between loss and lat(θ), loss represents the loss function between the true label and the search prediction, y represents the true label value, and x represents the true label value. l p(x) represents the feature input of the l-th layer. l ;W,θ) represents the output predicted by the l-th layer, the function lat(θ) represents the network latency function (mainly considering the long time spent searching the network, and the time spent searching the network should be reduced if deployed to mobile devices), n represents the number of feature extraction layers, W represents the set of weight parameters, and θ represents the optimal training parameters.

[0067] (2) In the formula, This represents the weight parameter in the j-th parallel functional module unit of the l-th layer. This represents the optimal training parameters when the j-th parallel functional module unit in the l-th layer is connected in parallel.

[0068] Furthermore, the weight parameters W and the optimal training parameters θ of the trained network are trained using the batch stochastic gradient descent algorithm, wherein the parameter update rules are as follows:

[0069]

[0070]

[0071] In equations (3) and (4), w t+1 Let w be the parameter value at the (t+1)th iteration. t The parameter value at the t-th iteration. For the target loss with respect to parameter w t The partial derivatives of θ t+1 Let θ be the parameter value at the (t+1)th iteration. t The parameter value at the t-th iteration. For the target loss with respect to parameter θ t The partial derivatives of η t This is the learning rate.

[0072] Learning rate η t The update is performed using a preheated cosine-simulated annealing strategy, with the following calculation rules:

[0073]

[0074] (5) In the formula, η0 is the maximum learning rate, t is the current iteration number, t0 is the warm-up iteration number, and t n Let π represent the total number of iterations, and π be the value of pi.

[0075] Furthermore, the step of judging the newly input image and classifying it into a certain subclass dataset includes:

[0076] The newly input image is decomposed using a quadtree to find the correspondence between the pixel value at each position and the pixel value of its neighbors. The image type label is then used to determine the degree of sparseness, scale, and contrast with the background of the target in the image.

[0077] Based on the sparsity of the objects in the image, their scale, and their contrast with the background, newly input images are assigned to a specific subclass dataset.

[0078] The second embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a cross-scale adaptive target detection method based on neural architecture search as described in any of the first embodiments.

[0079] A third embodiment of a computer device includes:

[0080] Memory is used to store instructions.

[0081] A processor is configured to execute the instructions, causing the computer device to perform operations of a cross-scale adaptive object detection method based on neural architecture search as described in any of the first embodiments.

[0082] Fourth embodiment:

[0083] Unlike existing methods, this invention is based on an existing object detection network architecture. By searching for enhancement modules at the backbone that can improve the network performance, it is used to extract weak features of small-scale targets. At the same time, a switch network is designed to match the optimal object detection model according to the changes in the target scale of the dataset images, so as to achieve full control over the implementation of the related structural functions of the backbone and neck parts, thereby improving the detection performance of existing networks for cross-scale targets.

[0084] The functional module units searched in this invention are modules composed of multiple atomic operations such as convolution and pooling, which can perform a specific function. Multiple such modules are embedded in the existing target detection network architecture in a certain connection manner. The best enhancement module units at different positions are searched by NAS to construct a new network.

[0085] This invention extends existing object detection methods, primarily considering the realistic appearance of individuals in the real world. This is reflected in the dataset images by significant variations in object size, ranging from irregular point-like objects to regular planar objects. If a single, established detection model cannot adequately fit all data samples, its generalization ability is limited, inevitably leading to missed detections and false positives. Furthermore, current neural architecture search is mainly used for general object recognition tasks to find the optimal network architecture. It lacks the ability to automatically search for and design complete network architectures capable of performing specific detection functions at different locations based on the existing detection infrastructure, rather than manually, thus hindering its direct extension to multi-scale object detection tasks.

[0086] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A cross-scale adaptive target detection method based on neural architecture search, characterized in that: Includes the following steps: Step 1: Obtain the full-scene dataset and divide it into eight sub-datasets; The full-scene dataset is divided into eight sub-datasets, including: Define three dimensions of the target in the image: sparsity, scale, and contrast with the background; Each dimension has two standards: density (dense) and sparseness (sparse); scale (large) and small; and contrast with the background (bright) and dark. Based on three dimensions and two criteria for each dimension, the full-scene dataset is divided into eight sub-datasets: dark images of dense, small-scale targets, bright images of dense, small-scale targets, dark images of dense, large-scale targets, bright images of dense, large-scale targets, dark images of sparse, small-scale targets, bright images of sparse, small-scale targets, dark images of sparse, large-scale targets, and bright images of sparse, large-scale targets. Step 2: Use each subclass dataset in the eight subclass datasets as the training set, and use neural architecture search to obtain the functional module units that implement each detection task; Step 3: Embed the functional module units that implement each detection task into the existing target detection model to obtain the embedded target detection model. Then, train the embedded target detection model using the corresponding subclass datasets and loss functions to obtain eight optimal target detection models that correspond one-to-one with the subclass datasets. The loss function ,include: ； In the formula, express and The weights between them The loss function represents the difference between the true label and the search prediction. Represents the actual label value. Indicates the first Layer feature input, Indicates the first The output of the layer prediction function This represents the network delay function, where n represents the number of feature extraction layers. Represents the set of weight parameters. Indicates the optimal parameters for training; The weight parameters of the target detection model and training optimal parameters The batch stochastic gradient descent algorithm is used for training, and the parameter update rule is as follows: ； ； In the formula, For the first The parameter value at the +1st iteration. For the first Parameter values ​​at the next iteration For the target loss against parameters The partial derivatives, For the first The parameter value at the +1st iteration. For the first Parameter values ​​at the next iteration For the target loss against parameters The partial derivatives, The learning rate; Step 4: Analyze the new input image and assign it to a specific subclass dataset. Select the best object detection model for the subclass dataset, input the new input image into the best object detection model, and output the object detection result.

2. The cross-scale adaptive target detection method based on neural architecture search according to claim 1, characterized in that: The full-scene dataset includes: Multiple open-source target detection datasets were selected; The images were captured and labeled using various camera devices at different times, angles, and weather conditions. It was obtained by searching and downloading from the Internet using a search engine and then annotating the results.

3. The cross-scale adaptive target detection method based on neural architecture search according to claim 1, characterized in that: The neural architecture search uses DARTS.

4. The cross-scale adaptive target detection method based on neural architecture search according to claim 1, characterized in that: The process of embedding the functional modules that implement each detection task into the existing target detection model to obtain the embedded target detection model includes: The YOLO series of one-stage detection models were selected as the existing target detection models. With the goal of optimizing network performance, the functional modules for each detection task are embedded into the backbone and neck structures of the existing target detection model to obtain the embedded target detection model.

5. The cross-scale adaptive target detection method based on neural architecture search according to claim 1, characterized in that: The step of judging the newly input image and assigning it to a certain subclass dataset includes: The newly input image is decomposed using a quadtree to find the correspondence between the pixel value at each position in the image and the value of its neighboring pixels. The image type label is then used to determine the degree of sparsity, scale, and contrast with the background of the target in the image. Based on the sparsity of the objects in the image, their scale, and their contrast with the background, newly input images are assigned to a specific subclass dataset.

6. A computer-readable storage medium, characterized in that: It stores a computer program that, when executed by a processor, implements a cross-scale adaptive target detection method based on neural architecture search as described in any one of claims 1-5.

7. A computer device, characterized in that: include: Memory, used to store instructions; A processor for executing the instructions, causing the computer device to perform the operation of a cross-scale adaptive target detection method based on neural architecture search as described in any one of claims 1-5.

Images