Target detection method, storage medium method, and electronic device

Abstract

This invention provides a target detection method, a storage medium method, and an electronic device. The method includes: data preprocessing to increase the number of samples; constructing a hybrid domain-based attention module and embedding it into a backbone network to filter target object features; constructing a feature enhancement module and embedding it into the lateral connection channels of a feature pyramid structure to enhance the extracted feature information; and using a detection head to process the extracted feature map to predict the category of small objects, the center coordinates of the bounding box, and its width and height. This invention effectively solves the problems of limited original features for small objects and the tendency of deep convolutional neural networks to lose feature information, thus effectively solving the problem of small target detection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to the field of target detection technology. Background Technology

[0002] Object detection is one of the fundamental tasks of computer vision. In recent years, with the continuous advancement of deep learning algorithm research and the upgrading of hardware resources, many high-performance object detectors have emerged. Currently popular object detection algorithms can generally be divided into two types: two-stage detection algorithms based on candidate regions and single-stage detection algorithms based on regression. Two-stage detection algorithms have achieved superior detection accuracy, while single-stage detection algorithms have an advantage in detection speed. Both types of object detection algorithms can detect large or medium-sized objects relatively accurately, but they still have significant limitations in recognizing small objects.

[0003] There are two main reasons for this problem: First, small objects often have very low resolution, which cannot provide enough feature information to the detection model, making it easy for existing algorithms to confuse the target with the background and noise; Second, the current state-of-the-art methods are all based on deep convolutional networks. Although the features extracted by deeper network layers have a fuller receptive field and richer high-level semantic information, a large number of features will be lost after multiple downsamplings of the object.

[0004] To address the aforementioned issues, some mainstream detection algorithms currently employ attention mechanisms to help the network learn to focus on the features of small objects. These algorithms often use global average pooling or global max pooling to extract aggregated features and then leverage fully connected layers or convolutional layers to map these features to higher dimensions. Other detection algorithms introduce a pyramid structure between the backbone network and the detection head to reuse feature information from small objects. However, in practical applications, these methods of integrating spatial and channel features are relatively coarse and struggle to accurately uncover the intrinsic relationships between feature map elements. Therefore, small object detection algorithms still require further research and improvement. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a target detection method, a storage medium method, and an electronic device to solve the problems of small objects having few original features and deep convolutional neural networks easily losing feature information.

[0006] To achieve the above and other related objectives, the present invention provides a target detection method, the method comprising: data preprocessing to expand the number of samples; constructing an attention module based on a hybrid domain and embedding it into a backbone network to filter target object features; constructing a feature enhancement module and embedding it into the lateral connection channels of a feature pyramid structure to enhance the extracted feature information; and using a detection head to process the extracted feature map to predict the category of the small object, the center coordinates of the bounding box, and its width and height.

[0007] In one embodiment of the present invention, the data preprocessing includes: performing translation transformation, horizontal flipping, color transformation, and enhancing / expanding the number of images on the target image.

[0008] In one embodiment of the present invention, the hybrid domain-based attention includes batch-normalized channel attention and layer-normalized spatial attention.

[0009] In one embodiment of the present invention, the method for predicting the category of the small object, the center coordinates of the bounding box, and the width and height is as follows: the feature map is mapped into a vector containing the object category, the center coordinates of the bounding box, and the width and height, and then output.

[0010] In one embodiment of the present invention, the enhancement module consists of four pooling cores, one of which has a fixed size, while the width and height of the other three pooling cores are variable.

[0011] In one embodiment of the present invention, the variable width and height of the pooling kernel is achieved by: performing k-means clustering on the dataset to obtain multiple anchor boxes and multiple detection heads; multiplying the anchor boxes by the sampling rate to obtain the initial pooling kernel size; rounding the initial pooling kernel size up to an odd number to obtain a set of width and height of the pooling kernel; and obtaining the three sets with the largest area occupied by each set as the three pooling kernels in the enhancement module.

[0012] In one embodiment of the present invention, the feature enhancement module generates four independent information transmission paths and produces four output vectors, and configures trainable coefficients for the four output vectors.

[0013] To achieve the above and other related objectives, the present invention also provides a storage medium storing program instructions that, when executed, implement the steps of the target detection method described above.

[0014] To achieve the above and other related objectives, the present invention also provides an electronic device, including a memory for storing a computer program and a processor for running the computer program to implement the steps of the target detection method as described above.

[0015] To achieve the above and other related objectives, the present invention also provides a mobile robot that uses the electronic equipment described above.

[0016] As described above, the target detection method, storage medium method, and electronic device of the present invention have the following beneficial effects:

[0017] This invention can effectively solve the problems of small objects having few original features and deep convolutional neural networks easily losing feature information, thus effectively solving the problem of small object detection. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 The diagram shown is a schematic representation of the overall process of a target detection method in one embodiment of this application.

[0020] Figure 2 The diagram shows a target detection process in one embodiment of this application.

[0021] Figure 3 The diagram shows a channel attention flow based on batch normalization in a target detection method according to an embodiment of this application.

[0022] Figure 4 The diagram shows a spatial attention process based on layer standardization in a target detection method according to an embodiment of this application.

[0023] Figure 5 The diagram shown is a basic structural diagram of the enhancement module in a target detection method according to an embodiment of this application.

[0024] Figure 6 The diagram shown is a schematic diagram of the pooling kernel size design in a target detection method according to an embodiment of this application.

[0025] Figure 7 The diagram shown is an adaptive output fusion schematic diagram in a target detection method according to an embodiment of this application.

[0026] Figure 8 The diagram shown is a schematic block diagram of an electronic device according to an embodiment of this application.

[0027] Component designation explanation

[0028] 101 Electronic Devices

[0029] 1001 processor

[0030] 1002 Memory

[0031] S100~S300 Steps Detailed Implementation

[0032] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0033] The purpose of this embodiment is to provide a target detection method, a storage medium method, and an electronic device to solve problems such as the lack of original features of small objects and the easy loss of feature information in deep convolutional neural networks.

[0034] The purpose of this embodiment is to creatively propose a novel small target detection algorithm to overcome the shortcomings of existing technologies. Considering that existing attention mechanisms mostly use global average pooling or global max pooling to aggregate features, this embodiment first designs channel attention based on batch normalization and spatial attention based on layer normalization to achieve a more refined attention mechanism from the perspective of data distribution. Then, a feature enhancement module is proposed. Specifically, the basic structure of the enhancement module is designed according to the divide-and-conquer design idea, and the size setting of the pooling kernel in the module is improved according to the size characteristics of small targets. Then, an adaptive output fusion strategy is designed to integrate the outputs of multiple branches of the enhancement module. Finally, the detection head is used to map the extracted features into an output vector to obtain the class probability of small targets in the image, the center coordinates of the bounding box, and the width and height.

[0035] The following will elaborate on the principles and implementation methods of the target detection method, storage medium method, and electronic device of the present invention, so that those skilled in the art can understand the target detection method, storage medium method, and electronic device of the present invention without creative effort.

[0036] Example 1

[0037] This embodiment provides a target detection method, which detects objects that occupy a small number of pixels in a target image. Specifically, as shown... Figure 1 As shown, the target detection method in this embodiment includes:

[0038] Step S100: Data preprocessing to expand the sample size;

[0039] Step S200: Construct a hybrid domain-based attention module and embed it into the backbone network to filter target object features;

[0040] Step S300: Construct a feature enhancement module and embed it into the horizontal connection channel of the feature pyramid structure to enhance the extracted feature information;

[0041] In step S400, the extracted feature map is processed using a detection head to predict the category of the small object, the center coordinates of the bounding box, and its width and height.

[0042] The object detection method in this embodiment first uses translation transformation, horizontal flipping, color transformation and enhancement to expand the number of images. Then, it proposes a novel attention mechanism based on hybrid domain and embeds it into the backbone network to filter small object features. Then, it proposes a novel feature enhancement module and embeds it into the feature pyramid structure to expand the extracted object features. Finally, it uses a detection head to process the extracted feature map to predict the category of small objects, the center coordinates of the bounding box and the width and height.

[0043] The following combination Figures 2 to 6 Steps S100 to S400 of the target detection method in this embodiment will be described in detail.

[0044] Step S100: Data preprocessing to expand the sample size.

[0045] In this embodiment, as Figure 2 As shown, before embedding the target image into the backbone network, data preprocessing is performed. The data preprocessing includes: performing translation transformation, horizontal flipping, color transformation on the target image, and enhancing and expanding the number of images. In this embodiment, Mosaic data enhancement is used to expand the number of images.

[0046] Step S200: Construct an attention module based on a hybrid domain and embed it into the backbone network to filter target object features.

[0047] In this embodiment, DarkNet53 is selected as the backbone network, that is, DarkNet53 is used as the backbone network for extracting image features. In addition, the backbone network can be selected according to specific experimental results, including but not limited to ResNet, ResNext, DarkNet, CSPDarkNet, etc.

[0048] In this embodiment, attention based on hybrid domains is embedded in the output layer of the backbone network to help the backbone network focus on the features of small objects.

[0049] In this embodiment, the attention based on hybrid domains includes channel attention based on batch normalization and spatial attention based on layer normalization.

[0050] This embodiment proposes a novel attention mechanism based on hybrid domains, including channel attention based on batch normalization and spatial attention based on layer normalization.

[0051] The hybrid domain-based attention mechanism is a hybrid attention mechanism that includes channel attention and spatial attention. Inspired by batch normalization and layer normalization algorithms, the target detection method in this embodiment designs a channel attention based on batch normalization and a spatial attention based on layer normalization, based on the data distribution relationship between spatial and channel elements.

[0052] The channel attention process based on batch normalization is described in [link to documentation]. Figure 3 As shown, each convolutional kernel in a convolutional neural network corresponds to a channel in the output feature map, representing a certain feature involved in detection. These features have different levels of importance, and channel attention calculates the proportion of each channel feature in the network based on the inter-channel relationships of the input feature map. The batch normalization algorithm is shown in formula (1): The standardization process adjusts the overall distribution of the data, while the subsequent scaling transformation targets each channel of the input data, applying the scaling factor w in the formula. b This is considered as a weight allocation between channels, thereby achieving the filtering of feature channels. Specific attention implementation formulas are shown in (1) to (3):

[0053]

[0054] w c =Sigmoid(f normalize (w b (2)

[0055] feature out =f channel_wise (w c feature in (3)

[0056] The input feature map containing c channels is copied, and one copy is fed into the batch normalization layer (BatchNorm) to extract the batch normalization scaling coefficient matrix w. b The matrix has a size of c×1. The feature... in y and μ are the batch-normalized input and output, respectively. B and σ B These are the mean and standard deviation of elements in a specific channel within a batch of data. During model training, the scaling factor w... bThis matrix is ​​continuously updated, representing the scaling factor for each channel entering the next network layer. Therefore, we directly extract this coefficient matrix, normalize it by channel, and then use the Sigmoid function to non-linearly transform the normalized matrix to obtain the weight matrix w. c This enhances the representativeness of the coefficient matrix. Finally, the matrix w... c Multiplying the input feature map by another channel-weighted feature map yields the output feature map after channel attention processing. out .

[0057] The layer-standardized spatial attention process is described below. Figure 4 As shown. Spatial attention transforms the information of the spatial dimension of the original input feature map to another spatial domain of the same size, and the scaling transformation of layer normalization is oriented towards each element of the spatial dimension of the input data. See (4) to (6) for specific formulas:

[0058]

[0059] w s =Sigmoid(f mean (f normalize (w l ))) (5)

[0060] feature out =f pixel_wise (w s feature in (6)

[0061] First, duplicate the input feature map containing C channels. Use one copy for layer normalization and extract the scaling factor matrix w of the layer normalization. l Its dimensions are C×H×W. The feature... in y and y are the input and output feature maps of the layer standardization, respectively; μ and σ are the mean and standard deviation of all channels of a sample, respectively; w l This represents the scaling factor matrix. During model training, w... l The coefficient matrix is ​​continuously updated, representing the scaling ratio of each element entering the next network layer in the spatial dimension. Therefore, this algorithm directly extracts this coefficient matrix and normalizes it along the spatial dimension to obtain a C×H×W weight matrix. Then, the average value of each pixel along the channel dimension is calculated to compress the channel dimension. The compressed matrix is ​​then encoded using the Sigmoid function to obtain the spatial attention matrix w. s The dimension is 1×H×W. Finally, w... s Multiplying the input feature map by spatially weighted pixels yields the output feature map after spatial attention encoding. out .

[0062] Step S300: Construct a feature enhancement module and embed it into the horizontal connection channel of the feature pyramid structure to enhance the extracted feature information.

[0063] This embodiment designs a feature enhancement module based on the divide-and-conquer approach, inserting it into the channel between the backbone network and the feature pyramid to optimize the transmission of feature information in the model. In this embodiment, the feature pyramid structure refers to the structure of the Feature Pyramid Network (FPN) used to further fuse and process multi-scale features. The feature enhancement module specifically includes three parts: 1) module structure design; 2) pooling kernel size design; and 3) adaptive output fusion strategy.

[0064] 1) Modular structure design

[0065] In this embodiment, the enhancement module consists of four pooling cores, one of which has a fixed size, while the width and height of the other three pooling cores are variable.

[0066] The basic structure of the enhancement module is as follows: Figure 5 The diagram shows a system consisting of four pooling kernels, one of which is fixed at 1×1 size, while the other three kernels have variable widths and heights. Pooling, rather than convolution, is chosen as the intermediate connection layer between the main body and branches to avoid adding excessive model parameters. The fixed 1×1 pooling kernel acts as an identity mapping of the input features, ensuring the model maintains its original information transmission channels after the addition of the enhancement module. The variable-size pooling kernels in the remaining three branches of this enhancement module allow it to find features more representative of small objects within a larger feature search space.

[0067] 2) Pooling core size design

[0068] In this embodiment, the variable width and height of the pooling kernel is achieved by: performing k-means clustering on the dataset to obtain multiple anchor boxes and multiple detection heads; multiplying the anchor boxes by the sampling rate to obtain the initial pooling kernel size; rounding the initial pooling kernel size up to an odd number to obtain a set of width and height of the pooling kernel; and obtaining the three sets with the largest area occupied by each set as the three pooling kernels in the enhancement module.

[0069] It should be noted that this embodiment uses the publicly available Small Object Dataset. The Small Object Dataset is a dataset specifically designed for small object detection, where the resolution of the objects to be detected does not exceed a certain limit. For example, this dataset contains 4925 images and 8393 object instances, categorized into 10 classes: "mouse," "telephone," "switch," "socket," "clock," "toilet paper," "tissue box," "faucet," "plate," and "jar."

[0070] In other words, in this embodiment, the feature enhancement module has three variable pooling kernels. This embodiment improves the size setting of the pooling kernels according to the size characteristics of small targets to reduce the introduced noise information.

[0071] First, k-means clustering is performed on the dataset, resulting in 9 anchor boxes, each corresponding to one of the 3 detectors. Here, Intersection over Union (IOU) is used instead of Euclidean distance, and the distance between the sample box and the cluster center is calculated as shown in formula (7), where IOU(box,center) represents the area of ​​overlap between the sample box and the cluster center, and d(box,center) represents the distance between them.

[0072] d(box,center)=1-IOU(box,center) (7)

[0073] Typically, k downsampling operations mean that the image resolution is reduced to the original value. Therefore, maintaining the competitiveness of small target features relative to background information is crucial. Based on the anchor boxes generated in the preceding steps, this embodiment proposes a scaling strategy to design the size of the pooling kernel in the feature enhancement module. In actual use of this feature enhancement module, it is added after a certain output layer of the backbone network. Assuming that the image has undergone i downsampling steps by the time it reaches this output layer, the three anchor boxes of the corresponding layer are multiplied by the downsampling rate. The initial pooling kernel size is obtained, and then the initial size is rounded up to an odd number to keep the size of the feature map passing through the pooling layer unchanged and to introduce richer contextual information, as shown in formulas (8) and (9), where b ih and b iw The function b represents the height and width of the cluster anchor box corresponding to the i-th layer. ih This means multiplying its parameters by... The function f2 represents rounding its parameter up to an odd number, and its result is o. i It is a set of height and width, where n refers to the number of network layers involved in the calculation, and the function f3 represents the three sizes in which each set occupies the largest area. Figure 5This is a schematic diagram of the pooling core size design in this embodiment, as shown below. Figure 6 As shown, C3, C4, and C5 represent the three output layers of the backbone network, and their corresponding downsampling rates are... and This represents the height and width of the j-th anchor box corresponding to the i-th output layer. and This represents the height and width of the selected k-th pooling kernel.

[0074] o i =f2(f1(b ih ,b iw (8)

[0075] o = f3(o1, o2, ..., o n (9)

[0076] 3) Adaptive output fusion strategy

[0077] In this embodiment, the feature enhancement module generates four independent information transmission paths and produces four output vectors, and configures trainable coefficients for the four output vectors.

[0078] In other words, in this embodiment, the feature enhancement module generates four independent information transmission paths and produces four output vectors. This embodiment introduces trainable coefficient k. i (i = 1, 2, 3, 4) are used to process these four output vectors, see Figure 7 Unlike simple element-wise addition fusion, trainable coefficients give the model more options in deciding how to focus on paths that can detect small targets. Furthermore, it can sort these output vectors according to importance, thus generating better output combinations. Compared to fixed coefficients (i.e., when averaging), Compared to traditional models, trainable coefficients also make it easier for the model to adapt to different datasets.

[0079] In step S400, the extracted feature map is processed using a detection head to predict the category of the small object, the center coordinates of the bounding box, and its width and height.

[0080] In this embodiment, the method for predicting the category, center coordinates of the bounding box, and width and height of the small object is as follows: the feature map is mapped into a vector containing the object category, center coordinates of the bounding box, and width and height, and then output.

[0081] As described above, this embodiment considers that existing attention mechanisms often employ global average pooling or global max pooling to aggregate features. First, it designs channel attention based on batch normalization and spatial attention based on layer normalization to achieve a more refined attention mechanism from the perspective of data distribution. Then, it proposes a feature enhancement module. Specifically, the basic structure of the enhancement module is designed based on the divide-and-conquer approach. The size setting of the pooling kernel in the module is improved according to the size characteristics of small targets. An adaptive output fusion strategy is then designed to integrate the outputs of multiple branches of the enhancement module. Finally, a detection head is used to map the extracted features into an output vector, obtaining the class probability of small targets in the image, the center coordinates of the bounding box, and its width and height. The target detection method in this embodiment has a simple and easy-to-understand overall framework and can effectively solve the problem of small target detection.

[0082] Example 2

[0083] like Figure 8 As shown, this embodiment provides an electronic device 101, which includes a processor 1001 and a memory 1002. The memory 1002 is used to store computer programs. The processor 1001 is used to execute the computer programs stored in the memory 1002, so that the electronic device 101 performs the steps of the target detection method as described in Embodiment 1. Since the specific implementation process of the target detection method has been described in detail in Embodiment 1, it will not be repeated here.

[0084] Processor 1001 is a Central Processing Unit (CPU). Memory 1002 is connected to processor 1001 via a system bus and communicates with it. Memory 1002 stores computer programs, and processor 1001 runs the computer programs to execute the target detection method. Memory 1002 may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0085] Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by the processor 1001, implements the steps of the target detection method described in Embodiment 1. Embodiment 1 has already provided a detailed description of the target detection method, which will not be repeated here.

[0086] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented using computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0087] In summary, this invention effectively solves the problems of limited original features for small objects and the tendency of deep convolutional neural networks to lose feature information, thus effectively addressing the problem of small object detection. Therefore, this invention effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0088] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A target detection method characterized by: The method comprises: data preprocessing to expand the sample number; constructing a mixed domain-based attention module and embedding it into the backbone network to filter the target object features; the mixed domain-based attention includes batch normalization-based channel attention and layer normalization-based spatial attention; constructing a feature enhancement module and inserting it into the channel between the backbone network and the feature pyramid to enhance the extracted feature information; the enhancement module is composed of four parallel pooling kernels, one of which is fixed in size, and the other three have variable width and height; using a detection head to process the extracted feature map to predict the class, center coordinates and width and height of the bounding box of small objects; the implementation formula of the batch normalization-based channel attention is: w c = Sigmoid(f normalize (w b )) feature out = f channel_wise (w c , feature in ) where w b is the batch-normalized scaling factor matrix, feature in and y are the batch-normalized input and output, respectively, μ B and σ B are the mean and standard deviation of the elements of a certain channel within a batch, w c is the weight matrix, feature out is the output feature map, and Sigmoid is the matrix nonlinearization function. the implementation formula of the layer normalization-based spatial attention is: w s = Sigmoid(f mean (f normalize (w l ))) feature out = f pixel_wise (w s , feature in ) where w l is the scaling coefficient matrix, feature in and y are the layer-normalized input and output feature maps, respectively, μ and σ are the element mean and standard deviation of all channels of a certain sample, w s is the spatial attention matrix, feature out is the output feature map.

2. The object detection method of claim 1, wherein: the data preprocessing includes: performing translation transformation, horizontal flip, color transformation on the target image, and performing enhancement to expand the number of pictures.

3. The object detection method of claim 1, wherein: The way to predict the class, center coordinates and width and height of the bounding box of small objects is to map the feature map into a vector containing object class, bounding box center coordinates and width and height and output.

4. The object detection method of claim 1, wherein: The way to realize the variable width and height of the pooling kernel is: perform k-means clustering on the data set to obtain multiple anchor boxes and multiple detection heads; multiply the anchor boxes by the downsampling rate to obtain the initial pooling kernel size; round up the initial pooling kernel size to an odd number to obtain the set of width and height of the pooling kernel; get the top three sets with the largest area as the three pooling kernels in the enhancement module.

5. The object detection method of claim 1, wherein: The feature enhancement module generates four independent information transmission paths and produces four output vectors, and configures trainable coefficients for the four output vectors.

6. A storage medium storing program instructions, characterized in that: The program instructions, when executed, implement the steps of the target detection method according to any one of claims 1 to 5.

7. An electronic device, comprising: It includes a memory for storing a computer program, and a processor for running the computer program to implement the steps of the target detection method according to any one of claims 1 to 5.

Images