Uniformly distributed target counting method based on dual-path feature enhancement network

Abstract

The present invention discloses a method for counting uniformly distributed targets based on a dual-path feature enhancement network. First, a method for generating uniformly distributed labels is designed to better characterize the appearance of uniformly distributed targets. Secondly, a dual-path feature enhancement network is proposed, which includes a basic network and a dual-path feature enhancement module. The dual-path feature enhancement module refines density features through supervision and online learning. The present invention proposes a method for counting the number of targets for uniformly distributed targets, which can be applied to crop yield prediction in agriculture and forestry, etc.

Description

Technical Field

[0001] The invention relates to the technical field of image processing and pattern recognition, and in particular to a uniformly distributed target counting method based on a dual-path feature enhancement network. Background Art

[0002] The statistics of target numbers can be applied to many fields, among which the most widely studied are crowd counting and wheat counting. The appearance of the targets targeted by these research directions is complex and changeable. For example, in crowd counting, the research object is the regression or positioning of the density of the human head area, but the head area can be the face, the top of the head, or the tail of the head. Secondly, different target head areas have different presentations, such as some heads wearing hats. For this reason, in crowd counting tasks, Gaussian kernels are usually used to describe the head area.

[0003] However, in high-density prediction tasks, some objects appear to be evenly distributed, such as dates, grapes, cherries, etc. Therefore, it is difficult for Gaussian kernels to better characterize these objects. Furthermore, in real scenes, the objects are noisy and the background is changeable. How to extract more robust features is still a huge challenge.

[0004] The present invention constructs a new label generation method to describe uniformly distributed targets, which can better characterize the distribution of the target area. At the same time, a dual-path feature enhancement module is proposed to better refine network features and improve the network's robust feature extraction capabilities.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The uniformly distributed target counting method based on the dual-path feature enhancement network includes the following steps:

[0007] Step S10: Obtain samples, generate uniformly distributed labels and mask labels, and divide them into a training set, a validation set, and a test set;

[0008] Step S20: Building a dual-path feature enhancement network; the dual-path feature enhancement network consists of a basic network and a dual-path feature enhancement module;

[0009] Step S30: Train the network using the established training set, use the distribution labels and mask labels as supervisory signals, optimize the network parameters, and during the training process, use the validation set to select the optimal network parameters for saving;

[0010] Step S40: Use the optimal network parameters to test the test set and obtain the sample counting results.

[0011] Preferably, the step S10 includes:

[0012] Step S10-1: Use LabelMe annotation software to mark the targets in the sample one by one to obtain sample annotations;

[0013] Step S10-2: Convert the sample annotations of step S10-1 into uniformly distributed labels;

[0014] Step S10-3: converting the uniformly distributed labels of step S10-2 into mask labels;

[0015] Preferably, in step S10-2, the uniform distribution labels are converted, and the calculation formula is as follows:

[0016]

[0017] In the formula, x∈i 2 Represents the coordinates of a given image, p j represents the jth labeled target position (l x j ,l y j ), d i is the set of labeled points corresponding to the sample, δ represents the field given by the current target position, which is an odd number by default, and x(δ) represents the labeled target position p j The corresponding set of pixel points in the δ neighborhood;

[0018] Preferably, in step S10-3, the uniformly distributed labels are converted into mask labels, and the calculation formula is as follows:

[0019]

[0020] Where Dm i (y) represents the value of the pixel at coordinate y in the uniformly distributed label, M i (y) represents the value of the pixel at coordinate y in the mask label;

[0021] Preferably, in step S20, the base network uses the first 10 layers of VGG 16, wherein the parameters of these 10 layers of convolution are initialized using parameters pre-trained on the ImageNet dataset;

[0022] Preferably, the dual-path feature enhancement module in step S20 includes the following steps:

[0023] Step S20-1: Transform the features obtained by the basic network through the shared convolutional layer to obtain the feature tensor F share ;

[0024] Step S20-2: Output F of step S20-1 share Send it to the convolutional layer to obtain the mask estimation output K out, supervised learning is performed through the mask label obtained in step S10-3;

[0025] Step S20-3: The output M of step S20-2 is out , generate mask feature representation F through convolutional learning M ;

[0026] Step S20-4: Output F of step S20-1 share , the single-channel spatial attention F is generated through the convolutional learning operation att ;

[0027] Step S20-5: The features obtained by the basic network are transformed through a series of convolutional layers to obtain the feature tensor F p ;

[0028] Step S20-6: The outputs of S20-3, S20-4 and S20-5 are enhanced by the following formula:

[0029] Dm p =F att *g(cat(F p ,F M ))+g(cat(F p ,F M ))

[0030] In the formula, g(·) represents the convolution operation, cat(·,·) represents the concatenation along the channel dimension;

[0031] Preferably, the loss function of supervised learning by mask labels in step S20-2 is specifically expressed as follows:

[0032]

[0033] Where λ1 is a hyperparameter;

[0034] Preferably, the step S20-6 performs learning of uniformly distributed labels through mean square error loss, and the loss function is specifically expressed as follows:

[0035]

[0036] Where λ2 is a hyperparameter;

[0037] Preferably, in step S30, the parameters of the network are optimized, and the network parameters are learned by an Adam optimizer;

[0038] Preferably, the optimal network parameters are selected and saved by using the validation set in step S30 by calculating the mean absolute error between the network output and the true label, and selecting the network parameters with the smallest mean absolute error for saving.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. The present invention designs a new label generation method to characterize uniformly distributed targets. The generation method is simple, but can better characterize the target area;

[0041] 2. The present invention designs a dual-path feature enhancement module, generates refined regions of interest through supervision and online learning, and improves the performance of network prediction by sequential refining. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a framework diagram of the deep network model in the present invention. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0044] Reference Figure 1 , a uniformly distributed target counting method based on a dual-path feature enhancement network, comprising the following steps:

[0045] Step S10: Obtain samples, generate uniformly distributed labels and mask labels, and divide them into a training set, a validation set, and a test set;

[0046] Step S20: Building a dual-path feature enhancement network; the dual-path feature enhancement network consists of a basic network and a dual-path feature enhancement module;

[0047] Step S30: Train the network using the established training set, use the distribution labels and mask labels as supervisory signals, optimize the network parameters, and during the training process, use the validation set to select the optimal network parameters for saving;

[0048] Step S40: Use the optimal network parameters to test the test set and obtain the sample counting results.

[0049] Preferably, the step S10 includes:

[0050] Step S10-1: Use LabelMe labeling software to label the targets in the sample one by one to obtain sample labels;

[0051] Step S10-2: Convert the sample annotations of step S10-1 into uniformly distributed labels;

[0052] Step S10-3: converting the uniformly distributed labels of step S10-2 into mask labels;

[0053] Preferably, in step S10-2, the uniform distribution labels are converted, and the calculation formula is as follows:

[0054]

[0055] In the formula, x∈i 2 Represents the coordinates of a given image, p j represents the jth labeled target position (l x j ,l y j ), d i is the set of labeled points corresponding to the sample, δ represents the area given by the current target position, and the default value is an odd number. Indicates the target position p j The corresponding set of pixel points in the δ neighborhood;

[0056] Preferably, in step S10-3, the uniformly distributed labels are converted into mask labels, and the calculation formula is as follows:

[0057]

[0058] Where Dm i (y) represents the value of the pixel at coordinate y in the uniformly distributed label, M i (y) represents the value of the pixel at coordinate y in the mask label;

[0059] Preferably, in step S20, the base network uses the first 10 layers of VGG 16, wherein the parameters of these 10 layers of convolution are initialized using parameters pre-trained on the ImageNet dataset;

[0060] Preferably, the dual-path feature enhancement module in step S20 includes the following steps:

[0061] Step S20-1: Transform the features obtained by the basic network through the shared convolutional layer to obtain the feature tensor F share ;

[0062] Step S20-2: Output F of step S20-1 share Send it to the convolutional layer to obtain the mask estimation output K out , supervised learning is performed through the mask label obtained in step S10-3;

[0063] Step S20-3: The output M of step S20-2 is out , generate mask feature representation F through convolutional learning M ;

[0064] Step S20-4: Output F of step S20-1 share, the single-channel spatial attention F is generated through the convolutional learning operation att ;

[0065] Step S20-5: The features obtained by the basic network are transformed through a series of convolutional layers to obtain the feature tensor F p ;

[0066] Step S20-6: The outputs of S20-3, S20-4 and S20-5 are enhanced by the following formula:

[0067] Dm p =F att *g(cat(F p ,F M ))+g(cat(F p ,F M ))

[0068] In the formula, g(·) represents the convolution operation, cat(·,·) represents the concatenation along the channel dimension;

[0069] Preferably, the loss function of supervised learning by mask labels in step S20-2 is specifically expressed as follows:

[0070]

[0071] Where λ1 is a hyperparameter;

[0072] Preferably, the step S20-6 performs learning of uniformly distributed labels through mean square error loss, and the loss function is specifically expressed as follows:

[0073]

[0074] Where λ2 is a hyperparameter;

[0075] Preferably, in step S30, the parameters of the network are optimized, and the network parameters are learned by an Adam optimizer;

[0076] Preferably, the optimal network parameters are selected and saved by using the validation set in step S30 by calculating the mean absolute error between the network output and the true label, and selecting the network parameters with the smallest mean absolute error for saving;

[0077] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A uniformly distributed target counting method based on a dual-path feature enhancement network, characterized in that: The method comprises the following steps: Step S10: Obtain samples, generate uniformly distributed labels and mask labels, and divide the data into a training set, a validation set, and a test set; Step S20: Building a dual-path feature enhancement network; the dual-path feature enhancement network consists of a basic network and a dual-path feature enhancement module; Step S30: Train the network using the established training set, use the distribution labels and mask labels as supervisory signals, optimize the network parameters, and during the training process, use the validation set to select the optimal network parameters for saving; Step S40: Use the optimal network parameters to test the test set and obtain the sample counting results; In step S20, the base network uses the first 10 layers of VGG 16, and the parameters of these 10 layers of convolution are initialized using parameters pre-trained on the ImageNet dataset; The dual-path feature enhancement module in step S20 is specifically as follows: Step S20-1: Transform the features obtained by the basic network through the shared convolutional layer to obtain the feature tensor F share ; Step S20-2: Output F of step S20-1 share Send it to the convolutional layer to obtain the mask estimation output K out , supervised learning is performed through the mask label obtained in step S10-3; Step S20-3: The output M of step S20-2 is out , generate mask feature representation F through convolutional learning M ; Step S20-4: Output F of step S20-1 share , the single-channel spatial attention F is generated through the convolutional learning operation att ; Step S20-5: The features obtained by the basic network are transformed through a series of convolutional layers to obtain the feature tensor F p ; Step S20-6: The outputs of S20-3, S20-4 and S20-5 are enhanced by the following formula: Dm p =F att *g(cat(F p ,F M ))+g(cat(F p ,F M )) Where g(·) represents the convolution operation and cat(·,·) represents the concatenation along the channel dimension.

2. The uniformly distributed target counting method based on a dual-path feature enhancement network according to claim 1 is characterized in that: In step S10, the details are as follows: Step S10-1: Use annotation software to mark the targets in the sample one by one to obtain sample annotations; Step S10-2: Convert the sample annotations of step S10-1 into uniformly distributed labels; Step S10 - 3 : Convert the uniformly distributed labels of step S10 - 2 into mask labels.

3. The uniformly distributed target counting method based on a dual-path feature enhancement network according to claim 2 is characterized in that: In step S10-2, the uniform distribution labels are converted, and the calculation formula is as follows: In the formula, Represents the coordinates of a given image, p j Indicates the jth labeled target position d i is the set of labeled points corresponding to the sample, δ represents the area given by the current target position, and the default value is an odd number. Indicates the target position p j The corresponding set of pixel points in the δ neighborhood.

4. The uniformly distributed target counting method based on a dual-path feature enhancement network according to claim 2 is characterized in that: In step S10-3, the uniformly distributed labels are converted into mask labels, and the calculation formula is as follows: Where Dm i (y) represents the value of the pixel at coordinate y in the uniformly distributed label, M i (y) represents the value of the pixel at coordinate y in the mask label.

5. The uniformly distributed target counting method based on a dual-path feature enhancement network according to claim 4 is characterized in that: Step S20-2 performs supervised learning loss via mask labels: Where λ1 is a hyperparameter.

6. The uniformly distributed target counting method based on a dual-path feature enhancement network according to claim 5 is characterized in that: Step S20-6 performs learning of uniformly distributed labels through mean square error loss, and the loss is given as follows: Where λ2 is a hyperparameter.

7. The uniformly distributed target counting method based on a dual-path feature enhancement network according to claim 1 is characterized in that: In step S30, the parameters of the network are optimized and the network parameters are learned by the Adam optimizer.

8. The uniformly distributed target counting method based on a dual-path feature enhancement network according to claim 1 is characterized in that: In step S30, the optimal network parameters are selected and saved using the validation set by calculating the mean absolute error between the network output and the true label, and the network parameters with the smallest mean absolute error are selected and saved.

Images