A method and device for counting people in images based on contextual attention

Abstract

The present invention discloses a method and device for counting people in an image based on contextual attention. The present invention uses part of the convolutional layer of VGG16 as the front-end network, generates four 64-channel density maps in the mid-end network and the back-end network, and introduces context feature sampling to form a 4-channel coefficient feature map. The four 64-channel density maps are further convolved and stacked in the fusion network to obtain a 4-channel intermediate density map. The intermediate density map and the coefficient feature map after Sigmoid or Softmax operation are multiplied pixel by pixel and fused to obtain the final crowd density map, and finally the number of people is obtained according to the integral accumulation of the crowd density map. Compared with the head counting based on multi-column neural network, the present invention can improve the counting efficiency to a certain extent, and compared with the existing counting, the present invention has a significant improvement in accuracy.

Description

Technical Field

[0001] The present invention relates to a head counting technology for images, and in particular to a head counting technology for images based on deep learning of convolutional neural networks. Background Art

[0002] Patent document CN 112651390 A discloses a method and device for counting people in pictures based on convolutional neural networks. The method for counting people in pictures based on convolutional neural networks uses a single-column convolutional neural network. There are many methods for counting people in pictures based on multi-column convolutional neural networks. Compared with these methods for counting people in pictures based on multi-column convolutional neural networks, the above-mentioned method for counting people in pictures based on a single-column convolutional neural network can effectively reduce the amount of model parameters, thereby improving the counting efficiency. At the same time, this method has a higher accuracy rate when counting people in pictures under general density scenarios. However, this method has the problem of poor accuracy and large error in counting people in pictures under high-density crowd scenarios. Summary of the invention

[0003] The problem to be solved by the present invention is to improve the accuracy, stability and efficiency of picture head counting.

[0004] To solve the above problems, the solution adopted by the present invention is as follows:

[0005] According to a method for counting people in an image based on contextual attention of the present invention, the method comprises a model calculation step; the model calculation step is used to calculate a crowd density map through a contextual attention network model;

[0006] The contextual attention network model includes: a front-end network, a mid-end network, a back-end network and a fusion network;

[0007] The front-end network, the mid-end network and the back-end network respectively include five convolutional layer groups consisting of convolutional layers; the convolutional layer group includes a plurality of convolutional layers;

[0008] The fusion network includes a channel dimension layer, a fusion stacking layer and a fusion product layer;

[0009] In the front-end network, five convolutional layer groups are sequentially connected in series through a pooling layer, and the pooling layer gradually halves the size of the image output by the five convolutional layer groups, and the output of the last convolutional layer group is connected to the first convolutional layer group in the mid-end network;

[0010] In the mid-end network, the outputs of the preceding convolutional layer groups are doubled in image size through the doubling layer, and the convolutional layer groups with the same image size as the outputs of the front-end network are stacked in the channel dimension through the stacking layer and connected to the subsequent convolutional layer groups. The outputs of the five convolutional layer groups are respectively connected to the five convolutional layer groups of the back-end network; the stacking layer is used to stack the input images in the channel dimension;

[0011] In the back-end network, the convolution layer group whose input is connected to the last-order convolution layer group of the mid-end network outputs a feature map with the same image size as the input image and the number of channels as 4, and the output is connected to the channel dimension layer of the fusion network; the other four convolution layer groups are each interleaved with a different number of doubling layers, so that the four convolution layer groups output feature maps with the same image size as the input image and the number of channels as 64, and the outputs of the four convolution layer groups are respectively connected to the fusion stacking layer of the fusion network through the convolution layer RR;

[0012] The convolution layer RR is used to process the feature map with the same image size as the input image and the number of channels as 64, and then output the feature map with the same image size as the input image and the number of channels as 1; the convolution kernel size of the convolution layer RR is 1*1, the padding is 0, and the step size is 1;

[0013] The fusion stacking layer is used to stack four feature maps with a channel number of 1 in dimension, and the output image size is the same as the input image and the channel number is 4.

[0014] The channel dimension layer is used to perform Sigmoid or Softmax operations on the input feature map to generate a coefficient feature map with 4 channels and the same image size as the input image;

[0015] The input of the fused multiplication layer is connected to the output of the channel dimension layer and the fused stacking layer, and is used to multiply the coefficient feature map and the fused feature map pixel by pixel, so as to obtain a crowd density map with 4 channels and the same image size as the input image;

[0016] Each convolutional layer of the front-end network, mid-end network and back-end network is equipped with an activation function ReLU.

[0017] Further, according to the method of image head counting based on contextual attention of the present invention,

[0018] In the front-end network, the five convolutional layer groups are respectively marked as convolutional layer group RF1, convolutional layer group RF2, convolutional layer group RF3, convolutional layer group RF4 and convolutional layer group RF5; the convolutional layer group RF1 includes convolutional layer RF11 and convolutional layer RF12 connected in series; the convolutional layer group RF2 includes convolutional layer RF21 and convolutional layer RF22 connected in series; the convolutional layer group RF3 includes convolutional layer RF31, convolutional layer RF32 and convolutional layer RF33 connected in series; the convolutional layer group RF4 includes convolutional layer RF41, convolutional layer RF42 and convolutional layer RF43 connected in series; the convolutional layer group RF5 includes convolutional layer RF51, convolutional layer RF52 and convolutional layer RF 53; the convolution kernel size of convolution layer RF11, convolution layer RF12, convolution layer RF21, convolution layer RF22, convolution layer RF31, convolution layer RF32, convolution layer RF33, convolution layer RF41, convolution layer RF42, convolution layer RF43, convolution layer RF51, convolution layer RF52, and convolution layer RF53 is 3*3, the step size is 1, the padding is 1, the number of input channels is: 3, 64, 64, 128, 128, 256, 256, 256, 512, 512, 512, 512, 512, and the number of output channels is: 64, 64, 128, 128, 256, 256, 256, 512, 512, 512, 512, 512, 512;

[0019] The four pooling layers connecting the five convolutional layer groups of the front-end network all use a pooling layer with a pooling kernel size of 2*2, a step size of 2, a padding of 0, and a maximum pooling;

[0020] In the mid-end network, the five convolutional layer groups are respectively marked as convolutional layer group RM1, convolutional layer group RM2, convolutional layer group RM3, convolutional layer group RM4 and convolutional layer group RM5; the convolutional layer group RM1 includes convolutional layer RM11 and convolutional layer RM12 connected in series; the convolutional layer group RM2 includes convolutional layer RM21 and convolutional layer RM22 connected in series; the convolutional layer group RM3 includes convolutional layer RM31 and convolutional layer RM32 connected in series; the convolutional layer group RM4 includes convolutional layer RM41 and convolutional layer RM42 connected in series; the convolutional layer group RM5 includes convolutional layer RM51 and convolutional layer RM52 connected in series. 2; The convolution kernel size of convolution layer RM11, convolution layer RM12, convolution layer RM21, convolution layer RM22, convolution layer RM31, convolution layer RM32, convolution layer RM41, convolution layer RM42, convolution layer RM51, and convolution layer RM52 is 3*3, the step size is 1, the padding is 1, the number of input channels is: 512, 1024, 1024, 512, 512, 256, 256, 128, 128, 64, and the number of output channels is: 1024, 512, 512, 256, 256, 128, 128, 64, 64, 64; the doubling layer uses bilinear interpolation to double the image size;

[0021] In the back-end network, each convolutional layer group includes four convolutional layers; the five convolutional layer groups are respectively marked as convolutional layer group RE1, convolutional layer group RE2, convolutional layer group RE3, convolutional layer group RE4 and convolutional layer group RE5; the convolutional layer group RE1 includes a quadruple layer KE11, a convolutional layer RE11, a convolutional layer RE12, a doubling layer KE12, a convolutional layer RE13, a convolutional layer RE14 and a doubling layer KE13 connected in series in sequence; the convolutional layer group RE2 includes a doubling layer KE21, a convolutional layer RE21, a convolutional layer RE22, a doubling layer KE22, a convolutional layer RE23, a convolutional layer R E24 and doubling layer KE23; the convolution layer group RE3 includes doubling layer KE31, convolution layer RE31, convolution layer RE32, doubling layer KE32, convolution layer RE33 and convolution layer RE34 connected in series in sequence; the convolution layer group RE4 includes doubling layer KE41, convolution layer RE41, convolution layer RE42, convolution layer RE43 and convolution layer RE44 connected in series in sequence; the convolution layer group RE5 includes convolution layer RE51, convolution layer RE52, convolution layer RE53 and convolution layer RE54 connected in series in sequence; convolution layer RE11, convolution layer RE12, convolution layer RE 13. The convolution kernel size of convolution layer RE14, convolution layer RE21, convolution layer RE22, convolution layer RE23, convolution layer RE24, convolution layer RE31, convolution layer RE32, convolution layer RE33, convolution layer RE34, convolution layer RE41, convolution layer RE42, convolution layer RE43, convolution layer RE44, convolution layer RE51, convolution layer RE52, convolution layer RE53, and convolution layer RE54 is 3*3, the step size is 1, and the number of input channels is 512, 512, 256, 128, 256, 256, 256, 128, 128, and 128 respectively. , 128, 128, 64, 64, 64, 64, 64, 64, 32, 16, and the number of output channels are 512, 256, 128, 64, 256, 256, 128, 64, 128, 128, 64, 64, 64, 64, 64, 64, 32, 16, 4 respectively; among them, convolutional layer RE12, convolutional layer RE14, convolutional layer RE22, convolutional layer RE24, convolutional layer RE32, convolutional layer RE34, convolutional layer RE42 and convolutional layer RE44 are all convolutional layers with a void rate of 2 and a padding of 2, and the other convolutional layers have a padding of 1;

[0022] The doubling layer KE12, the doubling layer KE13, the doubling layer KE21, the doubling layer KE22, the doubling layer KE23, the doubling layer KE31, the doubling layer KE32, and the doubling layer KE41 are used to double the image size by bilinear interpolation;

[0023] The quadruple layer KE11 is used to quadruple the image size by bilinear interpolation.

[0024] Further, according to the method for counting people in an image based on contextual attention of the present invention, the method further comprises a model initialization step, a data initialization step, an image preprocessing step and a head counting step;

[0025] The model initialization step is used to: initialize the contextual attention network model;

[0026] The data initialization step is used to: obtain the model feature matrix data obtained by training the contextual attention network model, and then load the model feature matrix data into the contextual attention network model;

[0027] The image preprocessing step is used to: trim the input image size by cutting off the edges so that the length and width of the image are both multiples of 16, and standardize the pixel values ​​on three channels with means of 0.485, 0.456, 0.406 and standard deviations of 0.229, 0.224, 0.225 so that the image is normalized to the interval [0, 1] on the three channels to obtain a preprocessed image FP;

[0028] The head counting step is used to: input the pre-processed image FP into the contextual attention network model after the data is initialized, and obtain a crowd density map through calculation by the contextual attention network model; then obtain the number of people in the input picture by integrating and accumulating the crowd density map;

[0029] The following formula is used for the integration accumulation through the crowd density map:

[0030] ;

[0031] Among them, Sum is the total number of people in the picture, P(xi) represents the pixel value of the i-th pixel in the crowd density map, and M is the number of pixels in the crowd density map.

[0032] Furthermore, according to the method for counting head numbers in images based on contextual attention of the present invention, the method also includes a step of obtaining a training data set and a model training step; the model training step is used to input the images in the training data set into the contextual attention network model for model calculation after the image preprocessing step, so as to obtain the model feature matrix data.

[0033] Further, according to the method for counting people in images based on contextual attention of the present invention, the following loss function is used in the model training step to evaluate the model feature matrix data:

[0034] ;

[0035] Wherein, θ is the model feature matrix data, Ii is the i-th picture in the training data set, Dp(Ii, θ) is the crowd density map obtained by the head counting step for the i-th picture in the training data set, Dgi is the expected density map obtained by manual editing for the i-th picture in the training data set, and N is the number of pictures in the training data set.

[0036] According to the present invention, a device for counting people in an image based on contextual attention includes a model calculation module; the model calculation module is used to calculate a crowd density map through a contextual attention network model;

[0037] The contextual attention network model includes: a front-end network, a mid-end network, a back-end network and a fusion network;

[0038] The front-end network, the mid-end network and the back-end network respectively include five convolutional layer groups consisting of convolutional layers; the convolutional layer group includes a plurality of convolutional layers;

[0039] The fusion network includes a channel dimension layer, a fusion stacking layer and a fusion product layer;

[0040] In the front-end network, five convolutional layer groups are sequentially connected in series through a pooling layer, and the pooling layer gradually halves the size of the image output by the five convolutional layer groups, and the output of the last convolutional layer group is connected to the first convolutional layer group in the mid-end network;

[0041] In the mid-end network, the outputs of the preceding convolutional layer groups are doubled in image size through the doubling layer, and the convolutional layer groups with the same image size as the outputs of the front-end network are stacked in the channel dimension through the stacking layer and connected to the subsequent convolutional layer groups. The outputs of the five convolutional layer groups are respectively connected to the five convolutional layer groups of the back-end network; the stacking layer is used to stack the input images in the channel dimension;

[0042] In the back-end network, the convolution layer group whose input is connected to the last-order convolution layer group of the mid-end network outputs a feature map with the same image size as the input image and the number of channels as 4, and the output is connected to the channel dimension layer of the fusion network; the other four convolution layer groups are each interleaved with a different number of doubling layers, so that the four convolution layer groups output feature maps with the same image size as the input image and the number of channels as 64, and the outputs of the four convolution layer groups are respectively connected to the fusion stacking layer of the fusion network through the convolution layer RR;

[0043] The convolution layer RR is used to process the feature map with the same image size as the input image and the number of channels as 64, and then output the feature map with the same image size as the input image and the number of channels as 1; the convolution kernel size of the convolution layer RR is 1*1, the padding is 0, and the step size is 1;

[0044] The fusion stacking layer is used to stack four feature maps with a channel number of 1 in dimension, and the output image size is the same as the input image and the channel number is 4.

[0045] The channel dimension layer is used to perform Sigmoid or Softmax operations on the input feature map to generate a coefficient feature map with 4 channels and the same image size as the input image;

[0046] The input of the fused multiplication layer is connected to the output of the channel dimension layer and the fused stacking layer, and is used to multiply the coefficient feature map and the fused feature map pixel by pixel, so as to obtain a crowd density map with 4 channels and the same image size as the input image;

[0047] Each convolutional layer of the front-end network, mid-end network and back-end network is equipped with an activation function ReLU.

[0048] Further, according to the apparatus for counting people in pictures based on contextual attention of the present invention,

[0049] In the front-end network, the five convolutional layer groups are respectively marked as convolutional layer group RF1, convolutional layer group RF2, convolutional layer group RF3, convolutional layer group RF4 and convolutional layer group RF5; the convolutional layer group RF1 includes convolutional layer RF11 and convolutional layer RF12 connected in series; the convolutional layer group RF2 includes convolutional layer RF21 and convolutional layer RF22 connected in series; the convolutional layer group RF3 includes convolutional layer RF31, convolutional layer RF32 and convolutional layer RF33 connected in series; the convolutional layer group RF4 includes convolutional layer RF41, convolutional layer RF42 and convolutional layer RF43 connected in series; the convolutional layer group RF5 includes convolutional layer RF51, convolutional layer RF52 and convolutional layer RF 53; the convolution kernel size of convolution layer RF11, convolution layer RF12, convolution layer RF21, convolution layer RF22, convolution layer RF31, convolution layer RF32, convolution layer RF33, convolution layer RF41, convolution layer RF42, convolution layer RF43, convolution layer RF51, convolution layer RF52, and convolution layer RF53 is 3*3, the step size is 1, the padding is 1, the number of input channels is: 3, 64, 64, 128, 128, 256, 256, 256, 512, 512, 512, 512, 512, and the number of output channels is: 64, 64, 128, 128, 256, 256, 256, 512, 512, 512, 512, 512, 512;

[0050] The four pooling layers connecting the five convolutional layer groups of the front-end network all use a pooling layer with a pooling kernel size of 2*2, a step size of 2, a padding of 0, and a maximum pooling;

[0051] In the mid-end network, the five convolutional layer groups are respectively marked as convolutional layer group RM1, convolutional layer group RM2, convolutional layer group RM3, convolutional layer group RM4 and convolutional layer group RM5; the convolutional layer group RM1 includes convolutional layer RM11 and convolutional layer RM12 connected in series; the convolutional layer group RM2 includes convolutional layer RM21 and convolutional layer RM22 connected in series; the convolutional layer group RM3 includes convolutional layer RM31 and convolutional layer RM32 connected in series; the convolutional layer group RM4 includes convolutional layer RM41 and convolutional layer RM42 connected in series; the convolutional layer group RM5 includes convolutional layer RM51 and convolutional layer RM52 connected in series. 2; The convolution kernel size of convolution layer RM11, convolution layer RM12, convolution layer RM21, convolution layer RM22, convolution layer RM31, convolution layer RM32, convolution layer RM41, convolution layer RM42, convolution layer RM51, and convolution layer RM52 is 3*3, the step size is 1, the padding is 1, the number of input channels is: 512, 1024, 1024, 512, 512, 256, 256, 128, 128, 64, and the number of output channels is: 1024, 512, 512, 256, 256, 128, 128, 64, 64, 64; the doubling layer uses bilinear interpolation to double the image size;

[0052] In the back-end network, each convolutional layer group includes four convolutional layers; the five convolutional layer groups are respectively marked as convolutional layer group RE1, convolutional layer group RE2, convolutional layer group RE3, convolutional layer group RE4 and convolutional layer group RE5; the convolutional layer group RE1 includes a quadruple layer KE11, a convolutional layer RE11, a convolutional layer RE12, a doubling layer KE12, a convolutional layer RE13, a convolutional layer RE14 and a doubling layer KE13 connected in series in sequence; the convolutional layer group RE2 includes a doubling layer KE21, a convolutional layer RE21, a convolutional layer RE22, a doubling layer KE22, a convolutional layer RE23, a convolutional layer R E24 and doubling layer KE23; the convolution layer group RE3 includes doubling layer KE31, convolution layer RE31, convolution layer RE32, doubling layer KE32, convolution layer RE33 and convolution layer RE34 connected in series in sequence; the convolution layer group RE4 includes doubling layer KE41, convolution layer RE41, convolution layer RE42, convolution layer RE43 and convolution layer RE44 connected in series in sequence; the convolution layer group RE5 includes convolution layer RE51, convolution layer RE52, convolution layer RE53 and convolution layer RE54 connected in series in sequence; convolution layer RE11, convolution layer RE12, convolution layer RE 13. The convolution kernel size of convolution layer RE14, convolution layer RE21, convolution layer RE22, convolution layer RE23, convolution layer RE24, convolution layer RE31, convolution layer RE32, convolution layer RE33, convolution layer RE34, convolution layer RE41, convolution layer RE42, convolution layer RE43, convolution layer RE44, convolution layer RE51, convolution layer RE52, convolution layer RE53, and convolution layer RE54 is 3*3, the step size is 1, and the number of input channels is 512, 512, 256, 128, 256, 256, 256, 128, 128, and 128 respectively. , 128, 128, 64, 64, 64, 64, 64, 64, 32, 16, and the number of output channels are 512, 256, 128, 64, 256, 256, 128, 64, 128, 128, 64, 64, 64, 64, 64, 64, 32, 16, 4 respectively; among them, convolutional layer RE12, convolutional layer RE14, convolutional layer RE22, convolutional layer RE24, convolutional layer RE32, convolutional layer RE34, convolutional layer RE42 and convolutional layer RE44 are all convolutional layers with a void rate of 2 and a padding of 2, and the other convolutional layers have a padding of 1;

[0053] The doubling layer KE12, the doubling layer KE13, the doubling layer KE21, the doubling layer KE22, the doubling layer KE23, the doubling layer KE31, the doubling layer KE32, and the doubling layer KE41 are used to double the image size by bilinear interpolation;

[0054] The quadruple layer KE11 is used to quadruple the image size by bilinear interpolation.

[0055] Further, according to the apparatus for counting people in images based on contextual attention of the present invention, the apparatus further includes a model initialization module, a data initialization module, an image preprocessing module and a head counting module;

[0056] The model initialization module is used to: initialize the contextual attention network model;

[0057] The data initialization module is used to: obtain the model feature matrix data obtained by training the contextual attention network model, and then load the model feature matrix data into the contextual attention network model;

[0058] The image preprocessing module is used to: trim the input image size by cutting off the edges so that the length and width of the image are both multiples of 16, and standardize the pixel values ​​on three channels with means of 0.485, 0.456, 0.406 and standard deviations of 0.229, 0.224, 0.225 so that the image is normalized to the interval [0, 1] on the three channels to obtain a preprocessed image FP;

[0059] The head counting module is used to: input the pre-processed image FP into the contextual attention network model after the data is initialized, and obtain a crowd density map through calculation by the contextual attention network model; and then obtain the number of people in the input picture by integrating and accumulating the crowd density map;

[0060] The following formula is used for the integration accumulation through the crowd density map:

[0061] ;

[0062] Among them, Sum is the total number of people in the picture, P(xi) represents the pixel value of the i-th pixel in the crowd density map, and M is the number of pixels in the crowd density map.

[0063] Furthermore, according to the device for counting head numbers in images based on contextual attention of the present invention, the device also includes a module for acquiring a training data set and a model training module; the model training module is used to input the images in the training data set into the contextual attention network model for model calculation after passing through the image preprocessing module to obtain the model feature matrix data.

[0064] Further, according to the apparatus for counting people in images based on contextual attention of the present invention, the model training module uses the following loss function to evaluate the model feature matrix data:

[0065] ;

[0066] Wherein, θ is the model feature matrix data, Ii is the i-th picture in the training data set, Dp(Ii, θ) is the crowd density map obtained by the head counting module for the i-th picture in the training data set, Dgi is the expected density map obtained by manual editing for the i-th picture in the training data set, and N is the number of pictures in the training data set.

[0067] The technical effects of the present invention are as follows:

[0068] 1. Compared with the existing image head counting method based on multi-column convolutional neural network, the image head counting method using contextual attention in the present invention can effectively reduce the amount of model parameters, thereby improving the efficiency of image head counting.

[0069] 2. Compared with the existing technology, whether it is the existing image head counting method based on multi-column convolutional neural network or the existing image head counting method based on single-column convolutional neural network, the method of the present invention has a certain degree of improvement in accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a schematic diagram of the overall structure of the contextual attention network model of an embodiment of the present invention.

[0071] Figure 2 It is a structural diagram of the front-end network of the contextual attention network model of an embodiment of the present invention.

[0072] Figure 3 It is a structural diagram of the mid-end network of the contextual attention network model of an embodiment of the present invention.

[0073] Figure 4 It is a structural diagram of the backend network of the contextual attention network model of an embodiment of the present invention.

[0074] Figure 5 It is a schematic diagram of the overall structure of the pedestrian flow analysis system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The present invention is further described in detail below with reference to the accompanying drawings.

[0076] This embodiment relates to a human flow analysis system, such as Figure 5 As shown, the system includes a server 100 arranged in a computer room and a plurality of front-end cameras 200 arranged at monitoring points. The front-end cameras 200 are connected to the server 100 via a network 300. The server 100 obtains a real-time image of the scene at the monitoring point through the front-end camera 200, and then analyzes the obtained real-time image through a method of counting people in an image based on contextual attention of the present invention, counts the number of people, and then calculates the flow of people.

[0077] The method for counting people in images based on contextual attention of the present invention is a method implemented by executing a computer software program by the server 100. The method comprises the following steps: a model initialization step, a data initialization step, a picture preprocessing step and a head counting step. Among them, the model initialization step is used to initialize the contextual attention network model. The data initialization step is used to obtain the model feature matrix data obtained by pre-training the contextual attention network model, and then load the model feature matrix data into the contextual attention network model. The picture preprocessing step is used to trim the input picture size by edge cutting so that the length and width of the picture are both multiples of 16, and standardize the pixel values ​​on three channels with mean 0.485, 0.456, 0.406 and standard deviation 0.229, 0.224, 0.225, so that the image is normalized to the interval [0,1] on three channels to obtain the preprocessed image FP. The head counting step is used to input the preprocessed image FP into the contextual attention network model after data initialization, and the crowd density map is calculated by the contextual attention network model; then the crowd density map is integrated and accumulated to obtain the number of people in the input image.

[0078] Among them, the "input image" in the image preprocessing step is the image. "Trimming the input image size by edge cutting so that the length and width of the image are both multiples of 16" is expressed in mathematical formulas: Wo=16*int(Wi / 16), Ho=16*int(Hi / 16). Wherein Wi and Hi are the width and height of the input image, Wo and Ho are the width and height of the image after edge cutting, and int is a rounding function. The three channels in "standardized pixel values ​​on three channels" generally refer to the RGB channels of the image or picture. In the head counting step, "inputting the preprocessed image FP into the contextual attention network model after data initialization, and calculating the crowd density map through the contextual attention network model" is the model calculation step referred to in the present invention. The model calculation step of the present invention calculates the crowd density map through the contextual attention network model. The contextual attention network model of the present invention is a model constructed in a convolutional neural network model platform system, and the convolutional neural network model platform system provides interfaces such as model initialization, model training, model feature matrix data output, model feature matrix data loading, and model calculation. The aforementioned model initialization steps and data initialization steps are implemented by calling the interface provided by the convolutional neural network model platform system.

[0079] like Figure 1As shown, the contextual attention network model of the present invention includes: a front-end network, a mid-end network, a back-end network and a fusion network. Among them, the front-end network, the mid-end network and the back-end network respectively include five convolutional layer groups composed of convolutional layers. The five convolutional layer groups in the front-end network are convolutional layer group RF1, convolutional layer group RF2, convolutional layer group RF3, convolutional layer group RF4 and convolutional layer group RF5. The five convolutional layer groups in the mid-end network are convolutional layer group RM1, convolutional layer group RM2, convolutional layer group RM3, convolutional layer group RM4 and convolutional layer group RM5. The five convolutional layer groups in the back-end network are convolutional layer group RE1, convolutional layer group RE2, convolutional layer group RE3, convolutional layer group RE4 and convolutional layer group RE5.

[0080] Convolutional layer group RF1, convolutional layer group RF2, convolutional layer group RF3, convolutional layer group RF4 and convolutional layer group RF5 are sequentially connected through the pooling layer. The pooling parameters of the pooling layer are: the pooling kernel size is 2*2, the step size is 2, the padding is 0, and the maximum pooling is used.

[0081] The input of convolution layer group RM1 is connected to the output of convolution layer group RF5. The input of convolution layer group RM2 is connected to the output of convolution layer group RM1 and convolution layer group RF4 through a doubling stacking group. The input of convolution layer group RM3 is connected to the output of convolution layer group RM2 and convolution layer group RF3 through a doubling stacking group. The input of convolution layer group RM4 is connected to the output of convolution layer group RM3 and convolution layer group RF2 through a doubling stacking group. The input of convolution layer group RM5 is connected to the output of convolution layer group RM4 and convolution layer group RF1 through a doubling stacking group.

[0082] The outputs of convolutional layer group RM1, convolutional layer group RM2, convolutional layer group RM3, convolutional layer group RM4 and convolutional layer group RM5 are respectively connected to the inputs of convolutional layer group RE1, convolutional layer group RE2, convolutional layer group RE3, convolutional layer group RE4 and convolutional layer group RE5, that is, the input of convolutional layer group RE1 is connected to the output of convolutional layer group RM1; the input of convolutional layer group RE2 is connected to the output of convolutional layer group RM2; the input of convolutional layer group RE3 is connected to the output of convolutional layer group RM3; the input of convolutional layer group RE4 is connected to the output of convolutional layer group RM4; the input of convolutional layer group RE5 is connected to the output of convolutional layer group RM5.

[0083] The fusion network includes a channel dimension layer, a fusion stack layer, a fusion convolution layer, and four convolution layers RR. The input of the fusion stack layer is connected to the outputs of convolution layer groups RE1, RE2, RE3, and RE4 through four convolution layers RR. The input of the channel dimension layer is connected to the output of convolution layer group RE5. The input of the fusion convolution layer is connected to the output of the channel dimension layer and the fusion stack layer. The image output by the fusion convolution layer is the final crowd density map.

[0084] Front-end network, see Figure 2 , including 13 convolutional layers in total. In each convolutional layer, the convolution kernel size is 3*3, the convolution step is 1, and the padding is 1. Among them, the convolutional layer group RF1 and the convolutional layer group RF2 each contain two convolutional layers; the convolutional layer group RF3, the convolutional layer group RF4 and the convolutional layer group RF5 each contain three convolutional layers. Specifically, the convolutional layer group RF1 includes the convolutional layer RF11 and the convolutional layer RF12 connected in series. The convolutional layer group RF2 includes the convolutional layer RF21 and the convolutional layer RF22 connected in series. The convolutional layer group RF3 includes the convolutional layer RF31, the convolutional layer RF32 and the convolutional layer RF33 connected in series. The convolutional layer group RF4 includes the convolutional layer RF41, the convolutional layer RF42 and the convolutional layer RF43 connected in series. The convolutional layer group RF5 includes the convolutional layer RF51, the convolutional layer RF52 and the convolutional layer RF53 connected in series. The sequential connection here means that the output of the previous layer is connected to the input of the next layer. Combined with the pooling layer connecting convolutional layer group RF1, convolutional layer group RF2, convolutional layer group RF3, convolutional layer group RF4 and convolutional layer group RF5, the specific connection of each convolutional layer is as follows:

[0085] The input of convolutional layer RF12 is connected to the output of convolutional layer RF11. The input of convolutional layer RF11 is also the input of the contextual attention network model. The output of convolutional layer RF12 is also the output of convolutional layer group RF1, and the output feature map is marked as feature map FF1. The number of input channels of convolutional layer RF11 is 3, and the number of output channels is 64; the number of input channels of convolutional layer RF12 is 64, and the number of output channels is 64. Therefore, the number of channels of feature map FF1 is 64, and the image size is the same as the input image. The input image here is the initial input image of the contextual attention network model, and the following is similar and will not be repeated.

[0086] The input of convolutional layer RF22 is connected to the output of convolutional layer RF21, the input of convolutional layer RF21 is connected to the output of the pooling layer between convolutional layer group RF1 and convolutional layer group RF2, and the input of the pooling layer between convolutional layer group RF1 and convolutional layer group RF2 is connected to the output of convolutional layer RF12. The output of convolutional layer RF22 is also the output of convolutional layer group RF2, and the output feature map is marked as feature map FF2. The number of input channels of convolutional layer RF21 is 64, and the number of output channels is 128; the number of input channels of convolutional layer RF22 is 128, and the number of output channels is 128. Therefore, the number of channels of feature map FF2 is 128. Due to the processing of the pooling layer with a step size of 2, the image size of feature map FF2 is 1 / 2 of the input image. The image size here represents the length and width of the image. The image size of 1 / 2 of the input image means that the length and width are 1 / 2 of the length and width of the input image respectively. The following is similar and will not be repeated.

[0087] The input of convolutional layer RF33 is connected to the output of convolutional layer RF32, the input of convolutional layer RF32 is connected to the output of convolutional layer RF31, the input of convolutional layer RF31 is connected to the output of the pooling layer between convolutional layer group RF2 and convolutional layer group RF3, and the input of the pooling layer between convolutional layer group RF2 and convolutional layer group RF3 is connected to the output of convolutional layer RF22. The output of convolutional layer RF33 is also the output of convolutional layer group RF3, and the output feature map is marked as feature map FF3. The number of input channels of convolutional layer RF31 is 128, and the number of output channels is 256; the number of input channels of convolutional layer RF32 is 256, and the number of output channels is 256; the number of input channels of convolutional layer RF33 is 256, and the number of output channels is 256. Therefore, the number of channels of feature map FF3 is 256. Due to the processing of the pooling layer, the image size of feature map FF3 is 1 / 2 of feature map FF2, that is, 1 / 4 of the input image.

[0088] The input of convolution layer RF43 is connected to the output of convolution layer RF42, the input of convolution layer RF42 is connected to the output of convolution layer RF41, the input of convolution layer RF41 is connected to the output of the pooling layer between convolution layer group RF3 and convolution layer group RF4, and the input of the pooling layer between convolution layer group RF3 and convolution layer group RF4 is connected to the output of convolution layer RF33. The output of convolution layer RF43 is also the output of convolution layer group RF4, and the output feature map is marked as feature map FF4. The number of input channels of convolution layer RF41 is 256, and the number of output channels is 512; the number of input channels of convolution layer RF42 is 512, and the number of output channels is 512; the number of input channels of convolution layer RF43 is 512, and the number of output channels is 512. Therefore, the number of channels of feature map FF4 is 512. Due to the processing of the pooling layer, the image size of feature map FF4 is 1 / 2 of feature map FF3, that is, 1 / 8 of the input image.

[0089] The input of convolution layer RF53 is connected to the output of convolution layer RF52, the input of convolution layer RF52 is connected to the output of convolution layer RF51, the input of convolution layer RF51 is connected to the output of the pooling layer between convolution layer group RF4 and convolution layer group RF5, and the input of the pooling layer between convolution layer group RF4 and convolution layer group RF5 is connected to the output of convolution layer RF43. The output of convolution layer RF53 is also the output of convolution layer group RF5, and the output feature map is marked as feature map FF5. The number of input channels of convolution layer RF51 is 256, and the number of output channels is 512; the number of input channels of convolution layer RF52 is 512, and the number of output channels is 512; the number of input channels of convolution layer RF53 is 512, and the number of output channels is 512. Therefore, the number of channels of feature map FF5 is 512. Due to the processing of the pooling layer, the image size of feature map FF5 is 1 / 2 of feature map FF4, that is, 1 / 16 of the input image.

[0090] The front-end network outputs five feature maps through the above connection, namely, feature map FF1, feature map FF2, feature map FF3, feature map FF4 and feature map FF5. The number of channels of feature map FF1, feature map FF2, feature map FF3, feature map FF4 and feature map FF5 are 64, 128, 256, 512 and 512 respectively, and the image sizes are 1, 1 / 2, 1 / 4, 1 / 8 and 1 / 16 of the input image respectively. Here, "1" means that the image size is the same as the input image.

[0091] Mid-range network, see Figure 3 , including five convolutional layer groups each consisting of two convolutional layers connected in series, four doubling stacking groups, and a total of 10 convolutional layers. In each convolutional layer in the five convolutional layer groups, the convolution kernel size is 3*3, the convolution step is 1, and the padding is 1. The five convolutional layer groups are: convolutional layer group RM1, convolutional layer group RM2, convolutional layer group RM3, convolutional layer group RM4, and convolutional layer group RM5. Convolutional layer group RM1 includes convolutional layer RM11 and convolutional layer RM12 connected in series. Convolutional layer group RM2 includes convolutional layer RM21 and convolutional layer RM22 connected in series. Convolutional layer group RM3 includes convolutional layer RM31 and convolutional layer RM32 connected in series. Convolutional layer group RM4 includes convolutional layer RM41 and convolutional layer RM42 connected in series. Convolutional layer group RM5 includes convolutional layer RM51 and convolutional layer RM52 connected in series. The doubling stacking group includes a doubling layer and a stacking layer, and thus, the four doubling stacking groups include four doubling layers and four stacking layers. The four convolutional layers are doubling layer K1, doubling layer K2, doubling layer K3 and doubling layer K4. The four stacking layers are stacking layer D1, stacking layer D2, stacking layer D3 and stacking layer D4. The stacking layer is used to stack the input image in the channel dimension and output it. The doubling layer is used to double the image size by bilinear interpolation. Combining doubling layer K1, doubling layer K2, doubling layer K3 and doubling layer K4, as well as stacking layer D1, stacking layer D2, stacking layer D3 and stacking layer D4, the specific connections of the layers in the mid-end network are as follows:

[0092] The input of convolutional layer RM11 is connected to the output of convolutional layer RF5, that is, the input of convolutional layer RM11 is feature map FF5. The output of convolutional layer RM11 is connected to the input of convolutional layer RM12. The output of convolutional layer RM12, that is, the output of convolutional layer group RM1, the output feature map is marked as feature map FM1. The number of input channels of convolutional layer RM11 is 512, and the number of output channels is 1024; the number of input channels of convolutional layer RM12 is 1024, and the number of output channels is 512. Therefore, the number of channels of feature map FM1 is 512, and the image size is the same as that of feature map FF5, that is, 1 / 16 of the input image.

[0093] The input of the convolution layer RM21 is connected to the output of the stacking layer D1, and the output is connected to the input of the convolution layer RM22. The output of the convolution layer RM22, that is, the output of the convolution layer group RM2, is marked as the feature map FM2. The input of the stacking layer D1 is connected to the output of the doubling layer K1 and the convolution layer RF43. The input of the doubling layer K1 is connected to the output of the convolution layer RM12. The feature map FM1 is doubled by the doubling layer K1, and the feature map FM1 outputs a feature map with a channel number of 512 and an image size of 1 / 8 of the input image. Therefore, the feature map with a channel number of 512 and an image size of 1 / 8 of the input image input by the doubling layer K1 and the feature map FF4 with a channel number of 512 and an image size of 1 / 8 of the input image output by the convolution layer RF43 are stacked in the channel dimension, and the output feature map with a channel number of 1024 and an image size of 1 / 8 of the input image is used as the input of the convolution layer RM21. The number of input channels of the convolutional layer RM21 is 1024, and the number of output channels is 512; the number of input channels of the convolutional layer RM22 is 512, and the number of output channels is 256. Therefore, the number of channels of the feature map FM2 is 256, and the image size is the same as that of the feature map FF4, that is, 1 / 8 of the input image.

[0094] The input of the convolution layer RM31 is connected to the output of the stacking layer D2, and the output is connected to the input of the convolution layer RM32. The output of the convolution layer RM32, that is, the output of the convolution layer group RM3, is a feature map marked as feature map FM3. The input of the stacking layer D2 is connected to the output of the doubling layer K2 and the convolution layer RF33. After the image size of the feature map FM2 is doubled by the doubling layer K2, the output channel number is 256 and the image size is 1 / 4 of the input image. Therefore, the feature map with a channel number of 256 and an image size of 1 / 4 of the input image input by the doubling layer K2 and the feature map FF3 with a channel number of 256 and an image size of 1 / 4 of the input image output by the convolution layer RF33 are stacked in the channel dimension, and the output channel number is 512. The feature map with an image size of 1 / 4 of the input image is used as the input of the convolution layer RM31. The number of input channels of the convolutional layer RM31 is 512, and the number of output channels is 256; the number of input channels of the convolutional layer RM32 is 256, and the number of output channels is 128. Therefore, the number of channels of the feature map FM3 is 128, and the image size is the same as that of the feature map FF3, that is, 1 / 4 of the input image.

[0095] The input of the convolution layer RM41 is connected to the output of the stacking layer D3, and the output is connected to the input of the convolution layer RM42. The output of the convolution layer RM42, that is, the output of the convolution layer group RM4, is a feature map marked as feature map FM4. The input of the stacking layer D3 is connected to the output of the doubling layer K3 and the convolution layer RF22. After the image size is doubled by the doubling layer K3, the feature map FM3 outputs a feature map with 128 channels and an image size of 1 / 2 of the input image. Therefore, the feature map with 128 channels and an image size of 1 / 2 of the input image input by the doubling layer K3 and the feature map FF2 with 128 channels and an image size of 1 / 2 of the input image output by the convolution layer RF22 are stacked in the channel dimension, and the output feature map with 256 channels and an image size of 1 / 2 of the input image is used as the input of the convolution layer RM41. The number of input channels of the convolutional layer RM41 is 256, and the number of output channels is 128; the number of input channels of the convolutional layer RM42 is 128, and the number of output channels is 64. Therefore, the number of channels of the feature map FM4 is 64, and the image size is the same as that of the feature map FF2, that is, 1 / 2 of the input image.

[0096] The input of the convolutional layer RM51 is connected to the output of the stacking layer D4, and the output is connected to the input of the convolutional layer RM52. The output of the convolutional layer RM52, that is, the output of the convolutional layer group RM5, is a feature map marked as feature map FM5. The input of the stacking layer D4 is connected to the output of the doubling layer K4 and the convolutional layer RF12. The feature map FM4 outputs a feature map with 64 channels and the same image size as the input image after the image size is doubled by the doubling layer K4. Therefore, the feature map with 64 channels and the same image size as the input image input by the stacking layer D4 through the doubling layer K4 and the feature map FF1 with 64 channels and the same image size as the input image output by the convolutional layer RF12 are stacked in the channel dimension, and the output feature map with 128 channels and the same image size as the input image is used as the input of the convolutional layer RM51. The input channel number of the convolutional layer RM51 is 128, and the output channel number is 64; the input channel number of the convolutional layer RM52 is 64, and the output channel number is 64. Therefore, the number of channels of the feature map FM5 is 64, and the image size is the same as that of the feature map FF1, that is, the image size is the same as the input image.

[0097] The mid-end network outputs five feature maps through the above connection, namely, feature map FM1, feature map FM2, feature map FM3, feature map FM4 and feature map FM5. The number of channels of feature map FM1, feature map FM2, feature map FM3, feature map FM4 and feature map FM5 are 512, 256, 128, 64 and 64 respectively, and the image sizes are 1 / 16, 1 / 8, 1 / 4, 1 / 2 and 1 of the input image respectively. Here, "1" means that the image size is the same as the input image.

[0098] Backend network, see Figure 4 , including five convolutional layer groups each consisting of four convolutional layers connected in series, namely: convolutional layer group RE1, convolutional layer group RE2, convolutional layer group RE3, convolutional layer group RE4 and convolutional layer group RE5. Convolutional layer group RE1, convolutional layer group RE2, convolutional layer group RE3, convolutional layer group RE4 and convolutional layer group RE5 are independent of each other. Convolutional layer group RE1 includes quadruple layer KE11, convolutional layer RE11, convolutional layer RE12, doubling layer KE12, convolutional layer RE13, convolutional layer RE14 and doubling layer KE13 connected in series in sequence. Convolutional layer group RE2 includes doubling layer KE21, convolutional layer RE21, convolutional layer RE22, doubling layer KE22, convolutional layer RE23, convolutional layer RE24 and doubling layer KE23 connected in series in sequence. The convolution layer group RE3 includes a doubling layer K31, a convolution layer RE31, a convolution layer RE32, a doubling layer KE32, a convolution layer RE33 and a convolution layer RE34 connected in series. The convolution layer group RE4 includes a doubling layer KE41, a convolution layer RE41, a convolution layer RE42, a convolution layer RE43 and a convolution layer RE44 connected in series. The convolution layer group RE5 includes a convolution layer RE51, a convolution layer RE52, a convolution layer RE53 and a convolution layer RE54 connected in series. Among them, the doubling layer KE12, the doubling layer KE13, the doubling layer KE21, the doubling layer KE22, the doubling layer KE23, the doubling layer KE31, the doubling layer KE32 and the doubling layer KE41 are all used to double the image size by bilinear interpolation. The quadruple layer KE11 is used to quadruple the image size through bilinear interpolation, which is equivalent to two doubling layers connected in sequence, and can also be regarded as the concatenation of two doubling layers. The specific connections of the backend network layers are as follows:

[0099] In the convolution layer group RE1, the input of the quadruple layer KE11 is connected to the output of the convolution layer RM12. That is, the input of the quadruple layer KE11 is the feature map FM1. The input of the convolution layer RE11 is connected to the output of the quadruple layer KE11, and the output is connected to the input of the convolution layer RE12. That is, the image size of the feature map input to the convolution layer RE11 is four times that of the feature map FM1, that is, 1 / 4 of the input image. The output of the convolution layer RE12 is connected to the input of the doubling layer KE12. The input of the convolution layer RE13 is connected to the output of the doubling layer KE12, and the output is connected to the input of the convolution layer RE14. That is, the image size of the feature map input to the convolution layer RE13 is eight times that of the feature map FM1, that is, 1 / 2 of the input image. The output of the convolution layer RE14 is connected to the input of the doubling layer KE13. The output of the doubling layer KE13 is also the output of the convolution layer group RE1, and its output feature map is marked as feature map FE1. The configurations of each convolution layer are as follows: the number of input channels of convolution layer RE11 is 512, the number of output channels is 512, and the padding is 1; the number of input channels of convolution layer RE12 is 512, the number of output channels is 256, the hole rate is 2, and the padding is 2; the number of input channels of convolution layer RE13 is 256, the number of output channels is 128, and the padding is 1; the number of input channels of convolution layer RE14 is 128, the number of output channels is 64, the hole rate is 2, and the padding is 2. The convolution kernel size of convolution layer RE11, convolution layer RE12, convolution layer RE13 and convolution layer RE14 is 3*3, and the convolution step size is 1. Therefore, the number of channels of feature map FE1 output by convolution layer group RE1 is 64, and the image size is the same as the input image.

[0100] In the convolution layer group RE2, the input of the doubling layer KE21 is connected to the output of the convolution layer RM22. That is, the input of the convolution layer RE21 is the feature map FM2. The input of the convolution layer RE21 is connected to the output of the doubling layer KE21, and the output is connected to the input of the convolution layer RE22. That is, the image size of the feature map input to the convolution layer RE21 is twice the feature map FM2, that is, 1 / 4 of the input image. The output of the convolution layer RE22 is connected to the input of the doubling layer KE22. The input of the convolution layer RE23 is connected to the output of the doubling layer KE22, and the output is connected to the input of the convolution layer RE24. That is, the image size of the feature map input to the convolution layer RE23 is four times the feature map FM2, that is, 1 / 2 of the input image. The output of the convolution layer RE24 is connected to the input of the doubling layer KE23. The output of the doubling layer KE23 is also the output of the convolution layer group RE2, and its output feature map is marked as feature map FE2. Each convolution layer is configured as follows: the number of input channels of convolution layer RE21 is 256, the number of output channels is 256, and the padding is 1; the number of input channels of convolution layer RE22 is 256, the number of output channels is 256, the hole rate is 2, and the padding is 2; the number of input channels of convolution layer RE23 is 256, the number of output channels is 128, and the padding is 1; the number of input channels of convolution layer RE24 is 128, the number of output channels is 64, the hole rate is 2, and the padding is 2. The convolution kernel size of convolution layer RE21, convolution layer RE22, convolution layer RE23 and convolution layer RE24 is 3*3, and the convolution step size is 1. Therefore, the number of channels of feature map FE2 output by convolution layer group RE2 is 64, and the image size is the same as the input image.

[0101] In the convolution layer group RE3, the input of the doubling layer KE31 is connected to the output of the convolution layer RM32. That is, the input of the convolution layer RE31 is the feature map FM3. The input of the convolution layer RE31 is connected to the output of the doubling layer KE31, and the output is connected to the input of the convolution layer RE32. In other words, the image size of the feature map input to the convolution layer RE31 is twice the feature map FM3, that is, 1 / 2 of the input image. The output of the convolution layer RE32 is connected to the input of the doubling layer KE32. The input of the convolution layer RE33 is connected to the output of the doubling layer KE32, and the output is connected to the input of the convolution layer RE34. In other words, the image size of the feature map input to the convolution layer RE33 is four times the feature map FM3, the same as the input image. The output of the convolution layer RE34 is also the output of the convolution layer group RE3, and its output feature map is marked as feature map FE3. Each convolution layer is configured as follows: the number of input channels of convolution layer RE31 is 128, the number of output channels is 128, and the padding is 1; the number of input channels of convolution layer RE32 is 128, the number of output channels is 128, the hole rate is 2, and the padding is 2; the number of input channels of convolution layer RE33 is 128, the number of output channels is 128, and the padding is 1; the number of input channels of convolution layer RE34 is 128, the number of output channels is 64, the hole rate is 2, and the padding is 2. The convolution kernel size of convolution layer RE31, convolution layer RE32, convolution layer RE33 and convolution layer RE34 is 3*3, and the convolution step size is 1. Therefore, the number of channels of feature map FE3 output by convolution layer group RE3 is 64, and the image size is the same as the input image.

[0102] In the convolution layer group RE4, the input of the doubling layer KE41 is connected to the output of the convolution layer RM42. That is, the input of the convolution layer RE41 is the feature map FM4. The input of the convolution layer RE41 is connected to the output of the doubling layer KE41, and the output is connected to the input of the convolution layer RE42. In other words, the image size of the feature map input to the convolution layer RE41 is twice that of the feature map FM4, that is, the same as the input image. The input of the convolution layer RE43 is connected to the output of the convolution layer RE42, and the output is connected to the input of the convolution layer RE44. The output of the convolution layer RE44 is also the output of the convolution layer group RE4, and its output feature map is marked as the feature map FE4. Each convolution layer is configured as follows: the number of input channels of convolution layer RE41 is 64, the number of output channels is 64, and the padding is 1; the number of input channels of convolution layer RE42 is 64, the number of output channels is 64, the hole rate is 2, and the padding is 2; the number of input channels of convolution layer RE43 is 64, the number of output channels is 64, and the padding is 1; the number of input channels of convolution layer RE44 is 64, the number of output channels is 64, the hole rate is 2, and the padding is 2. The convolution kernel size of convolution layer RE41, convolution layer RE42, convolution layer RE43 and convolution layer RE44 is 3*3, and the convolution step size is 1. Therefore, the number of channels of feature map FE4 output by convolution layer group RE4 is 64, and the image size is the same as the input image.

[0103] In the convolution layer group RE5, the input of convolution layer RE51 is connected to the output of convolution layer RM52. That is, the input of convolution layer RE51 is feature map FM5. The output of convolution layer RE51 is connected to the input of convolution layer RE52. The output of convolution layer RE52 is connected to the input of convolution layer RE53. The output of convolution layer RE53 is connected to the input of convolution layer RE54. The output of convolution layer RE54 is the output of convolution layer group RE5, and its output feature map is marked as feature map FE5. Each convolution layer is configured as follows: the number of input channels of convolution layer RE51 is 64, and the number of output channels is 64; the number of input channels of convolution layer RE52 is 64, and the number of output channels is 32; the number of input channels of convolution layer RE53 is 32, and the number of output channels is 16; the number of input channels of convolution layer RE54 is 16, and the number of output channels is 4; the convolution kernel size of convolution layer RE51, convolution layer RE52, convolution layer RE53 and convolution layer RE54 is 3*3, the convolution step is 1, and the padding is 1. The number of channels of feature map FE5 output by convolution layer group RE5 is 4, and the image size is the same as the input image.

[0104] Therefore, the back-end network outputs five feature maps through the above connection, namely, feature map FE1, feature map FE2, feature map FE3, feature map FE4 and feature map FE5. The number of channels of feature map FE1, feature map FE2, feature map FE3, feature map FE4 and feature map FE5 are 64, 64, 64, 64 and 4 respectively, and the image size is the same as the input image.

[0105] In the fusion network, the convolution layer RR convolves an image with 64 channels into an image with 1 channel. That is, the input channel number of the convolution layer RR is 64 and the output channel number is 1. The convolution kernel size of the convolution layer RR is 1*1, the step size is 1, and the padding is 0. The feature maps FE1, FE2, FE3, and FE4 output by the convolution layer group RE1, the convolution layer group RE2, the convolution layer group RE3, and the convolution layer group RE4 are convolved by the convolution layer RR and output four feature maps with 1 channel and the same image size as the input image. The four feature maps with 1 channel are stacked in the channel dimension by the fusion stacking layer to form a fusion feature map with 4 channels and the same image size as the input image.

[0106] The channel dimension layer is a module of Sigmoid or Softmax operation, which is used to generate a coefficient feature map with 4 channels and the same image size as the input image by Sigmoid or Softmax operation on the feature map FE5 output by the convolution layer group RE5. The Sigmoid here means that each pixel value of each channel of the image is converted by the Sigmoid function. The Sigmoid function is familiar to counting personnel in this field and is defined as follows: Sigmoid(x)= invmul (1+exp(-x)). The Softmax here means that each pixel value of the image is converted by the Softmax function. The Softmax function is defined as follows:

[0107] Softmax (x_i)= exp(-x_i)*invmul (exp(-x_1)+ exp(-x_2) +exp(-x_3)+ exp(-x_4));

[0108] Among them, x_i represents the pixel value of pixel x on the i-th channel, x_1, x_2, x_3, and x_4 are the pixel values ​​of pixel x on the four channels respectively, and here i takes the value of 1, 2, 3, or 4.

[0109] In the above formula, invmul represents the inverse function, and exp represents the exponential function with the natural constant e as the base.

[0110] The channel dimension layer is for the conversion processing of pixel values, so the number of channels and image size of the output feature map remain unchanged, that is, the number of channels of the output coefficient feature map is 4, and the image size is the same as the input image.

[0111] Therefore, the fused feature map and the coefficient feature map input by the fused convolution layer are the same in terms of the number of channels and image size. The fused convolution layer is used to multiply the coefficient feature map and the fused feature map pixel by pixel, so as to obtain a crowd density map with 4 channels and the same image size as the input image. Specifically, the fused convolution layer multiplies the two input feature maps one by one in a pixel-by-pixel manner to obtain the pixel value of the corresponding pixel in the crowd density map. In the present invention, the coefficient feature map represents the feature of contextual attention, which is also the origin of the name of the present invention.

[0112] Through the configuration of the above convolutional layers, it can be known that the convolution kernel size of each convolutional layer of the front-end network, mid-end network and back-end network of the present invention is 3*3, the step size is 1, and if the convolutional layer is a dilated convolutional layer with a dilation rate of 2, the padding is 2, otherwise the dilation rate is 1 and the padding is 1.

[0113] In addition, each convolutional layer of the front-end network, mid-end network, and back-end network is followed by an activation function ReLU.

[0114] In addition, it should be noted that the contextual attention network model in the present invention is a module of machine calculation. The front-end network, mid-end network, back-end network and fusion network in the contextual attention network model are equivalent to a collection of machine calculation modules, and each layer in the model, including the pooling layer and the convolution layer, is equivalent to a step of calculation execution.

[0115] After the model is calculated, a crowd density map is obtained, and then the number of people in the picture can be calculated based on the crowd density map. The number of people in the picture is calculated based on the crowd density map by integrating and accumulating, specifically:

[0116] ;

[0117] in, Sum is the total number of people in the picture, P ( x i ) represents the first i The pixel value of a pixel point, M is the number of pixels in the crowd density map.

[0118] As before, the model feature matrix data in the data initialization step is obtained through model training learning. In order to train and obtain the above-mentioned model feature matrix data, the method of this embodiment also includes a step of obtaining a training data set and a model training step. The model training step is used to input the images in the training data set into the context attention network model for model calculation after the aforementioned image preprocessing step to obtain the model feature matrix data. The following loss function is used in the model training step to evaluate the model feature matrix data:

[0119] ;

[0120] in, θ is the model feature matrix data, I i The training data set i pictures, D p ( I i , θ ) is the first i The crowd density map obtained by the head counting step is shown in the picture. D gi The training data set i The expected density map of the image is obtained by manual editing. N is the number of images in the training dataset. The expected density map here can also be regarded as the real crowd density map.

[0121] This embodiment uses the UCF_CC_50 dataset, the ShanghaiTech dataset, and the Mall dataset as training and test datasets. The UCF_CC_50 dataset comes from the Internet, contains a total of 50 pictures, a total of 63,974 people, and the number of people in each picture ranges from 94 to 4,543. Since the scene of this dataset is extremely dense and the number of pictures is small, it is one of the most challenging datasets at present. The ShanghaiTech dataset is a dataset published by Shanghai University of Science and Technology and is divided into two parts, Part_A and Part_B. There are 1,198 pictures in total, and a total of 330,165 people are marked. The Mall dataset contains a total of 2,000 pictures, a total of 62,325 people, and the number of people in each picture ranges from 13 to 53, which comes from surveillance cameras installed in shopping malls. In the UCF_CC_50 dataset, the average error of the head count obtained by the experimental test according to the method of this embodiment is 238.3, and the root mean square error is 363.1. The average error of head counting in the experimental test on the ShanghaiTech dataset Part_A is 62.2, and the root mean square error is 100.6. The average error of head counting in the experimental test on the ShanghaiTech dataset Part_B is 7.9, and the root mean square error is 12.1. The average error of head counting in the experimental test on the Mall dataset is 1.31, and the root mean square error is 1.59, with an accuracy of 96%.

[0122] Compared with the method disclosed in patent document CN 112651390 A, the average error of this method on ShanghaiTech dataset Part_A is 87.0, the root mean square error is 134.8, and the average error on Part_B is 14.2, the root mean square error is 23.3. Obviously, the present invention has a great improvement over the method disclosed in patent document CN 112651390 A.

[0123] Compare with the dense scene recognition network based on multi-column convolutional neural network proposed by Li et al., namely, Congested Scene Recognition Network, referred to as CSRNet. The dense scene recognition network is a method proposed after the patent document CN 112651390A. Under the method of dense scene recognition network, the average error of head counting in the UCF_CC_50 dataset is 266.0, and the root mean square error is 397.7; the average error of head counting experimentally tested on the ShanghaiTech dataset Part_A is 68.2, and the root mean square error is 115.0. The average error of head counting experimentally tested on the ShanghaiTech dataset Part_B is 10.6, and the root mean square error is 15.8. Obviously, the accuracy of the method of the present invention is greatly improved compared with CSRNet.

Claims

1. A method for counting people in images based on contextual attention, characterized in that: The method includes a model calculation step; the model calculation step is used to calculate a crowd density map through a contextual attention network model; The contextual attention network model includes: a front-end network, a mid-end network, a back-end network and a fusion network; The front-end network, the mid-end network and the back-end network respectively include five convolutional layer groups consisting of convolutional layers; the convolutional layer group includes a plurality of convolutional layers; The fusion network includes a channel dimension layer, a fusion stacking layer and a fusion product layer; In the front-end network, five convolutional layer groups are sequentially connected in series through a pooling layer, and the pooling layer gradually halves the size of the image output by the five convolutional layer groups, and the output of the last convolutional layer group is connected to the first convolutional layer group in the mid-end network; In the mid-end network, the outputs of the preceding convolutional layer groups are doubled in image size through the doubling layer, and the convolutional layer groups with the same image size as the outputs of the front-end network are stacked in the channel dimension through the stacking layer and connected to the subsequent convolutional layer groups. The outputs of the five convolutional layer groups are respectively connected to the five convolutional layer groups of the back-end network; the stacking layer is used to stack the input images in the channel dimension; In the back-end network, the convolution layer group whose input is connected to the last-order convolution layer group of the mid-end network outputs a feature map with the same image size as the input image and the number of channels as 4, and the output is connected to the channel dimension layer of the fusion network; the other four convolution layer groups are each interleaved with a different number of doubling layers, so that the four convolution layer groups output feature maps with the same image size as the input image and the number of channels as 64, and the outputs of the four convolution layer groups are respectively connected to the fusion stacking layer of the fusion network through the convolution layer RR; The convolution layer RR is used to process the feature map with the same image size as the input image and the number of channels as 64, and then output the feature map with the same image size as the input image and the number of channels as 1; the convolution kernel size of the convolution layer RR is 1*1, the padding is 0, and the step size is 1; The fusion stacking layer is used to stack four feature maps with a channel number of 1 in dimension, and the output image size is the same as the input image and the channel number is 4. The channel dimension layer is used to perform Sigmoid or Softmax operations on the input feature map to generate a coefficient feature map with 4 channels and the same image size as the input image; The input of the fused multiplication layer is connected to the output of the channel dimension layer and the fused stacking layer, and is used to multiply the coefficient feature map and the fused feature map pixel by pixel, so as to obtain a crowd density map with 4 channels and the same image size as the input image; Each convolutional layer of the front-end network, mid-end network and back-end network is equipped with an activation function ReLU.

2. The method for counting people in an image based on contextual attention as claimed in claim 1, characterized in that: In the front-end network, the five convolutional layer groups are respectively marked as convolutional layer group RF1, convolutional layer group RF2, convolutional layer group RF3, convolutional layer group RF4 and convolutional layer group RF5; the convolutional layer group RF1 includes convolutional layer RF11 and convolutional layer RF12 connected in series; the convolutional layer group RF2 includes convolutional layer RF21 and convolutional layer RF22 connected in series; the convolutional layer group RF3 includes convolutional layer RF31, convolutional layer RF32 and convolutional layer RF33 connected in series; the convolutional layer group RF4 includes convolutional layer RF41, convolutional layer RF42 and convolutional layer RF43 connected in series; the convolutional layer group RF5 includes convolutional layer RF51, convolutional layer RF52 and convolutional layer RF 53; the convolution kernel size of convolution layer RF11, convolution layer RF12, convolution layer RF21, convolution layer RF22, convolution layer RF31, convolution layer RF32, convolution layer RF33, convolution layer RF41, convolution layer RF42, convolution layer RF43, convolution layer RF51, convolution layer RF52, and convolution layer RF53 is 3*3, the step size is 1, the padding is 1, the number of input channels is: 3, 64, 64, 128, 128, 256, 256, 256, 512, 512, 512, 512, 512, and the number of output channels is: 64, 64, 128, 128, 256, 256, 256, 512, 512, 512, 512, 512, 512; The four pooling layers connecting the five convolutional layer groups of the front-end network all use a pooling layer with a pooling kernel size of 2*2, a stride of 2, a padding of 0, and a maximum pooling; In the mid-end network, the five convolutional layer groups are respectively marked as convolutional layer group RM1, convolutional layer group RM2, convolutional layer group RM3, convolutional layer group RM4 and convolutional layer group RM5; the convolutional layer group RM1 includes convolutional layer RM11 and convolutional layer RM12 connected in series; the convolutional layer group RM2 includes convolutional layer RM21 and convolutional layer RM22 connected in series; the convolutional layer group RM3 includes convolutional layer RM31 and convolutional layer RM32 connected in series; the convolutional layer group RM4 includes convolutional layer RM41 and convolutional layer RM42 connected in series; the convolutional layer group RM5 includes convolutional layer RM51 and convolutional layer RM52 connected in series. 2; The convolution kernel size of convolution layer RM11, convolution layer RM12, convolution layer RM21, convolution layer RM22, convolution layer RM31, convolution layer RM32, convolution layer RM41, convolution layer RM42, convolution layer RM51, and convolution layer RM52 is 3*3, the step size is 1, the padding is 1, the number of input channels is: 512, 1024, 1024, 512, 512, 256, 256, 128, 128, 64, and the number of output channels is: 1024, 512, 512, 256, 256, 128, 128, 64, 64, 64; the doubling layer uses bilinear interpolation to double the image size; In the back-end network, each convolutional layer group includes four convolutional layers; the five convolutional layer groups are respectively marked as convolutional layer group RE1, convolutional layer group RE2, convolutional layer group RE3, convolutional layer group RE4 and convolutional layer group RE5; the convolutional layer group RE1 includes a quadruple layer KE11, a convolutional layer RE11, a convolutional layer RE12, a doubling layer KE12, a convolutional layer RE13, a convolutional layer RE14 and a doubling layer KE13 connected in series in sequence; the convolutional layer group RE2 includes a doubling layer KE21, a convolutional layer RE21, a convolutional layer RE22, a doubling layer KE22, a convolutional layer RE23, a convolutional layer R E24 and doubling layer KE23; the convolution layer group RE3 includes doubling layer KE31, convolution layer RE31, convolution layer RE32, doubling layer KE32, convolution layer RE33 and convolution layer RE34 connected in series in sequence; the convolution layer group RE4 includes doubling layer KE41, convolution layer RE41, convolution layer RE42, convolution layer RE43 and convolution layer RE44 connected in series in sequence; the convolution layer group RE5 includes convolution layer RE51, convolution layer RE52, convolution layer RE53 and convolution layer RE54 connected in series in sequence; convolution layer RE11, convolution layer RE12, convolution layer RE 13. The convolution kernel size of convolution layer RE14, convolution layer RE21, convolution layer RE22, convolution layer RE23, convolution layer RE24, convolution layer RE31, convolution layer RE32, convolution layer RE33, convolution layer RE34, convolution layer RE41, convolution layer RE42, convolution layer RE43, convolution layer RE44, convolution layer RE51, convolution layer RE52, convolution layer RE53, and convolution layer RE54 is 3*3, the step size is 1, and the number of input channels is 512, 512, 256, 128, 256, 256, 256, 128, 128, and 128 respectively. , 128, 128, 64, 64, 64, 64, 64, 64, 32, 16, and the number of output channels are 512, 256, 128, 64, 256, 256, 128, 64, 128, 128, 64, 64, 64, 64, 64, 64, 32, 16, 4 respectively; among them, convolutional layer RE12, convolutional layer RE14, convolutional layer RE22, convolutional layer RE24, convolutional layer RE32, convolutional layer RE34, convolutional layer RE42 and convolutional layer RE44 are all convolutional layers with a void rate of 2 and a padding of 2, and the other convolutional layers have a padding of 1; The doubling layer KE12, the doubling layer KE13, the doubling layer KE21, the doubling layer KE22, the doubling layer KE23, the doubling layer KE31, the doubling layer KE32, and the doubling layer KE41 are used to double the image size by bilinear interpolation; The quadruple layer KE11 is used to quadruple the image size by bilinear interpolation.

3. The method for counting people in an image based on contextual attention as claimed in claim 1 or 2, characterized in that: The method also includes a model initialization step, a data initialization step, an image preprocessing step and a head counting step; The model initialization step is used to: initialize the context attention network model; The data initialization step is used to: obtain the model feature matrix data obtained by training the contextual attention network model, and then load the model feature matrix data into the contextual attention network model; The image preprocessing step is used to: trim the input image size by cutting off the edges so that the length and width of the image are both multiples of 16, and standardize the pixel values ​​on three channels with means of 0.485, 0.456, 0.406 and standard deviations of 0.229, 0.224, 0.225 so that the image is normalized to the interval [0, 1] on the three channels to obtain a preprocessed image FP; The head counting step is used to: input the pre-processed image FP into the contextual attention network model after the data is initialized, and obtain a crowd density map through calculation by the contextual attention network model; then obtain the number of people in the input picture by integrating and accumulating the crowd density map; The following formula is used for the integration accumulation through the crowd density map: ； in, Sum is the total number of people in the picture, P ( x i ) represents the first i The pixel value of a pixel point, M is the number of pixels in the crowd density map.

4. The method for counting people in an image based on contextual attention as claimed in claim 3, characterized in that: The method also includes a step of obtaining a training data set and a model training step; the model training step is used to input the images in the training data set into the context attention network model for model calculation after the image preprocessing step, so as to obtain the model feature matrix data.

5. The method for counting people in an image based on contextual attention as claimed in claim 4, characterized in that: The following loss function is used in the model training step to evaluate the model feature matrix data: ； in, θ is the model feature matrix data, I i is the training data set i pictures, D p ( I i , θ ) is the first i The picture is a crowd density map obtained by the head counting step, D gi is the training data set i The expected density map of the image is obtained by manual editing. N is the number of images in the training dataset.

6. A device for counting people in an image based on contextual attention, characterized in that: The device includes a model calculation module; The model calculation module is used to calculate the crowd density map through the context attention network model; The contextual attention network model includes: a front-end network, a mid-end network, a back-end network and a fusion network; The front-end network, the mid-end network and the back-end network respectively include five convolutional layer groups consisting of convolutional layers; the convolutional layer group includes a plurality of convolutional layers; The fusion network includes a channel dimension layer, a fusion stacking layer and a fusion product layer; In the front-end network, five convolutional layer groups are sequentially connected in series through a pooling layer, and the pooling layer gradually halves the size of the image output by the five convolutional layer groups, and the output of the last convolutional layer group is connected to the first convolutional layer group in the mid-end network; In the mid-end network, the outputs of the preceding convolutional layer groups are doubled in image size through the doubling layer, and the convolutional layer groups with the same image size as the outputs of the front-end network are stacked in the channel dimension through the stacking layer and connected to the subsequent convolutional layer groups. The outputs of the five convolutional layer groups are respectively connected to the five convolutional layer groups of the back-end network; the stacking layer is used to stack the input images in the channel dimension; In the back-end network, the convolution layer group whose input is connected to the last-order convolution layer group of the mid-end network outputs a feature map with the same image size as the input image and the number of channels as 4, and the output is connected to the channel dimension layer of the fusion network; the other four convolution layer groups are each interleaved with a different number of doubling layers, so that the four convolution layer groups output feature maps with the same image size as the input image and the number of channels as 64, and the outputs of the four convolution layer groups are respectively connected to the fusion stacking layer of the fusion network through the convolution layer RR; The convolution layer RR is used to process the feature map with the same image size as the input image and the number of channels as 64, and then output the feature map with the same image size as the input image and the number of channels as 1; the convolution kernel size of the convolution layer RR is 1*1, the padding is 0, and the step size is 1; The fusion stacking layer is used to stack four feature maps with a channel number of 1 in dimension, and the output image size is the same as the input image and the channel number is 4. The channel dimension layer is used to perform Sigmoid or Softmax operations on the input feature map to generate a coefficient feature map with 4 channels and the same image size as the input image; The input of the fused multiplication layer is connected to the output of the channel dimension layer and the fused stacking layer, and is used to multiply the coefficient feature map and the fused feature map pixel by pixel, so as to obtain a crowd density map with 4 channels and the same image size as the input image; Each convolutional layer of the front-end network, mid-end network and back-end network is equipped with an activation function ReLU.

7. The apparatus for counting people in images based on contextual attention as claimed in claim 6, characterized in that: In the front-end network, the five convolutional layer groups are respectively marked as convolutional layer group RF1, convolutional layer group RF2, convolutional layer group RF3, convolutional layer group RF4 and convolutional layer group RF5; the convolutional layer group RF1 includes convolutional layer RF11 and convolutional layer RF12 connected in series; the convolutional layer group RF2 includes convolutional layer RF21 and convolutional layer RF22 connected in series; the convolutional layer group RF3 includes convolutional layer RF31, convolutional layer RF32 and convolutional layer RF33 connected in series; the convolutional layer group RF4 includes convolutional layer RF41, convolutional layer RF42 and convolutional layer RF43 connected in series; the convolutional layer group RF5 includes convolutional layer RF51, convolutional layer RF52 and convolutional layer RF 53; the convolution kernel size of convolution layer RF11, convolution layer RF12, convolution layer RF21, convolution layer RF22, convolution layer RF31, convolution layer RF32, convolution layer RF33, convolution layer RF41, convolution layer RF42, convolution layer RF43, convolution layer RF51, convolution layer RF52, and convolution layer RF53 is 3*3, the step size is 1, the padding is 1, the number of input channels is: 3, 64, 64, 128, 128, 256, 256, 256, 512, 512, 512, 512, 512, and the number of output channels is: 64, 64, 128, 128, 256, 256, 256, 512, 512, 512, 512, 512, 512; The four pooling layers connecting the five convolutional layer groups of the front-end network all use a pooling layer with a pooling kernel size of 2*2, a stride of 2, a padding of 0, and a maximum pooling; In the mid-end network, the five convolutional layer groups are respectively marked as convolutional layer group RM1, convolutional layer group RM2, convolutional layer group RM3, convolutional layer group RM4 and convolutional layer group RM5; the convolutional layer group RM1 includes convolutional layer RM11 and convolutional layer RM12 connected in series; the convolutional layer group RM2 includes convolutional layer RM21 and convolutional layer RM22 connected in series; the convolutional layer group RM3 includes convolutional layer RM31 and convolutional layer RM32 connected in series; the convolutional layer group RM4 includes convolutional layer RM41 and convolutional layer RM42 connected in series; the convolutional layer group RM5 includes convolutional layer RM51 and convolutional layer RM52 connected in series. 2; The convolution kernel size of convolution layer RM11, convolution layer RM12, convolution layer RM21, convolution layer RM22, convolution layer RM31, convolution layer RM32, convolution layer RM41, convolution layer RM42, convolution layer RM51, and convolution layer RM52 is 3*3, the step size is 1, the padding is 1, the number of input channels is: 512, 1024, 1024, 512, 512, 256, 256, 128, 128, 64, and the number of output channels is: 1024, 512, 512, 256, 256, 128, 128, 64, 64, 64; the doubling layer uses bilinear interpolation to double the image size; In the back-end network, each convolutional layer group includes four convolutional layers; the five convolutional layer groups are respectively marked as convolutional layer group RE1, convolutional layer group RE2, convolutional layer group RE3, convolutional layer group RE4 and convolutional layer group RE5; the convolutional layer group RE1 includes a quadruple layer KE11, a convolutional layer RE11, a convolutional layer RE12, a doubling layer KE12, a convolutional layer RE13, a convolutional layer RE14 and a doubling layer KE13 connected in series in sequence; the convolutional layer group RE2 includes a doubling layer KE21, a convolutional layer RE21, a convolutional layer RE22, a doubling layer KE22, a convolutional layer RE23, a convolutional layer R E24 and doubling layer KE23; the convolution layer group RE3 includes doubling layer KE31, convolution layer RE31, convolution layer RE32, doubling layer KE32, convolution layer RE33 and convolution layer RE34 connected in series in sequence; the convolution layer group RE4 includes doubling layer KE41, convolution layer RE41, convolution layer RE42, convolution layer RE43 and convolution layer RE44 connected in series in sequence; the convolution layer group RE5 includes convolution layer RE51, convolution layer RE52, convolution layer RE53 and convolution layer RE54 connected in series in sequence; convolution layer RE11, convolution layer RE12, convolution layer RE 13. The convolution kernel size of convolution layer RE14, convolution layer RE21, convolution layer RE22, convolution layer RE23, convolution layer RE24, convolution layer RE31, convolution layer RE32, convolution layer RE33, convolution layer RE34, convolution layer RE41, convolution layer RE42, convolution layer RE43, convolution layer RE44, convolution layer RE51, convolution layer RE52, convolution layer RE53, and convolution layer RE54 is 3*3, the step size is 1, and the number of input channels is 512, 512, 256, 128, 256, 256, 256, 128, 128, and 128 respectively. , 128, 128, 64, 64, 64, 64, 64, 64, 32, 16, and the number of output channels are 512, 256, 128, 64, 256, 256, 128, 64, 128, 128, 64, 64, 64, 64, 64, 64, 32, 16, 4 respectively; among them, convolutional layer RE12, convolutional layer RE14, convolutional layer RE22, convolutional layer RE24, convolutional layer RE32, convolutional layer RE34, convolutional layer RE42 and convolutional layer RE44 are all convolutional layers with a void rate of 2 and a padding of 2, and the other convolutional layers have a padding of 1; The doubling layer KE12, the doubling layer KE13, the doubling layer KE21, the doubling layer KE22, the doubling layer KE23, the doubling layer KE31, the doubling layer KE32, and the doubling layer KE41 are used to double the image size by bilinear interpolation; The quadruple layer KE11 is used to quadruple the image size by bilinear interpolation.

8. The apparatus for counting people in images based on contextual attention as claimed in claim 7, characterized in that: The device also includes a model initialization module, a data initialization module, an image preprocessing module and a head counting module; The model initialization module is used to: initialize the contextual attention network model; The data initialization module is used to: obtain the model feature matrix data obtained by training the contextual attention network model, and then load the model feature matrix data into the contextual attention network model; The image preprocessing module is used to: trim the input image size by cutting off the edges so that the length and width of the image are both multiples of 16, and standardize the pixel values ​​on three channels with means of 0.485, 0.456, 0.406 and standard deviations of 0.229, 0.224, 0.225 so that the image is normalized to the interval [0, 1] on the three channels to obtain a preprocessed image FP; The head counting module is used to: input the pre-processed image FP into the contextual attention network model after the data is initialized, and obtain a crowd density map through calculation by the contextual attention network model; and then obtain the number of people in the input picture by integrating and accumulating the crowd density map; The following formula is used for the integration accumulation through the crowd density map: ； in, Sum is the total number of people in the picture, P ( x i ) represents the first i The pixel value of a pixel point, M is the number of pixels in the crowd density map.

9. The apparatus for counting people in images based on contextual attention as claimed in claim 8, characterized in that: The device also includes a module for acquiring a training data set and a model training module; the model training module is used to input the images in the training data set into the context attention network model for model calculation after the images are processed by the image preprocessing module to obtain the model feature matrix data.

10. The apparatus for counting people in images based on contextual attention as claimed in claim 9, characterized in that: The model training module uses the following loss function to evaluate the model feature matrix data: ； in, θ is the model feature matrix data, I i is the training data set i pictures, D p ( I i , θ ) is the first i The crowd density map obtained by the head counting module is shown in FIG. D gi is the training data set i The expected density map of the image is obtained by manual editing. N is the number of images in the training dataset.

Images