RGB-T crowd counting method based on cross-modal multi-scale collaborative network

Abstract

The invention discloses an RGB-T crowd counting method based on a cross-modal multi-scale collaborative network, and belongs to the field of computer vision, and the method comprises the steps: employing a VGG as a backbone network to construct an encoder part; the modal collaborative fusion module uses RGB-T output pairs of each stage of the encoder as input pairs to realize cross-modal interactive fusion, and modal fusion features of each stage are obtained; the scale collaborative fusion module adopts modal fusion feature output of the third stage, the fourth stage and the fifth stage as input to realize integration of multi-scale features, and deeply mines crowd distribution information from local and global crowd features to obtain scale fusion features; sending the scale fusion features into a regression head to obtain a prediction density map; and adding the predicted density map pixel by pixel to obtain the predicted total number of people. According to the method, the counting problem caused by insufficient cross-modal feature interaction, multi-scale and insufficient context information perception can be effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to an RGB-T crowd counting method based on a cross-modal multi-scale collaborative network. Background Art

[0002] As population density continues to rise, various large-scale gatherings are also showing a rapid growth trend. In this context, in order to more effectively protect social public safety and rationally carry out urban construction planning, the research on crowd counting technology has gradually become a key area of ​​concern in the academic community.

[0003] Although traditional crowd counting techniques have made significant progress, most of them rely on a single processing stream to process static RGB images, which exposes its limitations under extreme lighting conditions. For example, in low-light or high-contrast environments, image quality is often severely impaired by factors such as uneven lighting, shadows, reflections, or overexposure, which directly affects the accuracy of crowd counting networks. In response to this problem, the research community has begun to explore cross-modal crowd counting techniques. These techniques significantly improve the scene adaptability and counting accuracy of the network by fusing sensor data from different modalities. In particular, in order to overcome the instability of image optical features, researchers introduced heat maps to complement RGB images, giving rise to RGB-T crowd counting methods. Although RGB-T crowd counting technology shows great potential in improving counting accuracy, there are several problems in the field of RGB-T crowd counting that need to be solved: First, cross-modal feature interaction is insufficient. Existing methods only consider fusing the features of two modalities in the process of cross-modal feature fusion, but do not fully consider using cross-modal fusion features to feed back to enhance the features of each modality. Second, the perception of contextual information is insufficient. Existing methods only consider aggregating crowd features at multiple scales, but do not consider combining local and global crowd features to further extract contextual information. The above problems significantly affect the accuracy and robustness of the RGB-T crowd counting method. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an RGB-T crowd counting method based on a cross-modal multi-scale collaborative network. Firstly, the method can deeply analyze and utilize the complementary characteristics between different modalities, realize cross-modal feature fusion, and utilize cross-modal fusion features to feed back and enhance the features of each modality; secondly, the method can effectively integrate multi-scale features, and fully mine the crowd distribution information in local and global features, and realize comprehensive perception of crowd density information.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A RGB-T crowd counting method based on a cross-modal multi-scale collaborative network includes the following steps:

[0007] Step 1: Input the paired RGB images and thermal images into the backbone encoder composed of VGG respectively to extract the feature maps of RGB mode and thermal mode;

[0008] Step 2: Input the feature maps of RGB mode and heat mode processed by each stage of the backbone encoder into the modality collaborative fusion module for cross-modal feature fusion and interaction. Through the cross-modal interactive fusion mechanism, the complementarity between different modal features is deeply explored to achieve cross-modal feature fusion. At the same time, the cross-modal fusion features are used to further enhance the features of each modality, and the modal fusion feature map and the enhanced feature map of each modality are obtained.

[0009] Step 3: The scale collaborative fusion module uses the modal fusion feature outputs of the third, fourth, and fifth stages as input to achieve the integration of multi-scale features, and deeply mines the crowd distribution information from local and global crowd features to obtain scale fusion features.

[0010] Step 4: Send the scale fusion features into the regression head to obtain the predicted density map;

[0011] Step 5: Add the predicted density map pixel by pixel to get the predicted total number of people.

[0012] A further improvement of the technical solution of the present invention is that in step 1, the backbone encoder composed of VGG is divided into 5 stages.

[0013] A further improvement of the technical solution of the present invention is that in step 2, the following steps are specifically included:

[0014] S2.1, the feature maps of the RGB modality and the heat modality processed by each stage of the backbone encoder are connected in the channel dimension, so as to merge the features of different modalities into a unified feature space to obtain the preliminary fusion feature map of the modality;

[0015] S2.2, the modality preliminary fusion feature map is processed by convolution with a kernel size of 3×3 to expand the network attention area and realize feature alignment to obtain an aligned fusion feature map;

[0016] S2.3, the aligned fusion feature map is activated by the Sigmoid function to obtain a fusion weight matrix;

[0017] S2.4, multiplying the fusion weight matrix with the input of each modality, and performing summation and ReLU function activation processing on the feature map after the multiplication to obtain the modality fusion feature map;

[0018] S2.5, averaging the modality fusion feature map and the input feature map of each modality pixel by pixel to obtain the RGB modality enhanced feature map and the T modality enhanced feature map;

[0019] The process of cross-modal feature extraction and interaction realized by the modal collaborative fusion module is expressed as:

[0020]

[0021] E RGB,i =(F Fused,i +F RGB,i ) / 2

[0022] E T,i =(F Fused,i +F T,i ) / 2

[0023] In the formula, is the fusion weight matrix; F Fused is the modality fusion feature map; E RGB,i E is the RGB modality enhanced feature map; T,i is the T modal enhancement feature map; i=1,2,3,4,5 correspond to the modal collaborative fusion module after the 1st,2nd,3rd,4th,5th stage of the backbone encoder part respectively.

[0024] A further improvement of the technical solution of the present invention is that in step 3, the scale collaborative fusion module integrates feature information of different scales through a context information aggregation unit, and deeply mines crowd distribution information from local crowd characteristics and global crowd characteristics.

[0025] A further improvement of the technical solution of the present invention is that in step 3, the following steps are specifically included:

[0026] S3.1, align the number of channels of the modal fusion feature map output by the third modal collaborative fusion module through 1×1 convolution, and upsample the modal fusion feature maps output by the fourth and fifth modal collaborative fusion modules to obtain three scale feature maps to be fused;

[0027] S3.2, the three scale feature maps to be fused are processed by the feature pyramid network for scale perception, and the semantic information of high-level features and the detailed information of low-level features are deeply mined to obtain three scale enhanced feature maps; through this operation, individuals in dense areas can be counted more accurately on high-resolution feature maps, especially in highly crowded scenes; and on low-resolution feature maps, the overall crowd distribution and individuals in sparse areas can be better grasped;

[0028] S3.3, the three scale-enhanced feature maps are fused through the context information aggregation unit to obtain a scale-fused feature map.

[0029] A further improvement of the technical solution of the present invention is that in S3.3, the following steps are specifically included:

[0030] S3.3.1, sum the three scale-enhanced feature maps to obtain a preliminary scale-fused feature map;

[0031] S3.3.2, the scale preliminary fusion feature map is processed by 1×1 convolution operation and batch normalization, and then nonlinearity is introduced using ReLU activation function. The number of channels is aligned again by 1×1 convolution, and the local feature map of the crowd is obtained by batch normalization;

[0032] S3.3.3, the scale preliminary fusion feature map is processed by adaptive maximum pooling to obtain a preliminary global feature map. Then, the preliminary global feature map is further processed by 1×1 convolution operation and batch normalization, and its nonlinear expression ability is enhanced under the action of ReLU activation function. Then, the number of channels is adjusted by 1×1 convolution, and the global feature map of the crowd is obtained by batch normalization;

[0033] S3.3.4, adding the crowd local feature map and the crowd global feature map, and obtaining the context aggregation weight matrix after Sigmoid activation processing;

[0034] S3.3.5, multiplying the context aggregation weight matrix with the scale preliminary fusion feature map to obtain a scale fusion feature map, thereby achieving effective fusion of multi-scale features;

[0035] The process of realizing multi-scale feature fusion by the scale collaborative fusion module is expressed as:

[0036] Y1,Y2,Y3=FPN(X1,X2,X3)

[0037] Y=Y1+Y2+Y3

[0038] G=BN(Conv 1×1 (ReLU(BN(Conv 1×1 (AAP(Y))))))

[0039] L=BN(Conv 1×1 (ReLU(BN(Conv 1×1 (Y)))))

[0040] Z=Y·Sigmoid(G+L)

[0041] Where X1, X2, and X3 are the fused feature maps after the 3rd, 4th, and 5th stages respectively; Y1, Y2, and Y3 are the 3 scale-enhanced feature maps respectively; G is the global feature map of the crowd; L is the local feature map of the crowd; and Z is the scale-fused feature map.

[0042] Due to the adoption of the above technical solution, the technical progress achieved by the present invention is:

[0043] 1. This paper proposes a cross-modal multi-scale collaborative network for RGB-T crowd counting. By deeply exploring and making full use of cross-modal collaborative reasoning and multi-scale collaborative reasoning, the network gives full play to the complementary advantages between different modalities, deeply integrates multi-scale information, and greatly improves the accuracy and robustness of crowd counting. It can effectively solve the counting problems caused by insufficient cross-modal feature interaction, multi-scale and insufficient contextual information perception.

[0044] 2. This paper proposes a novel modal collaborative fusion module, which aims to promote effective interaction between cross-modal features. Through the cross-modal interactive fusion mechanism, the module deeply explores the complementary relationship between different modal features, realizes cross-modal feature fusion, and uses cross-modal fusion features to enhance the features of each modality in turn, thereby effectively solving the problem of insufficient cross-modal feature interaction.

[0045] 3. The present invention proposes a novel scale collaborative fusion module to realize the effective fusion of multi-scale crowd information. This module integrates feature information of different scales, deeply mines crowd distribution information from local and global crowd characteristics, and significantly enhances the network's perception of crowd density, thereby effectively alleviating the counting problem caused by insufficient perception of multi-scale and contextual information. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 is a flow chart of an RGB-T crowd counting method based on a cross-modal multi-scale collaborative network provided in an embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of the overall structure of an RGB-T crowd counting method based on a cross-modal multi-scale collaborative network provided in an embodiment of the present invention;

[0049] Figure 3 is a detailed schematic diagram of a novel scale collaborative fusion module in an embodiment of the present invention;

[0050] Figure 4 is a detailed schematic diagram of a novel context information aggregation unit in an embodiment of the present invention;

[0051] Figure 5 It is a schematic diagram of a crowd counting method based on a cross-modal multi-scale collaborative network for RGB-T crowd counting provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatus.

[0053] The present invention is further described in detail below with reference to the accompanying drawings and embodiments:

[0054] Figure 1 A flowchart of an RGB-T crowd counting method based on a cross-modal multi-scale collaborative network provided by the present invention. Figure 2 The overall structure diagram of the RGB-T crowd counting method based on the cross-modal multi-scale collaborative network provided by the present invention is shown in Figure 1, wherein the cross-modal multi-scale collaborative network includes two core modules: the modality collaborative fusion module and the scale collaborative fusion module. Figure 1 and Figure 2 , elaborate on the specific steps of an RGB-T crowd counting method based on a cross-modal multi-scale collaborative network:

[0055] Step 1: Input the paired RGB images and thermal images into the backbone encoder composed of VGG respectively to extract the feature maps of RGB mode and thermal mode. The backbone encoder composed of VGG is divided into 5 stages.

[0056] In step 2, the feature maps of the RGB modality and the heat modality processed by each stage of the backbone encoder are respectively input into the novel modality collaborative fusion module for cross-modal feature fusion and interaction. Through the cross-modal interactive fusion mechanism, the complementarity between the features of different modalities is deeply explored, and the fusion of cross-modal features is realized. At the same time, the cross-modal fusion features are used to further enhance the features of each modality, and the modality fusion feature map and the enhanced feature map of each modality are obtained.

[0057] Combine the following Figure 3 The modality collaborative fusion module used in the present invention is described as follows:

[0058] like Figure 3As shown in the figure, the detailed schematic diagram of the novel modality collaborative fusion module is used to obtain the modality fusion feature map extracted by the novel modality collaborative fusion module and the enhanced feature map of each modality. First, the RGB modality input and the T modality input are connected in the channel dimension, so as to merge the features of different modalities into a unified feature space to obtain the modality preliminary fusion feature map; secondly, the modality preliminary fusion feature map is processed by convolution with a kernel size of 3×3 to expand the network attention area and realize feature alignment to obtain the aligned fusion feature map; thirdly, the aligned fusion feature map is activated by the Sigmoid function to obtain the fusion weight matrix The fusion weight matrix Multiply it with the input of each modality, and sum the multiplied feature maps and activate them with the ReLU function to obtain the modality fusion feature map F Fused ; Finally, the modality fusion feature map F Fused The RGB modality enhanced feature map E is obtained by averaging the input feature maps of each modality pixel by pixel. RGB and T modality enhanced feature map E T The process of cross-modal feature extraction and interaction achieved by the modal collaborative fusion module can be expressed as:

[0059]

[0060] E RGB,i =(F Fused,i +F RGB,i ) / 2

[0061] E T,i =(F Fused,i +F T,i ) / 2

[0062] Where i=1, 2, 3, 4, 5 correspond to the modality collaborative fusion modules after stages 1, 2, 3, 4, and 5 of the backbone encoder, respectively.

[0063] Step 3: After aligning the modal fusion feature maps output by the last three modal collaborative fusion modules in various dimensions, the modal fusion feature maps are input into the scale collaborative fusion module to obtain the scale fusion feature map. The scale collaborative fusion module integrates feature information of different scales through the context information aggregation unit, and deeply mines crowd distribution information from local crowd features and global crowd features.

[0064] Combine the following Figure 2 and Figure 4 The novel scale collaborative fusion module used in the present invention is described as follows:

[0065] like Figure 2As shown in the figure, the scale fusion feature map extracted by the novel scale collaborative fusion module is obtained. First, the fused feature maps X1, X2, and X3 after the 3rd, 4th, and 5th stages are scale-aware through the feature pyramid network, and the semantic information of high-level features and the detail information of low-level features are deeply mined to obtain three scale-enhanced feature maps Y1, Y2, and Y3. Through this operation, on the high-resolution feature map, individuals in dense areas can be counted more accurately, especially in scenes with highly crowded crowds. On the low-resolution feature map, the overall crowd distribution and individuals in sparse areas can be better grasped. Secondly, the three scale-enhanced feature maps are fused through the context information aggregation unit. As shown Figure 4 As shown in Figure 1, specifically, the context information aggregation unit obtains a scale preliminary fusion feature map through a summation operation. In order to obtain local crowd features, the scale preliminary fusion feature map is processed by 1×1 convolution operation and batch normalization, and then the ReLU activation function is used to introduce nonlinearity, and the number of channels is aligned through 1×1 convolution, and the local crowd feature map L is obtained through batch normalization. In order to obtain the global crowd feature map, the scale preliminary fusion feature map is processed by adaptive maximum pooling to obtain a preliminary global feature map. Subsequently, the preliminary global feature map is further processed by 1×1 convolution operation and batch normalization, and its nonlinear expression ability is enhanced under the action of ReLU activation function, and the number of channels is adjusted by 1×1 convolution, and the global crowd feature map G is obtained through batch normalization. Finally, the local crowd feature map and the global crowd feature map are added, and the context aggregation weight matrix is ​​obtained through Sigmoid activation processing, which is multiplied with the scale preliminary fusion feature map to obtain the scale fusion feature map Z, realizing the effective fusion of multi-scale features. The process of realizing multi-scale feature fusion by the scale collaborative fusion module can be expressed as:

[0066] Y1,Y2,Y3=FPN(X1,X2,X3)

[0067] Y=Y1+Y2+Y3

[0068] G=BN(Conv 1×1 (ReLU(BN(Conv 1×1 (AAP(Y))))))

[0069] L=BN(Conv 1×1 (ReLU(BN(Conv 1×1 (Y)))))

[0070] Z=Y·Sigmoid(G+L)

[0071] Step 4: Send the scale fusion feature map output by the scale collaborative fusion module to the regression head and regress to obtain the predicted density map.

[0072] Step 5: Add the predicted density map pixel by pixel to get the predicted total number of people.

[0073] Figure 5 Schematic diagram of crowd counting using the RGB-T crowd counting method based on a cross-modal multi-scale collaborative network provided by the present invention. Figure 5 When the crowd counting method of the present invention is used for crowd counting, the RGB image and the thermal image of the target scene are used for crowd counting using the RGB-T crowd counting method based on the cross-modal multi-scale collaborative network, and the obtained predicted density map is added pixel by pixel to obtain the predicted total number of people.

[0074] In summary, the present invention aims at the counting problems caused by insufficient cross-modal feature interaction, multi-scale and insufficient context information perception, and proposes an RGB-T crowd counting method based on a cross-modal multi-scale collaborative network to achieve cross-modal interaction and multi-scale fusion. By inputting paired RGB and heat images into the backbone encoder composed of VGG, the feature maps of each modality are extracted; the modality collaborative fusion module uses the feature maps of the RGB modality and heat modality processed by each stage of the backbone encoder as input for cross-modal feature fusion and interaction; the modality fusion feature maps output by the last three modality collaborative fusion modules are aligned in various dimensions and then input into the scale collaborative fusion module, which integrates feature information of different scales through the feature pyramid network and the context information aggregation unit, and deeply mines crowd distribution information from local crowd features and global crowd features to obtain a scale fusion feature map; the scale fusion feature map output by the scale collaborative fusion module is sent to the regression head, and the predicted density map is obtained by regression; the predicted density map is added pixel by pixel to obtain the predicted total number of people. The RGB-T crowd counting method based on a cross-modal multi-scale collaborative network proposed in the present invention can achieve effective interaction of existing cross-modal features, full perception of multi-scale and contextual information, thereby greatly improving the accuracy and robustness of crowd counting.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A RGB-T crowd counting method based on a cross-modal multi-scale collaborative network, characterized in that: The steps include: Step 1: Input the paired RGB images and thermal images into the backbone encoder composed of VGG respectively to extract the feature maps of RGB mode and thermal mode; Step 2: Input the feature maps of RGB mode and heat mode processed by each stage of the backbone encoder into the modality collaborative fusion module for cross-modal feature fusion and interaction. Through the cross-modal interactive fusion mechanism, the complementarity between different modal features is deeply explored to achieve cross-modal feature fusion. At the same time, the cross-modal fusion features are used to further enhance the features of each modality, and the modal fusion feature map and the enhanced feature map of each modality are obtained. Step 3: The scale collaborative fusion module uses the modal fusion feature outputs of the third, fourth, and fifth stages as input to achieve the integration of multi-scale features, and deeply mines the crowd distribution information from local and global crowd features to obtain scale fusion features. Step 4: Send the scale fusion features into the regression head to obtain the predicted density map; Step 5: Add the predicted density map pixel by pixel to get the predicted total number of people.

2. According to claim 1, a RGB-T crowd counting method based on a cross-modal multi-scale collaborative network is characterized in that: In step 1, the backbone encoder composed of VGG is divided into 5 stages.

3. According to claim 1, a RGB-T crowd counting method based on a cross-modal multi-scale collaborative network is characterized in that: In step 2, the following steps are specifically included: S2.1, the feature maps of the RGB modality and the heat modality processed by each stage of the backbone encoder are connected in the channel dimension, so as to merge the features of different modalities into a unified feature space to obtain the preliminary fusion feature map of the modality; S2.2, the modality preliminary fusion feature map is processed by convolution with a kernel size of 3×3 to expand the network attention area and realize feature alignment to obtain an aligned fusion feature map; S2.3, the aligned fusion feature map is activated by the Sigmoid function to obtain a fusion weight matrix; S2.4, multiplying the fusion weight matrix with the input of each modality, and performing summation and ReLU function activation processing on the feature map after the multiplication to obtain the modality fusion feature map; S2.5, averaging the modality fusion feature map and the input feature map of each modality pixel by pixel to obtain the RGB modality enhanced feature map and the T modality enhanced feature map; The process of cross-modal feature extraction and interaction realized by the modal collaborative fusion module is expressed as: E RGB,i =(F Fused,i +F RGB,i ) / 2 E T,i =(F Fused,i +F T,i ) / 2 In the formula, is the fusion weight matrix; F Fused is the modality fusion feature map; E RGB,i E is the RGB modality enhanced feature map; T,i is the T modal enhancement feature map; i=1,2,3,4,5 correspond to the modal collaborative fusion module after the 1st,2nd,3rd,4th,5th stage of the backbone encoder part respectively.

4. According to claim 1, a RGB-T crowd counting method based on a cross-modal multi-scale collaborative network is characterized in that: In step 3, the scale collaborative fusion module integrates feature information of different scales through a context information aggregation unit, and deeply mines crowd distribution information from local crowd features and global crowd features.

5. According to claim 4, the RGB-T crowd counting method based on cross-modal multi-scale collaborative network is characterized in that: In step 3, the following steps are specifically included: S3.1, align the number of channels of the modal fusion feature map output by the third modal collaborative fusion module through 1×1 convolution, and upsample the modal fusion feature maps output by the fourth and fifth modal collaborative fusion modules to obtain three scale feature maps to be fused; S3.2, the three scale feature maps to be fused are processed by the feature pyramid network for scale perception, and the semantic information of high-level features and the detailed information of low-level features are deeply mined to obtain three scale enhanced feature maps; through this operation, individuals in dense areas can be counted more accurately on high-resolution feature maps, especially in highly crowded scenes; and on low-resolution feature maps, the overall crowd distribution and individuals in sparse areas can be better grasped; S3.3, the three scale-enhanced feature maps are fused through the context information aggregation unit to obtain a scale-fused feature map.

6. According to claim 3, the RGB-T crowd counting method based on a cross-modal multi-scale collaborative network is characterized in that: In S3.3, the following steps are specifically included: S3.3.1, sum the three scale-enhanced feature maps to obtain a preliminary scale-fused feature map; S3.3.2, the scale preliminary fusion feature map is processed by 1×1 convolution operation and batch normalization, and then nonlinearity is introduced using ReLU activation function. The number of channels is aligned again by 1×1 convolution, and the local feature map of the crowd is obtained by batch normalization; S3.3.3, the scale preliminary fusion feature map is processed by adaptive maximum pooling to obtain a preliminary global feature map. Then, the preliminary global feature map is further processed by 1×1 convolution operation and batch normalization, and its nonlinear expression ability is enhanced under the action of ReLU activation function. Then, the number of channels is adjusted by 1×1 convolution, and the global feature map of the crowd is obtained by batch normalization; S3.3.4, adding the crowd local feature map and the crowd global feature map, and obtaining the context aggregation weight matrix after Sigmoid activation processing; S3.3.5, multiplying the context aggregation weight matrix with the scale preliminary fusion feature map to obtain a scale fusion feature map, thereby achieving effective fusion of multi-scale features; The process of realizing multi-scale feature fusion by the scale collaborative fusion module is expressed as: Y1,Y2,Y3=FPN(X1,X2,X3) Y=Y1+Y2+Y3 G=BN(Conv 1×1 (ReLU(BN(Conv 1×1 (AAP(Y)))))) L=BN(Conv 1×1 (ReLU(BN(Conv 1×1 (Y))))) Z=Y·Sigmoid(G+L) Where X1, X2, and X3 are the fused feature maps after the 3rd, 4th, and 5th stages respectively; Y1, Y2, and Y3 are the 3 scale-enhanced feature maps respectively; G is the global feature map of the crowd; L is the local feature map of the crowd; and Z is the scale-fused feature map.

Images