Multi-dimensional dynamic perception and progressive focusing network model for crowd counting

Abstract

The invention provides a multi-dimensional dynamic perception and progressive focusing network model for crowd counting, and relates to the technical field of computer vision, and the model comprises a front-end sub-network which is used for carrying out the shallow image feature extraction of a preprocessed input image, and obtaining the shallow features of the image; the main body sub-network is used for realizing multi-branch feature interaction based on image shallow features, generating deep features and cross-layer features, performing cross attention processing and multi-scale feature fusion on the deep features and the cross-layer features, performing cross-dimensional attention modeling on the fused features, and outputting the deep features through layer normalization; and the rear terminal network is used for performing high-dimensional feature construction and spatial resolution adjustment on the depth features to obtain intermediate features, performing feature optimization on the intermediate features, performing attention fusion and density decoding, and outputting a density map and a target attention map. Through the network model, crowd counting in a wide-area scene can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a multi-dimensional dynamic perception and progressive focusing network model for crowd counting. BACKGROUND

[0002] Crowd counting aims to improve the statistical ability of the number of people in public places, traffic sites and commercial areas through the application of computer vision technology, so as to meet the actual needs of urban planning, traffic control, public safety and the like. Therefore, crowd counting has important research value in the academic and industrial fields.

[0003] At present, the traditional manual feature crowd counting method depends on specific features of the crowd area, and mainly realizes rough statistics of individuals in the target area through simple mathematical models and network architectures, which is usually suitable for relatively simple scenes or small-scale environments, and has a narrow range of application. SUMMARY

[0004] Therefore, the present application provides a multi-dimensional dynamic perception and progressive focusing network model for crowd counting.

[0005] In a first aspect, the present application provides a multi-dimensional dynamic perception and progressive focusing network model for crowd counting, comprising a front-end sub-network, a main body sub-network and a back-end sub-network. The front-end sub-network comprises a preprocessing module and a feature extraction module, the preprocessing module is used for preprocessing an input image, and the feature extraction module is used for shallow image feature extraction on the preprocessed input image to obtain image shallow features. The main body sub-network comprises an Inception network module, an attention coupling module and a heterogeneous attention collaborator, the Inception network module is used for realizing multi-branch feature interaction based on the image shallow features to generate deep features and cross-layer features; the attention coupling module is used for cross-attention processing and multi-scale feature fusion on the deep features and the cross-layer features; and the heterogeneous attention collaborator is used for attention modeling across dimensions on the fused features, and cross-channel distribution calibration is implemented through layer normalization to output deep features. The back-end sub-network comprises an adjustment module, an optimization module and a decoding module, the adjustment module is used for high-dimensional feature construction and spatial resolution adjustment on the deep features to obtain intermediate features, the optimization module is used for feature optimization on the intermediate features to obtain an initial attention map and optimized features, and the decoding module is used for attention fusion and density decoding based on the optimized features and the initial attention map to output a density map and a target attention map.

[0006] In an embodiment, the feature extraction module comprises a first convolutional submodule and a second convolutional submodule. The first convolutional submodule is configured to extract features of the preprocessed input image through multiple convolutional layers, and output initial shallow layer features, wherein each convolutional layer is processed using a batch normalization algorithm and an activation function after processing. The second convolutional submodule is configured to process the initial shallow layer features through a dynamic adaptive convolutional unit to obtain image shallow layer features with enhanced shallow layer details.

[0007] In an embodiment, the Inception network module comprises an InceptionA structure group, an InceptionB structure group and an InceptionC structure group connected in sequence, the InceptionA structure group comprises a plurality of cascaded InceptionA structures, the InceptionB structure group comprises at least one InceptionB structure, and the InceptionC structure group comprises a plurality of cascaded InceptionC structures, wherein the InceptionA structure, the InceptionB structure and the InceptionC structure belong to an Inception-v3 structure respectively. The InceptionA structure is configured to extract features using a four-way parallel structure, and to concatenate the four-way extracted features along the channel dimension; the InceptionB structure is configured to perform spatial down-sampling on the output features of the InceptionA structure group using three-way heterogeneous convolution branches respectively, and to concatenate the three-way output features along the channel dimension; the InceptionC structure is internally provided with a double-path 7×7 convolution decomposition into a 1×7 and a 7×1 convolution pair, which is combined with a 1×1 convolution for channel regulation, and the output features of the last two InceptionC structures in the InceptionC structure group are used as the cross-layer features and the deep layer features.

[0008] In an embodiment, the attention coupling module comprises a feature concatenation submodule, an attention calculation submodule and an optimization submodule. The feature concatenation submodule is configured to concatenate the deep layer features and the cross-layer features along the channel dimension to generate concatenated features, and then perform feature compression through a preprocessing convolution group to output compressed features. The attention calculation submodule is configured to input the compressed features into a multi-head mixed convolution attention layer after reshaping into a preset format, and to process the input features through four groups of deep convolution branches with dynamic kernel sizes in the multi-head mixed convolution attention layer, and then perform element-wise multiplication between the four groups of features after channel recombination and a value vector after linear projection to generate first fusion features. The optimization submodule is configured to process the first fusion feature through a multi-branch dilated convolution pyramid, and generate a second fusion feature through 1x1 convolution after four-way output is spliced through a channel and cross-scale fusion is performed;

[0009] In an embodiment, the heterogeneous attention coordinator comprises an attention modeling submodule and a calibration submodule; The attention modeling submodule is configured to perform cross-dimension attention modeling through two parallel paths based on the input second fusion feature, and adaptively fuse weights of the two parallel paths through a learnable parameter to output a third fusion feature; The calibration submodule is configured to perform cross-channel distribution calibration on the third fusion feature through layer normalization to output the deep feature.

[0010] In an embodiment, the attention modeling submodule comprises a channel attention path, a spatial attention path, and a fusion path. The channel attention path is configured to perform a global average pooling operation on the input feature to compress a spatial dimension into a single pixel point, generate a channel energy vector, perform cross-channel interaction on the channel energy vector through one-dimensional convolution with an adaptive kernel size, and generate a channel dimension weight after Sigmoid function normalization. The spatial attention path is configured to perform maximum pooling and average pooling on the input feature to extract high response features of a head significant area and overall spatial distribution features, respectively, splice the two features along a channel dimension to form a joint feature map, perform spatial context modeling through 7x7 large kernel convolution, and generate a spatial dimension weight after batch normalization and Sigmoid activation. The fusion path is configured to dynamically weight fuse the channel dimension weight and the spatial dimension weight through a learnable coefficient, add a preset residual connection ratio of a residual feature, and output the third fusion feature.

[0011] In an embodiment, the adjustment module comprises an InceptionD structure, an InceptionE structure, and an upsampling submodule, and the InceptionD structure and the InceptionE structure respectively belong to an Inception-v3 structure. The InceptionD structure is configured to perform channel dimension expansion on the deep feature through a three-way heterogeneous convolution strategy to generate a first high-dimensional feature. The InceptionE structure is used to implement multi-branch feature interaction processing, and outputs a second high-dimensional feature after channel splicing of four output features. The up-sampling sub-module generates the intermediate feature by performing 2 times spatial up-sampling on the second high-dimensional feature through a nearest neighbor interpolation algorithm.

[0012] In an embodiment, the optimization module includes a first feature optimization branch and a second feature optimization branch. The first feature optimization branch is used to perform channel compression operation on the intermediate feature, generate an initial spatial weight matrix, and output a normalized initial attention map after Sigmoid function activation. The second feature optimization branch generates the optimized feature by performing semantic optimization on the intermediate feature through the InceptionE structure.

[0013] In an embodiment, the decoding module includes an attention fusion sub-module and a density decoding sub-module. The attention fusion sub-module is used to perform element-wise multiplication operation on the optimized feature and the initial attention map to generate an attention fusion feature. The density decoding sub-module is used to perform channel compression on the attention fusion feature to generate an initial density response map, and perform activation processing on the initial density response map through a ReLU activation function, reshape the output feature map and the initial attention map into two-dimensional spatial distribution forms respectively, and output the density map and the target attention map.

[0014] In a second aspect, the present application further provides a network model training method, which is used to train the multi-dimensional dynamic perception and progressive focusing network model for crowd counting according to the first aspect; the network model training method includes: obtaining a training sample set and the multi-dimensional dynamic perception and progressive focusing network model for crowd counting; setting a weight value of each sample in the training sample set according to a dynamic threshold and a sample density value, the dynamic threshold being positively correlated with a training period, the sample density value being inversely correlated with the weight value when the sample density value is greater than the dynamic threshold, and the weight value being 1 when the sample density value is not greater than the dynamic threshold; training a network model according to the weight value of each sample, the training sample set, and a target loss function to obtain a target training model, wherein the target loss function is a sum function of a density map loss function and an attention map loss function.

[0015] The multi-dimensional dynamic perception and progressive focusing network model for crowd counting according to the present application has the following beneficial effects relative to related technologies: 1、In this application, the multi-dimensional dynamic perception and progressive focusing network model for crowd counting is a deep learning model, and the crowd counting is realized through the deep learning model. Since the deep learning-based method has better generalization, it performs better in handling scene adaptability, scale diversity and other problems. Moreover, since the features are self-learned and do not need to be manually selected, the counting effect can be significantly improved. Therefore, the network model of the application can be applied to crowd counting in complex scenes, and has a wide range of applications.

[0016] 2、The network model provided in the application extracts image shallow features through the front-end sub-network. The main body sub-network realizes multi-branch feature interaction based on the image shallow features, generates deep features and cross-layer features, and performs cross-attention processing and multi-scale feature fusion on the deep features and the cross-layer features. The fused features are modeled with cross-dimensional attention, and the cross-channel distribution is calibrated through layer normalization to output deep features, realize multi-granularity feature fusion of local details and global semantics, improve the capture ability of the model for dense small targets, and improve the accuracy and robustness of crowd counting. Finally, the deep features are processed by the back-end sub-network to obtain intermediate features through high-dimensional feature construction and spatial resolution adjustment. The initial attention map and the optimized features are obtained by optimizing the intermediate features, and the optimized features and the optimized features are fused and decoded to output the density map and the target attention map. Based on the density map, the crowd counting can be realized, and based on the target attention map, the model can be optimized and updated, thereby further improving the accuracy and robustness of crowd counting. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the application or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 It is an embodiment of the application for the structure diagram of the multi-dimensional dynamic perception and progressive focusing network model for crowd counting. Figure 2 It is an embodiment of the application for the structure diagram of the multi-dimensional dynamic perception and progressive focusing network model for crowd counting. Figure 3 It is an embodiment of the application for the structure diagram of the dynamic adaptive convolution unit. Figure 4 It is an embodiment of the application for the structure diagram of the InceptionA structure. Figure 5FIG. 1 is a structural schematic diagram of an InceptionB structure in an embodiment of the present application; Figure 6 FIG. 2 is a structural schematic diagram of an InceptionC structure in an embodiment of the present application; Figure 7 FIG. 3 is a structural schematic diagram of an attention coupling module in an embodiment of the present application; Figure 8 FIG. 4 is a structural schematic diagram of a heterogeneous attention collaborator in an embodiment of the present application; Figure 9 FIG. 5 is a structural schematic diagram of an InceptionD structure in an embodiment of the present application; Figure 10 FIG. 6 is a structural schematic diagram of an InceptionE structure in an embodiment of the present application; Figure 11 FIG. 7 is a flow schematic diagram of a network model training method in an embodiment of the present application.

[0019] Legend of reference signs: 1 - front-end subnetwork, 11 - preprocessing module, 12 - feature extraction module, 121 - first convolutional sub-module, 122 - second convolutional sub-module, 2 - main subnetwork, 21 - Inception network module, 22 - attention coupling module, 23 - heterogeneous attention collaborator, 231 - attention modeling sub-module, 232 - calibration sub-module, 3 - rear-end subnetwork, 31 - adjustment module, 32 - optimization module, 33 - decoding module. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0021] In some embodiments, as shown in FIG. 1, the present application provides a multi-dimensional dynamic perception and progressive focusing network model (MDDPN) for crowd counting, which includes a front-end subnetwork 1, a main subnetwork 2 and a rear-end subnetwork 3. Figure 1

[0022] ​The front-end sub-network 1 comprises a preprocessing module 11 and a feature extraction module 12, the preprocessing module 11 is configured to preprocess an input image, and the feature extraction module 12 is configured to perform shallow image feature extraction on the preprocessed input image to obtain image shallow features. In an application, the preprocessing module 11 can preprocess a data set in batches. The preprocessing can include data enhancement and standardization processing. The feature extraction module 12 can process the preprocessed input image through multi-layer convolution to obtain image shallow features.

[0023] Exemplarily, the processing process of data enhancement can be as follows: first, the input image is subjected to grayscale detection, if it is a grayscale image, it is converted into an RGB image, then each picture is subjected to random horizontal flipping with a probability of 50%, and then random position cropping is performed, and k sub-images are taken out from each picture. The process of standardization processing is as follows: the statistical mean and standard deviation of the data set are used as the standardization parameters, the image is scaled to MxN as the input of the feature extraction module 12, and finally the input parameters are (X, Y, M, N), representing batch dimension, channel number, height and width respectively. X is the batch number, Y is the channel number of the input image, generally 3 for an RGB image, consistent with the sample used in the present scheme. The height and width need to be multiples of 8 to avoid rounding errors. The dimension of the image shallow features can be (X, 192, M / 2, N / 2).

[0024] The main body sub-network 2 comprises an Inception network module 21, an attention coupling module 22 and a heterogeneous attention collaborator 23, the Inception network module 21 is configured to realize multi-branch feature interaction based on the image shallow features to generate deep features and cross-layer features; the attention coupling module 22 is configured to perform cross-attention processing and multi-scale feature fusion on the deep features and the cross-layer features, and the heterogeneous attention collaborator 23 is configured to perform cross-dimensional attention modeling on the fused features, and implement cross-channel distribution calibration through layer normalization to output deep features.

[0025] The Inception network module 21 includes multiple Inception structures, which improve the multi-scale fusion and perception ability of the model base and construct a multi-scale feature pyramid, realize multi-branch feature interaction, and generate deep features and cross-layer features. The attention coupling module 22 realizes the multi-granularity association of local details and global context by cross-attention processing and multi-scale feature fusion on deep features and cross-layer features, significantly improves the feature response strength of dense small targets, and is beneficial to improve the accuracy of crowd counting. The heterogeneous attention coordinator 23 can be decoupled and optimized through channel-space dual domains, balanced the attention contribution degree of channel domain and space domain through hierarchical normalization strategy, and made the model dynamically focus on the head saliency region while suppressing the feature deviation caused by interference factors such as illumination change and complex background. The dimension of the deep feature is (X, 768, M / 4, N / 4).

[0026] The backend sub-network 3 includes an adjustment module 31, an optimization module 32, and a decoding module 33. The adjustment module 31 is configured to perform high-dimensional feature construction and spatial resolution adjustment on the deep feature to obtain an intermediate feature. The optimization module 32 is configured to perform feature optimization on the intermediate feature to obtain an initial attention map and an optimized feature. The decoding module 33 is configured to perform attention fusion and density decoding based on the optimized feature and the initial attention map, and output a density map and a target attention map. The dimension of the density map can be X*M / 4*N / 4, and the dimension of the target attention map can be X*M / 4*N / 4.

[0027] It can be understood that the backend sub-network 3 realizes prediction output, and after the backend sub-network 3 outputs the density map, density integration is performed on the density map to obtain the final crowd counting value, thereby realizing crowd counting. The target attention map can be applied to focused training of the model, and the model can be updated based on the target attention map, thereby further improving the accuracy and robustness of crowd counting.

[0028] The multi-dimensional dynamic perception and progressive focusing network model for crowd counting described above is a deep learning model, which realizes crowd counting through a deep learning model. Since the method based on deep learning has better generalization, it performs better in handling scene adaptability and scale diversity. Since the features are self-learned and do not need to be manually selected, the counting effect can be significantly improved. Therefore, the network model of the present application can be applied to crowd counting in complex scenes, and has a wide range of applications.

[0029] In addition, the network model provided in the application extracts image shallow features through the front-end sub-network 1. The main body sub-network 2 realizes multi-branch feature interaction based on the image shallow features, generates deep features and cross-layer features, and performs cross-attention processing and multi-scale feature fusion on the deep features and the cross-layer features. The fused features are subjected to cross-dimension attention modeling and cross-channel distribution calibration through layer normalization, and output deep features, realizing multi-granularity feature fusion of local details and global semantics, improving the capturing ability of the model for dense small targets, and improving the accuracy and robustness of crowd counting. Finally, the back-end sub-network 3 performs high-dimensional feature construction and spatial resolution adjustment on the deep features to obtain intermediate features, performs feature optimization on the intermediate features to obtain initial attention maps and optimized features, and performs attention fusion and density decoding on the optimized features and the optimized features, and outputs a density map and a target attention map. Based on the density map, crowd counting can be realized, and based on the target attention map, the model can be optimized and updated, thereby further improving the accuracy and robustness of crowd counting.

[0030] In some embodiments, as shown in FIG. 1, the feature extraction module 12 includes a first convolutional sub-module 121 and a second convolutional sub-module 122. Figure 2

[0031] The first convolutional sub-module 121 is configured to extract features from the preprocessed input image through multiple convolutional layers, and output initial shallow features. Each convolutional layer is processed using a batch normalization algorithm and an activation function.

[0032] In some embodiments, the first convolutional sub-module 121 can include m convolutional layers, and m can be in the range of 3-5, for example, m can be 5.

[0033] For example, m = 5, the input RGB image P is first processed through 5 layers of ordinary convolution, and the output feature D1 is obtained. The specific parameters of the convolutional layers are as follows: the first layer Conv2d_1a, the kernel size is 3x3, the step is 2, the padding is 1, and the output channel number is 32; the second layer Conv2d_2a, the kernel size is 3x3, the step is 1, the padding is 1, and the output channel number is 32; the third layer Conv2d_3a, the kernel size is 3x3, the step is 1, the padding is 1, and the output channel number is 64; the fourth layer Conv2d_4a, the kernel size is 1x1, the step is 1, the padding is 0, and the output channel number is 80; the fifth layer Conv2d_5a, the kernel size is 3x3, the step is 1, the padding is 1, and the output channel number is 192. Each convolutional layer is processed using a batch normalization bn and a RELU activation function.

[0034] The second convolutional sub-module 122 is configured to process the initial shallow features through a dynamic adaptive convolution unit to obtain image shallow features with enhanced shallow details. ​

[0035] Exemplarily, as shown in Figure 3 The first convolutional sub-module 121 outputs the feature D1 to a dynamic adaptive convolution unit (DACU), and finally outputs the image shallow feature C2 with enhanced shallow details. The specific process can be as follows: first, D1 generates a skip connection D1_s to retain high-resolution edge details, and then uses two layers of 3x3 convolution and batch normalization to generate an intermediate feature D2, wherein the convolution parameters can be: the first layer Conv2d_1b, the kernel size is 3x3, the step is 1, the padding is 1, and the output channel number is 192; the second layer Conv2d_2b, the kernel size is 3x3, the step is 1, the padding is 1, and the output channel number is 192; batch normalization bn and activation function RELU are used after each layer of convolution. The generated feature D2 is globally averaged pooled to generate a channel weight vector and normalized by a Sigmoid function, and a shallow dynamic attention weighted feature D3 is obtained by performing channel-by-channel multiplication on D2. Finally, D3 and the input feature D1_s are added through residual connection, and the optimized feature C2, i.e. the image shallow feature, is output after ReLU activation.

[0036] In this embodiment, to solve the scale sensitivity problem caused by the dependence of related methods on single-level abstract features, a dynamic adaptive convolution unit is embedded in the front-end network, and a double-path calibration strategy of residual skip connection and composite attention mechanism is adopted, which not only retains the high-resolution edge, corner and other detail information in the shallow feature, but also adjusts and enhances the texture robustness of the occluded area through attention-guided local receptive field, effectively solving the detail loss and semantic fragmentation problem caused by excessive downsampling of traditional single-branch network. In addition, the adaptive attention mechanism is used to enhance the expression of shallow texture details, so that the model can focus on the texture detail part in the shallow feature, effectively capture the subtle differences of edge, texture and other basic visual information, and improve the richness and delicacy of feature representation. This mechanism not only enhances the description ability of shallow features to texture details, providing better basic input for subsequent deep feature extraction, but also suppresses the interference of irrelevant areas through adaptive weight allocation, optimizing feature expression while retaining key texture information, which finally helps to improve the crowd recognition accuracy and improve the overall performance of the model.

[0037] In an embodiment, as shown in Figure 2 The Inception network module 21 includes an InceptionA structure group, an InceptionB structure group and an InceptionC structure group connected in sequence, as shown in Figures 4 to 6As shown, the InceptionA structure group includes a plurality of cascaded InceptionA structures, the InceptionB structure group includes at least one InceptionB structure, and the InceptionC structure group includes a plurality of cascaded InceptionC structures, wherein the InceptionA structure, the InceptionB structure and the InceptionC structure respectively belong to the Inception-v3 structure.

[0038] The InceptionA structure is used for feature extraction in a four-way parallel structure, and the four-way extracted features are spliced along the channel dimension; the InceptionB structure is used for spatial down-sampling of the output features of the InceptionA structure group by three heterogeneous convolution branches respectively, and the three-way output features are spliced along the channel dimension; the double 7x7 convolution inside the InceptionC structure is decomposed into a 1x7 and a 7x1 convolution pair, and a 1x1 convolution is used for channel regulation, and the output features of the last two InceptionC structures in the InceptionC structure group are used as cross-layer features and deep features.

[0039] Among them, the multi-branch feature interaction can be realized by x cascaded InceptionA structures, y InceptionB structures and z InceptionC structures, and deep features with cross-scale perception ability are generated. The InceptionA structure is a basic structure for shallow feature extraction, and the number x ranges from 2 to 4. The InceptionB structure is a down-sampling structure, and considering the accuracy of generating a density map, y is generally taken as 1-2. The InceptionC structure is a large receptive field structure, and z is taken as 4-6. It should be noted that the Inception network module 21 retains the middle layer feature information by cross-layer skip connection in the processing process.

[0040] Exemplarily, x = 3, y = 1, z = 4, the processing procedure of the Inception network module 21 is as follows: the image shallow layer feature C2 is input into the first level InceptionA structure, and the first level InceptionA structure internally adopts a four-way parallel structure to perform feature extraction: the first way uses 1x1 convolution to compress the channel to 64 dimensions; the second way performs 1x1 convolution to reduce the dimension and then performs 5x5 convolution to expand to 64 channels; the third way adopts 1x1 convolution and double-layer 3x3 convolution stacking to generate 96 channel features; and the fourth way performs 3x3 maximum pooling and then performs 1x1 convolution to output 32 channels. The four-way features are spliced along the channel dimension to output a 256-dimensional feature map E1. The second level InceptionA structure expands the fourth way pooling branch channel to 64 on the basis of the previous level InceptionA structure, and outputs a 288-dimensional feature E2. The third level InceptionA structure maintains the same structure, outputs a 288-dimensional feature E3, and completes the transition from the shallow layer feature to the middle layer feature. The feature E3 is input into the InceptionB structure to perform spatial down-sampling, and the InceptionB structure adopts a three-way heterogeneous convolution strategy: the first way directly down-samples by 3x3 convolution with a step of 2; the second way performs 1x1 convolution and 3x3 convolution stacking to realize dimension reduction with a step of 2; and the third way uses 3x3 maximum pooling with a step of 2. The three-way outputs are spliced along the channel to generate a 768-dimensional feature E4. Subsequently, four InceptionC structures process E4 in a cascaded manner: each InceptionC structure internally sets a double-way 7x7 convolution to be decomposed into a 1x7 and a 7x1 convolution pair, cooperates with 1x1 convolution to perform channel regulation, and enhances the head feature response in the horizontal and vertical directions while maintaining 768 channels. The output features of the last two InceptionC structures are used as the cross-layer feature E5 and the deep layer feature E6.

[0041] In this embodiment, the InceptionA structure, the InceptionB structure and the InceptionC structure correspond to the requirements of shallow layer feature extraction, down-sampling optimization and large receptive field feature capture respectively, and then through the cooperation of the InceptionA structure group, the InceptionB structure group and the InceptionC structure group, the attention coupling module 22 can be provided with hierarchical rich feature input.

[0042] In some embodiments, the attention coupling module 22 includes a feature splicing sub-module, an attention calculation sub-module and an optimization sub-module.

[0043] The feature splicing sub-module is used for splicing the deep layer feature and the cross-layer feature along the channel dimension to generate a spliced feature, and then performing feature compression on the spliced feature through a preprocessing convolution group to output a compressed feature. The feature splicing sub-module is used for realizing multi-scale feature splicing. Exemplarily, as shown in FIG. 6, the feature splicing sub-module includes a first preprocessing convolution group, a second preprocessing convolution group and a third preprocessing convolution group. Figure 7As shown, the feature concatenation submodule can concatenate the deep feature E6 and the cross-layer feature E5 along the channel dimension to generate the concatenated feature E7_a. Then the feature is compressed by the pre-processing convolution group. The specific process can be: first, a 3x3 grouped convolution is used to implement spatial context modeling on E7_a, and then batch normalization and GELU activation function are used to generate nonlinear features E7_b; then, the channel number is compressed by 1x1 convolution, and the compressed feature E7_c is output.

[0044] The attention calculation submodule is used to input the compressed feature into the multi-head mixed convolution attention layer after reshaping into a preset format, and process the input feature through four groups of deep convolution branches with dynamic kernel size in the multi-head mixed convolution attention layer. After channel reorganization of the four groups of features, element-wise multiplication is performed with the value vector after linear projection to generate the first fusion feature.

[0045] The attention calculation submodule is used to implement multi-head mixed convolution attention calculation. Exemplarily, the feature E7_c is reshaped into the form of BxNxC (N=HxW, B is the batch dimension, and C is the channel dimension), and is input into the multi-head mixed convolution attention layer. The layer includes four groups of deep convolution branches with dynamic kernel size: the first branch uses a 3x3 convolution kernel (padding=1) to extract local details; the second branch uses a 5x5 convolution kernel (padding=2) to capture medium-range context; the third branch deploys a 7x7 convolution kernel (padding=3) to model long-range dependencies; and the fourth branch applies a 9x9 convolution kernel (padding=4) to establish global associations. After channel reorganization of the four groups of features, element-wise multiplication is performed with the value vector after linear projection to generate the first fusion feature E7_d.

[0046] The optimization submodule is used to process the first fusion feature through a multi-branch dilated convolution pyramid, and the four-way output is channel-pasted and fused through 1x1 convolution to generate the second fusion feature; wherein the first branch of the multi-branch dilated convolution pyramid deploys average pooling, 1x1 convolution, and up-sampling to model global semantics; the second branch uses 1x1 convolution to retain the original feature as an identity mapping; the third branch uses a 3x3 dilated convolution with an expansion rate of 2 to extract medium receptive field features; and the fourth branch uses a 3x3 dilated convolution with an expansion rate of 4 to capture large-range context.

[0047] The optimization sub-module is configured to implement multi-branch dilated convolution optimization. For example, the first fusion feature E7_d is input into a multi-branch dilated convolution pyramid, and the following operations are performed in parallel: a first path is configured to deploy average pooling, 1x1 convolution, and up-sampling to model global semantics; a second path is configured to use 1x1 convolution to retain the original feature as an identity mapping; a third path is configured to use a 3x3 dilated convolution with an expansion rate of 2 to extract medium receptive field features; and a fourth path is configured to use a 3x3 dilated convolution with an expansion rate of 4 to capture a large range of context. The outputs of the four paths are spliced in the channel direction, and then cross-scale fusion is performed through 1x1 convolution to generate a second fusion feature E7.

[0048] It can be understood that the cross-resolution bidirectional interaction of deep and shallow layer features is achieved through the multi-branch mixed convolution, and the local-global perception network is constructed by combining the multi-expansion rate dilated convolution, which expands the field of view of feature fusion, captures multi-scale head distribution patterns by using spatial pyramid pooling, and significantly improves the feature response strength of dense small targets.

[0049] In this embodiment, the parallel cross-attention coupling module 22 takes the deep layer feature E6 (containing global semantic information) and the cross-layer feature E5 (retaining medium layer local details) as inputs by constructing a multi-granularity association mechanism of local details and global context, and performs cross-attention calculation of the two in the parallel structure inside the module, so that the global semantic guides the feature focusing of the local details, and the local detail information refuels the semantic refinement of the global context, thereby dynamically balancing the feature contributions of different granularities in the multi-scale feature fusion process, and finally generating a feature E7 that has rich local detail representation and global semantic association, effectively enhancing the hierarchical expression ability and cross-scale interaction efficiency of the feature.

[0050] In some embodiments, as shown in FIG. 2B, the heterogeneous attention coordinator 23 includes an attention modeling sub-module 231 and a calibration sub-module 232. Figure 2 and Figure 8 The attention modeling sub-module 231 performs cross-dimensional attention modeling based on the input second fusion feature through two parallel paths, and adaptively fuses the weights of the two parallel paths through learnable parameters to output a third fusion feature.

[0051] The calibration sub-module 232 is configured to perform cross-channel distribution calibration on the third fusion feature through layer normalization to output a deep feature.

[0052] The calibration sub-module 232 is configured to perform cross-channel distribution calibration on the third fusion feature through layer normalization to output a deep feature.

[0053] In this embodiment, attention modeling submodule 231 performs cross-dimension attention modeling (such as channel and spatial dimension attention mechanisms) on the second fused feature E7 through two parallel paths, optimizes the channel-spatial dual domain by decoupling, balances the attention contribution of the channel domain and the spatial domain through the hierarchical normalization strategy, enables the model to dynamically focus on the head saliency region, and at the same time suppresses the feature deviation caused by interference factors such as illumination changes and complex backgrounds. In addition, the dependence and interaction mode of the features in different dimensions are captured through different paths, and then the attention weights output by the two paths are adaptively weighted and fused by means of learnable parameters a and b, outputting a third fused feature, so that the model can dynamically adjust the contribution proportion of the two paths according to the task requirements. The third fused feature is processed by layer normalization (LayerNorm) to calibrate the cross-channel feature distribution and ensure the numerical stability of each channel feature. Finally, a deep feature C3 with multi-dimensional attention correlation and normalized distribution is output, effectively enhancing the semantic expression ability and cross-dimension collaboration of the feature.

[0054] In some embodiments, attention modeling submodule 231 includes a channel attention path, a spatial attention path, and a fusion path.

[0055] The channel attention path is used to perform a global average pooling operation on the input feature, compress the spatial dimension into a single pixel point, generate a channel energy vector, perform cross-channel interaction on the channel energy vector through one-dimensional convolution with an adaptive kernel size (based on the number of channels to dynamically calculate the convolution kernel size, ensuring that the weight calculation of each channel can fuse the context information of the adjacent four channels, avoiding information loss caused by dimension reduction of the fully connected layer), and generate a channel dimension weight after Sigmoid function normalization. The numerical distribution of the channel dimension weight (channel attention mask matrix) Mc presents a significantly high response to the corresponding channel of the head key region, while the background interference channel is suppressed.

[0056] The spatial attention path adopts a dual-modal pooling strategy. The spatial attention path is used to perform maximum pooling and average pooling on the input feature at the same time, extract high response features of the head salient region and overall spatial distribution features, respectively, and concatenate the two along the channel dimension to form a joint feature map. The spatial context is modeled through 7x7 large kernel convolution, batch normalization, and Sigmoid to perform maximum pooling and average pooling on the input feature at the same time, respectively extract the activated head salient region, and generate a spatial dimension weight.

[0057] It can be understood that the joint feature map can effectively associate the geometric constraint relationship between the hairline contour and the shoulder position through 7x7 large kernel convolution for spatial context modeling. After batch normalization and Sigmoid activation, the spatial attention mask generates a gradient transition zone at the occlusion boundary and maintains a high activation value in the center area of the head to generate a spatial dimension weight (channel attention mask matrix) Ms, forming accurate pixel-level positioning guidance.

[0058] The fusion path dynamically weights and fuses the channel dimension weight and the spatial dimension weight through a learnable coefficient, adds a preset residual connection ratio of residual features, and performs layer normalization processing to output the third fusion feature.

[0059] In applications, the attention masks output by the two paths are dynamically weighted and fused through a learnable coefficient , In the fusion, a certain residual connection ratio of the original features is retained, the coefficient is , to prevent feature distortion caused by excessive filtering and enhance the robustness of the module to illumination changes. The fused features are processed by layer normalization, and the third fusion feature C3' is finally output. The calculation formula is as follows:

[0060] wherein represents a channel-by-space broadcast product. The third fusion feature is processed by layer normalization (LayerNorm) to implement cross-channel distribution calibration, and finally outputs the depth feature C3.

[0061] In some embodiments, as shown in Figure 9 and Figure 10 , the adjustment module 31 includes an InceptionD structure, an InceptionE structure, and an upsampling sub-module, and the InceptionD structure and the InceptionE structure belong to an Inception-v3 structure respectively.

[0062] The InceptionD structure is used to expand the channel dimension of the depth feature through a three-way heterogeneous convolution strategy to generate a first high-dimensional feature. The InceptionE structure is used to implement multi-branch feature interaction processing and outputs a second high-dimensional feature after concatenating the four output features along the channel. The upsampling sub-module performs 2x spatial upsampling on the second high-dimensional feature through a nearest neighbor interpolation algorithm to generate an intermediate feature.

[0063] Exemplarily, the input feature C3 (dimension X X 768 X M / 4 X N / 4) is first subjected to deep semantic enhancement by an InceptionD block, wherein the InceptionD module maintains the spatial resolution at 32 X 32 by a three-way heterogeneous convolution strategy, and expands the channel dimension to 1280, to generate a first high-dimensional feature F1. Subsequently, F1 is input into an InceptionE module to implement multi-branch feature interaction: the first branch adopts 1 X 1 convolution to compress the channel to 320 dimensions; the second branch uses a 1 X 3 and 3 X 1 convolution pair, and cooperates with 1 X 1 convolution to generate 384-channel features by channel regulation and stacking; the third branch uses 1 X 1 convolution to connect 3 X 3 convolution and a 1 X 3 and 3 X 1 convolution pair to output 384 channels; the fourth branch directly performs 3 X 3 max-pooling on the input feature and adjusts the channel to 192 by 1 X 1 convolution. The four-way features are spliced along the channel to output a second high-dimensional feature F2 (dimension 32 X 2048 X 16 X 16). The second high-dimensional feature F2 is subjected to 2 times spatial up-sampling by a nearest neighbor interpolation algorithm to generate an intermediate feature F3 (dimension X X 2048 X M / 4 X N / 4). By finally generating an intermediate feature with adaptive spatial size and rich channel information, an input feature with high-dimensional semantics and reasonable spatial resolution is provided for the subsequent optimization module 32.

[0064] In some embodiments, as shown in FIG. 3, the optimization module 32 includes a first feature optimization branch and a second feature optimization branch. Figure 2 The first feature optimization branch is used to implement a channel compression operation on the intermediate feature to generate an initial spatial weight matrix, and output a normalized initial attention map after Sigmoid function activation. The second feature optimization branch performs semantic optimization on the intermediate feature by an InceptionE structure to generate an optimized feature.

[0065] The first feature optimization branch is used to implement a channel compression operation on the intermediate feature to generate an initial spatial weight matrix, and output a normalized initial attention map after Sigmoid function activation. The second feature optimization branch performs semantic optimization on the intermediate feature by an InceptionE structure to generate an optimized feature.

[0066] Exemplarily, after the intermediate feature F3 is input into the optimization module 32, it is divided into a first feature optimization branch and a second feature optimization branch. The first feature optimization branch implements a channel compression operation on F3: using 1 X 1 convolution to reduce the 2048-dimensional channel to 1 dimension to generate an initial spatial weight matrix F4 (dimension X X 1 X M / 4 X N / 4), and output a normalized initial attention map C4 (dimension X X 1 X M / 4 X N / 4, numerical range 0-1) after Sigmoid function activation; the second feature optimization branch F3 is input into InceptionE for final semantic optimization, to finally generate an optimized feature F5 (dimension X X 2048 X M / 4 X N / 4).

[0067] In this embodiment, the optimization module 32 includes a first feature optimization branch and a second feature optimization branch. The first branch performs a channel compression operation on the intermediate feature to generate an initial spatial weight matrix, and then performs normalization processing through a Sigmoid function activation to output an initial attention map of a focused spatial key region. The second branch performs semantic enhancement optimization on the intermediate feature by means of a multi-branch feature interaction mechanism of an InceptionE structure to generate optimized features that fuse multi-scale context information. The two branches process the intermediate feature from the perspectives of spatial attention modeling and semantic feature optimization, respectively, to provide the subsequent decoding module 33 with an initial attention map with spatial weight guidance and semantic-rich optimized features.

[0068] In some embodiments, as shown in FIG. 3, the decoding module 33 includes an attention fusion sub-module and a density decoding sub-module. Figure 2

[0069] The attention fusion sub-module is configured to perform an element-wise multiplication operation on the optimized feature and the initial attention map to generate attention fusion features. The density decoding sub-module is configured to perform channel compression on the attention fusion features to generate an initial density response map, and to perform activation processing on the initial density response map through a ReLU activation function. The output feature map and the initial attention map are reshaped into two-dimensional spatial distribution forms, respectively, to output a density map and a target attention map.

[0070] The attention fusion sub-module performs an element-wise multiplication operation on the optimized feature F5 and the initial attention map C4 to generate attention fusion features F6, and the calculation formula is F6 = F5 C4, where represents broadcast multiplication along the spatial dimension. The density decoding sub-module performs channel compression and non-linear mapping on the feature F6 to generate a density map and a target attention map. First, a 1x1 convolution is performed to reduce the 2048-dimensional channel to 1 dimension to generate an initial density response map F7 (dimension Xx1xM / 4xN / 4). Then, the ReLU activation function is used to eliminate negative value interference and ensure the non-negativity of the physical meaning of the density value. Finally, the feature map and the attention map are reshaped into two-dimensional spatial distribution forms to output a density map C5 (dimension XxM / 4xN / 4) and a target attention map C6 (dimension XxM / 4xN / 4).

[0071] ​In this embodiment, the attention fusion sub-module generates attention fusion features with spatial attention guidance by performing element-wise multiplication operation between the optimized features and the initial attention map, so that the optimized features focus on the key regions according to the weight distribution of the initial attention map in the spatial dimension; the density decoding sub-module first performs channel compression (such as reducing the channel dimension by global pooling or 1x1 convolution) on the attention fusion features to generate an initial density response map, then enhances the nonlinear response through the ReLU activation function, and finally reshapes the output feature map and the initial attention map into two-dimensional spatial distribution forms respectively, so as to output the density map (representing the target density distribution) and the target attention map (highlighting the spatial weight distribution of the key region), and complete the decoding process from the features to the final target output.

[0072] In some embodiments, as shown in FIG. 13, the present application also provides a network model training method. The network model training method is used for training the multi-dimensional dynamic perception and progressive focusing network model for crowd counting according to any one of the above solutions. The network model training method comprises the following steps: S1101 to S1103. Figure 11

[0073] S1101: Obtain a training sample set and a multi-dimensional dynamic perception and progressive focusing network model for crowd counting.

[0074] S1102: Set a weight value for a sample in the training sample set according to a dynamic threshold and a sample density value. The dynamic threshold is positively correlated with a training period. In a case where the sample density value is greater than the dynamic threshold, the sample density value is inversely correlated with the weight value. In a case where the sample density value is not greater than the dynamic threshold, the weight value is 1.

[0075] wherein the dynamic threshold th is first defined as a curriculum learning controller:

[0076] The threshold maintains a low value at the beginning of training, forcing the model to prioritize learning the prediction task in low-density, simple regions. As the training period epoch increases linearly, th gradually expands the coverage of difficult samples. Then the weight calculation function regulates the weight of samples with different density values through the smooth transition of Sigmoid and the activation of Relu:

[0077] wherein weights represents the weight value assigned to the sample when calculating the loss, γ is a transition factor that controls the weight decay slope from easy to difficult samples. labels is the label count value, which is the total number of people in the training label map. The operation mechanism is as follows: ​When labels≤th, ReLU(labels-th)=0, and weights=1, the loss contribution of simple samples is completely preserved. When labels>th, the Sigmoid term σ(·) generates a decay coefficient between 0 and 1, so that the loss weight of difficult samples is nonlinearly reduced with the difference between labels and th, and the greater the difference, the smaller the weight.

[0078] S1103: Training the network model according to the weight values of each sample, the training sample set and the target loss function to obtain a target training model, wherein the target loss function is a sum function of the density map loss function and the attention map loss function.

[0079] It can be understood that the final model loss is composed of two parts: the density map loss and the attention map loss.

[0080] First, the density map loss, the tensor calculation method of the dynamically weighted density loss is as follows:

[0081] The MSE loss is calculated for the predicted density map tensor and the label tensor, and the result is weighted element by element with the weight tensor weightsij, and then summed and normalized by dividing by the sum of all values of the weight tensor. Where i, j are the row and column numbers of the tensor, weightsij represents the weight value of the i-th row and j-th column of the weight tensor weights. Labelsij represents the label count value of the i-th row and j-th column of the label count tensor weights, and C3ij represents the depth feature value of the i-th row and j-th column of the depth feature C3.

[0082] Then, the attention map segmentation loss is:

[0083] Where fbs is the binary attention label obtained by processing the label map using a Gaussian kernel. The attention map generated by the model is constrained to focus on the effective head region in the spatial dimension.

[0084] The target loss function is the sum of the two loss functions, and the formula is as follows:

[0085] Wherein, is an adjustable hyperparameter.

[0086] It can be understood that by supervising the model training in the above manner, a target training model is obtained, so that a high-quality density map can be generated by the target training model and its density integral can be used to obtain the final count value, thereby achieving accurate crowd counting.

[0087] The network model training method adopts an intelligent progressive focusing training mechanism and a multi-task loss optimization, in a density map regression process, a dynamic threshold is defined as a course learning controller to construct a dynamic difficulty regulation mechanism, training stages are automatically divided according to the complexity of the density distribution, the feature learning of sparse areas is strengthened in the early training stage to establish a basic perception ability, then dense samples are progressively introduced and an adaptive gradient reweighting strategy is adopted, the contribution weights of losses in different density intervals are adjusted to balance the model convergence speed, thereby systematically solving the optimization deviation problem caused by the gradient suppression of the traditional mean square error function on the sparse area, and the generalization performance breakthrough of the model in different crowd density scenes is realized.

[0088] In some embodiments, the electronic device provided by the embodiments of the present application includes a processor and a memory; the memory has a computer program stored therein, wherein the computer program, when executed by the processor, implements the functions of the multi-dimensional dynamic perception and progressive focusing network model for crowd counting or the network model training method.

[0089] Specifically, the processor may, for example, include a general-purpose microprocessor, an instruction set processor, and / or a related chipset, and / or a special-purpose microprocessor (such as an application-specific integrated circuit (ASIC)), etc. The processor can also include on-board memory for cache use. The processor can be a single processing unit for executing the network model functions of the embodiments of the present application or a plurality of processing units, and can also be a single processing unit or a plurality of processing units for different actions according to the method processes of the embodiments of the present application.

[0090] The memory may, for example, be any medium capable of containing, storing, communicating, propagating or transmitting instructions. For example, the memory may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices, devices or propagation media. Specific examples of the memory include magnetic storage devices such as magnetic tapes or hard disk drives (HDDs); optical storage devices such as compact discs (CD-ROMs); and also random access memories (RAMs) or flash memories; and / or wired / wireless communication links.

[0091] The present application also provides a computer readable medium having a computer program stored thereon, which, when executed by a processor, implements the functions of the multi-dimensional dynamic perception and progressive focusing network model for crowd counting or the network model training method. The computer readable medium can be included in the devices / apparatuses / systems described in the above embodiments; or it can exist separately without being assembled into the devices / apparatuses / systems. The computer readable medium carries one or more programs, which, when executed, implement the functions of the multi-dimensional dynamic perception and progressive focusing network model for crowd counting or the network model training method.

[0092] It will be appreciated by those skilled in the art that features of the various embodiments and / or claims of the present application can be combined and / or interchanged, even though such combinations and / or interchanges are not expressly disclosed in the present application. In particular, the features of the various embodiments and / or claims of the present application can be combined and / or interchanged, even though such combinations and / or interchanges are not expressly disclosed in the present application. All of these combinations and / or interchanges are within the scope of the present application. Accordingly, the scope of the present application should not be limited to the above-described embodiments, but should be determined by the appended claims and their equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A multidimensional dynamic sensing and progressive focusing network model for crowd counting, characterized in that, include: Front-end subnet, main subnet, and rear-end subnet; The front-end subnetwork includes a preprocessing module and a feature extraction module. The preprocessing module is used to preprocess the input image, and the feature extraction module is used to extract shallow image features from the preprocessed input image to obtain shallow image features. The main sub-network includes: an Inception network module, an attention coupling module, and a heterogeneous attention coordinator. The Inception network module is used to realize multi-branch feature interaction based on the shallow features of the image to generate deep features and cross-layer features. The attention coupling module is used to perform cross-attention processing and multi-scale feature fusion on the deep features and the cross-layer features. The heterogeneous attention coordinator is used to perform cross-dimensional attention modeling on the fused features, and perform cross-channel distribution calibration through layer normalization to output deep features. The back-end subnet includes an adjustment module, an optimization module, and a decoding module. The adjustment module is used to construct high-dimensional features and adjust the spatial resolution of the deep features to obtain intermediate features. The optimization module is used to optimize the intermediate features to obtain an initial attention map and optimized features. The decoding module is used to perform attention fusion and density decoding based on the optimized features and the initial attention map to output a density map and a target attention map.

2. The multidimensional dynamic sensing and progressive focusing network model for crowd counting as described in claim 1, characterized in that, The feature extraction module includes: a first convolutional submodule and a second convolutional submodule; The first convolutional submodule is used to extract features from the preprocessed input image through multiple convolutional layers and output initial shallow features. Each convolutional layer is processed using a batch normalization algorithm and an activation function. The second convolutional submodule is used to process the initial shallow features through a dynamic adaptive convolutional unit to obtain shallow image features with enhanced shallow details.

3. The multidimensional dynamic sensing and progressive focusing network model for crowd counting as described in claim 1, characterized in that, The Inception network module includes an InceptionA structure group, an InceptionB structure group, and an InceptionC structure group connected in sequence. The InceptionA structure group includes multiple cascaded InceptionA structures, the InceptionB structure group includes at least one InceptionB structure, and the InceptionC structure group includes multiple cascaded InceptionC structures. The InceptionA, InceptionB, and InceptionC structures belong to the Inception-v3 structure. The InceptionA structure is used for feature extraction using a four-way parallel structure, and the extracted features from the four paths are concatenated along the channel dimension. The InceptionB structure is used to perform spatial downsampling on the output features of the InceptionA structure group using three heterogeneous convolutional branches, and the three output features are concatenated along the channel dimension. The dual 7×7 convolutions set inside the InceptionC structure are decomposed into 1×7 and 7×1 convolution pairs, and channel control is performed in conjunction with 1×1 convolutions. The output features of the last two InceptionC structures in the InceptionC structure group serve as the cross-layer features and the deep features.

4. The multidimensional dynamic sensing and progressive focusing network model for crowd counting as described in claim 3, characterized in that, The attention coupling module includes a feature splicing submodule, an attention calculation submodule, and an optimization submodule; The feature splicing submodule is used to splice the deep features and the cross-layer features along the channel dimension to generate spliced ​​features, and then perform feature compression through a preprocessing convolution group to output compressed features; The attention calculation submodule is used to reshape the compressed features into a preset format and input them into a multi-head hybrid convolutional attention layer. The input features are then processed by the four sets of depth convolution branches with dynamic kernel sizes of the multi-head hybrid convolutional attention layer. After channel recombination, the four sets of features are multiplied element-wise with the linearly projected value vector to generate the first fusion feature. The optimization submodule is used to process the first fused feature through a multi-branch dilated convolutional pyramid. The four outputs are concatenated through channels and then fused across scales through 1×1 convolution to generate the second fused feature. The first branch of the multi-branch dilated convolutional pyramid deploys average pooling, 1×1 convolution, and upsampling to model global semantics. The second branch uses 1×1 convolution to retain the original features as an identity mapping. The third branch uses a 3×3 dilated convolution with a dilation rate of 2 to extract moderate receptive field features; the fourth branch uses a 3×3 dilated convolution with a dilation rate of 4 to capture a wide range of context.

5. The multidimensional dynamic sensing and progressive focusing network model for crowd counting as described in claim 4, characterized in that, The heterogeneous attention coordinator includes an attention modeling submodule and a calibration submodule; The attention modeling submodule performs cross-dimensional attention modeling based on the second fusion feature as input, and adaptively fuses the weights of the two parallel paths through learnable parameters to output the third fusion feature. The calibration submodule is used to perform cross-channel distribution calibration on the third fused feature after layer normalization, and output the depth feature.

6. The multidimensional dynamic sensing and progressive focusing network model for crowd counting as described in claim 5, characterized in that, The attention modeling submodule includes channel attention path, spatial attention path, and fusion path; The channel attention path is used to perform global average pooling on the input features, compressing the spatial dimension into a single pixel, generating a channel energy vector, and performing cross-channel interaction on the channel energy vector through a one-dimensional convolution with an adaptive kernel size. After normalization by the Sigmoid function, channel-dimensional weights are generated. The spatial attention path is used to simultaneously perform max pooling and average pooling on the input features, extracting high-response features of the salient head region and overall spatial distribution features, respectively. The two are concatenated along the channel dimension to form a joint feature map. Spatial context modeling is performed through 7×7 large kernel convolution. After batch normalization and Sigmoid activation, spatial dimension weights are generated. The fusion path dynamically weights and fuses the channel dimension weights and the spatial dimension weights using learnable coefficients, and then adds the result to the preset residual connection ratio of the residual features to output the third fusion feature.

7. The multidimensional dynamic sensing and progressive focusing network model for crowd counting as described in claim 1, characterized in that, The adjustment module includes an InceptionD structure, an InceptionE structure, and an upsampling submodule. The InceptionD structure and the InceptionE structure belong to the Inception-v3 structure, respectively. The InceptionD structure is used to expand the channel dimension of the deep features through a three-way heterogeneous convolution strategy to generate a first high-dimensional feature. The InceptionE structure is used to implement multi-branch feature interaction processing, and outputs a second high-dimensional feature by concatenating the four output features along the channels. The upsampling submodule performs a 2x spatial upsampling on the second high-dimensional feature using a nearest neighbor interpolation algorithm to generate the intermediate feature.

8. The multidimensional dynamic sensing and progressive focusing network model for crowd counting as described in claim 1, characterized in that, The optimization module includes: a first feature optimization branch and a second feature optimization branch; The first feature optimization branch is used to perform channel compression on the intermediate features, generate an initial spatial weight matrix, and output a normalized initial attention map after activation by the Sigmoid function. The second feature optimization branch performs semantic optimization on the intermediate features through the InceptionE structure to generate the optimized features.

9. The multidimensional dynamic sensing and progressive focusing network model for crowd counting as described in claim 8, characterized in that, The decoding module includes: an attention fusion submodule and a density decoding submodule; The attention fusion submodule is used to perform element-wise multiplication of the optimized features with the initial attention map to generate attention fusion features; The density decoding submodule is used to perform channel compression on the attention fusion features to generate an initial density response map, and to activate the initial density response map through the ReLU activation function, thereby reshaping the output feature map and the initial attention map into a two-dimensional spatial distribution form, and outputting the density map and the target attention map.

10. A method for training a network model, characterized in that, The network model training method is used to train the multidimensional dynamic perception and progressive focusing network model for crowd counting as described in any one of claims 1 to 9; The network model training method includes: Obtain the training sample set and the multidimensional dynamic perception and progressive focusing network model for crowd counting; Weight values ​​are assigned to the samples in the training sample set based on a dynamic threshold and a sample density value. The dynamic threshold is positively correlated with the training period. When the sample density value is greater than the dynamic threshold, the sample density value is negatively correlated with the weight value. When the sample density value is not greater than the dynamic threshold, the weight value is 1. The network model is trained based on the weight values ​​of each sample, the training sample set, and the target loss function to obtain the target training model, wherein the target loss function is the sum of the density map loss function and the attention map loss function.

Images