A crowd counting method based on semantic information optimization

Abstract

The present invention relates to the field of computer vision technology, and specifically to a crowd counting method based on semantic information optimization. It includes using CNN and Transformer to form a dual-backbone feature extraction module, and extracting features containing global crowd semantic information and local crowd semantic information at the same time; designing a semantic information fusion module to effectively fuse the extracted crowd features; designing a multi-scale residual attention regression module, including a multi-scale convolution branch, an attention branch and a residual branch, which is used to decode the fused features to obtain a predicted density map; the multi-scale convolution branch uses at least four parallel branches, and the parallel branch includes two convolution layers, which contain convolution kernels of different sizes; the attention branch includes a convolution layer with a ReLU activation function, and a convolution layer with a Sigmoid function; the residual branch includes a convolution layer with BN. This technical solution can improve the adaptability and robustness to complex crowd scenes and improve the accuracy of crowd counting.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a crowd counting method based on semantic information optimization. Background Art

[0002] As an important research direction in the field of computer vision, crowd counting has received extensive attention in recent years. With the acceleration of urbanization, real-time monitoring and management of crowd density has become increasingly important, and has important applications in public safety, traffic planning, and business intelligence. Existing crowd counting methods are mainly based on deep learning technologies such as convolutional neural networks (CNN) or Vision-Transformer, which estimate the number of people by extracting image features.

[0003] Although these methods have achieved good results in extracting feature maps, they have limitations in further improving counting accuracy. Specifically, existing technologies mainly focus on how to improve the quality of feature maps, while ignoring the importance of effectively utilizing these feature maps. As a result, the potential of feature maps cannot be fully utilized, which limits the further improvement of counting accuracy.

[0004] To address the above problems, those skilled in the art have tried to improve the performance of crowd counting through various optimization strategies, such as multi-scale semantic optimization strategy, hard example focusing, and adaptive label density map generator, etc. However, the above methods still face the problems of how to more effectively integrate local and global information, reduce scale changes and background noise interference, etc. Summary of the invention

[0005] The purpose of the present invention is to propose a crowd counting method based on semantic information optimization, which can improve the adaptability and robustness to complex crowd scenes and improve the accuracy of crowd counting.

[0006] To achieve the above objectives, the present disclosure provides a crowd counting method based on semantic information optimization, including:

[0007] A dual-backbone feature extraction module consisting of CNN and Transformer is used to extract features containing global crowd semantic information and local crowd semantic information at the same time;

[0008] Design a semantic information fusion module to fuse the crowd features extracted by the dual-backbone feature extraction module to obtain fused features;

[0009] Design a multi-scale residual attention regression module, including a multi-scale convolution branch, an attention branch, and a residual branch, to decode the fused features and obtain a predicted density map;

[0010] Among them, the multi-scale convolution branch adopts at least four parallel branches, and the parallel branch includes two convolution layers, which contain convolution kernels of different sizes; the attention branch includes a convolution layer with a ReLU activation function and a convolution layer containing a Sigmoid function; the residual branch includes a convolution layer with BN.

[0011] Beneficial effects of the basic solution: By adopting a dual-backbone feature extraction module consisting of CNN and Transformer, it is possible to effectively extract global crowd semantic information and local crowd semantic information in the image at the same time, thereby better understanding the crowd scene. The self-attention mechanism of Transformer can establish long-distance dependencies and establish a global receptive field, enabling the network to better focus on the crowd area from a global perspective, while CNN can better capture local information in the image through convolution operations. The dual-backbone feature extraction module can be more adaptable and improve adaptability and robustness to complex crowd scenes.

[0012] The semantic information fusion module can effectively fuse feature maps from different sources, which helps to extract richer and more robust crowd semantic information, thereby improving the accuracy of crowd counting.

[0013] The multi-scale residual attention regression module enables this method to adapt to changes in crowd density at different scales. The multi-scale convolution branch has multiple parallel branches, and the use of convolution kernels of different sizes can extract detailed information at different levels, that is, capture the head features of crowds of different sizes. The attention branch helps the model focus on important crowd areas, while the residual branch can reduce the counting interference caused by scale changes and background noise, and improve the accuracy of the predicted density map. The multi-scale residual attention regression module enhances the robustness of the model for crowd scenes of different densities, especially in scenes with severe crowding and occlusion, the model can still maintain a high counting accuracy at different viewing angles.

[0014] As an implementable preferred solution, a loss function is defined to supervise network training, which is used to measure the difference between the model's predicted probability and the true label, and the mixed supervision loss function , the formula is as follows:

[0015]

[0016] in, represents the predicted density map, represents the label density map, represents the counting loss, represents the optimal transmission loss, is the weight of the optimal transmission loss, represents the MSE loss, is the weight of the MSE loss.

[0017] As an implementable preferred solution, CNN and Transformer are used to form a dual-backbone feature extraction module, including the following contents:

[0018] The basic version of Twins Transformer is used as the first backbone model branch, and the first ten layers of VGG16-BN are used as the second backbone model branch. They are pre-trained on ImageNet-1K. The Adam-W optimizer is used as the optimizer, and random cropping and random flipping data enhancement methods are used to expand the sample space.

[0019] As an implementable preferred solution, a semantic information fusion module is designed, including the following contents:

[0020] The output features of the 2nd, 3rd, and 4th stages of TwinsTransformer , , to merge; to and Adjust to The resolution is obtained and ;

[0021] right , , Perform a convolution with a kernel size of 3 3 convolution operations to obtain features with shallow / deep global context information , , , the calculation formula is as follows:

[0022]

[0023]

[0024]

[0025] in, Representative 3 3 convolution operations, Represents an upsampling operation.

[0026] As an implementable preferred solution, , and The resolution is uniformly adjusted to 1 / 8 of the input image size.

[0027] As an implementable preferred solution, the semantic information fusion module is designed, which also includes:

[0028] right and Perform element-by-element addition to obtain features ,right Perform a convolution with a kernel size of 3 3 convolution operation, and and Add together to get the fusion feature with global contextual semantic information , the calculation formula is as follows:

[0029]

[0030]

[0031] in, Represents a pixel-level addition operation.

[0032] As an implementable preferred solution, the semantic information fusion module is designed, which also includes:

[0033] Extract features with stronger local semantic information through the second backbone model branch ,Will and Perform feature fusion to obtain , Input for the multi-scale residual attention regression module.

[0034] As an implementable preferred solution, the multi-scale convolution branch uses four parallel branches , , , , branch Including convolution kernel With convolution kernel , branch Contains two convolution kernels , branch Including convolution kernel With convolution kernel , branch Contains two convolution kernels , the outputs of the four branches are summed to obtain the feature map ;

[0035] As an implementable preferred solution, the attention branch is designed to include the following contents:

[0036] With the ReLU activation function Convolutional layer to reduce the number of channels, and then use a 1 1 convolution layer to maintain the parameters, and then use a Sigmoid function to get the global attention map ; The feature map and the global attention map Perform element-by-element multiplication to obtain the weighted feature map .

[0037] As an implementable optimal solution, the residual branch is designed, including The convolutional layer consists of the following:

[0038] Feature map As input and output ; and Perform element-by-element addition to obtain the final features ;right Execute a 1 1. The convolution operation is regressed to obtain the final predicted density map. The formula is as follows:

[0039]

[0040] in, represents an element-wise multiplication operation, Represents an element-wise addition operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A logical schematic diagram of a crowd counting method based on semantic information optimization according to the present invention;

[0042] Figure 2 It is a schematic diagram of a dual-backbone feature extraction module in an embodiment of a crowd counting method based on semantic information optimization of the present invention;

[0043] Figure 3 A schematic diagram of a semantic information fusion module in an embodiment of a crowd counting method based on semantic information optimization of the present invention;

[0044] Figure 4 A schematic diagram of a multi-scale residual attention regression module in an embodiment of a crowd counting method based on semantic information optimization of the present invention;

[0045] Figure 5 This is a visualization diagram of a crowd counting method based on semantic information optimization according to the present invention;

[0046] Figure 6 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the technical solution and advantages of the present application clearer, the technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings. It will be understood that the specific embodiments described herein are only partial embodiments of the present invention, which are only used to explain the present application, rather than to limit the present application. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered isolated, and they can be combined with each other to achieve better technical effects. The same reference numerals appearing in the drawings of the following embodiments represent the same features or components, which can be applied to different embodiments.

[0048] In addition, unless otherwise defined, the technical terms or scientific terms used in the description of the present invention should have the common meanings understood by those skilled in the art in the art to which the present invention belongs.

[0049] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0050] Explanation of reference numerals: electronic device 500 , processor 501 , communication interface 502 , memory 503 , bus 504 .

[0051] Reference Figure 1 , a crowd counting method based on semantic information optimization, comprising:

[0052] Step S100, refer to Figure 2 , using CNN (convolutional neural network) and Transformer (self-attention network model) to form a dual-backbone feature extraction module, which is used to extract features containing global crowd semantic information and local crowd semantic information, including:

[0053] Step S101, introduce the basic version of TwinsTransformer into the crowd counting task and use it as a branch of the dual backbone feature extraction module, i.e. the first backbone model branch, and pre-train on ImageNet-1K.

[0054] Step S102, using the first ten layers of VGG16-BN as another branch of the dual-backbone feature extraction module, i.e., the second backbone model branch, and pre-training on ImageNet-1K.

[0055] In this embodiment, the optimizer uses the Adam-W optimizer, the initial learning rate is set to 1e-5, the weight decay is 1e-4, the batch size is set to 8, and the number of iterations is 2000.

[0056] In the training phase, random cropping and random flipping are used to expand the sample space. Specifically, first, image patches of size are randomly cropped for multi-batch training. Table 1 shows the values ​​of different data sets. Second, the images are randomly flipped horizontally with a probability of 0.5.

[0057] Table 1

[0058]

[0059] Step S200, refer to Figure 3 , design a semantic information fusion module to effectively fuse the crowd features extracted by the dual-backbone feature extraction module composed of CNN and TwinsTransformer, specifically including:

[0060] Step S201, TwinsTransformer consists of four stages, taking the output features of the 2nd, 3rd, and 4th stages , , , the output features of the last three stages of TwinsTransformer are fused to avoid the lack of information in the first stage that affects the technical accuracy. and Adjust to The resolution is obtained and , aligned with the output feature resolution of the CNN branch, thereby achieving higher counting accuracy. In this embodiment, , and The resolution is uniformly adjusted to 1 / 8 of the input image size to make it compatible with both CNN and Transformer branches. As the resolution of the feature map decreases, the amount of computation for the model decreases. A resolution of 1 / 8 can reduce the computational burden while maintaining sufficient details. Moreover, reducing the resolution to 1 / 8 means that each pixel represents a larger physical space, making it easier to capture global semantic information.

[0061] Step S202: after the uniform resolution and Each performs a convolution with a kernel size of 3 3 convolution operation, the number of input channels is 1024 and 512 respectively, and the number of output channels is uniformly adjusted to 512. Perform the same convolution operation and adjust the number of channels to 256 to obtain features with shallow / deep global context information , , , the calculation formula is as follows:

[0062]

[0063]

[0064]

[0065] in, Representative 3 3 convolution operations, Represents an upsampling operation.

[0066] Step S203, and Perform element-by-element addition to obtain features , in order to provide more detailed information, Perform the same convolution operation and unify the number of channels to 256. and Add together to get the fusion feature with global contextual semantic information , which will add more detailed information, the calculation formula is as follows:

[0067]

[0068]

[0069] in, Represents a pixel-level addition operation.

[0070] Step S204: extracting features with stronger local semantic information through the second backbone model branch , thereby improving Transformer's ability to obtain local detail information. and Perform feature fusion to obtain , thereby improving the locality and globality of CNN and Transformer when processing visual tasks.

[0071] Step S300, refer to Figure 4 , design a multi-scale residual attention regression module, the output of the semantic information fusion module is used to reduce the interference caused by scale changes and background noise, and finally output the corresponding prediction density map, specifically including:

[0072] Step S301, design a multi-scale convolution branch, using four parallel branches ( , , , ), each branch contains two convolutional layers with convolution kernels of different sizes ( ), in order to cope with the change of the scale of the crowd's heads. In this embodiment, the branch Including convolution kernel With convolution kernel , branch Contains two convolution kernels , branch Including convolution kernel With convolution kernel , branch Contains two convolution kernels .

[0073] After each branch undergoes convolution, the dimension of the feature map is reduced to 1 / 4 of the original, and the outputs of the four branches are summed to obtain the feature map .

[0074] Step S302, design the attention branch to feature map As input, the attention map As output, it includes:

[0075] Step S302-1, using the ReLU activation function Convolutional layer to reduce the number of channels, and then use a 1 1 convolution layer to maintain the parameters, and then use a Sigmoid function to get the global attention map .

[0076] Step S302-2: feature map and the global attention map Perform element-by-element multiplication to obtain the weighted feature map , to focus on the crowd area and suppress the interference of background noise.

[0077] Step S303, designing a residual branch, the residual branch includes a BN (Batch Normalization) Convolutional layers are used to reduce information loss and enhance the stability of the model, including:

[0078] Step S303-1, using feature map As input and output . and Perform element-by-element addition to obtain the final features .

[0079] Step S303-2: Execute a simple 1. The convolution operation is regressed to obtain the final predicted density map. The formula is as follows:

[0080]

[0081] in, represents an element-wise multiplication operation, Represents an element-wise addition operation.

[0082] Step S400, define a loss function to supervise network training, which is used to measure the difference between the predicted probability of the model and the true label, including technical loss, optimal transmission loss and MSE loss, and mixed supervision loss function , the formula is as follows:

[0083]

[0084] in, and Represent the prediction density map and label density map respectively. represents the counting loss, represents the optimal transmission loss, is the weight of the optimal transmission loss, represents the MSE loss, is the weight of the MSE loss.

[0085] In this embodiment, Set to 0.01, Perform ablation experiments to obtain appropriate values. Combining count-level annotations and location-level annotations further improves the accuracy of the method model in estimating the number of people, allowing the model to learn data features more comprehensively during training.

[0086] Step S500, evaluation and analysis are performed.

[0087] Determine the evaluation indicators, including Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). They are used to indicate the accuracy and robustness between the predicted number of people and the actual number of people. The lower the MAE and RMSE values, the better the model counting performance. The formula is as follows:

[0088]

[0089]

[0090] in, represents the number of crowd images in the test sample, and Respectively represent The predicted and true number of people in a crowd image.

[0091] In order to verify the effectiveness of the crowd counting method in optimizing semantic information, experiments were carried out on five mainstream crowd datasets (ShanghaiTechPartA, ShanghaiTechPartB, UCF-CC-50, UCF-QNRF, and NWPU-Crowd), and the experimental results were compared with the existing crowd counting methods.

[0092] Table 2 shows the comparison of the counting errors of the models on the ShanghaiTech, UCF-QNRF and UCF-CC-50 datasets. A crowd counting method based on semantic information optimization of the present invention is comprehensively compared with several existing crowd counting methods. In ShanghaiTechPartA and ShanghaiTechPartB, the crowd counting method of the present invention achieved the second best results, with only a small gap compared with the best results. The reason for the analysis may be that the model's fitting ability for sparse scenes is insufficient. Most of the images in UCF-QNRF and UCF-CC-50 belong to crowded scenes, and there are serious perspective occlusions and scale changes. The crowd counting method of the present invention has achieved the best MAE on both datasets. In particular, in UCF-CC-50, the MAE result of the crowd counting method of the present invention is 4.1% higher than that of the second best method. The crowd counting method of the present invention achieved the second best RMSE in UCF-CC-50, only 1.1 points behind the best result. This strongly proves the effectiveness and robustness of the fusion feature operation of the crowd counting method in crowded scenes with high population density.

[0093] Table 2

[0094]

[0095] The comparison of counting errors of the NWPU-Crowd dataset is shown in Table 3. NWPU-Crowd is the dataset with the most images, the widest scenes, and the most uniform density distribution in the field of crowd counting. The crowd counting method achieved the best MAE (55.8) and RMSE (167.5) on the validation set, where the RMSE was 32.0% higher than the second best method. On the test set, the crowd counting method obtained the second best result. This highlights that the feature fusion method in the crowd counting method can significantly improve the accuracy of the model in crowded scenes.

[0096] Table 3

[0097]

[0098] The proposed SIRNet is partially visualized on the ShanghaiTechPartA&PartB and UCF-QNRF datasets to verify the model counting performance from a visualization perspective. Figure 5 , the first column is the image to be counted, the second column is the corresponding label density map, the third column is the predicted density map for visualization, GT_Count and EST_Count represent the real number of people and the number of people predicted by the model respectively. From the counting results in the figure, it can be seen that whether it is a sparse scene or a dense scene, this crowd counting method can achieve relatively accurate counting performance.

[0099] Step S600: In order to verify the effectiveness of the crowd counting method based on semantic information optimization of the present invention, an ablation experiment is performed on the ShanghaiTechPartA and UCF-QNRF datasets for comparative analysis.

[0100] Step S601, performing an ablation experiment on the dual-backbone model.

[0101] Three sets of ablation experiments were conducted on the ShanghaiTechPartA and UCF-QNRF datasets, using the loss function When the first 10 layers of VGG16-BN or TwinsTransformer are used as the backbone model, the semantic information fusion module does not need to fuse local and global semantic information, and the multi-scale residual attention regression module remains unchanged. The three combinations are set as follows:

[0102] VGG16-BN: Only the first 10 layers of VGG16-BN are used as the backbone model.

[0103] TwinsTransformer: Use TwinsTransformer only as the backbone model.

[0104] Dual-backbone model: A dual-backbone model consisting of the first 10 layers of VGG16-BN and TwinsTransformer.

[0105] The experimental results are shown in Table 4. When the dual-backbone model is used, the MAE and RMSE on both datasets are optimal. On the ShanghaiTechPartA dataset, the dual-backbone model achieves MAE and RMSE of 53.8 and 88.2, which are 2.5 points and 9.3 points lower than the backbone model using only CNN, and 0.8 points and 4 points lower than the backbone model using only Transformer. On the UCF-QNRF dataset, the dual-backbone model is reduced to MAE (83.8) / RMSE (156.4), which is 18.8 points and 17.1 points lower than the backbone model based on CNN, and 5.8 points and 7 points lower than the backbone model using only Transformer. The experimental data demonstrates the powerful feature extraction capability of TwinsTransformer and fully demonstrates the effectiveness of the fusion of local / global semantic information in improving counting performance when using a dual-backbone model.

[0106] Table 4

[0107]

[0108] Step S602: Verify the hybrid supervision loss function on the ShanghaiTechPartA dataset The effectiveness of the experiment will be used to select the appropriate Three sets of ablation experiments will be conducted, and the loss functions are Euclidean loss function , DM-Count loss function and the mixed supervision loss function , when using hour, The values ​​are set to 0.01, 0.1 and 1 respectively.

[0109] The experimental results are shown in Table 5. and When the loss function is set to 0.1, the optimal MAE (53.8) and RMSE (88.2) are obtained. The MAE and RMSE are reduced by 6.5% and 11.6% respectively compared with the Euclidean loss function. It is proved that the Euclidean loss is more suitable for the pixel-level regression task based on density map than the total variant loss.

[0110] Table 5

[0111]

[0112] Step S603, performing ablation experiments on the multi-scale residual attention regression module on the ShanghaiTechPartA and UCF-QNRF datasets to verify the effectiveness.

[0113] Two sets of experiments were conducted, one of which used a multi-scale residual attention regression module, and the other replaced the multi-scale residual attention regression module with a regression head consisting of three layers of 3×3 convolution cascades, while the rest remained consistent.

[0114] The experimental results are shown in Table 6. When the multi-scale residual attention regression module is used, the model significantly improves the counting performance. On the ShanghaiTechPartA dataset, MAE and RMSE are reduced by 2 points and 4.4 points respectively; on the UCF-QNRF dataset, MAE and RMSE are reduced by 7.4% and 5.6% respectively. The ablation experiment results strongly prove the adaptability of the multi-scale residual attention module in the crowd counting method to scale changes and the anti-interference ability to background noise in non-crowd areas, which fully confirms the effectiveness of the multi-scale residual attention regression module in improving counting performance.

[0115] Table 6

[0116]

[0117] The disclosed embodiment also provides a crowd counting system based on semantic information optimization, which uses the above-mentioned crowd counting method based on semantic information optimization.

[0118] The embodiment of the present disclosure further provides a storage medium, in which a computer program is stored. When the computer program is executed by a processor, all steps of the above-mentioned crowd counting method based on semantic information optimization can be implemented.

[0119] A person of ordinary skill in the art can understand that the implementation of all or part of the process in a crowd counting method based on semantic information optimization can be completed by instructing related hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include a process of various embodiments of a crowd counting method based on semantic information optimization. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0120] The embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the crowd counting method based on semantic information optimization are implemented. In the embodiment of the present application, the processor is the control center of the computer system, which can be a processor of a physical machine or a processor of a virtual machine.

[0121] Reference Figure 6 The electronic device 500 includes: at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one bus 504. The bus 504 is used to realize the connection and communication between these components, the communication interface 502 is used to communicate signaling or data with other node devices, and the memory 503 stores machine-readable instructions executable by the processor 501. When the electronic device 500 is running, the processor 501 communicates with the memory 503 through the bus 504, and when the machine-readable instructions are called by the processor 501, the steps of the crowd counting method based on semantic information optimization are executed.

[0122] The above contents are only embodiments of the present invention. The common sense such as the known specific structures and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for the ordinary technicians in the relevant field to implement this application. It should be pointed out that for the technicians in this field, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to explain the content of the claims.

Claims

1. A crowd counting method based on semantic information optimization, Features: include: A dual-backbone feature extraction module consisting of CNN and Transformer is used to extract features containing global crowd semantic information and local crowd semantic information at the same time; Design a semantic information fusion module to fuse the crowd features extracted by the dual-backbone feature extraction module to obtain fusion features. Includes the following: The output features of the 2nd, 3rd, and 4th stages of TwinsTransformer , , to merge; to and Adjust to The resolution is obtained and ; right , , Perform a convolution with a kernel size of 3 3 convolution operations to obtain features with shallow / deep global context information , , , the calculation formula is as follows: in, Representative 3 3 convolution operations, represents an upsampling operation; right and Perform element-by-element addition to obtain features ,right Perform a convolution with a kernel size of 3 3 convolution operation, and and Add together to get the fusion feature with global contextual semantic information , the calculation formula is as follows: in, represents pixel-level addition operation; Extract features with stronger local semantic information through the second backbone model branch ,Will and Perform feature fusion to obtain , Input for the multi-scale residual attention regression module; Design a multi-scale residual attention regression module, including a multi-scale convolution branch, an attention branch, and a residual branch, to decode the fused features and obtain a predicted density map; Design multi-scale convolution branches, including: Using four parallel branches , , , , branch Including convolution kernel With convolution kernel , branch Contains two convolution kernels , branch Including convolution kernel With convolution kernel , branch Contains two convolution kernels , the outputs of the four branches are summed to obtain the feature map ; Design the attention branch, including: With the ReLU activation function Convolutional layer to reduce the number of channels, and then use a 1 1 convolution layer to maintain the parameters, and then use a Sigmoid function to get the global attention map ; The feature map and the global attention map Perform element-by-element multiplication to obtain the weighted feature map ; Design residual branches, including those with BN Convolutional layers, including: Feature map As input and output ; and Perform element-by-element addition to obtain the final features ;right Execute a 1 1. The convolution operation is regressed to obtain the final predicted density map. The formula is as follows: in, represents an element-wise multiplication operation, Represents an element-by-element addition operation.

2. A crowd counting method based on semantic information optimization according to claim 1, Features: Also includes: Define a loss function to supervise network training, which is used to measure the difference between the model's predicted probability and the true label, mixed supervision loss function , the formula is as follows: in, represents the predicted density map, represents the label density map, represents the counting loss, represents the optimal transmission loss, is the weight of the optimal transmission loss, represents the MSE loss, is the weight of the MSE loss.

3. A crowd counting method based on semantic information optimization according to claim 2, Features: The dual-backbone feature extraction module is composed of CNN and Transformer, which includes the following contents: The basic version of Twins Transformer is used as the first backbone model branch, and the first ten layers of VGG16-BN are used as the second backbone model branch. They are pre-trained on ImageNet-1K. The Adam-W optimizer is used as the optimizer, and random cropping and random flipping data enhancement methods are used to expand the sample space.

4. The crowd counting method based on semantic information optimization according to claim 1, Features: , and The resolution is uniformly adjusted to 1 / 8 of the input image size.

Images