Crowd counting method and device based on deep learning, and readable storage medium
By adopting deep learning methods in the population counting algorithm, combining VAN network and CBAM attention mechanism for feature extraction, and performing expansion convolution and multi-scale feature fusion, the problem of low population counting accuracy in the prior art is solved, and higher counting accuracy and real-time performance are achieved.
Patent Information
- Application Number
- CN202411864410.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-04-15
AI Technical Summary
The existing population counting algorithms have the problem of low accuracy of results, especially when it comes to excessive changes in the population scale, serious background noise, serious overlap of people, and perspective distortion, it is difficult to accurately count.
The population counting method based on deep learning is adopted, and feature extraction is performed through the visual attention network VAN network and the convolution block attention module CBAM attention mechanism, combining expanded convolution and multi-scale feature fusion to reduce background interference and perspective distortion effects and improve counting accuracy.
By effectively suppressing the interference of background noise and overlap of people, reducing the impact of perspective distortion on counting results, significantly improving the accuracy of crowd counting, and accurately counting large-scale populations in real-time image processing.
Smart Images
Figure CN119832490A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a crowd counting method, device and readable storage medium based on deep learning. Background Art
[0002] Crowd counting is a research direction in the field of computer vision, which refers to the use of image processing and machine learning techniques to estimate the number of people in an image or video scene. This technology has important application value in many fields such as social management, public safety, and business analysis.
[0003] At present, the application of crowd counting algorithms mainly includes the following aspects:
[0004] Public safety monitoring: At large events such as sporting events, concerts, rallies, etc., crowd counting helps ensure crowd safety and prevent dangerous situations caused by crowding.
[0005] Business analysis: In a retail environment, counting the number of customers can help merchants analyze customer flow and optimize resource allocation and marketing strategies.
[0006] Traffic Management: At transportation hubs, such as subway stations, railway stations, etc., crowd counting helps in traffic planning and flow control.
[0007] However, existing crowd counting algorithms suffer from low result accuracy. Summary of the invention
[0008] The technical problem to be solved by the present application is to provide a crowd counting method, device and readable storage medium based on deep learning to solve the problems existing in the prior art in view of the above-mentioned deficiencies in the prior art.
[0009] In a first aspect, the present application provides a crowd counting method based on deep learning, the method comprising:
[0010] S1, obtaining the target image to be detected;
[0011] S2, extracting features of the target image through a visual attention network (VAN) network and a convolutional block attention module (CBAM) attention mechanism to obtain attention features at multiple different stages;
[0012] S3, performing dilated convolution, downsampling and feature fusion processing according to the attention features of multiple different stages to obtain multiple dilated features;
[0013] S4, performing multi-scale feature fusion processing according to the multiple attention features at different stages and the multiple expansion features to obtain fusion features;
[0014] S5. Perform recognition and counting processing based on the fusion features to obtain a crowd counting result of the target image.
[0015] In some embodiments, S2 includes:
[0016] Extracting features of the target image through a VAN network to obtain initial features;
[0017] The initial features are passed into the CBAM attention mechanism, the corresponding channel attention is obtained through the channel attention module, and the first-stage attention features are obtained through the spatial attention module. The first-stage attention features are the spatial attention of the mixed channel attention;
[0018] The first-stage attention features are continuously downsampled to obtain attention features of multiple different stages.
[0019] In some embodiments, feature extraction is performed on the target image through a VAN network to obtain initial features, including:
[0020] The target image is sequentially subjected to feature extraction through a deep convolution module, a deep extended convolution module, and a 1×1 convolution module in the VAN network to obtain initial features.
[0021] In some embodiments, the total number of the attention features is N, where N is an integer greater than or equal to 2, and the attention features of the latter stage are obtained by downsampling the attention features of the previous stage;
[0022] S3, including:
[0023] Taking the first-stage attention feature as the target object, performing a first-multiple dilated convolution process on the target object, and performing a downsampling process on the dilated result to obtain a first dilated downsampling feature; performing a second-multiple dilated convolution process on the first dilated downsampling feature, and performing a downsampling process on the dilated result to obtain a second dilated downsampling feature, and repeating the above process until a preset number of dilation times is reached;
[0024] From the multiple dilated downsampled features of the first-stage attention features, determine the target dilated downsampled features with the same number of channels as the second-stage attention features;
[0025] The target dilated down-sampled features are fused with the second-stage attention features, and the fusion result is used as the target object again, and the above process is repeated until the dilated down-sampled features of the N-1th stage attention features are obtained;
[0026] The down-sampled features corresponding to the dilation results with the largest dilation multiple of the attention features in each stage in the dilated convolution process are taken as dilated features, and N-1 dilated features are obtained.
[0027] In some embodiments, in the first stage attention feature to the N-1th stage attention feature, the number of expansions of the i-th stage attention feature is Ni, and the expansion multiples of each expansion of the i-th stage attention feature are different;
[0028] Here, i is an integer greater than or equal to 1 and less than or equal to N-1.
[0029] In some embodiments, S4 includes:
[0030] The Nth stage attention feature among the multiple different stages of attention features is subjected to multi-scale feature fusion processing with N-1 expansion features to obtain a fused feature.
[0031] In some embodiments, the initial features are passed into the CBAM attention mechanism, the corresponding channel attention is obtained through the channel attention module, and the first-stage attention features are obtained through the spatial attention module, including:
[0032] The initial features are respectively subjected to global maximum pooling and global average pooling by the channel attention module, and the obtained global maximum pooling results and global average pooling results are transmitted to the shared perceptron to obtain channel attention;
[0033] The channel attention is globally averaged and globally maximized pooled along the channel direction through the spatial attention module, and the global maximum pooling result and the global average pooling result are concatenated in the channel direction to obtain the first-stage attention feature.
[0034] In a second aspect, the present application provides a crowd counting device based on deep learning, the device comprising:
[0035] An image acquisition module, which is configured to acquire a target image to be detected;
[0036] A feature extraction module is configured to extract features of the target image through a visual attention network (VAN) network and a convolutional block attention module (CBAM) attention mechanism to obtain attention features at multiple different stages;
[0037] An expansion processing module, which is configured to perform expansion convolution, downsampling, and feature fusion processing according to the attention features of multiple different stages to obtain multiple expansion features;
[0038] A feature fusion module, configured to perform multi-scale feature fusion processing according to the attention features at different stages and the expansion features to obtain a fusion feature;
[0039] The counting processing module is configured to perform identification and counting processing based on the fusion features to obtain a crowd counting result of the target image.
[0040] In a third aspect, the present application provides a crowd counting device based on deep learning, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the crowd counting method based on deep learning described in the first aspect above.
[0041] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the crowd counting method based on deep learning described in the first aspect above is implemented.
[0042] The present application provides a crowd counting method, device and readable storage medium based on deep learning, the method comprising: obtaining a target image to be detected; extracting features of the target image through a visual attention network VAN network and a convolution block attention module CBAM attention mechanism to obtain attention features of multiple different stages; performing dilated convolution, downsampling and feature fusion processing according to the attention features of multiple different stages to obtain multiple dilated features; performing multi-scale feature fusion processing according to the attention features of the multiple different stages and the multiple dilated features to obtain fusion features; performing recognition counting processing based on the fusion features to obtain the crowd counting result of the target image. The present application proposes a crowd counting algorithm based on deep learning, which extracts features by using a VAN network that combines a convolutional neural network and a Transformer, so that feature values with long-range dependence can be effectively obtained, and the running speed block of the convolutional neural network is characterized by the block. Adding the CBAM attention mechanism in the convolution process reduces the interference of the overlap of background objects and crowds on model extraction, and superimposes the feature values of each stage to obtain multi-scale feature information, thereby reducing the impact of perspective distortion and improving the accuracy of crowd counting results. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0044] Figure 1 A flowchart of a crowd counting method based on deep learning provided in an embodiment of the present application;
[0045] Figure 2 A schematic diagram of the CBAM attention mechanism provided in an embodiment of the present application;
[0046] Figure 3A schematic diagram of the channel attention mechanism architecture provided for an embodiment of the present application;
[0047] Figure 4 A schematic diagram of the spatial attention mechanism architecture provided for an embodiment of the present application;
[0048] Figure 5 An example diagram of counting processing performed by the solution of the present application provided in the embodiment of the present application;
[0049] Figure 6 A schematic diagram of the structure of a crowd counting device based on deep learning provided in an embodiment of the present application;
[0050] Figure 7 A schematic diagram of the structure of another crowd counting device based on deep learning provided in an embodiment of the present application.
[0051] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0052] In order to enable those skilled in the art to better understand the technical solution of the present application, the implementation mode of the present application will be further described in detail below with reference to the accompanying drawings.
[0053] It should be understood that the specific embodiments and drawings described herein are only used to explain the present application, rather than to limit the present application.
[0054] It can be understood that, in the absence of conflict, the various embodiments in the present application and the various features in the embodiments can be combined with each other.
[0055] It can be understood that, for the convenience of description, the drawings of the present application only show the parts related to the present application, while the parts unrelated to the present application are not shown in the drawings.
[0056] It can be understood that each unit and module involved in the embodiments of the present application may correspond to only one physical structure, or may be composed of multiple physical structures, or multiple units and modules may be integrated into one physical structure.
[0057] It can be understood that the terms "first", "second", etc. in the embodiments of the present application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.
[0058] It is understandable that, in the absence of conflict, the functions and steps marked in the flowcharts and block diagrams of the present application may occur in an order different from that marked in the drawings.
[0059] It is understood that the flowcharts and block diagrams of the present application illustrate the possible architectures, functions, and operations of the systems, devices, equipment, and methods according to the various embodiments of the present application. Among them, each box in the flowchart or block diagram may represent a unit, module, program segment, code, which contains executable instructions for implementing the specified functions. Moreover, each box or combination of boxes in the block diagram and flowchart may be implemented by a hardware-based system that implements the specified functions, or by a combination of hardware and computer instructions.
[0060] It can be understood that the units and modules involved in the embodiments of the present application can be implemented by software or hardware, for example, the units and modules can be located in a processor.
[0061] Existing crowd counting algorithms can be divided into two categories: counting algorithms based on traditional methods and counting algorithms based on deep learning.
[0062] 1. Counting algorithm based on traditional methods
[0063] Early traditional work mainly used detection-based methods to detect specific parts of the human body for crowd counting. However, due to severe occlusion of the crowd in actual scenes, it is impossible to accurately identify the human body. In addition, due to the serious overlap between background objects and the crowd, background objects are easily misidentified as people, which affects the performance of the method and makes it impossible to accurately count.
[0064] 2. Counting algorithm based on deep learning
[0065] Deep learning, especially convolutional neural networks, has achieved outstanding results in computer vision related fields such as target detection and image segmentation. For example, applying deep learning technology to crowd counting tasks by combining convolutional neural networks with density maps has significantly surpassed the performance of traditional methods. This method can efficiently and accurately count the number of large crowds and show excellent counting results. For another example, by using dense dilated convolution blocks, image information at multiple scales can be obtained, and more contextual information can be obtained through dense connections, so that images can be better recognized.
[0066] However, existing crowd counting algorithms have the following disadvantages:
[0067] 1. The scale of the crowd changes too much: Due to large-scale crowd gatherings and other reasons, the scale of the portrait changes too much during the image processing process, which increases the difficulty of recognition. The existing methods mainly use convolutional neural networks to process images, but the extraction ability is weak in the feature extraction process and cannot deal with this problem well. With the rise of the Transformer model, more feature information can be obtained through long-range dependencies for processing, but because the model is too complex, its deployment and training require relatively more resources and time, making it difficult to perform real-time image processing, and its actual application scenarios have certain limitations.
[0068] 2. Severe background noise and serious crowd overlap interfere with recognition: Due to the presence of a large number of background objects in image processing, there is a large amount of background noise in the feature extraction process, which brings a lot of interference to the recognition and processing of the model, thereby reducing the recognition ability of the model. In addition, due to the serious overlap of the crowd during the shooting process, it is impossible to effectively identify a single person from the overlapping part during the crowd recognition process, which causes certain difficulties in recognition.
[0069] 3. Serious perspective distortion: During the shooting process, different people are at different distances from the shooting point, and the same person will change the distance from the shooting point over time, so people will have different degrees of deformation during the shooting process. After the model extracts features from the image, it cannot fully utilize the extracted feature information, which affects the recognition effect and its accuracy.
[0070] In response to the above shortcomings, this application proposes a crowd counting algorithm based on deep learning. By adopting a convolutional neural network with long-range dependencies, it fully combines the operating efficiency of the convolutional neural network and the advantages of the long-range dependencies of the Transformer to improve the feature extraction ability of the model. A hybrid attention mechanism is used to ensure that background noise can be effectively suppressed in the feature extraction stage to ensure that effective information can be obtained and reduce the interference caused by crowd overlap. By adopting a multi-scale feature fusion operation to fuse feature values of different scales, feature information of different scales can be obtained to reduce the impact of perspective distortion and improve the accuracy of crowd counting results.
[0071] This application can conduct a comprehensive evaluation of the recognition effect on large-scale crowds, and ultimately be able to conduct real-time statistics on the flow of people, thereby ensuring urban governance and public safety.
[0072] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0073] The present application provides a crowd counting method based on deep learning. The working process of the method can be implemented by electronic devices, such as computers, handheld smart terminals, etc. For the convenience of explanation, the embodiments of the present application are described with the method execution subject being a computer.
[0074] Figure 1 A schematic diagram of a crowd counting method based on deep learning provided in an embodiment of the present application, such as Figure 1 As shown, the present application provides a crowd counting method based on deep learning, and the method includes S1-S5, which are as follows:
[0075] S1, obtaining the target image to be detected;
[0076] The target image to be detected is an image containing people who need to be counted.
[0077] S2, extracting features of the target image through a visual attention network (VAN) network and a convolutional block attention module (CBAM) attention mechanism to obtain attention features at multiple different stages;
[0078] In the present application, the network architecture for image processing includes a feature extractor based on a visual attention network (Visual Attention Network, VAN network), which processes images by combining the advantages of convolutional neural networks and Transformers.
[0079] In crowd counting tasks, convolutional neural networks are often used as backbone networks, but they lack a self-attention mechanism and cannot cope with large changes in crowd size compared to Transformer. As a new backbone network, Transformer's unique self-attention mechanism module can obtain long-range dependencies during image processing, but its parameters are too large and requires a lot of resources to train, which limits the practical application of the model. VAN network is a neural network architecture that aims to simulate the attention mechanism in the human visual system to better process information in images and videos. In the field of artificial intelligence, especially in computer vision, VAN helps models focus on and parse key parts of visual scenes more effectively, thereby improving the performance of tasks such as image classification, object detection, and image segmentation.
[0080] VAN has a simple hierarchical structure. Taking a four-level sequence as an example, its resolution and channel changes are as follows: the output spatial resolution decreases, and is H / 4x W / 4, H / 8xW / 8, H / 16x W / 16, and H / 32x W / 32. H and W represent the height and width of the input image. As the resolution decreases, the number of output channels increases.
[0081] In addition, the present application adds a CABM (Convolutional Block Attention Module) attention mechanism in each layer to suppress adverse factors such as background interference in the feature extraction process.
[0082] In some embodiments, S2 includes:
[0083] Extracting features of the target image through a VAN network to obtain initial features;
[0084] The initial features are passed into the CBAM attention mechanism, the corresponding channel attention is obtained through the channel attention module, and the first-stage attention features are obtained through the spatial attention module. The first-stage attention features are the spatial attention of the mixed channel attention;
[0085] The first-stage attention features are continuously downsampled to obtain attention features of multiple different stages.
[0086] In some embodiments, feature extraction is performed on the target image through a VAN network to obtain initial features, including:
[0087] The target image is sequentially subjected to feature extraction through a deep convolution module, a deep extended convolution module, and a 1×1 convolution module in the VAN network to obtain initial features.
[0088] Specifically, the VAN network combines the long-range dependency of self-attention with the convolutional neural network by introducing large kernel attention to improve the feature extraction ability of the model. Large kernel attention can fully obtain the feature information of polyps while also obtaining the long-range dependency in the polyp image by using large kernel convolution.
[0089] The large core attention splits it into deep convolution, deep dilated convolution and 1×1 convolution modules, and extracts features of the polyp image through deep convolution, deep dilated convolution and 1×1 convolution modules respectively. The calculation formula is as follows:
[0090] Attention=Conv 1×1 (DW-D-Conv(DW-conv(F)))
[0091] Output=Attention×F
[0092] Among them, DW-Conv is a deep convolution, DW-D-Conv is a deep extended convolution, Conv 1×1 is a 1×1 channel convolution. F∈R C×H×W is the input feature value, Attention∈RC×H×W represents the attention feature map, that is, the initial feature, and × is the dot product.
[0093] In some embodiments, the initial features are passed into the CBAM attention mechanism, the corresponding channel attention is obtained through the channel attention module, and the first-stage attention features are obtained through the spatial attention module, including:
[0094] The initial features are respectively subjected to global maximum pooling and global average pooling by the channel attention module, and the obtained global maximum pooling results and global average pooling results are transmitted to the shared perceptron to obtain channel attention;
[0095] The channel attention is globally averaged and globally maximized pooled along the channel direction through the spatial attention module, and the global maximum pooling result and the global average pooling result are concatenated in the channel direction to obtain the first-stage attention feature.
[0096] Since there are a lot of background objects that interfere with the crowd during the feature extraction process, there is a lot of interference information in the feature extraction process, which reduces the accuracy of the model. In addition, due to the serious overlap of people, it is difficult to identify people. Therefore, in order to reduce the interference caused by the overlap of background objects and people, the CBAM attention mechanism is introduced into the model. As a mainstream hybrid attention mechanism, the CBAM attention mechanism can be embedded in the VAN network to effectively suppress the interference problem in the downsampling process by combining channel attention with spatial attention, so that the network pays more attention to crowd information and improves the accuracy of the network.
[0097] Figure 2 A schematic diagram of the CBAM attention mechanism provided in an embodiment of the present application, Figure 3 A schematic diagram of the channel attention mechanism architecture provided in an embodiment of the present application, Figure 4 A schematic diagram of the spatial attention mechanism architecture provided in the embodiment of the present application, such as Figures 2 to 4 As shown in the figure, the specific process of the CBAM attention mechanism is as follows:
[0098] First, the channel attention module performs global maximum pooling and global average pooling on the input eigenvalues to generate two eigenvalues. The two generated eigenvalues are passed to the shared perceptron to obtain channel information. The two output eigenvalues of the shared perceptron are added together and the channel attention vector is generated through the Sigmoid function. The channel attention formula is as follows:
[0099]
[0100] Among them, F is the input eigenvalue, M c(F) is channel attention, AvgPool is average pooling, MaxPool is maximum pooling, MLP is shared perceptron, σ is Sigmoid function, M c is the output channel attention feature value.
[0101] Then, the channel attention vector generated by the channel attention mechanism is globally averaged and globally max-pooled along the channel direction to reduce the channel dimension, thereby paying more attention to spatial information. The two pooled feature values obtained are concatenated in the channel direction, and then a 7×7 convolution is performed to extract higher-level feature values. Finally, the spatial attention vector is generated through the Sigmoid function. The formula is as follows:
[0102]
[0103] Among them, F is the input feature value (i.e. channel attention), M e (F) is spatial attention, AvgPool is average pooling, MaxPool is maximum pooling, f 7×7 is a 7×7 convolution, and δ is a Sigmoid function.
[0104] S3, performing dilated convolution, downsampling and feature fusion processing according to the attention features of multiple different stages to obtain multiple dilated features;
[0105] In some embodiments, the total number of the attention features is N, where N is an integer greater than or equal to 2, and the attention features of the latter stage are obtained by downsampling the attention features of the previous stage;
[0106] S3, including:
[0107] Taking the first-stage attention feature as the target object, performing a first-multiple dilated convolution process on the target object, and performing a downsampling process on the dilated result to obtain a first dilated downsampling feature; performing a second-multiple dilated convolution process on the first dilated downsampling feature, and performing a downsampling process on the dilated result to obtain a second dilated downsampling feature, and repeating the above process until a preset number of dilation times is reached;
[0108] From the multiple dilated downsampled features of the first-stage attention features, determine the target dilated downsampled features with the same number of channels as the second-stage attention features;
[0109] The target dilated down-sampled features are fused with the second-stage attention features, and the fusion result is used as the target object again, and the above process is repeated until the dilated down-sampled features of the N-1th stage attention features are obtained;
[0110] The down-sampled features corresponding to the dilation results with the largest dilation multiple of the attention features in each stage in the dilated convolution process are taken as dilated features, and N-1 dilated features are obtained.
[0111] In some embodiments, in the first stage attention features to the N-1 stage attention features, the number of expansions of the i-th stage attention features is Ni, and the expansion multiples of each expansion of the i-th stage attention features are different; wherein i is an integer greater than or equal to 1 and less than or equal to N-1.
[0112] Specifically, the detail information decreases as the downsampling continues to deepen, but the rich detail information can effectively help the model recognize crowd information. Therefore, in order to obtain more feature information, this application performs dilated convolution processing of different proportions on downsampling feature values of different depths to obtain different perceptual domains, so as to obtain more feature information therefrom.
[0113] For example, the expansion ratios of the eigenvalues from shallow to deep are {1, 2, 3}, {1, 2}, {1}, etc.
[0114] The eigenvalues after dilated convolution processing are added to the eigenvalues of the lower layer to perform multi-scale feature fusion. Finally, the eigenvalues of each layer are spliced to better integrate the information of shallow features and deep features, so as to better deal with the problem of perspective distortion.
[0115] In this embodiment, the expanded down-sampling features after each layer extraction are fused with the corresponding stage attention features to obtain multi-scale feature values, which can better cope with the recognition difficulties caused by perspective distortion problems.
[0116] S4, performing multi-scale feature fusion processing according to the multiple attention features at different stages and the multiple expansion features to obtain fusion features;
[0117] In some embodiments, S4 includes:
[0118] The Nth stage attention feature among the multiple different stages of attention features is subjected to multi-scale feature fusion processing with N-1 expansion features to obtain a fused feature.
[0119] S5. Perform recognition and counting processing based on the fusion features to obtain a crowd counting result of the target image.
[0120] After the fusion features are obtained, identification and counting processing is performed through a related counting algorithm to obtain a crowd counting result of the target image.
[0121] This application proposes a crowd counting algorithm based on deep learning. By using a VAN network that combines a convolutional neural network and a Transformer to extract features, it can effectively obtain feature values with long-range dependencies and has the characteristics of a convolutional neural network running speed block. Adding a CBAM attention mechanism in the convolution process reduces the interference of background objects and crowds on model extraction, and superimposing the feature values of each stage to obtain multi-scale feature information, thereby reducing the impact of perspective distortion and improving the accuracy of crowd counting results.
[0122] Figure 5 An example diagram of counting processing by the solution of the present application is provided in the embodiment of the present application, such as Figure 5 As shown, the overall technical solution of this application is as follows:
[0123] (1) The target image to be detected is cropped to 512×512, the learning rate is set to 0.00001, the weight decay is 0.0001, and training is performed for 1000 rounds.
[0124] (2) The image is passed into the feature extraction network for feature extraction. Through the VAN network + CBAM attention mechanism, the first-stage attention features with a channel number of 32 are obtained, and then passed to the subsequent stages for further downsampling to obtain the second-stage attention features, third-stage attention features, and fourth-stage attention features with channel sizes of 64, 160, and 256, respectively.
[0125] (3) After obtaining the attention features of each stage, dilated convolution, downsampling and feature fusion processing are performed.
[0126] First, the attention features of the first stage are subjected to dilated convolution with dilation multiples of 1, 2, and 3, and are continuously downsampled during the dilation process to obtain three dilated downsampled features.
[0127] Secondly, the eigenvalues with the same number of channels in the second-stage attention features and the eigenvalues downsampled during the expansion process of the first-stage attention features are added, and then the eigenvalues after the addition are dilated and convolved with expansion multiples of 1 and 2, and then downsampled.
[0128] Then, the third-stage attention feature is added with the feature values with the same number of channels that are downsampled during the expansion process of the second-stage attention feature, and then a 1-fold expansion convolution is performed and downsampled.
[0129] Finally, the down-sampled features corresponding to the dilation results with the largest dilation multiples of the attention features in each stage in the dilated convolution processing are used as dilated features, the three dilated features are added to the attention features of the fourth stage, the fused feature values are processed, and finally the final counting recognition results are generated.
[0130] It should be understood that, although the various steps in the flowcharts in the above-described embodiments are sequentially displayed according to the indications of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily to be carried out sequentially, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0131] Figure 6 A schematic diagram of a crowd counting device based on deep learning provided in an embodiment of the present application, such as Figure 6 As shown, the present application provides a crowd counting device based on deep learning, the device comprising:
[0132] An image acquisition module 11 is configured to acquire a target image to be detected;
[0133] A feature extraction module 12 is configured to extract features of the target image through a visual attention network (VAN) network and a convolutional block attention module (CBAM) attention mechanism to obtain attention features at multiple different stages;
[0134] An expansion processing module 13, which is configured to perform expansion convolution, downsampling and feature fusion processing according to the attention features of multiple different stages to obtain multiple expansion features;
[0135] A feature fusion module 14, which is configured to perform multi-scale feature fusion processing according to the attention features of the multiple different stages and the multiple expansion features to obtain a fusion feature;
[0136] The counting processing module 15 is configured to perform identification and counting processing based on the fusion features to obtain a crowd counting result of the target image.
[0137] Regarding the limitation of the crowd counting device based on deep learning, reference may be made to the limitation of the crowd counting method based on deep learning in the above embodiments of the present application, which will not be elaborated in this embodiment.
[0138] Figure 7 Another schematic diagram of a crowd counting device based on deep learning provided in an embodiment of the present application is shown in FIG. Figure 7As shown, in some embodiments, the present application provides a crowd counting device based on deep learning, including a memory 22 and a processor 21, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the crowd counting method based on deep learning in the above-mentioned embodiments of the present application.
[0139] The memory is connected to the processor, the memory may be a flash memory or a read-only memory or other memory, and the processor may be a central processing unit or a single-chip microcomputer.
[0140] In some embodiments, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the crowd counting method based on deep learning in the above-mentioned embodiments of the present application is implemented.
[0141] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.
[0142] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present application, but the present application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of the present application, and these modifications and improvements are also considered to be within the scope of protection of the present application.
Claims
1. A crowd counting method based on deep learning, characterized in that: The method comprises: S1, obtaining the target image to be detected; S2, extracting features of the target image through a visual attention network (VAN) network and a convolutional block attention module (CBAM) attention mechanism to obtain attention features at multiple different stages; S3, performing dilated convolution, downsampling and feature fusion processing according to the attention features of multiple different stages to obtain multiple dilated features; S4, performing multi-scale feature fusion processing according to the multiple attention features at different stages and the multiple expansion features to obtain fusion features; S5. Perform recognition and counting processing based on the fusion features to obtain a crowd counting result of the target image.
2. The crowd counting method based on deep learning according to claim 1, characterized in that: S2, including: Extracting features of the target image through a VAN network to obtain initial features; The initial features are passed into the CBAM attention mechanism, the corresponding channel attention is obtained through the channel attention module, and the first-stage attention features are obtained through the spatial attention module. The first-stage attention features are the spatial attention of the mixed channel attention; The first-stage attention features are continuously downsampled to obtain attention features of multiple different stages.
3. The crowd counting method based on deep learning according to claim 2, characterized in that: The target image is subjected to feature extraction through the VAN network to obtain initial features, including: The target image is sequentially subjected to feature extraction through a deep convolution module, a deep extended convolution module, and a 1×1 convolution module in the VAN network to obtain initial features.
4. The crowd counting method based on deep learning according to claim 2, characterized in that: The total number of the attention features is N, where N is an integer greater than or equal to 2, and the attention features of the latter stage are obtained by downsampling the attention features of the previous stage; S3, including: Taking the first-stage attention feature as the target object, performing a first-multiple dilated convolution process on the target object, and performing a downsampling process on the dilated result to obtain a first dilated downsampling feature; performing a second-multiple dilated convolution process on the first dilated downsampling feature, and performing a downsampling process on the dilated result to obtain a second dilated downsampling feature, and repeating the above process until a preset number of dilation times is reached; From the multiple dilated downsampled features of the first-stage attention features, determine the target dilated downsampled features with the same number of channels as the second-stage attention features; The target dilated down-sampled features are fused with the second-stage attention features, and the fusion result is used as the target object again, and the above process is repeated until the dilated down-sampled features of the N-1th stage attention features are obtained; The down-sampled features corresponding to the dilation results with the largest dilation multiple of the attention features in each stage in the dilated convolution process are taken as dilated features, and N-1 dilated features are obtained.
5. The crowd counting method based on deep learning according to claim 4, characterized in that: In the first stage attention features to the N-1 stage attention features, The number of expansions of the attention feature in the i-th stage is Ni, and the expansion multiples of each expansion of the attention feature in the i-th stage are different; Here, i is an integer greater than or equal to 1 and less than or equal to N-1.
6. The crowd counting method based on deep learning according to claim 4, characterized in that: S4, including: The Nth stage attention feature among the multiple different stages of attention features is subjected to multi-scale feature fusion processing with N-1 expansion features to obtain a fused feature.
7. The crowd counting method based on deep learning according to claim 2, characterized in that: The initial features are passed into the CBAM attention mechanism, the corresponding channel attention is obtained through the channel attention module, and the first-stage attention features are obtained through the spatial attention module, including: The initial features are respectively subjected to global maximum pooling and global average pooling by the channel attention module, and the obtained global maximum pooling results and global average pooling results are transmitted to the shared perceptron to obtain channel attention; The channel attention is globally averaged and globally maximized pooled along the channel direction through the spatial attention module, and the global maximum pooling result and the global average pooling result are concatenated in the channel direction to obtain the first-stage attention feature.
8. A crowd counting device based on deep learning, characterized in that: The device comprises: An image acquisition module, which is configured to acquire a target image to be detected; A feature extraction module is configured to extract features of the target image through a visual attention network (VAN) network and a convolutional block attention module (CBAM) attention mechanism to obtain attention features at multiple different stages; An expansion processing module, which is configured to perform expansion convolution, downsampling, and feature fusion processing according to the attention features of multiple different stages to obtain multiple expansion features; A feature fusion module, configured to perform multi-scale feature fusion processing according to the attention features at different stages and the expansion features to obtain a fusion feature; The counting processing module is configured to perform identification and counting processing based on the fusion features to obtain a crowd counting result of the target image.
9. A crowd counting device based on deep learning, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the crowd counting method based on deep learning as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the crowd counting method based on deep learning as described in any one of claims 1 to 7 is implemented.