Method for generating domain adaptive crowd counting data set based on SA-CycleGAN

Abstract

The invention discloses a method for generating a domain adaptive crowd counting data set based on SA-CycleGAN, and the method comprises the following steps: building a virtual scene based on a simulation platform, and collecting a virtual crowd counting data set in a source domain; automatically marking pedestrian images in the virtual crowd counting data set; performing style migration on the marked virtual crowd counting data set in the source domain by using the trained domain adaptation network model to generate a real crowd counting data set with a target domain style; and the domain adaptive network model is constructed on the basis of an SA-CycleGAN network structure. According to the method, cross-domain self-adaption can be realized in different scenes, and the problem that the performance of a data set is greatly reduced when other scenes are checked can be relieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and crowd counting, and more specifically to a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN. Background Art

[0002] Crowd counting is a branch of crowd analysis and plays an important role in video surveillance, public area planning, traffic flow monitoring, etc. The goal of this task is to predict density maps and estimate the number of people in a crowd scene. Currently, many CNN and GAN based methods have achieved amazing performance on existing datasets.

[0003] However, the above mainstream deep learning methods require a large amount of accurately labeled and diverse data. However, the current datasets cannot fully meet the needs, which also leads to two thorny problems in crowd counting. First, the existing datasets lack the diversity of scenes and lack comprehensive coverage of different times, weather, lighting conditions and large-scale crowd gatherings. This single scene limits the model's generalization ability to the complex and changeable real world. In addition, the scarcity of labeled data also makes many crowd counting algorithms prone to overfitting during the training process. This overfitting problem greatly limits the generalization ability of the algorithm, causing their performance to drop significantly when facing new scenes or complex environments, making it difficult to count accurately. In addition, there is an inherent problem with dense crowd datasets: the labels are not very accurate.

[0004] Therefore, how to improve the generalization ability of crowd data sets in order to improve the accuracy of crowd counting is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of the above problems, the present invention provides a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN to at least solve some of the technical problems mentioned in the above background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] The present invention provides a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN, comprising the following steps:

[0008] S1. Build a virtual scene based on the simulation platform and collect virtual crowd counting datasets in the source domain;

[0009] S2, automatically labeling pedestrian images in the virtual crowd counting dataset;

[0010] S3. Use the trained domain adaptation network model to perform style transfer on the virtual crowd counting dataset annotated in the source domain to generate a real crowd counting dataset with the style of the target domain.

[0011] The domain adaptation network model is built based on the SA-CycleGAN network structure.

[0012] Furthermore, it also includes:

[0013] S4. The fusion quality of the real crowd counting dataset is tested and evaluated through quantitative quality evaluation indicators, and the SA-CycleGAN network is optimized according to the evaluation results.

[0014] Furthermore, it also includes:

[0015] S5. Verify the feasibility of the real crowd counting data set in terms of crowd counting effect through crowd counting evaluation indicators.

[0016] Furthermore, the step S1 specifically includes:

[0017] Build city-related and field-related scenarios based on the simulation platform;

[0018] Render different scenes in different weather conditions;

[0019] By changing the scene season and scene time, the changes in different light intensities are simulated; the visible light camera and depth camera are turned on, and the drone is controlled to switch between different positions and perspectives to collect a virtual crowd counting data set.

[0020] Furthermore, the SA-CycleGAN network structure includes an improved generator and an improved discriminator;

[0021] The improved generator retains the original encoder, conversion module and decoder in the CycleGAN network generator and adds the Shuffle Attention mechanism.

[0022] The improved discriminator adopts the Auto-Encoder network structure to replace the original structure of the CycleGAN network discriminator.

[0023] Furthermore, the principles of the Shuffle Attention mechanism include:

[0024] Divide the input feature map into multiple groups, each group is a sub-feature;

[0025] Each sub-feature is divided into a channel attention branch and a spatial attention branch along the channel dimension, and the channel attention result and spatial attention result corresponding to each sub-feature are obtained;

[0026] The channel attention result and spatial attention result of each sub-feature are combined to form the attention result of the sub-feature.

[0027] Furthermore, the improved discriminator uses an expansion and convolution operation instead of a deconvolution during upsampling.

[0028] Furthermore, the total loss of the SA-CycleGAN network includes adversarial loss, cycle consistency loss and mapping consistency loss; it is expressed as:

[0029]

[0030] L SA-CycleGAN =L GAN (G,D Y ,X,Y)+L GAN (F,D X ,Y,X)+λL cyc (G,F)+μL Identity

[0031] Among them, L GAN (G,D Y ,X,Y) represents the adversarial loss of the improved generator G; L GAN (F,D X ,X,Y) represents the adversarial loss of the improved generator F; L cyc (G, F) represents cycle consistency loss; L Identity (G, F) represents the mapping consistency loss; L SA-CycleGAN represents the total loss of the SA-CycleGAN network; X represents the source domain; Y represents the target domain; x represents the image sample in the virtual crowd counting dataset; y represents the image sample in the real crowd counting dataset; D Y (y) represents the improved discriminator D Y Image samples obtained after encoding and decoding; D X (x) represents the improved discriminator D X The image sample obtained after encoding and decoding; G(x) represents the image sample x in the source domain X mapped by the improved generator G to generate the image sample in the target domain Y, which is recorded as the translated image sample G(x); D Y (G(x)) represents the translation image sample G(x) through the improved discriminator D Y The image obtained after encoding and decoding; ||·||1 indicates the calculation of L1 loss, that is, the calculation of squared absolute error; G(y) indicates that the image samples in the target domain Y are mapped to the image samples in the source domain X through the improved generator G; D X (G(y)) represents G(y) through the improved discriminator D XThe image obtained after encoding and decoding; F(G(x)) means that the translation image sample G(x) is remapped by the improved generator F to generate the image sample in the source domain X; F(y) means that the target domain Y is mapped by the improved generator F to generate the image sample in the source domain X, recorded as the translation image sample F(y); G(F(y)) means that the translation image sample F(y) is mapped by the improved generator G to generate the image in the target domain Y; represents image samples taken from a real crowd counting dataset; Represents the image samples obtained from the virtual crowd counting dataset; λ represents the weight of the cycle consistency loss; μ represents the weight of the mapping consistency loss in the total loss; F(x) represents the generation of image samples in the target domain Y by mapping the image samples in the source domain X through the improved generator F.

[0032] Furthermore, the quality evaluation index includes: structural similarity and peak signal-to-noise ratio; specifically expressed as:

[0033]

[0034] Among them, SSIM represents structural similarity; PSNR represents peak signal-to-noise ratio; μ1 represents the average pixel value of the source domain image; μ2 represents the average pixel value of the target domain image; Represents the pixel variance of the source domain image; represents the pixel variance of the target domain image; σ 12 represents the pixel covariance of the source domain image and the target domain image; I(i,j) represents the pixel at the corresponding position of the source domain image; K(i,j) represents the pixel at the corresponding position of the target domain image; i and j represent the horizontal and vertical coordinate points of the image respectively; m represents the width of the image; n represents the height of the image; C1 and C2 are both constants.

[0035] Furthermore, the crowd counting evaluation index includes mean absolute error and root mean square error; expressed as:

[0036]

[0037] Among them, MAE represents mean absolute error; RMSE represents root mean square error; h represents the number of samples; y e represents the actual number of people in the e-th image; is the predicted value in the e-th image.

[0038] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN, which has the following beneficial effects:

[0039] The present invention uses SA-CycleGAN to generate adversarial networks to convert source domain images into target domain styles, and can obtain the generated target domain dataset without manual labeling; it saves the manpower and financial resources required to collect and label images from real scenes, and alleviates the problem of significant performance degradation of the dataset when tested in other scenes.

[0040] The present invention uses the generated data set to fine-tune the original crowd counting model, and can obtain a detection model suitable for new crowd counting scenarios, which greatly reduces the time and cost of annotation, alleviates the problem of a significant drop in performance of the data set when testing in other scenarios, and thus improves the versatility of the crowd technology model.

[0041] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1 A schematic flow chart of a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN provided in an embodiment of the present invention.

[0044] Figure 2 A schematic diagram of the generator structure provided in an embodiment of the present invention.

[0045] Figure 3 A schematic diagram of the discriminator structure provided in an embodiment of the present invention.

[0046] Figure 4 The original picture provided by the embodiment of the present invention.

[0047] Figure 5 A schematic diagram of a density map of an original image provided by an embodiment of the present invention.

[0048] Figure 6 A schematic diagram of a density map predicted by a crowd counting algorithm using an unprocessed virtual crowd counting dataset image as input provided by an embodiment of the present invention.

[0049] Figure 7 A schematic diagram of a density map predicted by a crowd counting algorithm after the virtual crowd counting dataset image provided in an embodiment of the present invention is trained by the original CycleGAN and then input into the crowd counting algorithm.

[0050] Figure 8The virtual crowd counting data set image provided for the embodiment of the present invention is trained by SA-CycleGAN and then input into the density map predicted by the crowd counting algorithm. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. In addition, various serial numbers are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0052] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In the description of the present invention, it should be noted that: in some processes described in the specification and drawings of this application, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel.

[0054] See also Figure 1 As shown, an embodiment of the present invention discloses a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN, characterized in that it includes the following steps:

[0055] S1. Build a virtual scene based on the simulation platform and collect virtual crowd counting datasets in the source domain;

[0056] S2, automatically annotate pedestrian images in the virtual crowd counting dataset;

[0057] S3. Use the trained domain adaptation network model to perform style transfer on the virtual crowd counting dataset annotated in the source domain to generate a real crowd counting dataset with the style of the target domain.

[0058] The domain adaptation network model is built based on the SA-CycleGAN network structure.

[0059] This method can achieve cross-domain adaptation in different scenarios, and can alleviate the problem of a significant drop in data set performance when testing in other scenarios.

[0060] Next, each of the above steps will be described in detail.

[0061] In the above step S1, city-related scenes and field-related scenes are built based on the simulation platform; different weather conditions (such as rain, snow, fog, wind and sand, etc.) are rendered for different scenes; the change of different light intensities is simulated by changing the scene season and scene time; sensors such as visible light cameras and depth cameras are turned on, and the drone is controlled to switch to different positions and perspectives to collect virtual crowd counting data sets; this process can ensure the diversity of the collected virtual crowd counting data sets;

[0062] The simulation platform may adopt the AirSim platform and / or UE4 (Unreal Engine 4).

[0063] In the above step S2, the pedestrian images in the virtual crowd counting data set are automatically labeled; specifically: by using the depth images corresponding to the images in the collected virtual crowd counting data set, using a special data collector and labeler for crowd counting, the pedestrian images in the collected virtual crowd counting data set are automatically labeled.

[0064] In the above step S3, the trained domain adaptation network model is used to perform style transfer on the virtual crowd counting dataset annotated in the source domain to generate a real crowd counting dataset with the style of the target domain; the above domain adaptation network model is built based on the SA-CycleGAN network structure; the SA-CycleGAN network structure includes an improved generator, an improved discriminator and a total loss; since the SA-CycleGAN in the embodiment of the present invention is composed of two GAN networks, it includes two improved generators and two improved discriminators; namely, the improved generator G and the improved generator F, the improved discriminator D X And the improved discriminator D Y ; The purpose of improving the generator is to learn the mapping between two different domains (i.e., source domain and target domain), i.e., G:x→y and F:y→x; D X is to judge the difference between x and G(x), D Y is used to distinguish between y and F(y); specifically:

[0065] (1) Improve the generator

[0066] The improved generator in the embodiment of the present invention retains the original encoder, conversion module and decoder in the CycleGAN network generator, and adds the Shuffle Attention mechanism (SA); specifically, the Shuffle Attention mechanism is used to simultaneously construct channel attention and spatial attention for each sub-feature group, and embeds it into the generative adversarial network to enhance the features, so that the network can pay more attention to the important features in the image, while suppressing irrelevant information; thereby improving the ability of domain adaptation; the location of the Shuffle Attention mechanism can be specifically seen in Figure 2 As shown; its working principle specifically includes:

[0067] 1) Shuffle Attention mechanism divides the input feature map into multiple groups, each group is a sub-feature, denoted by X k ; Specifically, the input feature map is divided into S groups along the channel dimension, expressed as C represents the number of channels, H and W represent the height and width of the feature map, respectively, where each sub-feature X k As training progresses, a specific semantic information will be gradually captured;

[0068] 2) The sub-features will be divided into two branches, still divided along the channel dimension, and the two sub-feature branches are represented as like Figure 2 The middle part; the upper branch realizes channel attention to exploit the dependency between channels, and the lower branch captures the spatial dependency between features to generate a spatial attention map. In this way, the model simultaneously completes the attention of semantic and position information; specifically:

[0069] ① In the channel attention branch, a single-layer transformation of GAP+Scale+Sigmoid is used, which is expressed as:

[0070]

[0071] σ(·)=sigmod(·)

[0072] Among them, s represents the statistical information of the channel generation method; is the intermediate calculation function in the formula; represents the transformation function; X′ k1 represents the channel attention result; there are only two transformation parameters in these two formulas, namely and W1 represents the weight on the channel attention branch; b1 represents the bias on the channel attention branch; i and j represent the horizontal and vertical coordinate points of the image respectively;

[0073] ② In the spatial attention branch, the input feature map is first normalized (Group Norm), and then a transformation is performed To enhance the representation of the input, the specific formula is as follows:

[0074] X′ k2 =σ(W2·GN(X k2 )+b2)·X k2

[0075] Among them, X′ k2 Represents the spatial attention result; the sizes of W2 and b2 are W2 represents the weight on the spatial attention branch; b2 represents the bias on the spatial attention branch; GN represents normalization processing;

[0076] 3) The above channel attention result X′ k1 And the spatial attention result X′ k2 Put together to form

[0077] (2) Improve the discriminator

[0078] The improved discriminator uses the Auto-Encoder network structure to replace the original structure of the CycleGAN network discriminator. The core advantage of using the Auto-Encoder network structure on the discriminator is that the training of the discriminator is no longer constrained by the generator. The discriminator can be trained first, and the training of the improved generator can be stimulated by the optimization of the discriminator, thereby solving the training imbalance problem caused by the original model. The principle is: suppose there are three pictures now, one is the input picture x, one is the picture D(x) after being encoded and decoded by the discriminator D, and the other is generated by the generator G and then by the discriminator. The image D(G(y)) obtained after D encoding and decoding; with the training of the network, the generator G and the discriminator D gradually reach Nash equilibrium, that is, (||D(x)-x||-||D(x)-D(G(y))||)→0; the discriminator D is trained with real data distribution, which makes the distribution of D(x) infinitely close to the distribution of x, that is, ||D(x)-x||→0; through the idea of ​​limit, we can get ||D(x)-D(G(y))||→0; at this time, the distribution of x will be infinitely close to the distribution of G(y), and the generator G will learn the mapping relationship from the source distribution of y to the source distribution of x, and the generator G and the discriminator D will reach Nash equilibrium;

[0079] See also Figure 3As shown in the figure, the improved discriminator uses the expansion and convolution operation instead of deconvolution in the upsampling process; the square module in the figure represents the convolution layer, the convolution kernel size is w, d = (a, b) indicates that the number of channels of the feature map changes from a to b; Fully connected indicates the fully connected layer; Subsampling indicates the average pooling operation; NN Upsampling indicates the neighboring point filling layer, which fills the surroundings of the image with the pixel content of the corresponding position of the image.

[0080] (3) Total loss

[0081] The total loss of the SA-CycleGAN network includes adversarial loss, cycle consistency loss, and mapping consistency loss; it is expressed as:

[0082]

[0083] L SA-CycleGAN =L GAN (G,D Y ,X,Y)+L GAN (F,D X ,Y,X)+λL cyc (G,F)+μL Identity

[0084] Among them, L GAN (G,D Y ,X,Y) represents the adversarial loss of the improved generator G; L GAN (F,D X ,X,Y) represents the adversarial loss of the improved generator F; L cyc (G, F) represents cycle consistency loss; L Identity (G, F) represents the mapping consistency loss; L SA-CycleGAN represents the total loss of the SA-CycleGAN network; X represents the source domain; Y represents the target domain; x represents the image sample in the virtual crowd counting dataset; y represents the image sample in the real crowd counting dataset; D Y (y) represents the improved discriminator D Y Image samples obtained after encoding and decoding; D X (x) represents the improved discriminator D X The image sample obtained after encoding and decoding; G(x) represents the image sample x in the source domain X mapped by the improved generator G to generate the image sample in the target domain Y, which is recorded as the translated image sample G(x); D Y (G(x)) represents the translation image sample G(x) through the improved discriminator D YThe image obtained after encoding and decoding; ||·||1 indicates the calculation of L1 loss, that is, the calculation of squared absolute error; G(y) indicates that the image samples in the target domain Y are mapped to the image samples in the source domain X through the improved generator G; D X (G(y)) represents G(y) through the improved discriminator D X The image obtained after encoding and decoding; F(G(x)) means that the translation image sample G(x) is remapped by the improved generator F to generate the image sample in the source domain X; F(y) means that the target domain Y is mapped by the improved generator F to generate the image sample in the source domain X, recorded as the translation image sample F(y); G(F(y)) means that the translation image sample F(y) is mapped by the improved generator G to generate the image in the target domain Y; represents image samples taken from a real crowd counting dataset; represents the image samples obtained from the virtual crowd counting dataset; λ represents the weight of the cycle consistency loss; μ represents the weight of the mapping consistency loss in the total loss; F(x) represents the image samples in the source domain X mapped to the image samples in the target domain Y by the improved generator F; where the improved discriminator D X Used to distinguish the translated image sample G(x) from the real sample y; Improve the discriminator D Y Used to distinguish the translated image sample F(y) from the real sample y.

[0085] In another embodiment, it also includes:

[0086] S4. The fusion quality of the real crowd counting dataset is tested and evaluated through quantitative quality evaluation indicators, and the SA-CycleGAN network is optimized based on the evaluation results;

[0087] Among them, the quality evaluation indicators include: structural similarity and peak signal-to-noise ratio; specifically expressed as:

[0088]

[0089] Among them, SSIM represents structural similarity; PSNR represents peak signal-to-noise ratio; μ1 represents the average pixel value of the source domain image; μ2 represents the average pixel value of the target domain image; Represents the pixel variance of the source domain image; represents the pixel variance of the target domain image; σ 12 represents the pixel covariance of the source domain image and the target domain image; I(i,j) represents the pixel at the corresponding position of the source domain image; K(i,j) represents the pixel at the corresponding position of the target domain image; i and j represent the corresponding horizontal and vertical coordinate points of the coordinate system established with the upper left corner of the image as the origin; m represents the width of the image; n represents the height of the image; C1 and C2 represents a constant to avoid instability caused by the denominator being close to 0. In this embodiment, after the image is generated, the above-mentioned index is used to evaluate the quality of the generated image, and the image is compared with other CycleGAN algorithms, and the parameters of the network are modified to improve the quality of the generated image.

[0090] In another embodiment, it also includes:

[0091] S5. The feasibility of the crowd counting effect of the real crowd counting dataset is verified through the crowd counting evaluation index; specifically, the images generated after training are used as the training set of the crowd counting algorithm, and the original images are used as the verification set. After the training is completed, the crowd counting evaluation indicators: mean absolute error (MAE) and root mean square error (RMSE) are used to illustrate that this method can alleviate the problem of a significant drop in performance of the dataset when tested in other scenarios; the mean absolute error and root mean square error are expressed as:

[0092]

[0093] Among them, MAE represents mean absolute error; RMSE represents root mean square error; h represents the number of samples; y e represents the actual number of people in the e-th image; is the predicted value in the e-th image.

[0094] Figure 4 is the original image; Figure 5 is the density map of the original image; Figure 6 It is the density map predicted by the crowd counting algorithm using the unprocessed virtual crowd counting dataset image input; Figure 7 It is the density map predicted by the crowd counting algorithm after the virtual crowd counting dataset image is trained by the original CycleGAN. Figure 8 The virtual crowd counting dataset image is trained with SA-CycleGAN and then input into the density map predicted by the crowd counting algorithm. It can be seen that the difference between the predicted density map and the original density map is not large, indicating that the present invention can alleviate the problem of a significant drop in performance of the dataset when tested in other scenarios and improve the versatility of the algorithm.

[0095] In summary, the embodiments of the present invention provide a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN. The present invention can save the manpower and financial resources required to collect and annotate images from real scenes, and alleviate the problem of a significant drop in performance of the dataset when tested in other scenes.

[0096] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0097] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN, characterized in that: The steps include: S1. Build a virtual scene based on the simulation platform and collect virtual crowd counting datasets in the source domain; S2, automatically labeling pedestrian images in the virtual crowd counting dataset; S3. Use the trained domain adaptation network model to perform style transfer on the virtual crowd counting dataset annotated in the source domain to generate a real crowd counting dataset with the style of the target domain. The domain adaptation network model is built based on the SA-CycleGAN network structure.

2. According to claim 1, a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN is characterized in that: Also includes: S4. The fusion quality of the real crowd counting dataset is tested and evaluated through quantitative quality evaluation indicators, and the SA-CycleGAN network is optimized according to the evaluation results.

3. According to claim 1, a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN is characterized in that: Also includes: S5. Verify the feasibility of the real crowd counting data set in terms of crowd counting effect through crowd counting evaluation indicators.

4. According to claim 1, a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN is characterized in that: The step S1 specifically includes: Build city-related and field-related scenarios based on the simulation platform; Render different scenes in different weather conditions; By changing the scene season and scene time, the changes in different light intensities are simulated; the visible light camera and depth camera are turned on, and the drone is controlled to switch between different positions and perspectives to collect a virtual crowd counting data set.

5. According to claim 1, a method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN is characterized in that: The SA-CycleGAN network structure includes an improved generator and an improved discriminator; The improved generator retains the original encoder, conversion module and decoder in the CycleGAN network generator and adds the Shuffle Attention mechanism. The improved discriminator adopts the Auto-Encoder network structure to replace the original structure of the CycleGAN network discriminator.

6. The method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN according to claim 5, characterized in that: The principles of the ShuffleAttention mechanism include: Divide the input feature map into multiple groups, each group is a sub-feature; Each sub-feature is divided into a channel attention branch and a spatial attention branch along the channel dimension, and the channel attention result and spatial attention result corresponding to each sub-feature are obtained; The channel attention result and spatial attention result of each sub-feature are combined to form the attention result of the sub-feature.

7. The method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN according to claim 5, characterized in that: The improved discriminator uses dilation and convolution operations instead of deconvolution during upsampling.

8. The method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN according to claim 1, characterized in that: The total loss of the SA-CycleGAN network includes adversarial loss, cycle consistency loss and mapping consistency loss; it is expressed as: L GAN (G,D Y ,X,Y)=E y～Pdata(y) [logD Y (y)]+E x～Pdata(x) [log(1-D Y (G(x))] L GAN (F,D X ,X,Y)=E x～Pdata(x) [logD X (x)]+E y～Pdata(y) [log(1-D X (G(y))] L cyc (G,F)=E x～Pdata(x) [||F(G(x))-x||1]+E y～Pdata(y) [||G(F(y))-y||1] L Identity (G,F)=E x～Pdata(x) [||F(x)-x||]+E y～Pdata(y) [||F(y)-y||] L SA-CycleGAN =L GAN (G,D Y ,X,Y)+L GAN (F,D X ,Y,X)+λL cyc (G,F)+μL Identity Among them, L GAN (G,D Y ,X,Y) represents the adversarial loss of the improved generator G; L GAN (F,D X ,X,Y) represents the adversarial loss of the improved generator F; L cyc (G, F) represents cycle consistency loss; L Identity (G, F) represents the mapping consistency loss; L SA-CycleGAN represents the total loss of the SA-CycleGAN network; X represents the source domain; Y represents the target domain; x represents the image sample in the virtual crowd counting dataset; y represents the image sample in the real crowd counting dataset; D Y (y) represents the improved discriminator D Y Image samples obtained after encoding and decoding; D X (x) represents the improved discriminator D X The image sample obtained after encoding and decoding; G(x) represents the image sample x in the source domain X mapped by the improved generator G to generate the image sample in the target domain Y, which is recorded as the translated image sample G(x); D Y (G(x)) represents the translation image sample G(x) through the improved discriminator D Y The image obtained after encoding and decoding; ||·||1 indicates the calculation of L1 loss, that is, the calculation of squared absolute error; G(y) indicates that the image samples in the target domain Y are mapped to the image samples in the source domain X through the improved generator G; D X (G(y)) represents G(y) through the improved discriminator D X The image obtained after encoding and decoding; F(G(x)) means that the translation image sample G(x) is remapped by the improved generator F to generate the image sample in the source domain X; F(y) means that the target domain Y is mapped by the improved generator F to generate the image sample in the source domain X, recorded as the translation image sample F(y); G(F(y)) means that the translation image sample F(y) is mapped by the improved generator G to generate the image in the target domain Y; represents image samples taken from a real crowd counting dataset; Represents the image samples obtained from the virtual crowd counting dataset; λ represents the weight of the cycle consistency loss; μ represents the weight of the mapping consistency loss in the total loss; F(x) represents the generation of image samples in the target domain Y by mapping the image samples in the source domain X through the improved generator F.

9. The method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN according to claim 2, characterized in that: The quality evaluation indexes include: structural similarity and peak signal-to-noise ratio; specifically expressed as: Among them, SSIM represents structural similarity; PSNR represents peak signal-to-noise ratio; μ1 represents the average pixel value of the source domain image; μ2 represents the average pixel value of the target domain image; Represents the pixel variance of the source domain image; represents the pixel variance of the target domain image; σ 12 represents the pixel covariance of the source domain image and the target domain image; I(i,j) represents the pixel at the corresponding position of the source domain image; K(i,j) represents the pixel at the corresponding position of the target domain image; i and j represent the horizontal and vertical coordinate points of the image respectively; m represents the width of the image; n represents the height of the image; C1 and C2 are both constants.

10. The method for generating a domain adaptive crowd counting dataset based on SA-CycleGAN according to claim 3, characterized in that: The crowd counting evaluation indicators include mean absolute error and root mean square error; expressed as: Among them, MAE represents mean absolute error; RMSE represents root mean square error; h represents the number of samples; y e represents the actual number of people in the e-th image; is the predicted value in the e-th image.

Images