Method and related apparatus for determining a number of people

Abstract

The application provides a method for determining the number of people and a related device. In the method, when determining the number of people in an image based on a target neural network model, the target neural network model can perform fusion processing on feature maps output by different layers. Since the feature maps output by different layers can reflect features of different scales, when the target neural network model is trained using first training data, the target neural network model can better learn the scale change characteristics of people in diverse images, so that when a target image is input into the trained target neural network model, the trained target neural network model can more accurately output a density distribution map corresponding to the target image, thereby further improving the accuracy of the number of people in the target image determined based on the density distribution map corresponding to the target image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, and particularly relates to a method for determining a crowd quantity and a related device. BACKGROUND

[0002] It is very important to determine the crowd quantity in public places in time to avoid some unnecessary disasters.

[0003] At present, a method for determining the crowd quantity in public places is to use a neural network. Specifically, the method comprises: obtaining a target image; inputting the target image into a trained neural network to obtain a density distribution graph corresponding to the image, wherein the trained neural network model is used to receive an image and output a density distribution graph corresponding to the image; calculating the integral sum of the density distribution graph corresponding to the target image and determining the integral sum as the crowd quantity in the target image.

[0004] However, there is a problem of low accuracy when determining the crowd quantity based on the neural network. SUMMARY

[0005] The present application provides a method for determining a crowd quantity and a related device to improve the accuracy of determining the crowd quantity.

[0006] In a first aspect, the present application provides a method for training a neural network for determining a crowd quantity, comprising: obtaining first training data, wherein the first training data comprises a first image and a label, and the label indicates a first density distribution graph corresponding to the first image; training a target neural network model using the first training data; wherein the target neural network model comprises a feature extraction module, a first scale difference module, a first context fusion module and a prediction module, the feature extraction module is used to receive an image, perform feature extraction on the image and output the extracted feature map, the first scale difference module is used to receive the feature map, perform feature extraction on the feature map and output the extracted feature map, the first context fusion module is used to receive the feature map output by the first scale difference module and the feature map output by the first convolutional layer in the feature extraction module, perform fusion processing on the feature map output by the first scale difference module and the feature map output by the first convolutional layer and output the fused feature map, the prediction module is used to receive the fused feature map, generate a density distribution graph corresponding to the image based on the fused feature map and output the density distribution graph, and the integral sum of the density graph indicates the crowd quantity in the image.

[0007] In the method for determining the number of people provided in the present application, the target neural network model comprises a first context fusion module, which is capable of performing fusion processing on the feature maps output by the first convolution layer in the feature extraction layer and the feature maps output by the first scale difference module, i.e., the first context model is capable of performing fusion processing on the feature maps output by different layers. It should be understood that the feature maps output by the first convolution layer and the feature maps output by the first scale difference module can reflect features of different scales, and therefore, when the target neural network model is trained using the first training data, the target neural network model can better learn the features of the scale variation of people in diverse images, so that when the target image is input into the trained target neural network model, the trained target neural network model can more accurately output the density distribution map corresponding to the target image, thereby further improving the accuracy of the number of people in the target image determined based on the density distribution map corresponding to the target image.

[0008] In combination with the first aspect, in a possible implementation manner, when performing feature extraction on the feature maps output by the feature extraction module, the following is included: performing feature extraction on the feature maps output by the feature extraction module using N convolution kernels to obtain N feature maps, the N convolution kernels comprising at least two convolution kernels of different sizes, and N being a positive integer greater than 1; performing difference on every two adjacent feature maps in the N feature maps to obtain N-1 difference feature maps; merging the N-1 difference feature maps and the feature maps output by the feature extraction module; and continuing to perform feature extraction on the merged feature maps.

[0009] In combination with the first aspect, in a possible implementation manner, the convolution layer in the feature extraction module comprises the convolution layer in "VGG16".

[0010] In the second aspect, the present application provides a method for determining the number of people, which comprises: acquiring a target image; acquiring a target density distribution map corresponding to the target image using a target neural network model, the target neural network model being a target neural network model obtained by training using any one of the implementation manners in combination with the first aspect; and determining the number of people in the target image as the integral sum of the target density distribution map.

[0011] Thirdly, this application provides an apparatus for training a neural network to determine the number of people, comprising: an acquisition module for acquiring first training data, the first training data including a first image and a label, the label indicating a first density distribution map corresponding to the first image; and a processing module for training a target neural network model using the first training data; wherein the target neural network model includes a feature extraction module, a first scale difference module, a first context fusion module, and a prediction module; the feature extraction module is used to receive an image, perform feature extraction on the image, and output the extracted feature map; the first scale difference module is used to receive the feature map, perform feature extraction on the feature map, and output the extracted feature map; the first context fusion module is used to receive the feature map output by the first scale difference module and the feature map output by the first convolutional layer in the feature extraction module, perform fusion processing on the feature map output by the first scale difference module and the feature map output by the first convolutional layer, and output the fused feature map; and the prediction module is used to receive the fused feature map, generate a density distribution map corresponding to the image based on the fused feature map, and output the density distribution map, wherein the integral of the density map indicates the number of people in the image.

[0012] In conjunction with the third aspect, in one possible implementation, when the processing module extracts features from the feature map output by the feature extraction module, it specifically performs the following: It uses N convolutional kernels to extract features from the feature map output by the feature extraction module, obtaining N feature maps, where the N convolutional kernels include at least two kernels of different sizes, and N is a positive integer greater than 1; it calculates the difference between every two adjacent feature maps in the N feature maps, obtaining N-1 difference feature maps; it merges the N-1 difference feature maps with the feature map output by the feature extraction module; and it continues to extract features from the merged feature map.

[0013] In conjunction with the third aspect, in one possible implementation, the feature extraction module includes convolutional layers from "VGG16".

[0014] Fourthly, this application provides an apparatus for determining the number of people, comprising: an acquisition module for acquiring a target image; the acquisition module is further configured to: acquire a target density distribution map corresponding to the target image using a target neural network model, wherein the target neural network model is a target neural network model trained using any of the implementation methods described in the first aspect or in combination with the first aspect; a processing module for determining the integral sum of the target density distribution map; the processing module is further configured to determine the number of people in the target image as the integral sum.

[0015] Fifthly, this application provides an apparatus for training a neural network for determining the number of people, comprising: a processor coupled to a memory for storing a computer program, wherein when the processor invokes the computer program, the apparatus performs the method as described in the first aspect and any possible implementation thereof.

[0016] In a sixth aspect, this application provides an apparatus for determining the number of people, comprising: a processor coupled to a memory for storing a computer program, wherein when the processor invokes the computer program, the apparatus performs the method as described in the second aspect.

[0017] In a seventh aspect, this application provides a chip including at least one processor and a communication interface, the communication interface and the at least one processor being interconnected via a line, the at least one processor being configured to run a computer program or instructions to perform the methods described in the first aspect or the second aspect or any possible implementation thereof.

[0018] Eighthly, this application provides a computer-readable storage medium storing program code for computer execution, the program code including methods for performing the methods described in the first aspect or the second aspect or any possible implementation thereof.

[0019] Ninthly, this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the method as described in the first aspect or the second aspect or any possible implementation thereof. Attached Figure Description

[0020] Figure 1 A schematic diagram illustrating one application scenario provided in this application;

[0021] Figure 2 A flowchart illustrating the method for training a neural network to determine the size of a crowd, as provided in this application;

[0022] Figure 3 A schematic diagram of the target neural network model constructed for this application;

[0023] Figure 4 A schematic diagram of the first-scale difference module provided in this application;

[0024] Figure 5 A schematic diagram illustrating the fusion process of the first context fusion module provided in this application;

[0025] Figure 6 A flowchart illustrating the method for determining population size provided in this application;

[0026] Figure 7 A schematic diagram of a device for training to determine the size of a crowd, provided in this application;

[0027] Figure 8 This is a schematic diagram of a device for training to determine the number of people, provided in this application. Detailed Implementation

[0028] As urbanization progresses, more people are flocking to cities, putting greater pressure on urban infrastructure. For example, medical facilities, public transportation, and supermarkets often experience extreme overcrowding. Therefore, timely determination of the number of people in public places is crucial to preventing unnecessary disasters.

[0029] Determining crowd size based on vision has become a research hotspot. Currently, one method for determining crowd size based on vision uses neural networks. Specifically, the method includes: acquiring a target image; inputting the target image into a trained neural network to obtain the density distribution map corresponding to the image, wherein the trained neural network model is used to receive the image and output the density distribution map corresponding to the image; calculating the integral sum of the density distribution map corresponding to the target image and determining the crowd size in the target image based on the integral sum.

[0030] For example, Figure 1 This is a schematic diagram illustrating a process for determining the size of a population as provided in this application. Figure 1 As shown, the training server 101 can train the neural network model using training data to obtain a trained model. Each training data includes an image and the density distribution map corresponding to that image. Then, the training server 101 can send the trained model to the target device 102, so that the target device 102 can use the trained model to generate a new density distribution map corresponding to the target image when it receives a new target image, so as to further determine the number of people in the target image based on the density distribution map corresponding to the target image.

[0031] It should be noted that, in this application, determining the number of people specifically refers to determining the number of people in an image.

[0032] Analysis revealed that when using neural network models to determine population size, variations in population scale significantly impact the density distribution map output by the model, thus affecting the accuracy of population size determination based on the density distribution map. However, current methods for determining population size using neural network models do not consider variations in population scale, resulting in low accuracy.

[0033] In view of this, this application provides a method and related apparatus for determining the number of people. The method first acquires first training data, specifically, the first training data includes a first image and a label, where the label indicates a first density distribution map corresponding to the first image; then, a target neural network is trained using the first training data to obtain a trained neural network that can be used to determine the density distribution map of an image.

[0034] Then, for a new target image, the trained neural network used to determine the density distribution map of the image is used to obtain the target density distribution map of the new target image. Then, the number of people in the target image is calculated based on the target density distribution map, that is, the number of people in the target image is calculated based on the target density distribution map.

[0035] In its specific implementation, the target neural network model constructed in this application includes a feature extraction module, a first-scale difference module, a first-context fusion module, and a prediction module. The feature extraction module receives an image, extracts features from the image, and outputs the extracted feature map. The first-scale difference module receives the feature map, extracts features from the feature map, and outputs the extracted feature map. The first-context fusion module receives the feature map output by the first-scale difference module and the feature map output by the first convolutional layer in the feature extraction module, fuses the feature map output by the first-scale difference module and the feature map output by the first convolutional layer, and outputs the fused feature map. The prediction module receives the fused feature map, generates a density distribution map corresponding to the image based on the fused feature map, and outputs the density distribution map. In the method provided by this application, because it includes a first-context fusion module, this first-context module can be used to fuse feature maps output from different layers. It should be understood that feature maps output from different layers can reflect features at different scales. Therefore, when the target neural network model is trained using the first training data, the target neural network model can better learn the scale variation features of crowds in diverse images. This allows the trained target neural network model to output the density distribution map corresponding to the target image more accurately when the target image is input into it, thereby further improving the accuracy of determining the number of people in the target image based on the density distribution map corresponding to the target image.

[0036] Below, in conjunction with Figure 2 This application provides a detailed description of the method for training neural networks.

[0037] like Figure 2 As shown, the method of this application may include S201 and S202.

[0038] S201, Obtain first training data, which includes a first image and a label, wherein the label indicates the first density distribution map corresponding to the first image.

[0039] In order to obtain a target neural network model that can be used to determine the number of people, it is first necessary to obtain training data for training the target neural network model, which is also referred to as the first training data in this application.

[0040] It should be understood that when training a neural network model, the training data usually consists of two parts: one part is the input data, and the other part is the target value that the input data should output after being input into the neural network model.

[0041] In this method, the first training data also includes two parts: a first image and a label. Specifically, the label indicates the density distribution map (i.e., the first density distribution map) corresponding to the first image.

[0042] In practical implementation, the label of the first image can be obtained using the following formula:

[0043]

[0044] Where G(x) is the density map, x i G is the position of the i-th person in the first image. σ It is a Gaussian kernel.

[0045] It should be noted that this application does not limit the type of the first image.

[0046] In one possible scenario, the first image is a grayscale image. In another possible scenario, the first image is a color image.

[0047] S202, Train the target neural network model using the first training data.

[0048] Once the first training data, including the first image and the corresponding first density distribution map, is obtained, the first training data can be used to train the target neural network model, thereby obtaining a neural network model that can be used to determine the number of people.

[0049] The following explains how to use the first training data to train the target neural network model after obtaining the first training data.

[0050] 1) Build the target neural network model.

[0051] In this application, the target neural network model is constructed as follows: Figure 3 As shown, it includes a feature extraction module, a first scale difference module, a first context fusion module, and a prediction module.

[0052] Specifically, the feature extraction module in this application is used to receive the input image, extract features from the image, and output the extracted feature map.

[0053] In one implementation scheme, such as Figure 3As shown, the feature extraction module can be based on a VGG16 network pre-trained on the ImageNet dataset, which contains 13 convolutional layers and 3 fully connected layers. Additionally, there are 5 max-pooling layers for downsampling the feature map. For example, the feature extraction module in this application can extract the features of the input image by truncating the first 10 convolutional layers and 3 max-pooling layers (not shown in the figure) of the VGG16 network.

[0054] Specifically, such as Figure 3 As shown, the first scale difference module in this application is used to receive the feature map output by the feature extraction module, perform feature extraction on the feature map, and output the extracted feature map.

[0055] It should be noted that this application does not impose any restrictions on how the first scale difference module extracts features from the feature map output by the feature extraction module.

[0056] In one implementation, the first-scale difference module can use N convolutional kernels to extract features from the feature map output by the feature extraction module to obtain N feature maps; then, the difference between any two adjacent feature maps in the N feature maps is calculated to obtain N-1 difference feature maps; the N-1 difference feature maps are merged with the original feature map output by the feature extraction module; and feature extraction is then performed on the merged feature map.

[0057] As an example, such as Figure 4 As shown, the first scale difference module can include 5 branches. The first branch uses a 1*1 convolution kernel to convolve the original feature map output by the feature extraction module and outputs the feature map. Figure 1 The second branch uses a 3x3 convolution kernel to perform a convolution with a dilation rate of 1 on the original feature map output by the feature extraction module and outputs the feature map. Figure 2 The third branch uses a 3x3 convolution kernel to perform a 2-dilation convolution on the original feature map output by the feature extraction module and outputs the feature map. Figure 3 The fourth branch uses a 3x3 convolution kernel to perform a 3-dilation convolution on the original feature map output by the feature extraction module and outputs the feature map. Figure 4The fifth branch is the original feature map output by the feature extraction module. Then, the difference between the first and second branches is used to obtain the first difference feature map (e.g., FD1), the difference between the second and third branches is used to obtain the second difference feature map (e.g., FD2), and the difference between the third and fourth branches is used to obtain the third difference feature map (e.g., FD3). Finally, the obtained difference feature maps and the original feature map output by the feature extraction module are concatenated, and a 1*1 convolution kernel is used to compress the number of channels to the number of channels in the original feature map. That is, in this example, the first-scale difference module can be specifically represented as:

[0058] FD1 = Conv3-1(Fi) - Conv1-1(Fi)

[0059] FD2 = Conv3-2(Fi) - Conv3-1(Fi)

[0060] FD3 = Conv3 - 3(Fi) - Conv3 - 2(Fi)

[0061] Fo=Conv1(FD1, FD2, FD3, Fi)

[0062] Where Fi represents the features input to the first scale difference module, Fo represents the features output by the first scale difference module, Conv3-1 is a convolutional layer with a kernel size of 3×3 and a dilation rate of 1, Conv1-1 is a convolutional layer with a kernel size of 1×1 and a dilation rate of 1, Conv3-2 is a convolutional layer with a kernel size of 3×3 and a dilation rate of 2, Conv3-3 is a convolutional layer with a kernel size of 3×3 and a dilation rate of 2, and Conv1 is a fusion convolutional layer.

[0063] As can be seen, in this application, the first scale difference module adopts a multi-branch structure to further learn the feature differences between different scales of the same layer feature map and infer scale changes, thus it can learn scale characteristics better.

[0064] Specifically, such as Figure 3 As shown, the first context fusion module in this application is used to receive the feature map output by the first scale difference module and the feature map output by the first convolutional layer in the feature extraction module, perform fusion processing on the feature map output by the first scale difference module and the feature map output by the first convolutional layer, and output the fused feature map. In other words, in this application, the first context fusion module fuses feature maps of different resolutions.

[0065] For example, such as Figure 3As shown, the first context fusion module is used to receive the feature map output by the first scale difference module and the feature map output by the 7th convolutional layer in the feature extraction module, and to perform fusion processing on the feature map output by the first scale difference module and the feature map output by the 7th convolutional layer and output the fused feature map.

[0066] For example, when fusing the feature map output by the first scale difference module and the feature map output by the first convolutional layer, one implementation is as follows: Figure 5 As shown: It includes two inputs, one is the Fo output from the first scale difference module, and the other is a feature map output from a convolutional layer in the feature extraction network (e.g., represented as F2). During fusion, F2 is sampled to the same spatial size as Fo, then pixel-wise added, and then initially fused using a 3×3 convolutional kernel. Next, deep fusion is performed using channel attention, which includes one global average pooling (GAP) layer, one convolutional layer, and one sigmoid activation function layer. Finally, the initially fused features and the deeply fused features are multiplied pixel-wise to obtain the final output. That is, in this... Figure 5 In the example shown, the first context fusion module can be specifically represented as:

[0067] Fc = Conv3(Fo + Up(F2))

[0068] Fd = Sigmoid(Conv1(GAP(Fc)))

[0069]

[0070] Up represents sampling, Conv3 represents a preliminary fusion convolutional layer, GAP represents a global pooling layer, Conv1 represents a deep fusion convolutional layer, and Sigmoid represents the activation function. This indicates pixel-level multiplication. Fc is the result of the initial fusion, Fd is the result of the deep fusion, and C is the final output of the first context fusion module.

[0071] Specifically, such as Figure 3 As shown, the prediction module in this application is used to receive the feature map fused by the first context module, generate a density distribution map corresponding to the image based on the fused feature map, and output a density distribution map, the integral of the density map, and the number of people in the image.

[0072] In one implementation, when the prediction module generates a density distribution map corresponding to the image based on the fused feature map, the prediction module may further include at least one second-scale difference module and at least one second-context fusion module. Each of the at least one second-scale difference modules functions identically to the first difference module, except that the input to the second-scale difference module is no longer the feature map output by the feature extraction module, but rather the feature map output by the previous layer. Similarly, each of the at least one second-context fusion modules functions identically to the first context fusion module, except that its input is no longer the feature map output by the feature extraction layer and the feature map output by the first convolutional layer in the feature extraction layer selected during fusion by the first context fusion module, but rather the feature map output by the previous layer and the feature maps output by other convolutional layers in the feature extraction layer besides the first convolutional layer.

[0073] As an example, such as Figure 3 As shown, the prediction module includes two second-scale difference modules (denoted by M) and one second-context fusion module (denoted by C). The input to the first second-scale difference module in the prediction module is the feature map output by the first context fusion module. Then, the first second-scale difference module can, for example, use... Figure 4 The method shown extracts features from the feature map output by the first context fusion module and outputs the extracted feature map. Then, the feature map output by the first second-scale difference module and the feature map output by the fourth convolutional layer in the feature extraction module are input into the second context fusion module. The second context fusion module can then use the feature map output by the first scale difference module and the feature map output by the fourth convolutional layer. Figure 5 The method shown performs fusion processing and outputs the fused feature map; similarly, the fused feature map output by the second context fusion module is input to the second scale difference module, which then continues to use it. Figure 4 The method shown extracts features from the feature map output by the second context fusion module and outputs the extracted feature map; finally, the feature map output by the second scale difference module in the prediction module is further subjected to two convolutions to obtain the density distribution map corresponding to the input image and outputs the density distribution map.

[0074] In this application, for ease of description, the first scale difference module, the first context fusion module, and the prediction module are collectively referred to as the density map estimation subnetwork.

[0075] 2) Load the parameters of the feature extraction module and the parameters of the density map estimation subnetwork for random initialization.

[0076] 3) Input the first training sample into the target neural network to train the target neural network model.

[0077] In this application, when training the target neural network model, the images in the first training sample data can be scaled, aligned, flipped, or randomly cropped to increase the number of training samples. For example, when performing random cropping, nine 256*256 image blocks can be randomly cropped from the first image.

[0078] Furthermore, in this step, when training the target neural network model, the training loss can be constructed based on MSE to optimize the model, specifically as follows:

[0079]

[0080] Where θ is the parameter that the proposed target neural network model needs to learn, and n is the number of training samples.

[0081] It should be understood that once the above-trained target neural network model is obtained, it can be used to determine the number of people in an image.

[0082] Specifically, Figure 6 A flowchart illustrating the method for determining population size provided in this application. Figure 6 As shown, the method includes S601, S602, S603 and S604.

[0083] S601, acquire the target image.

[0084] S602, Use the target neural network model to obtain the target density distribution map corresponding to the target image.

[0085] The target neural network model is one that uses... Figure 5 The target neural network model trained by the method shown.

[0086] Depend on Figure 5 It can be seen that, Figure 5 The target neural network model takes an image as input and outputs the density distribution map of that image. Therefore, in this application, when a target image is input into the target neural network model, the density distribution map of the target image (also known as the target density distribution map) can be obtained.

[0087] S603, determine the integral sum of the target density distribution map;

[0088] S604, determine the number of people in the target image as an integral sum.

[0089] As can be seen, in the method for determining the number of people provided in this application, the target neural network model includes a first context fusion module. This first context module can fuse the feature map output by the first convolutional layer in the feature extraction layer and the feature map output by the first scale difference module. That is, the first context model can fuse the feature maps output by different layers. It should be understood that the feature map output by the first convolutional layer and the feature map output by the first scale difference module can reflect features at different scales. Therefore, when the target neural network model is trained using the first training data, the target neural network model can better learn the scale variation features of the crowd in diverse images. This allows the trained target neural network model to output a density distribution map corresponding to the target image more accurately when the target image is input into it, thereby further improving the accuracy of determining the number of people in the target image based on the density distribution map corresponding to the target image.

[0090] The above, combined with Figures 2 to 6 This application describes the method for determining population size. Below, in conjunction with... Figure 7 and Figure 8 This application describes the relevant apparatus provided for determining the number of people.

[0091] Figure 7 One application of this application provides a training apparatus for determining the size of a population, the apparatus 700 comprising: an acquisition module 701 and a processing module 72.

[0092] In the first example, the device 700 can be used to train a neural network for determining the number of people.

[0093] Specifically, in the first example, the acquisition module 701 is used to acquire first training data, which includes a first image and a label, the label indicating a first density distribution map corresponding to the first image; the processing module 702 is used to train a target neural network model using the first training data; wherein, the target neural network model includes a feature extraction module, a first scale difference module, a first context fusion module, and a prediction module. The feature extraction module is used to receive an image, perform feature extraction on the image, and output the extracted feature map. The first scale difference module is used to receive the feature map, perform feature extraction on the feature map, and output the extracted feature map. The first context fusion module is used to receive the feature map output by the first scale difference module and the feature map output by the first convolutional layer in the feature extraction module, perform fusion processing on the feature map output by the first scale difference module and the feature map output by the first convolutional layer, and output the fused feature map. The prediction module is used to receive the fused feature map, generate a density distribution map corresponding to the image based on the fused feature map, and output the density distribution map. The integral of the density map indicates the number of people in the image.

[0094] In one possible implementation, when the processing module 702 extracts features from the feature map output by the feature extraction module, it specifically performs the following steps: extracting features from the feature map output by the feature extraction module using N convolutional kernels to obtain N feature maps, where the N convolutional kernels include at least two convolutional kernels of different sizes, and N is a positive integer greater than 1; subtracting each pair of adjacent feature maps from the N feature maps to obtain N-1 difference feature maps; merging the N-1 difference feature maps with the feature map output by the feature extraction module; and extracting features from the merged feature map.

[0095] In one possible implementation, the feature extraction module includes convolutional layers from "VGG16".

[0096] In the second example, the device 700 can determine the number of people.

[0097] Specifically, the acquisition module 701 is used to acquire a target image; the acquisition module 701 is also used to: acquire a target density distribution map corresponding to the target image using a target neural network model, wherein the target neural network model is a target density distribution map corresponding to the target image using a target neural network model. Figure 2 The method shown trains a target neural network model; processing module 702 is used to determine the integral sum of the target density distribution map; processing module 702 is also used to determine the number of people in the target image as the integral sum.

[0098] Figure 8 This is a structural schematic diagram of the device for determining the number of people provided in this application. Figure 8 The apparatus 800 shown can be used to perform the method described above.

[0099] like Figure 8 As shown, the device 800 includes a memory 801, a processor 802, a communication interface 803, and a bus 804. The memory 801, processor 802, and communication interface 803 are interconnected via the bus 804.

[0100] The memory 801 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 801 can store programs, and when the program stored in the memory 801 is executed by the processor 802, the processor 802 performs the execution... Figures 2 to 6 The steps of the method shown.

[0101] The processor 802 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, used to execute relevant programs to implement this application. Figures 2 to 6 The method shown.

[0102] The processor 802 can also be an integrated circuit chip with signal processing capabilities. In its implementation, this application... Figures 2 to 6 The various steps of the method can be accomplished through integrated logic circuits in the hardware of the processor 802 or through instructions in software form.

[0103] The processor 802 described above can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in this application. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0104] The steps of the method disclosed in this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 801. The processor 802 reads the information in memory 801 and, in conjunction with its hardware, completes the functions required by the units included in the device of this application. For example, it can execute... Figures 2 to 6 The steps / functions shown.

[0105] The communication interface 803 can use, but is not limited to, transceivers to enable communication between the device 800 and other devices or communication networks.

[0106] Bus 804 may include a pathway for transmitting information between various components of device 800 (e.g., memory 801, processor 802, communication interface 803).

[0107] It should be understood that the device 800 shown in this application may be an electronic device, or it may be a chip configured in an electronic device.

[0108] The above methods can be implemented, in whole or in part, through software, hardware, firmware, or any other combination thereof. When implemented in software, the above methods can be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in this application are generated, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0109] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0110] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0111] It should be understood that in the various methods of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the method of this application.

[0112] Those skilled in the art will recognize that the units and algorithm steps of the examples described in conjunction with the methods disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0113] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned methods, and will not be repeated here.

[0114] In the methods provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus described above is merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or units may be electrical, mechanical, or other forms.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this application according to actual needs.

[0116] In addition, the functional units in the various methods of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0117] If the aforementioned function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0118] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training a neural network to determine the size of a crowd, characterized in that, The method comprises: obtaining first training data, wherein the first training data comprises a first image and a label, and the label indicates a first density distribution map corresponding to the first image; training a target neural network model using the first training data; wherein the target neural network model comprises a feature extraction module, a first scale difference module, a first context fusion module and a prediction module, the feature extraction module is configured to receive an image, perform feature extraction on the image and output extracted feature maps, the first scale difference module is configured to receive the feature maps, perform feature extraction on the feature maps and output extracted feature maps, the first context fusion module is configured to receive the feature maps output by the first scale difference module and the feature maps output by a first convolutional layer in the feature extraction module, perform fusion processing on the feature maps output by the first scale difference module and the feature maps output by the first convolutional layer and output fused feature maps, and the prediction module is configured to receive the fused feature maps, generate a density distribution map corresponding to the image based on the fused feature maps and output the density distribution map, and the integral of the density map indicates the number of people in the image; when performing feature extraction on the feature maps output by the feature extraction module, comprising: performing feature extraction on the feature maps output by the feature extraction module using N convolutional kernels to obtain N feature maps, wherein the N convolutional kernels comprise at least two convolutional kernels of different sizes, and N is a positive integer greater than 1; performing difference on every adjacent two feature maps in the N feature maps to obtain N-1 difference feature maps; merging the N-1 difference feature maps and the feature maps output by the feature extraction module; continuing to perform feature extraction on the merged feature maps.

2. The method of claim 1, wherein, The feature extraction module comprises convolutional layers in "VGG16".

3. A method for determining the size of a population, characterized in that, The method comprises: obtaining a target image; obtaining a target density distribution map corresponding to the target image using a target neural network model, wherein the target neural network model is a target neural network model trained using the method of claim 1 or 2; determining an integral sum of the target density distribution map; determining the number of people in the target image as the integral sum.

4. An apparatus for training a neural network for determining a crowd count, the apparatus comprising: The method comprises: an obtaining module configured to obtain first training data, wherein the first training data comprises a first image and a label, and the label indicates a first density distribution map corresponding to the first image; a processing module configured to train a target neural network model using the first training data. The target neural network model comprises a feature extraction module, a first scale difference module, a first context fusion module and a prediction module. The feature extraction module is configured to receive an image, perform feature extraction on the image and output extracted feature maps. The first scale difference module is configured to receive the feature maps, perform feature extraction on the feature maps and output extracted feature maps. The first context fusion module is configured to receive the feature maps output by the first scale difference module and feature maps output by a first convolutional layer in the feature extraction module, perform fusion processing on the feature maps output by the first scale difference module and the feature maps output by the first convolutional layer, and output fused feature maps. The prediction module is configured to receive the fused feature maps, generate a density distribution map corresponding to the image based on the fused feature maps, and output the density distribution map. An integral of the density map indicates a number of people in the image. When performing feature extraction on the feature maps output by the feature extraction module, the processing module is specifically configured to: perform feature extraction on the feature maps output by the feature extraction module using N convolutional kernels to obtain N feature maps, wherein the N convolutional kernels comprise at least two convolutional kernels of different sizes, and N is a positive integer greater than 1; perform difference operation on every two adjacent feature maps in the N feature maps to obtain N-1 difference feature maps; merge the N-1 difference feature maps and the feature maps output by the feature extraction module; and continue to perform feature extraction on the merged feature maps.

5. The apparatus of claim 4, wherein, The feature extraction module comprises convolutional layers in "VGG16".

6. A device for determining the number of people, characterized in that, The method comprises: an acquisition module configured to acquire a target image; the acquisition module is further configured to acquire a target density distribution map corresponding to the target image using a target neural network model, wherein the target neural network model is a target neural network model trained using the method according to claim 1 or 2; a processing module configured to determine an integral sum of the target density distribution map; the processing module is further configured to determine a number of people in the target image as the integral sum.

7. A chip, characterized by The computer readable medium stores program codes for computer execution, and the program codes comprise instructions for executing the method according to any one of claims 1 to 3.

8. A computer readable medium characterized by The computer readable medium stores program codes for computer execution, and the program codes comprise instructions for executing the method according to any one of claims 1 to 3.

Images