Object-based quantity detection method, device, electronic device, and storage medium

Abstract

The embodiments of this specification provide a method, device, electronic device and storage medium for object-based quantity detection. The method includes: obtaining an original image containing multiple objects, extracting features from the original image using a residual network to obtain a combined feature; inputting the combined feature into a regression network model, mapping the combined feature using the regression network model to obtain a feature map, and mapping the feature map into a first density map; detecting the feature map to obtain a detection frame, determining the position and confidence score of each detection frame, and performing a convolution operation on the center point of the detection frame based on the position and confidence score of the detection frame to obtain a second density map; using an attention function to calculate the weights corresponding to the first density map and the second density map, and superimposing the pixels in the first density map and the second density map one by one based on the weights to obtain the number of objects in the original image. The present disclosure improves the accuracy of the object quantity detection results and expands the application scope of the object detection algorithm.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, electronic device, and storage medium for object-based quantity detection. Background Art

[0002] In the field of image recognition, images captured in specific scenes are used to estimate the number of objects in the image using image recognition algorithms. This object count detection method has been widely used in scenarios such as video surveillance and security. The following uses crowd density detection in public places as an example to introduce existing methods for crowd density detection based on image recognition technology and the existing problems.

[0003] Currently, mainstream crowd density detection methods are based on density map regression algorithms. However, counting methods based on regression-based density maps often overestimate the number of people in low-density situations. Therefore, they cannot effectively detect sparse crowds. In real-world scenarios, crowd sizes often vary across space and time. Therefore, determining crowd counts based solely on regression or other single detection methods cannot meet all requirements. Existing crowd detection methods based on density map regression suffer from inaccurate detection results and a limited range of application scenarios. Summary of the Invention

[0004] In view of this, the embodiments of the present disclosure provide an object-based quantity detection method, device, electronic device and storage medium to solve the problems of inaccurate object detection results and limited application scenarios in the prior art.

[0005] In a first aspect of an embodiment of the present disclosure, a method for object-based quantity detection is provided, comprising: obtaining an original image containing multiple objects, and using a preset residual network to perform feature extraction on the original image to obtain a combined feature; inputting the combined feature into a preset regression network model, using the regression network model to perform a mapping operation on the combined feature to obtain a feature map, and mapping the feature map to a first density map; performing a detection operation on the feature map to obtain a detection frame, and determining the position and confidence score corresponding to each detection frame, and based on the position and confidence score of the detection frame, performing a convolution operation on the center point of the detection frame to obtain a second density map; using a preset attention function to calculate the weights corresponding to the first density map and the second density map, respectively, and superimposing the pixels in the first density map and the second density map one by one based on the weights to obtain the number of objects in the original image.

[0006] According to a second aspect of an embodiment of the present disclosure, an object-based quantity detection device is provided, comprising: an extraction module configured to obtain an original image containing multiple objects, and use a preset residual network to perform feature extraction on the original image to obtain a combined feature; a regression module configured to input the combined feature into a preset regression network model, use the regression network model to perform a mapping operation on the combined feature to obtain a feature map, and map the feature map to a first density map; a detection module configured to perform a detection operation on the feature map to obtain a detection frame, and determine the position and confidence score corresponding to each detection frame, and based on the position and confidence score of the detection frame, perform a convolution operation on the center point of the detection frame to obtain a second density map; and an evaluation module configured to use a preset attention function to calculate the weights corresponding to the first density map and the second density map, respectively, and superimpose pixels in the first density map and the second density map one by one based on the weights to obtain the number of objects in the original image.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the program.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0009] At least one of the above technical solutions adopted in the embodiments of the present disclosure can achieve the following beneficial effects:

[0010] By obtaining an original image containing multiple objects, a preset residual network is used to extract features from the original image to obtain a combined feature; the combined feature is input into a preset regression network model, and a mapping operation is performed on the combined feature using the regression network model to obtain a feature map, and the feature map is mapped to a first density map; a detection operation is performed on the feature map to obtain a detection frame, and the position and confidence score corresponding to each detection frame are determined, and based on the position and confidence score of the detection frame, a convolution operation is performed on the center point of the detection frame to obtain a second density map; the weights corresponding to the first density map and the second density map are calculated using a preset attention function, and the pixels in the first density map and the second density map are superimposed one by one based on the weights to obtain the number of objects in the original image. The present disclosure combines the detection algorithm with the regression algorithm to make the object quantity detection result more accurate, and can simultaneously solve the crowd estimation problem in sparse crowd scenes and dense crowd scenes, thereby improving the application scope of the object detection algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 Schematic diagram of a network structure combining detection and regression provided by an embodiment of the present disclosure;

[0013] Figure 2 is a flow chart of an object-based quantity detection method provided by an embodiment of the present disclosure;

[0014] Figure 3 is a structural diagram of an object-based quantity detection device provided by an embodiment of the present disclosure;

[0015] Figure 4 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present disclosure with unnecessary detail.

[0017] Detecting the number of objects in a captured image based on image recognition technology is a common application in the field. For example, in a crowd image, for example, pedestrians are the subjects. Crowd counting uses an algorithm to automatically count the number of people in the image scene. This method is widely used in video surveillance, security, and other fields. In particular, real-time crowd estimation in public places such as shopping malls, stations, and tourist attractions facilitates crowd analysis and anomaly detection, effectively ensuring safety.

[0018] Public places are prone to crowds, which can easily create safety hazards. In addition, in crowded scenes, estimating the number of people can effectively suppress the continued gathering of people and provide users with corresponding dense crowd warnings, which can help people reduce unnecessary waiting time, such as the waiting time of users in the cafeteria when queuing for food.

[0019] Currently, there are two main methods for crowd size detection: detection algorithms and regression algorithms. Detection-based crowd size detection primarily targets scenes with sparse crowds. While detection algorithms can effectively obtain human detection frames, in densely populated scenes, human occlusions and poor image resolution for small objects can affect detection results. Density map estimation based on regression algorithms, on the other hand, cannot effectively detect sparse crowds, but it does produce relatively good detection results for image blocks with relatively high crowd density. Therefore, regression algorithms often overestimate crowd counts in sparse crowds or even in scenes without people.

[0020] In view of the above problems in the prior art, the present disclosure proposes a method for detecting the number of objects by converting the crowd counting task into a density map estimation problem. Since counting by a pure regression algorithm will lead to overestimation of the results when the crowd density is low, and counting by a detection algorithm is more accurate, the detection algorithm is more suitable for object detection scenarios with low crowd density. However, in actual scenarios, the number of people changes with space and time. Therefore, determining the crowd count based only on a regression algorithm or a detection algorithm cannot meet all situations. The embodiment of the present disclosure proposes a method that combines detection and regression to solve the problem of estimating the number of people in dense scenes, that is, using a detection module and a regression module at the same time to estimate the number of people, and then using an attention module to decide which module should be used to estimate the result for a specific pixel.

[0021] It should be noted that the embodiments of this disclosure are described using crowd size detection in public places as a specific application scenario, with pedestrians as the detection targets. The detection of crowd size in crowded places is performed through a combination of detection and regression. However, it should be understood that the embodiments of this disclosure are not limited to crowd size detection in public places, nor are the detection targets limited to pedestrians. Other scenarios where the number of specific objects is estimated using images are also applicable to this solution.

[0022] The following describes in detail the network structure combining detection and regression in the present disclosure with reference to the accompanying drawings. Figure 1 Schematic diagram of a network structure combining detection and regression provided by an embodiment of the present disclosure; Figure 1 As shown, the network structure may specifically include:

[0023] When detecting the number of people based on image recognition, large errors may occasionally occur in the counting results due to the oversized human bodies observed at close range. The present disclosure designs a residual learning framework to reformulate the global count regression into a learning residual function with reference to the sum of the densities in each frame of the picture. The network structure of the embodiment of the present disclosure uses the ResNet residual network as the backbone network, extracts features from the original picture based on the ResNet residual network (feature extraction network), and estimates the high feature resolution required for pixel-by-pixel density, adjusts and combines the features of the 2nd, 3rd, 4th, and 5th layers of the ResNet residual network, and then adds a convolution layer to perform feature reweighting after combining the feature quantity of the 1x1 kernel. The reweighted features can better distinguish foreground and background pixels.

[0024] Here, the network structure combining detection and regression in the embodiment of the present disclosure includes at least the following three branches, namely, a regression branch, a detection branch, and an evaluation branch. Each branch can be considered as a module that constitutes the network structure. The following will combine these three branches and specific embodiments to describe the technical solution of the present disclosure in detail.

[0025] Figure 2 It is a flowchart of the object-based quantity detection method provided by an embodiment of the present disclosure. Figure 2 The object-based number detection method can be executed by the server. Figure 2 As shown, the object-based quantity detection method may specifically include:

[0026] S201, obtaining an original image containing multiple objects, and extracting features from the original image using a preset residual network to obtain combined features;

[0027] S202, inputting the combined features into a preset regression network model, performing a mapping operation on the combined features using the regression network model to obtain a feature map, and mapping the feature map into a first density map;

[0028] S203: Perform a detection operation on the feature map to obtain a detection frame, determine the position and confidence score corresponding to each detection frame, and perform a convolution operation on the center point of the detection frame based on the position and confidence score of the detection frame to obtain a second density map;

[0029] S204: Calculate the weights corresponding to the first density map and the second density map respectively using a preset attention function, and superimpose pixels in the first density map and the second density map one by one based on the weights to obtain the number of objects in the original image.

[0030] Specifically, the original image is an image of pedestrians captured by a camera in a specific scene. The objects in the original image are the pedestrians in the image. A feature map is a graph data that combines multiple feature vectors. A density map can be thought of as a matrix divided into many boxes, with the value in each box representing the density.

[0031] Furthermore, residual networks, also known as ResNets, are based on the idea of adding direct connections to the network. By retaining a certain percentage of the output from previous layers, this allows the original input information to be passed directly to subsequent layers. Residual networks are characterized by ease of optimization and the ability to improve accuracy by increasing depth. The residual blocks within them use skip connections, alleviating the vanishing gradient problem associated with increasing depth in deep neural networks.

[0032] Compared to traditional convolutional or fully connected networks, which often experience information loss and degradation during information transmission, as well as vanishing or exploding gradients, which can render very deep networks untrainable, ResNet addresses this issue to a certain extent. By routing input information directly to the output, it protects information integrity. The entire network only needs to learn the difference between the input and output, simplifying the learning objective and difficulty.

[0033] According to the technical solution provided by the embodiment of the present disclosure, the present disclosure obtains an original image containing multiple objects, uses a preset residual network to extract features from the original image to obtain a combined feature; inputs the combined feature into a preset regression network model, uses the regression network model to perform a mapping operation on the combined feature to obtain a feature map, and maps the feature map to a first density map; performs a detection operation on the feature map to obtain a detection frame, and determines the position and confidence score corresponding to each detection frame, and based on the position and confidence score of the detection frame, performs a convolution operation on the center point of the detection frame to obtain a second density map; uses a preset attention function to calculate the weights corresponding to the first density map and the second density map, respectively, and superimposes the pixels in the first density map and the second density map one by one based on the weights to obtain the number of objects in the original image. The present disclosure combines the detection algorithm with the regression algorithm, making the object quantity detection result more accurate, and can simultaneously solve the crowd estimation problem in sparse crowd scenes and dense crowd scenes, thereby improving the application scope of the object detection algorithm.

[0034] In some embodiments, an original image containing multiple objects is obtained, and features of the original image are extracted using a preset residual network to obtain combined features, including: obtaining one or more original images, using the original images as input to a ResNet residual network, extracting features from the original images using the ResNet residual network, and reweighting the extracted features using a convolutional layer, and using the reweighted features as combined features.

[0035] Specifically, when using a network structure based on a combination of detection and regression to estimate crowd size, the original image must first be preprocessed to obtain the corresponding composite features. This involves extracting features from the original image using a feature extraction network (ResNet residual network), and then performing pixel-by-pixel density estimation to achieve the required high feature resolution. The features of layers 2, 3, 4, and 5 of the ResNet residual network are then adjusted and combined. A convolutional layer is then added to perform feature reweighting after combining the features with a 1x1 kernel. This reweighted feature can better distinguish foreground and background pixels.

[0036] In some embodiments, the combined features are input into a preset regression network model, the regression network model is used to perform a mapping operation on the combined features to obtain a feature map, and the feature map is mapped to a first density map, including: inputting the combined features into a deconvolution network of the regression network model, using the deconvolution network to map the combined features to a feature map of the same size as the original image, and using a convolution layer to map the feature map to a first density map.

[0037] Specifically, the generation of the first density map is achieved by the regression branch. The regression network model is used to obtain the combined features, which are then input into the deconvolution network. The deconvolution network contains six deconvolution layers, in which a 3x3 kernel is applied. The combined features are mapped back to the image size through the deconvolution layer, and a convolution layer with a 3x3 kernel is added, followed by a convolution layer with a 1x1 kernel, thereby mapping the feature map to the first density map.

[0038] Furthermore, the regression branch uses a regression network model to learn density estimation weight matrices corresponding to different image regions and embed geometric information. However, since the exact location of each pedestrian object in the image patch is not known in advance, the regression network model only determines crowd density based on the original image pixels. Therefore, it is necessary to further utilize the detection branch to generate a second density map.

[0039] In some embodiments, a detection operation is performed on the feature map to obtain a detection frame, and the position and confidence score corresponding to each detection frame are determined, including: using a preset feature generation network to detect the feature map to determine the position of the detection frame in the feature map, and scoring the confidence for each detection frame to obtain the confidence score corresponding to each detection frame.

[0040] Specifically, the feature map generated by the regression branch is input into the feature generation network, which corresponds to the detection branch mentioned above. The first stage of the detection branch is to use the feature generation network to determine the position of the detection box in the feature map and score the confidence of each detection box. The confidence score is used to indicate whether the position of the detection box is correct, that is, the credibility of the detection box position. In other words, the corresponding credible area (the area corresponding to the detection box) in the preprocessed feature map is obtained, as well as the confidence score corresponding to the credible area.

[0041] Furthermore, while the regression algorithm can accurately estimate crowd density in image blocks with relatively high crowd density, it tends to overestimate the number of people in sparse or even "empty" scenes. Therefore, the purpose of adding a detection branch after the regression branch is to address this issue in the regression network by generating a density map based on "location awareness." The detection branch is constructed by connecting a fully connected layer after the feature map, resulting in three detection branches. The following describes these three detection branches in detail, in conjunction with specific examples.

[0042] In some embodiments, based on the position and confidence score of the detection frame, a convolution operation is performed on the center point of the detection frame to obtain a second density map, including: a feature generation network is connected to a fully connected layer, and the fully connected layer includes a first detection branch, a second detection branch, and a third detection branch; wherein the first detection branch is used to obtain the confidence score and regression boundary of each detection frame, and use a Gaussian convolution layer to convolve the center point of the detection frame so as to concentrate the center point of the detection frame; the second detection branch is used to detect the number of actual object frames in the feature map in a predetermined sub-map, and the sub-map is a number of sub-maps obtained by segmenting the feature map, and each sub-map corresponds to an ROI value; the third detection branch is used to perform similarity detection between detection frames, and detect the degree of overlap between the detection frame and the actual object frame; the detection result of the first detection branch is adjusted based on the detection results of the second detection branch and the third detection branch to obtain a second density map.

[0043] Specifically, the first detection branch is used to obtain the detection score (ie, confidence score) and return boundary of each detection box, and add a Gaussian convolution layer; the Gaussian convolution layer uses a constant Gaussian function N det (p|μ=P,σ 2 ), for the center point of the detected bounding box Convolution is performed to concentrate the center point of the detection frame. Among them, the density map of the detection branch Expressed as:

[0044]

[0045] Furthermore, the second detection branch is used to detect the number of actual pedestrian frames (i.e., actual object frames) in the feature map within a given ROI. The third detection branch is used to evaluate the similarity between the detection frames and the degree of overlap between the detection frames and the actual frames, using the results of the third detection branch as auxiliary information for the detection branch. Utilizing these detection branches can address the issue of inaccurate estimation results caused by occlusion between people in the feature map.

[0046] In some embodiments, the weights corresponding to the first density map and the second density map are calculated using a preset attention function, and the pixels in the first density map and the second density map are superimposed one by one based on the weights to obtain the number of objects in the original image, including: performing an upsampling operation on the first density map and the second density map, and superimposing the upsampled first density map and the second density map, and inputting them into the full convolution layer and the Si layer. g In the network composed of MOID layers, the network processes the attention map, which is used as the attention function. The first density map and the second density map are scored respectively using the attention function, and weights are assigned to the first density map and the second density map according to the scores. The weights of the first density map and the second density map are weighted and summed according to the pixels in the first density map and the second density map to obtain the number of objects in the original image.

[0047] Specifically, quantity estimation is performed based on the first density map obtained by the regression branch and the second density map obtained by the detection branch, and the number of people in sparse density scenes is accurately estimated by locating the position of the human body. However, due to the low detection confidence caused by the small size of the objects and occlusion, counting by detection is not accurate in crowded occasions. On the contrary, the fully convolutional network based on the regression branch can capture rich contextual crowd density information, thus compensating for this problem. In the same scene, the density may vary significantly in different parts or time intervals. Therefore, the embodiment of the present disclosure proposes an attention module that uses the attention map to model the selection process of the best count estimate, and captures the importance weights of the first density map and the second density map by dynamically evaluating the quality of each pixel.

[0048] Furthermore, for a given image I i , the evaluation module first upsamples the results of the detection branch (i.e., the second density map) and the results of the regression branch (i.e., the first density map) to the same density map as I iThe same size. Then the three branches are stacked together as the network input with 5 channels. After four full convolution layers and one pixel-level Sigmoid layer, the output probability attention map is Att i (p|Ω eva , I i ). The attention map Att defined at pixel p i (p|Ω eva , I i ) reflects the density map based on the detection and regression counterparts Therefore, the evaluation module can determine the relative reliability between the detection branch and the regression branch. It is defined as the weighted sum of the density maps of the two branches, which is represented by the attention map Att i guide:

[0049]

[0050] Where ⊙ represents the Hadamard product of two matrices, and J is a matrix with the same scale Att i An all-1 matrix.

[0051] It can be seen that the attention map evaluates the scores of the two density maps, assigns weights to them, and obtains the final number of people through weighted summation. It should be noted that in actual applications, when using the evaluation module branch to estimate the number of people, the mapping result of the original image can also be used as the input of the model for estimation, that is, the mapping operation is performed on the pixels in the original image to obtain the mapping map, and the mapping map, the first density map and the second density map are combined. Figure 1 The three graphs are used as the output of the evaluation module branch, and the evaluation module is used to superimpose the three graphs to obtain the final estimation result.

[0052] In some embodiments, after obtaining the number of objects in the original image, the method further includes: determining the size of the objects in the original image and the distance between the objects, and determining the center point of the object based on the detection frame, performing a discretization operation on the center point to obtain a discrete feature map, and generating a third density map for quantifying the object density based on the discrete feature map, the size of the objects, and the distance between the objects.

[0053] Specifically, after determining the number of people in the original image, a final density map can be generated based on all the obtained human frames, and the density map can be used to quantify the density of people in the original image. Since the actual distances between different areas in the image are different, the distances need to be taken into account in the density estimation. Specifically:

[0054] By obtaining the center point of a human body in the original image, h(x) = ∑nd(x-xi) is used to obtain a pixel-level discrete feature map, representing an image with n human bodies. h(x) is then convolved with the Gaussian kernel G, F(x) = H(x) × G(x), to convert it into a continuous density function. Due to different shooting angles, the human bodies in the image vary in distance and size. To capture these differences, the human body's size is taken into account, as well as the average distance between each human body and its k neighboring bodies, i.e., the standard deviation a = rd of the Gaussian kernel in G(x), where r represents a hyperparameter. In other words, by discretizing the detection box, a final density estimate map is obtained.

[0055] It should be noted that this disclosure proposes a network that combines detection and regression, provides three detection branches for the detection branch, and designs a network structure for each detection branch. This not only solves the problem of crowd occlusion but also improves the accuracy of crowd counting. In practical applications, without considering the number of model parameters, the regression branch can be applied to a separate network for the backbone network; for the three detection branches in the multi-branch detection network, the second detection branch can be removed, retaining only the first and third detection branches, that is, the number of actual boxes in the ROI can be ignored.

[0056] The following are embodiments of the apparatus disclosed herein, which can be used to implement the method embodiments disclosed herein. For details not disclosed in the apparatus embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0057] Figure 3 Schematic diagram of the structure of the object-based number detection device provided by the embodiment of the present disclosure. Figure 3 As shown, the object-based quantity detection device includes:

[0058] The extraction module 301 is configured to obtain an original image containing multiple objects and extract features from the original image using a preset residual network to obtain combined features;

[0059] The regression module 302 is configured to input the combined features into a preset regression network model, perform a mapping operation on the combined features using the regression network model to obtain a feature map, and map the feature map into a first density map;

[0060] The detection module 303 is configured to perform a detection operation on the feature map to obtain a detection frame, determine the position and confidence score corresponding to each detection frame, and perform a convolution operation on the center point of the detection frame based on the position and confidence score of the detection frame to obtain a second density map;

[0061] The evaluation module 304 is configured to calculate the weights corresponding to the first density map and the second density map respectively using a preset attention function, and superimpose the pixels in the first density map and the second density map one by one based on the weights to obtain the number of objects in the original image.

[0062] In some embodiments, Figure 3 The extraction module 301 obtains one or more original pictures, uses the original pictures as input of the ResNet residual network, uses the ResNet residual network to extract features in the original pictures, and uses the convolution layer to reweight the extracted features, and uses the reweighted features as combined features.

[0063] In some embodiments, Figure 3 The combined features are input into the deconvolution network of the regression network model, the deconvolution network is used to map the combined features into a feature map of the same size as the original image, and the convolution layer is used to map the feature map into the first density map.

[0064] In some embodiments, Figure 3 The detection module 303 detects the feature map using a preset feature generation network to determine the position of the detection frame in the feature map, and performs a confidence score for each detection frame to obtain a confidence score corresponding to each detection frame.

[0065] In some embodiments, the feature generation network is connected to a fully connected layer, and the fully connected layer includes a first detection branch, a second detection branch, and a third detection branch; wherein the first detection branch is used to obtain a confidence score and a regression boundary for each detection box, and use a Gaussian convolution layer to convolve the center point of the detection box to concentrate the center point of the detection box; the second detection branch is used to detect the number of actual object boxes in a predetermined sub-image in the feature map, where the sub-image is a plurality of sub-images obtained by segmenting the feature map, and each sub-image corresponds to an ROI value; the third detection branch is used to perform similarity detection between the detection boxes and detect the degree of overlap between the detection box and the actual object box; Figure 3 The detection module 303 adjusts the detection results of the first detection branch based on the detection results of the second detection branch and the third detection branch to obtain a second density map.

[0066] In some embodiments, Figure 3The evaluation module 304 performs an upsampling operation on the first density map and the second density map, superimposes the upsampled first density map and the second density map, and inputs the upsampled first density map and the second density map into a network composed of a full convolution layer and a Sigmoid layer. The network processes the first density map and the second density map to obtain an attention map, and uses the attention map as an attention function. The attention function is used to score the first density map and the second density map respectively, and weights are assigned to the first density map and the second density map according to the scores. The weights of the first density map and the second density map are weightedly summed according to the pixels in the first density map and the second density map to obtain the number of objects in the original image.

[0067] In some embodiments, Figure 3 After obtaining the number of objects in the original image, the evaluation module 304 determines the size of the objects and the distance between the objects in the original image, determines the center point of the object based on the detection frame, performs a discretization operation on the center point to obtain a discrete feature map, and generates a third density map for quantifying the object density based on the discrete feature map, the size of the object, and the distance between the objects.

[0068] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0069] Figure 4 Schematic diagram of the structure of the electronic device 4 provided by the embodiment of the present disclosure. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps of the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0070] For example, computer program 403 may be divided into one or more modules / units, which are stored in memory 402 and executed by processor 401 to implement the present disclosure. One or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of computer program 403 in electronic device 4.

[0071] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include but is not limited to a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4It is only an example of the electronic device 4 and does not constitute a limitation of the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0072] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0073] Memory 402 can be an internal storage unit of electronic device 4, such as a hard drive or memory of electronic device 4. Memory 402 can also be an external storage device of electronic device 4, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on electronic device 4. Furthermore, memory 402 can include both an internal storage unit of electronic device 4 and an external storage device. Memory 402 is used to store computer programs and other programs and data required by the electronic device. Memory 402 can also be used to temporarily store data that has been output or is about to be output.

[0074] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0075] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0076] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0077] In the embodiments provided in the present disclosure, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely schematic. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of the apparatus or unit, which may be electrical, mechanical or other forms.

[0078] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0079] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0080] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0081] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure, and should all be included in the scope of protection of the present disclosure.

Claims

1. A method for detecting the number of objects, characterized in that: include: Obtain an original image containing multiple objects, and use a preset residual network to extract features from the original image to obtain combined features; Inputting the combined features into a preset regression network model, performing a mapping operation on the combined features using the regression network model to obtain a feature map, and mapping the feature map into a first density map; Performing a detection operation on the feature map to obtain a detection frame, determining a position and a confidence score corresponding to each detection frame, and performing a convolution operation on a center point of the detection frame based on the position of the detection frame and the confidence score to obtain a second density map; Calculating weights corresponding to the first density map and the second density map respectively using a preset attention function, and superimposing pixels in the first density map and the second density map one by one based on the weights to obtain the number of objects in the original image; The performing of a detection operation on the feature map to obtain a detection frame, and determining a position and a confidence score corresponding to each detection frame, includes: Detecting the feature map using a preset feature generation network to determine the position of the detection frame in the feature map, and performing a confidence score for each detection frame to obtain a confidence score corresponding to each detection frame; The step of performing a convolution operation on the center point of the detection frame based on the position of the detection frame and the confidence score to obtain a second density map includes: The feature generation network is connected to a fully connected layer, and the fully connected layer includes a first detection branch, a second detection branch, and a third detection branch; The first detection branch is used to obtain the confidence score and the return boundary of each detection frame, and use a Gaussian convolution layer to convolve the center point of the detection frame to concentrate the center point of the detection frame; The second detection branch is used to detect the number of actual object frames in the feature map in a predetermined sub-map, where the sub-map is a plurality of sub-maps obtained by segmenting the feature map, and each sub-map corresponds to an ROI value; The third detection branch is used to perform similarity detection between the detection frames and detect the degree of overlap between the detection frames and the actual object frame; The detection results of the first detection branch are adjusted based on the detection results of the second detection branch and the third detection branch to obtain the second density map.

2. The method according to claim 1, characterized in that The obtaining of an original image containing multiple objects and extracting features from the original image using a preset residual network to obtain combined features include: Obtain one or more original images, use the original images as input to a ResNet residual network, extract features from the original images using the ResNet residual network, reweight the extracted features using a convolutional layer, and use the reweighted features as the combined features.

3. The method according to claim 1, characterized in that The step of inputting the combined features into a preset regression network model, performing a mapping operation on the combined features using the regression network model to obtain a feature map, and mapping the feature map into a first density map includes: The combined features are input into the deconvolution network of the regression network model, the combined features are mapped into a feature map of the same size as the original image using the deconvolution network, and the feature map is mapped into a first density map using a convolution layer.

4. The method according to claim 1, wherein The calculating weights corresponding to the first density map and the second density map respectively by using a preset attention function, and superimposing pixels in the first density map and the second density map one by one based on the weights to obtain the number of objects in the original image includes: Performing an upsampling operation on the first density map and the second density map, superimposing the upsampled first density map and the second density map, and inputting the upsampled first density map and the second density map into a network consisting of a full convolutional layer and a sigmoid layer, processing the network to obtain an attention map, and using the attention map as the attention function; The first density map and the second density map are scored respectively using the attention function, weights are assigned to the first density map and the second density map according to the scores, and the weights of the first density map and the second density map are weighted summed according to the pixels in the first density map and the second density map to obtain the number of objects in the original image.

5. The method according to claim 1, wherein After obtaining the number of objects in the original picture, the method further includes: Determine the size of the objects and the distance between the objects in the original image, determine the center point of the object based on the detection frame, perform a discretization operation on the center point to obtain a discrete feature map, and generate a third density map for quantifying the object density based on the discrete feature map, the size of the objects, and the distance between the objects.

6. An object-based quantity detection device, characterized in that: include: An extraction module is configured to obtain an original image containing multiple objects and perform feature extraction on the original image using a preset residual network to obtain a combined feature; a regression module configured to input the combined features into a preset regression network model, perform a mapping operation on the combined features using the regression network model to obtain a feature map, and map the feature map into a first density map; a detection module configured to perform a detection operation on the feature map to obtain a detection frame, determine a position and a confidence score corresponding to each detection frame, and perform a convolution operation on a center point of the detection frame based on the position of the detection frame and the confidence score to obtain a second density map; an evaluation module configured to calculate weights corresponding to the first density map and the second density map respectively using a preset attention function, and superimpose pixels in the first density map and the second density map one by one based on the weights to obtain the number of objects in the original image; The detection module is configured to detect the feature map using a preset feature generation network to determine the position of the detection frame in the feature map, and to perform a confidence score for each detection frame to obtain a confidence score corresponding to each detection frame; The feature generation network is connected to the fully connected layer, and the fully connected layer includes a first detection branch, a second detection branch and a third detection branch; wherein the first detection branch is used to obtain the confidence score and the regression boundary of each detection frame, and use the Gaussian convolution layer to convolve the center point of the detection frame so as to concentrate the center point of the detection frame; the second detection branch is used to detect the number of actual object frames in the feature map in the predetermined sub-map, and the sub-map is a number of sub-maps obtained by segmenting the feature map, and each sub-map corresponds to an ROI value; the third detection branch is used to perform similarity detection between the detection frames, and detect the overlap between the detection frame and the actual object frame; the detection result of the first detection branch is adjusted based on the detection results of the second detection branch and the third detection branch to obtain the second density map.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images