Crowd positioning method and device, electronic device and storage medium

Abstract

The present disclosure relates to a crowd positioning method and apparatus, an electronic device, and a storage medium. The method comprises: performing crowd positioning on a crowd image to obtain at least two first positioning maps, each of which is used to indicate the preliminary positioning positions of people included in the crowd image, and each of which has a different size; and determining a final positioning map based on the at least two first positioning maps, the final positioning map indicating the final positioning positions of people included in the crowd image. Embodiments of the present disclosure can effectively improve the accuracy of crowd positioning.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a crowd positioning method and device, an electronic device, and a storage medium. Background Art

[0002] With population growth and accelerated urbanization, large crowd gatherings are becoming more frequent and larger in scale. Crowd analysis is crucial for public safety and urban planning. Common crowd analysis tasks include crowd counting, crowd behavior analysis, and crowd location. Crowd location is the foundation for all other crowd analysis tasks. Crowd location uses computer vision algorithms to estimate the positions of human bodies within an image or video. This determines the coordinates of the human bodies within the image or video, providing data for subsequent crowd analysis tasks such as crowd counting and crowd behavior analysis. The accuracy of crowd location directly impacts the precision of crowd counting and crowd behavior analysis. Therefore, a highly accurate crowd location method is urgently needed. Summary of the Invention

[0003] The present disclosure proposes a crowd positioning technology solution.

[0004] According to one aspect of the present disclosure, a crowd positioning method is provided, comprising: performing crowd positioning on a crowd image to obtain at least two first positioning maps, the first positioning maps being used to indicate preliminary positioning positions of human bodies included in the crowd image, the first positioning maps having different sizes; and determining a final positioning map based on the at least two first positioning maps, the final positioning map being used to indicate final positioning positions of human bodies included in the crowd image.

[0005] In one possible implementation, the crowd positioning on the crowd image to obtain at least two first positioning maps includes: performing feature extraction on the crowd image to obtain at least two first feature maps, each first feature map having a different size; and performing crowd positioning according to each of the first feature maps to obtain a first positioning map corresponding to each of the first feature maps.

[0006] In one possible implementation, determining the final positioning map based on the at least two first positioning maps includes: for any first positioning map, performing activation processing on the first positioning map to obtain a second feature map; performing threshold segmentation on the second feature map according to a preset feature threshold to obtain a third feature map, where the third feature map is a binary map consisting of 0 and 1; performing non-maximum suppression processing on the third feature map according to a preset neighborhood size to obtain a second positioning map; and determining the final positioning map based on the at least two second positioning maps.

[0007] In one possible implementation, determining the final positioning map based on the at least two second positioning maps includes: resizing each second positioning map to obtain at least two third positioning maps, where the sizes of the third positioning maps are the same as the sizes of the crowd images; and determining the final positioning map based on the at least two third positioning maps.

[0008] In one possible implementation, determining the final positioning map based on the at least two third positioning maps includes: determining any third positioning map among the at least two third positioning maps as a fourth positioning map; for any first pixel point with a value of 1 in the fourth positioning map, determining a first distance between the first pixel point and each second pixel point with a value of 1 in a fifth positioning map, where the fifth positioning map is any third positioning map among the at least two third positioning maps except the fourth positioning map; determining whether a second distance less than a preset distance threshold exists in the first distance; and if the second distance does not exist, fusing the at least two third positioning maps to obtain the final positioning map.

[0009] In one possible implementation, determining the final positioning map based on the at least two third positioning maps further includes: when the second distance exists, setting the value of the second pixel corresponding to the second distance in the fifth positioning map to 0 to obtain a sixth positioning map; and fusing the fourth positioning map and the sixth positioning map to obtain a final positioning map.

[0010] In a possible implementation, the third feature map is subjected to non-maximum suppression processing according to a preset neighborhood size to obtain a second positioning map, including: for any third pixel point with a value of 1 in the third feature map, determining the neighborhood of the third pixel point in the third feature map according to a preset neighborhood size; judging whether there is a fourth pixel point with a value of 1 in the neighborhood of the third pixel point; if the fourth pixel point exists, determining a first value corresponding to the third pixel point and a second value corresponding to the fourth pixel point from the first positioning map respectively; determining the minimum value between the first value and the second value; determining the pixel point corresponding to the minimum value among the third pixel point and the fourth pixel point as the target pixel point; and setting the value of the target pixel point in the third feature map to 0 to obtain a second positioning map.

[0011] In one possible implementation, the crowd positioning method is implemented by a crowd positioning neural network, which is used to perform crowd positioning on the crowd image to obtain at least two first positioning maps. The training set of the crowd positioning neural network includes multiple first crowd sample images and at least two real positioning maps corresponding to each of the first crowd sample images. The method also includes: for any first crowd sample image in the training set, using the crowd positioning neural network, performing crowd positioning on the first crowd sample image to obtain at least two first predicted positioning maps; activating the at least two first predicted positioning maps to obtain at least two second predicted positioning maps; determining the positioning loss corresponding to the first crowd sample image based on the at least two second predicted positioning maps and the at least two real positioning maps corresponding to the first crowd sample image; determining the network loss based on the positioning loss corresponding to each of the first crowd sample images in the training set; and optimizing the crowd positioning neural network based on the network loss.

[0012] In a possible implementation, the method further includes: for any first population sample image, determining a first annotation result of the first population sample image, the first annotation result including the coordinates of the human body in the first population sample image; performing size transformation on the first population sample image according to at least two preset sizes, to obtain second population sample images corresponding to each of the preset sizes; performing coordinate transformation on the first annotation result according to the at least two preset sizes, to obtain second annotation results corresponding to each of the preset sizes, the second annotation result including the coordinates of the human body in the corresponding second population sample image; determining a real positioning map corresponding to each of the preset sizes based on the second population sample image corresponding to each of the preset sizes and the second annotation result.

[0013] According to one aspect of the present disclosure, a crowd positioning device is provided, comprising: a first positioning module, configured to perform crowd positioning on a crowd image to obtain at least two first positioning maps, wherein the first positioning maps are configured to indicate preliminary positioning positions of people included in the crowd image, and the sizes of the first positioning maps are different;

[0014] The second positioning module is configured to determine a final positioning map based on the at least two first positioning maps, where the final positioning map is used to indicate a final positioning position of a human body included in the crowd image.

[0015] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0016] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above method is implemented.

[0017] In the disclosed embodiments, crowd positioning can be performed on a crowd image, obtaining at least two first positioning maps of different sizes. A final positioning map can then be determined based on the at least two first positioning maps of different sizes. Compared to existing crowd positioning methods based on positioning maps of a single size, the crowd positioning method of the disclosed embodiments can determine the final positioning map by fusing first positioning maps of multiple sizes, achieving crowd positioning. This allows crowd positioning to accommodate people of multiple sizes in the same or different scenes, effectively improving the accuracy of crowd positioning.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, rather than limiting the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0020] Figure 1 A flow chart of a crowd positioning method according to an embodiment of the present disclosure is shown.

[0021] Figure 2 A schematic diagram illustrating the processing procedure of the crowd positioning method according to an embodiment of the present disclosure.

[0022] Figure 3 A block diagram of a crowd positioning device according to an embodiment of the present disclosure is shown.

[0023] Figure 4 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0024] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0025] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0026] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0027] The term "and / or" herein simply describes an association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent the existence of three situations: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0028] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0029] The crowd positioning method of the embodiment of the present disclosure can be applied to scenarios requiring crowd positioning, such as intelligent video analysis, security, and crowd products (such as Tuling crowd products). The present disclosure does not limit the specific application scenarios of the crowd positioning method.

[0030] The crowd positioning method of the embodiments of the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The crowd positioning method can be implemented by a processor calling computer-readable instructions stored in a memory. Alternatively, the crowd positioning method can be executed by a server.

[0031] Figure 1 FIG. 1 is a flow chart showing a crowd positioning method according to an embodiment of the present disclosure. Figure 1 As shown, the crowd positioning method includes:

[0032] Step S11 , performing crowd positioning on the crowd image to obtain at least two first positioning maps.

[0033] Among them, the crowd image is an image containing a dense crowd, which can be obtained by an image acquisition device after capturing images of a dense crowd in a certain spatial range (such as squares, shopping malls, subway stations, tourist attractions and other places with large traffic flow), or it can be a key image frame containing a dense crowd obtained from a video, or it can be obtained by other means, and the present disclosure does not impose any restrictions on this.

[0034] Crowd localization in a crowd image refers to estimating the positions of human bodies within the crowd image. In step S11, crowd estimation can be performed on the crowd image using a computer vision algorithm to obtain at least two first localization maps. The first localization maps can be used to indicate the preliminary positions of human bodies within the crowd image. The sizes of the at least two first localization maps are different.

[0035] In one possible implementation, the human body can be represented by the center point of the human head. Then, the first positioning map can be used to indicate the preliminary positioning position of the center point of the human head included in the crowd image (i.e., the coordinates of the preliminary positioned center point of the human head). Those skilled in the art will appreciate that the human body can also be represented by other methods, and this disclosure does not limit this.

[0036] It should be noted that those skilled in the art may set the specific number and size of the first positioning map according to actual conditions, such as the accuracy and performance of the computer vision algorithm used, and the present disclosure does not limit this.

[0037] Step S12: determining a final positioning map based on the at least two first positioning maps.

[0038] Since the first positioning map is used to indicate the preliminary positioning positions of the human bodies included in the crowd image, after obtaining at least two first positioning maps, a final positioning map can be determined in step S12 based on the at least two first positioning maps. The final positioning map can be used to indicate the final positioning positions of the human bodies included in the crowd image. For example, in step S12, the at least two first positioning maps can be fused to obtain the final positioning map. The size of the final positioning map is the same as the size of the crowd image. When the human body is represented by the center point of the human head, the final positioning map can be used to indicate the final positioning position of the center point of the human head included in the crowd image (i.e., the coordinates of the center point of the human head at the final location).

[0039] In one possible implementation, the final positioning map may be a binary map consisting of 0 and 1. When the value of a pixel point (i, j) in the final positioning map is 1, the pixel point (i, j) in the crowd image may be considered to be the location of a human body; when the value of a pixel point (i, j) in the final positioning map is 0, the pixel point (i, j) in the crowd image may be considered to be a location outside the human body, where i and j are both integers greater than or equal to 0.

[0040] For example, assuming that the human body is represented by the center point of the human head, then if the value of pixel (i, j) in the final positioning map is 1, pixel (i, j) in the crowd image can be considered to be the center point of the human head, and the coordinates (i, j) of pixel (i, j) are the coordinates of the center point of the human head in the crowd image; if the value of pixel (i, j) in the final positioning map is 0, pixel (i, j) in the crowd image can be considered to be somewhere other than the center point of the human head. In other words, a pixel with a value of 1 in the final positioning map indicates that the corresponding pixel in the crowd image is the center point of the human head, and a pixel with a value of 0 in the final positioning map indicates that the corresponding pixel in the crowd image is somewhere other than the center point of the human head.

[0041] According to the embodiments of the present disclosure, crowd positioning can be performed on a crowd image, obtaining at least two first positioning maps of different sizes, and then determining a final positioning map based on the at least two first positioning maps of different sizes. Compared with existing crowd positioning methods based on positioning maps of a single size, the crowd positioning method of the embodiments of the present disclosure can determine the final positioning map based on the fusion of first positioning maps of multiple sizes to achieve crowd positioning. This allows crowd positioning to be compatible with people of multiple sizes in the same scene or different scenes, effectively improving the accuracy of crowd positioning.

[0042] In a possible implementation, step S11 may include: performing feature extraction on the crowd image to obtain at least two first feature maps, each of which has a different size; and performing crowd positioning according to each of the first feature maps to obtain a first positioning map corresponding to each of the first feature maps.

[0043] When performing crowd location on a crowd image, feature extraction can be performed on the crowd image to obtain at least two first feature maps, where each first feature map has a different size. For example, feature extraction can be performed on the crowd image to obtain two first feature maps of different sizes, namely, first feature map A1 and first feature map A2.

[0044] After obtaining at least two first feature maps, crowd positioning can be performed based on each first feature map, resulting in a first positioning map corresponding to each first feature map. For example, crowd positioning can be performed based on first feature map A1, resulting in a first positioning map B1 corresponding to first feature map A1, with the size of first positioning map B1 being the same as that of first feature map A1. Crowd positioning can be performed based on first feature map A2, resulting in a first positioning map B2 corresponding to first feature map A2, with the size of first positioning map B2 being the same as that of first feature map A2.

[0045] It should be noted that those skilled in the art may set the specific number and specific size of the first feature map according to actual conditions, and this disclosure does not limit this.

[0046] When locating a crowd in a crowd image, features are first extracted from the crowd image to obtain at least two first feature maps, and then the crowd is located based on each first feature map to obtain a first positioning map corresponding to each first feature map. This not only improves processing efficiency but also improves the accuracy of the first positioning map.

[0047] In one possible implementation, the crowd localization method of the embodiments of the present disclosure can be implemented using a crowd localization neural network. The crowd localization neural network can be used to localize crowds in crowd images, generating at least two first localization maps. The crowd localization neural network can be a convolutional neural network (CNN) or other type of neural network. The present disclosure does not limit the specific type of the crowd localization neural network.

[0048] Before using a crowd localization neural network to locate crowds in a crowd image, the neural network must be trained. The training set for the crowd localization neural network may include multiple first crowd sample images and at least two ground truth localization maps corresponding to each of the first crowd sample images. The ground truth localization maps corresponding to each of the first crowd sample images may have different sizes.

[0049] In a possible implementation, the crowd positioning method may also include a related processing process of determining at least two real positioning maps corresponding to each first crowd sample image, specifically: for any first crowd sample image, determining a first annotation result of the first crowd sample image, the first annotation result including the coordinates of the human body in the first crowd sample image; performing size transformation on the first crowd sample image according to at least two preset sizes, to obtain second crowd sample images corresponding to each preset size; performing coordinate transformation on the first annotation result according to the at least two preset sizes, to obtain second annotation results corresponding to each preset size, the second annotation result including the coordinates of the human body in the corresponding second crowd sample image; determining the real positioning map corresponding to each preset size based on the second crowd sample image corresponding to each preset size and the second annotation result.

[0050] For any first sample image of a population, a first annotation result of the first sample image of the population may be determined first. The first annotation result may include the coordinates of the human body in the first sample image of the population. For example, the first sample image I∈R H1 ×W1×3 , H1 and W1 are the height and width of the first population sample image I respectively, 3 is the number of channels of the first population sample image I (for example, RGB three channels), the human body included in the first population sample image I can be annotated to obtain the first annotation result of the first population sample image I Among them, a v is the coordinate of the vth human body in the first crowd sample image I, n is the number of human bodies included in the first crowd sample image I, and both n and v are positive integers.

[0051] At the same time, the first crowd sample image can be resized according to at least two preset sizes to obtain second crowd sample images corresponding to the respective preset sizes. For example, assuming that the size of the first crowd sample image is the same as the size of the input image of the crowd positioning neural network, both are 720P, and the crowd positioning neural network outputs two images, whose sizes are 1 / 8 (stride8, the size of the input image reduced by 8 times) and 1 / 16 (stride16, the size of the input image reduced by 16 times) of the input image, respectively. Then, the preset sizes may include stride8 and stride16. Then, according to these two preset sizes, the first crowd sample image is resized to obtain second crowd sample images corresponding to the respective preset sizes.

[0052] Afterwards, the first annotation result can be coordinate-transformed according to at least two preset sizes to obtain second annotation results corresponding to each preset size. The second annotation result can include the coordinates of the human body in the corresponding second population sample image. For example, when the first annotation result is coordinate-transformed according to the preset size stride8, the horizontal coordinate and the vertical coordinate of the first annotation result can be divided by 8 to obtain the second annotation result corresponding to the preset size stride8. The second annotation result corresponding to the preset size stride8 includes the coordinates of the human body in the second population sample image with a size of stride8 (i.e., corresponding to the preset size stride8); when the first annotation result is coordinate-transformed according to the preset size stride16, the horizontal coordinate and the vertical coordinate of the first annotation result can be divided by 16 to obtain the second annotation result corresponding to the preset size stride16. The second annotation result corresponding to the preset size stride16 includes the coordinates of the human body in the second population sample image with a size of stride16 (i.e., corresponding to the preset size stride16).

[0053] Then, the real positioning map corresponding to each preset size can be determined based on the second crowd sample image corresponding to each preset size and the second annotation result. For example, the second crowd sample image corresponding to the preset size stride8 is I′∈R H2 ×W2×3 , H2 and W2 are the height and width of the second crowd sample image I′ respectively, 3 is the number of channels of the second crowd sample image I′, and the second annotation result corresponding to the preset size stride8 is a′ v The coordinates of the vth person in the second crowd sample image I′ can be determined using the following formula (1): H2 ×W2 :

[0054]

[0055] in, x is the coordinate of the pixel with the same relative position in the second crowd sample image I′ and the true positioning map Y, K = [0, 0, 0; 0, 1, 0; 0, 0, 0] is the convolution kernel, ψ is the convolution result map, δ(·) is the multivariate delta function, and its specific form can be shown as the following formula (2):

[0056]

[0057] The size of the true localization map Y is the same as that of the second crowd sample image I′, both of which are stride 8. The true localization map Y is a binary image composed of 0 and 1, where the value of the pixel point corresponding to the human body coordinate is 1, and the value of the pixel point at other positions is 0.

[0058] In one possible implementation, the true positioning map corresponding to the preset size stride 16 can be determined by the above method, thereby obtaining two true positioning maps, namely, the true positioning map corresponding to the preset size stride 8 and the true positioning map corresponding to the preset size stride 16, and these two true positioning maps are determined as the two true positioning maps corresponding to the first population sample image I.

[0059] It should be noted that, in addition to using the above formula (1) to determine the true positioning map, other methods may also be used to determine the true positioning map, and the present disclosure does not limit this.

[0060] It should be noted that the above description uses only two real positioning maps as an example to exemplify the process of determining at least two real positioning maps corresponding to the first crowd sample image. If there are multiple real positioning maps corresponding to the first crowd sample image, the determination process is similar to the above process and will not be further described here.

[0061] In one possible implementation, if the size of the first crowd sample image is different from the size of the input image of the crowd positioning neural network, in order to make the size of the first crowd sample image the same as the size of the input image of the crowd positioning neural network, the first crowd sample image can be resized and the first annotation result can be coordinate-transformed based on the proportional relationship between the size of the first crowd sample image and the size of the input image of the crowd positioning neural network to obtain the transformed first crowd sample image and the transformed first annotation result, and then at least two real positioning maps corresponding to the first crowd sample image are determined through the above method.

[0062] For example, assuming that the size of the crowd sample image is 1080P, the size of the input image of the crowd positioning neural network is 720P, and the ratio between 1080P and 720P is 1.5, the first crowd sample image can be resized (reduced by 1.5 times) according to the ratio 1.5, and the first annotation result can be coordinate-transformed (both the horizontal and vertical coordinates are divided by 1.5) to obtain the transformed first crowd sample image and the transformed first annotation result, and then at least two real positioning maps corresponding to the first crowd sample image are determined through the above method.

[0063] In this embodiment, when determining at least two real positioning maps corresponding to the first population sample image, the first annotation result of the first population sample image is first determined, and the first population sample image is resized according to at least two preset sizes to obtain second population sample images corresponding to each preset size. Then, the first annotation result is coordinate-transformed according to at least two preset sizes to obtain second annotation results corresponding to each preset size. The second annotation result includes the coordinates of the human body in the corresponding second population sample image. Then, based on the second population sample image corresponding to each preset size and the second annotation result, the real positioning map corresponding to each preset size is determined, thereby obtaining at least two real positioning maps corresponding to the first population sample image.

[0064] In this way, the coordinates of the human body in the first population sample image can be used as the annotation result, and through size transformation, coordinate transformation, etc., at least two real positioning maps corresponding to the first population sample image can be obtained, which not only reduces the annotation cost but also improves the accuracy of the at least two real positioning maps corresponding to the first population sample image.

[0065] In one possible implementation, after obtaining at least two real positioning maps corresponding to each first crowd sample image, a training set for training a crowd positioning neural network can be established based on multiple first crowd sample images and at least two real positioning maps corresponding to each first crowd sample image.

[0066] In a possible implementation, the method may also include a related process of training a crowd localization neural network, specifically: for any first crowd sample image in the training set, crowd localization is performed on the first crowd sample image through the crowd localization neural network to obtain at least two first predicted localization maps; activation processing is performed on the at least two first predicted localization maps to obtain at least two second predicted localization maps; based on the at least second predicted localization map and at least two real localization maps corresponding to the first crowd sample image, the localization loss corresponding to the first crowd sample image is determined; based on the localization loss corresponding to each first crowd sample image in the training set, the network loss is determined; and based on the network loss, the crowd localization neural network is optimized.

[0067] When training a crowd localization neural network, any first crowd sample image in a training set can be input into the crowd localization neural network for processing to obtain at least two first predicted localization maps; then, the at least two first predicted localization maps are activated, for example, by activating each first predicted localization map through a Sigmoid function, mapping the value of each pixel point in each first predicted localization map to the (0, 1) interval, to obtain at least two second predicted localization maps, wherein the value of each pixel point in the second predicted localization map is in the (0, 1) interval; then, the localization loss corresponding to the first crowd sample image can be determined based on the at least two second predicted localization maps and at least two true localization maps corresponding to the first crowd sample image. For example, the binary cross entropy between the second predicted localization map and the true localization map of the same size can be calculated respectively, and the calculated binary cross entropies can be weightedly summed according to preset weights to obtain the localization loss corresponding to the first crowd sample image.

[0068] After determining the positioning loss corresponding to each first crowd sample image in the training set using the above method, the network loss can be determined based on the positioning loss corresponding to each first crowd sample image in the training set. For example, the sum of the positioning losses corresponding to each first crowd sample image in the training set can be determined as the network loss. The crowd localization neural network can then be optimized based on the network loss. When preset training termination conditions are met (e.g., the network loss converges within a certain interval, the number of training rounds reaches a preset round threshold, etc.), training can be terminated, resulting in a trained crowd localization neural network.

[0069] In this embodiment, the real positioning map in the training set is used as a supervisory signal. The positioning loss corresponding to each first crowd sample image in the training set is determined, and then the network loss is determined. The crowd positioning neural network is optimized according to the network loss, thereby obtaining a trained crowd positioning neural network to improve the accuracy of the crowd positioning neural network when used for crowd positioning.

[0070] After obtaining the trained crowd positioning neural network, when performing crowd positioning on a crowd image, the crowd image can be input into the trained crowd positioning neural network for processing to obtain at least two first positioning maps.

[0071] In one possible implementation, step S12 may include: for any first positioning map, activating the first positioning map to obtain a second feature map; performing threshold segmentation on the second feature map according to a preset feature threshold to obtain a third feature map; performing non-maximum suppression processing on the third feature map according to a preset neighborhood size to obtain a second positioning map; and determining a final positioning map based on at least two second positioning maps.

[0072] When determining the final positioning map based on at least two first positioning maps, for any first positioning map, the first positioning map can be first activated. For example, the first positioning map can be activated by a Sigmoid function, and the values ​​of each pixel in the first positioning map are mapped to the (0, 1) interval to obtain a second feature map, and the values ​​of each pixel in the second feature map are located in the (0, 1) interval; then, according to a preset feature threshold, the second feature map is threshold segmented to obtain a third feature map. For example, assuming that the feature threshold is 0.25, when the second feature map is threshold segmented, the values ​​of pixels in the second feature map with values ​​greater than 0.25 are set to 1, and the values ​​of pixels in the second feature map with values ​​less than or equal to 0.25 are set to 0, thereby obtaining a third feature map, which is a binary map consisting of 0 and 1; then, according to a preset neighborhood size, the third feature map can be non-maximum suppression processed to obtain a second positioning map.

[0073] In one possible implementation, when performing non-maximum suppression processing on the third feature map, for any third pixel point with a value of 1 in the third feature map, the neighborhood of the third pixel point can be determined in the third feature map based on a preset neighborhood size (for example, the neighborhood size is the pixel radius of the neighborhood is 2), and then it is determined whether there is a fourth pixel point with a value of 1 in the neighborhood of the third pixel point.

[0074] In the case that there is a fourth pixel point with a value of 1 in the neighborhood of the third pixel point, the first value corresponding to the third pixel point and the second value corresponding to the fourth pixel point are determined from the first positioning map, and the minimum value of the first value and the second value is determined. Then, the pixel point corresponding to the minimum value among the third pixel point and the fourth pixel point is determined as the target pixel point, and the value of the target pixel point in the third feature map is set to 0.

[0075] For example, when there is a fourth pixel with a value of 1 in the neighborhood of the third pixel, the first value corresponding to the third pixel in the first positioning map is 0.45, the second value corresponding to the fourth pixel in the first feature map is 0.72, and the minimum value of the first and second values ​​is 0.45. Then, the third pixel corresponding to the minimum value of 0.45 among the third and fourth pixels can be determined as the target pixel, and the value of the target pixel in the third feature map is set to 0.

[0076] The above processing can be performed on all third pixel points with a value of 1 in the third feature map. If there is no fourth pixel point with a value of 1 in the neighborhood of all third pixel points with a value of 1 in the third feature map, the third feature map can be determined as the second positioning map.

[0077] By performing non-maximum suppression processing on the third feature map in this way, false detection of adjacent positions of the same human body can be suppressed, thereby improving the accuracy of the second positioning map.

[0078] It should be noted that those skilled in the art may set specific values ​​of the feature threshold and the neighborhood size according to actual conditions, and this disclosure does not impose any restrictions on this.

[0079] In a possible implementation, after obtaining at least two second positioning maps in the above manner, a final positioning map may be determined based on the at least two second positioning maps.

[0080] In this embodiment, by performing activation, threshold segmentation, and non-maximum suppression on each first positioning map, a second positioning map is obtained. Compared with the first positioning map, the second positioning map has higher accuracy. Then, a final positioning map is determined based on at least two second positioning maps with higher accuracy, thereby improving the accuracy of the final positioning map.

[0081] In one possible implementation, determining the final positioning map based on the at least two second positioning maps includes: resizing each second positioning map to obtain at least two third positioning maps, where the sizes of the third positioning maps are the same as the sizes of the crowd images; and determining the final positioning map based on the at least two third positioning maps.

[0082] Because the sizes of the second positioning maps are different, when determining the final positioning map based on the at least two second positioning maps, the second positioning maps may first be resized to obtain at least two third positioning maps. The third positioning maps correspond to the second positioning maps and have the same size as the crowd image.

[0083] For example, assuming that the size of the second positioning map is stride 8 and the size of the crowd image is 1080P, then the horizontal and vertical coordinates of the pixel points with a value of 1 in the second positioning map can be first multiplied by 8 to obtain the horizontal and vertical coordinates of the pixel points with a value of 1 in the second positioning map at the 720P size. Then, the horizontal and vertical coordinates at the 720P size are multiplied by 1.5 to obtain the horizontal and vertical coordinates of the pixel points with a value of 1 in the second positioning map at the 1080P size. Then, based on the horizontal and vertical coordinates of the pixel points with a value of 1 in the second positioning map at the 1080P size, a third positioning map corresponding to the second positioning map is generated.

[0084] It should be noted that when the size of the second positioning map is other sizes (such as stride 16), the size conversion method is similar to the above, and will not be repeated here.

[0085] After obtaining at least two third positioning maps, a final positioning map may be determined based on the at least two third positioning maps. For example, the final positioning map may be obtained by performing image fusion on the at least two third positioning maps.

[0086] In this embodiment, when determining the final positioning map based on at least two second positioning maps, each second positioning map is first resized to obtain at least two third positioning maps, so that the size of each third positioning map is the same as the size of the crowd image. Then, the final positioning map is determined based on the at least two third positioning maps, thereby improving the accuracy of the final positioning map.

[0087] In one possible implementation, when determining a final positioning map based on at least two third positioning maps, any third positioning map in the at least two third positioning maps may be determined as a fourth positioning map, and any third positioning map other than the fourth positioning map in the at least two third positioning maps may be determined as a fifth positioning map. For any first pixel point with a value of 1 in the fourth positioning map, a first distance is determined between the first pixel point and each second pixel point with a value of 1 in the fifth positioning map. The first distance may be the Euclidean distance between the coordinates of the first pixel point and the coordinates of the second pixel point. A determination is then made as to whether any second distance in the first distance is less than a preset distance threshold (e.g., distance threshold = 20).

[0088] If there is no second distance less than the distance threshold in the first distance, the at least two third positioning maps are fused to obtain a final positioning map. For example, all pixels with a value of 1 in the at least two third positioning maps may be fused into a positioning map according to their coordinate positions, and the fused positioning map is determined as the final positioning map.

[0089] If there is a second distance smaller than the distance threshold in the first distance, the value of the second pixel corresponding to the second distance in the fifth positioning map is set to 0 to obtain a sixth positioning map, and then the fourth positioning map and the sixth positioning map are fused to obtain a final positioning map.

[0090] For example, assuming that there are two third positioning maps, namely, a third positioning map D1 with a size of stride 8 and a third positioning map D2 with a size of stride 16, the third positioning map D1 can be regarded as the fourth positioning map, and the third positioning map D2 can be regarded as the fifth positioning map. For any first pixel point with a value of 1 in the third positioning map D1 (fourth positioning map), a first distance between the first pixel point and each second pixel point with a value of 1 in the third positioning map D2 (fifth positioning map) is determined. Then, it is determined whether there is a second distance in the first distance that is less than a preset distance threshold (distance threshold = 20).

[0091] If there is no second distance less than the distance threshold in the first distances, that is, all first distances are greater than or equal to the distance threshold, then all pixel points with a value of 1 in the third positioning map D1 and the third positioning map D2 are fused into one positioning map according to their coordinate positions, and the fused positioning map is determined as the final positioning map.

[0092] If there is a second distance smaller than the distance threshold in the first distance, the value of the second pixel corresponding to the second distance in the third positioning map D2 is set to 0 to obtain a sixth positioning map. Then, all pixels with a value of 1 in the third positioning map D1 and the sixth positioning map are fused into one positioning map according to their coordinate positions, and the fused positioning map is determined as the final positioning map.

[0093] In this embodiment, when determining the final positioning map based on at least two third positioning maps, it is possible to determine whether there are two or more pixels in the at least two third positioning maps that have a value of 1 and are relatively close to each other (less than a distance threshold). If not, the at least two third positioning maps are directly fused to obtain the final positioning map. If so, the value of one of the two or more pixels that are relatively close is retained as 1, and the values ​​of the other pixels in the two or more pixels that are relatively close are set to 0. The modified at least two third positioning maps are then fused to obtain the final positioning map. In this way, the accuracy of the final positioning map can be improved.

[0094] Figure 2 A schematic diagram showing the processing process of the crowd positioning method according to an embodiment of the present disclosure is shown. Figure 2 As shown, the processing process of the crowd positioning method may include:

[0095] Step S21, using a crowd positioning neural network to perform crowd positioning on a crowd image, and obtaining at least two first positioning maps, each of which has a different size;

[0096] Step S22, performing activation, threshold segmentation, and non-maximum suppression processing on each first positioning map to obtain at least two second positioning maps;

[0097] Step S23, resizing each second positioning map to obtain at least two third positioning maps, where the size of each third positioning map is the same as the size of the crowd image;

[0098] Step S24, determining any third positioning map among the at least two third positioning maps as a fourth positioning map, and determining any third positioning map among the at least two third positioning maps except the fourth positioning map as a fifth positioning map;

[0099] Step S25: for any first pixel point whose value is 1 in the fourth positioning map, determine a first distance between the first pixel point and each second pixel point whose value is 1 in the fifth positioning map;

[0100] Step S26, determining whether there is a second distance in the first distance that is smaller than a preset distance threshold;

[0101] If not, executing step S27, fusing at least two third positioning maps to obtain a final positioning map;

[0102] If so, executing step S28, setting the value of the second pixel corresponding to the second distance in the fifth positioning map to 0, to obtain a sixth positioning map; and executing step S29, fusing the fourth positioning map and the sixth positioning map to obtain a final positioning map.

[0103] The disclosed embodiments can perform crowd positioning on crowd images, obtaining at least two first positioning maps of different sizes, and determining a final positioning map based on the at least two first positioning maps of different sizes. Compared to existing crowd positioning methods based on single-sized positioning maps, the crowd positioning method of the disclosed embodiments can determine the final positioning map by fusing multiple-sized first positioning maps to achieve crowd positioning. This allows crowd positioning to accommodate people of multiple sizes in the same or different scenes, effectively improving the accuracy of crowd positioning.

[0104] In one possible implementation, after obtaining the final positioning map, the positions of people included in the crowd image can be determined based on the final positioning map, achieving crowd positioning and providing data for other crowd analysis tasks such as crowd counting and group behavior analysis. For example, if a person is represented by the center point of the head, the number of pixels with a value of 1 in the final positioning map can be counted to determine the number of people included in the crowd image, thereby achieving crowd counting. Alternatively, the distribution of pixels with a value of 1 in the final positioning map can be counted to determine the behavioral trajectories of people included in the crowd image, thereby achieving crowd behavior analysis.

[0105] It is understood that the above-mentioned various method embodiments mentioned in this disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to space limitations, this disclosure will not go into details. It is understood by those skilled in the art that in the above-mentioned methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0106] In addition, the present disclosure also provides a crowd positioning device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any crowd positioning method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method section and will not be repeated here.

[0107] Figure 3 A block diagram of a crowd positioning device according to an embodiment of the present disclosure is shown as follows: Figure 3 As shown, the device includes:

[0108] A first positioning module 31 is configured to perform crowd positioning on a crowd image to obtain at least two first positioning maps, wherein the first positioning maps are used to indicate preliminary positioning positions of people included in the crowd image, and the sizes of the first positioning maps are different;

[0109] The second positioning module 32 is configured to determine a final positioning map based on the at least two first positioning maps, where the final positioning map is used to indicate the final positioning positions of the human bodies included in the crowd image.

[0110] In one possible implementation, the first positioning module 31 includes: a feature extraction submodule, used to extract features from the crowd image to obtain at least two first feature maps, each of which has a different size; and a first positioning submodule, used to locate the crowd based on each of the first feature maps to obtain a first positioning map corresponding to each of the first feature maps.

[0111] In one possible implementation, the second positioning module 32 includes: an activation submodule, for activating any first positioning map, to obtain a second feature map; a segmentation submodule, for performing threshold segmentation on the second feature map according to a preset feature threshold, to obtain a third feature map, wherein the third feature map is a binary map consisting of 0 and 1; a non-maximum suppression submodule, for performing non-maximum suppression on the third feature map according to a preset neighborhood size, to obtain a second positioning map; and a second positioning submodule, for determining a final positioning map based on at least two second positioning maps.

[0112] In a possible implementation, the second positioning submodule is configured to: perform size transformation on each second positioning map to obtain at least two third positioning maps, where the sizes of the third positioning maps are the same as the size of the crowd image; and determine a final positioning map based on the at least two third positioning maps.

[0113] In one possible implementation, determining the final positioning map based on the at least two third positioning maps includes: determining any third positioning map among the at least two third positioning maps as a fourth positioning map; for any first pixel point with a value of 1 in the fourth positioning map, determining a first distance between the first pixel point and each second pixel point with a value of 1 in a fifth positioning map, where the fifth positioning map is any third positioning map among the at least two third positioning maps except the fourth positioning map; determining whether a second distance less than a preset distance threshold exists in the first distance; and if the second distance does not exist, fusing the at least two third positioning maps to obtain the final positioning map.

[0114] In one possible implementation, determining the final positioning map based on the at least two third positioning maps further includes: when the second distance exists, setting the value of the second pixel corresponding to the second distance in the fifth positioning map to 0 to obtain a sixth positioning map; and fusing the fourth positioning map and the sixth positioning map to obtain a final positioning map.

[0115] In one possible implementation, the non-maximum suppression submodule is used to: for any third pixel point with a value of 1 in the third feature map, determine the neighborhood of the third pixel point in the third feature map according to a preset neighborhood size; determine whether there is a fourth pixel point with a value of 1 in the neighborhood of the third pixel point; if the fourth pixel point exists, determine the first value corresponding to the third pixel point and the second value corresponding to the fourth pixel point from the first positioning map respectively; determine the minimum value between the first value and the second value; determine the pixel point corresponding to the minimum value among the third pixel point and the fourth pixel point as the target pixel point; set the value of the target pixel point in the third feature map to 0 to obtain a second positioning map.

[0116] In one possible implementation, the crowd positioning device is implemented by a crowd positioning neural network, which is used to perform crowd positioning on the crowd image to obtain at least two first positioning maps. The training set of the crowd positioning neural network includes multiple first crowd sample images and at least two real positioning maps corresponding to each of the first crowd sample images.

[0117] The device also includes: a third positioning module, which performs crowd positioning on any first crowd sample image in the training set through the crowd positioning neural network to obtain at least two first predicted positioning maps; an activation module, which performs activation processing on the at least two first predicted positioning maps to obtain at least two second predicted positioning maps; a positioning loss determination module, which determines the positioning loss corresponding to the first crowd sample image based on the at least two second predicted positioning maps and at least two real positioning maps corresponding to the first crowd sample image; a network loss determination module, which determines the network loss based on the positioning losses corresponding to each first crowd sample image in the training set; and an optimization module, which optimizes the crowd positioning neural network based on the network loss.

[0118] In one possible implementation, the device further includes: a labeling determination module, which determines, for any first population sample image, a first labeling result of the first population sample image, wherein the first labeling result includes the coordinates of the human body in the first population sample image; a size transformation module, which is used to perform size transformation on the first population sample image according to at least two preset sizes, respectively, to obtain second population sample images corresponding to each of the preset sizes; a coordinate transformation module, which is used to perform coordinate transformation on the first labeling result according to the at least two preset sizes, respectively, to obtain second labeling results corresponding to each of the preset sizes, wherein the second labeling result includes the coordinates of the human body in the corresponding second population sample image; and a real positioning map determination module, which is used to determine the real positioning map corresponding to each of the preset sizes based on the second population sample image corresponding to each of the preset sizes and the second labeling result.

[0119] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0120] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0121] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0122] An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0123] The electronic device may be provided as a terminal, a server, or other forms of devices.

[0124] Figure 4 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, or other terminal device.

[0125] Reference Figure 4 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0126] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0127] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0128] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0129] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0130] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0131] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0132] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor assembly 814 can also detect changes in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0133] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as wireless network (Wi-Fi), second generation mobile communication technology (2G), third generation mobile communication technology (3G), fourth generation mobile communication technology (4G), long term evolution (LTE) of universal mobile communication technology, fifth generation mobile communication technology (5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0134] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0135] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions. The computer program instructions can be executed by the processor 820 of the electronic device 800 to perform the above method.

[0136] Figure 5 FIG1 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 can be provided as a server or a terminal device. Figure 5 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0137] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (Mac OSX TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open-source Unix-like operating system (FreeBSD TM ) or similar.

[0138] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0139] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0140] Computer-readable storage media can be a tangible device that can hold and store the instructions used by the instruction execution device. Computer-readable storage media can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. Computer-readable storage media used herein is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (for example, light pulses by fiber optic cables), or electrical signals transmitted by wires.

[0141] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0142] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0143] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0144] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0145] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0146] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0147] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0148] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0149] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0150] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0151] While various embodiments of the present disclosure have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A crowd positioning method, characterized in that: include: Performing crowd positioning on the crowd image to obtain at least two first positioning maps, wherein the first positioning maps are used to indicate preliminary positioning positions of people included in the crowd image, and the sizes of the first positioning maps are different; determining a final positioning map according to the at least two first positioning maps, wherein the final positioning map is used to indicate a final positioning position of a human body included in the crowd image; The determining of a final positioning map according to the at least two first positioning maps includes: For any first positioning map, performing activation processing on the first positioning map to obtain a second feature map; Performing threshold segmentation on the second feature map according to a preset feature threshold to obtain a third feature map, wherein the third feature map is a binary map consisting of 0 and 1; According to a preset neighborhood size, non-maximum suppression processing is performed on the third feature map to obtain a second positioning map; determining a final positioning map based on the at least two second positioning maps; Determining a final positioning map based on at least two second positioning maps includes: Performing size transformation on each second positioning map to obtain at least two third positioning maps, wherein the sizes of the third positioning maps are the same as the size of the crowd image; A final positioning map is determined according to the at least two third positioning maps.

2. The method according to claim 1, characterized in that The performing crowd positioning on the crowd image to obtain at least two first positioning maps includes: Performing feature extraction on the crowd image to obtain at least two first feature maps, each of the first feature maps having a different size; Crowd positioning is performed according to each of the first feature maps to obtain a first positioning map corresponding to each of the first feature maps.

3. The method according to claim 1, characterized in that Determining the final positioning map based on the at least two third positioning maps includes: Determining any one of the at least two third positioning maps as a fourth positioning map; For any first pixel point having a value of 1 in the fourth positioning map, determining a first distance between the first pixel point and each second pixel point having a value of 1 in a fifth positioning map, where the fifth positioning map is any third positioning map among the at least two third positioning maps except the fourth positioning map; Determining whether there is a second distance within the first distance that is less than a preset distance threshold; In the absence of the second distance, the at least two third positioning maps are fused to obtain a final positioning map.

4. The method according to claim 3, characterized in that The determining of the final positioning map based on the at least two third positioning maps further includes: If the second distance exists, setting the value of the second pixel corresponding to the second distance in the fifth positioning map to 0 to obtain a sixth positioning map; The fourth positioning map and the sixth positioning map are fused to obtain a final positioning map.

5. The method according to claim 1, characterized in that The step of performing non-maximum suppression processing on the third feature map according to a preset neighborhood size to obtain a second positioning map includes: For any third pixel point whose value is 1 in the third feature map, determining a neighborhood of the third pixel point in the third feature map according to a preset neighborhood size; Determine whether there is a fourth pixel point with a value of 1 in the neighborhood of the third pixel point; In a case where the fourth pixel point exists, determining a first value corresponding to the third pixel point and a second value corresponding to the fourth pixel point from the first positioning map respectively; determining a minimum value between the first value and the second value; Determine the pixel point corresponding to the minimum value among the third pixel point and the fourth pixel point as the target pixel point; The value of the target pixel in the third feature map is set to 0 to obtain a second positioning map.

6. The method according to any one of claims 1 to 5, characterized in that The crowd positioning method is implemented by a crowd positioning neural network, which is used to perform crowd positioning on the crowd image to obtain at least two first positioning maps. The training set of the crowd positioning neural network includes multiple first crowd sample images and at least two real positioning maps corresponding to each of the first crowd sample images. The method further comprises: For any first crowd sample image in the training set, perform crowd positioning on the first crowd sample image using the crowd positioning neural network to obtain at least two first predicted positioning maps; activating the at least two first predicted positioning maps to obtain at least two second predicted positioning maps; determining a positioning loss corresponding to the first crowd sample image based on the at least two second predicted positioning maps and the at least two real positioning maps corresponding to the first crowd sample image; determining a network loss according to a positioning loss corresponding to each first population sample image in the training set; The crowd localization neural network is optimized based on the network loss.

7. The method according to claim 6, characterized in that The method further comprises: For any first population sample image, determining a first annotation result of the first population sample image, where the first annotation result includes coordinates of a human body in the first population sample image; Resizing the first population sample image according to at least two preset sizes to obtain second population sample images corresponding to the preset sizes; Performing coordinate transformation on the first annotation results according to the at least two preset sizes to obtain second annotation results corresponding to the preset sizes, the second annotation results including coordinates of the human body in the corresponding second population sample image; According to the second crowd sample images corresponding to the respective preset sizes and the second annotation results, a true positioning map corresponding to the respective preset sizes is determined.

8. A crowd positioning device, characterized in that: include: a first positioning module, configured to perform crowd positioning on a crowd image to obtain at least two first positioning maps, wherein the first positioning maps are used to indicate preliminary positioning positions of people included in the crowd image, and the sizes of the first positioning maps are different; a second positioning module, configured to determine a final positioning map based on the at least two first positioning maps, wherein the final positioning map is used to indicate a final positioning position of a human body included in the crowd image; The second positioning module includes: an activation submodule for performing activation processing on any first positioning map to obtain a second feature map; a segmentation submodule for performing threshold segmentation on the second feature map according to a preset feature threshold to obtain a third feature map, wherein the third feature map is a binary map composed of 0 and 1; a non-maximum suppression submodule for performing non-maximum suppression processing on the third feature map according to a preset neighborhood size to obtain a second positioning map; and a second positioning submodule for determining a final positioning map based on at least two second positioning maps. The second positioning submodule is configured to: perform size conversion on each second positioning map to obtain at least two third positioning maps, wherein the sizes of the third positioning maps are the same as the size of the crowd image; and determine a final positioning map based on the at least two third positioning maps.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images