Crowd density detection method and apparatus, electronic device, and medium

Abstract

The present application provides a crowd density detection method and device, electronic equipment and medium, wherein the method comprises: acquiring a target picture collected by a camera device; identifying the target picture to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the head center position from the target picture; converting the position of the first pixel unit in the picture coordinate system to obtain the first position in the ground coordinate system according to the target conversion relationship between the picture coordinate system and the ground coordinate system of the target picture; converting the position of the second pixel unit in the picture coordinate system to obtain the second position relative to the ground coordinate system according to the target conversion relationship, so as to determine the standing position of the person on the ground based on the second position; and determining the crowd density according to the first position and the standing position. Thus, the crowd density is effectively determined based on the standing position of the person and the first position of the first pixel unit belonging to the ground in the ground coordinate system in the target picture.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and medium for detecting crowd density. Background Technology

[0002] In some large-scale events with dense crowds, the risk of overcrowding and stampedes is very high when people are too close together.

[0003] To prevent stampedes during large-scale events, real-time crowd density monitoring can be implemented to allow for timely evacuation if the density exceeds a set threshold. Therefore, real-time crowd density monitoring is essential for large-scale events. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose a crowd density detection method to realize the target transformation relationship between the image coordinate system and the ground coordinate system. This method can effectively determine the first pixel unit belonging to the ground and the second pixel unit belonging to the center position of the human head in the target image, respectively, at the first position and the standing position on the ground. Thus, the crowd density can be effectively determined based on the standing position and the first position of the person.

[0006] The second objective of this invention is to provide a population density detection device.

[0007] The third objective of this invention is to provide an electronic device.

[0008] The fourth objective of this invention is to provide a non-transient computer-readable storage medium.

[0009] The fifth objective of this invention is to provide a computer program product.

[0010] To achieve the above objectives, a first aspect of the present invention provides a method for detecting crowd density, comprising:

[0011] Acquire the target image captured by the camera device;

[0012] The target image is identified to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center of a human head.

[0013] Based on the target transformation relationship between the screen coordinate system and the ground coordinate system of the target image, the position of the first pixel unit in the screen coordinate system is transformed to obtain the first position in the ground coordinate system;

[0014] According to the target transformation relationship, the position of the second pixel unit in the image coordinate system is transformed to obtain a second position relative to the ground coordinate system, so as to determine the standing position of the person on the ground based on the second position;

[0015] The crowd density is determined based on the first position and the standing position.

[0016] Optionally, as a first possible implementation of the first aspect, the step of identifying the target image to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center position of a human head includes:

[0017] An instance segmentation algorithm is used to identify the target image to determine the first pixel unit belonging to the ground from the target image;

[0018] Based on the crowd scene category to which the target image belongs, the corresponding crowd detection algorithm is used to determine the second pixel unit belonging to the center position of the human head in the target image.

[0019] Optionally, as a second possible implementation of the first aspect, the crowd scene categories include sparse crowd scenes and dense crowd scenes;

[0020] The crowd detection algorithm corresponding to the sparse crowd scene is used to extract features of at least two different scales from the target image, and to make predictions based on the features of at least two different scales to obtain the second pixel unit belonging to the center position of the human head.

[0021] The crowd detection algorithm for the dense crowd scene is used to extract head features from the target image based on a convolutional neural network, generate a crowd density map based on the head features, and determine the second pixel unit belonging to the center position of the head through the crowd density map.

[0022] Alternatively, as a third possible implementation of the first aspect, the method further includes:

[0023] Obtain the target transformation relationship based on the intrinsic and extrinsic parameters of the camera device;

[0024] The internal and external parameters include installation point information and focal length information.

[0025] Optionally, as a fourth possible implementation of the first aspect, the step of transforming the position of the second pixel unit in the image coordinate system according to the target transformation relationship to obtain a second position relative to the ground coordinate system, so as to determine the standing position of the person on the ground based on the second position, includes:

[0026] Based on the target transformation relationship, a first transformation relationship is determined between the screen coordinate system P and the coordinate system H established based on the center position of the human head, wherein the coordinate system H is a two-dimensional plane that passes through the center position of the human head and is parallel to the plane R on which the ground is located;

[0027] Based on the first transformation relationship, the position of the second pixel unit in the image coordinate system P is transformed to obtain the second position of the second pixel unit in the coordinate system H;

[0028] Based on the offset between the center position of the human head and the standing position, a second transformation relationship between the coordinate system H and the ground coordinate system is determined;

[0029] Based on the second transformation relationship, the second position is transformed to determine the standing position.

[0030] Optionally, as a fifth possible implementation of the first aspect, the installation point information includes the installation height of the camera device and the angle between the optical axis of the camera device and the vertical line of the ground.

[0031] The crowd density detection method of this invention acquires a target image captured by a camera device; identifies the target image to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center position of a person's head; transforms the position of the first pixel unit in the image coordinate system to obtain a first position in the ground coordinate system based on the target transformation relationship between the image coordinate system and the ground coordinate system; transforms the position of the second pixel unit in the image coordinate system to obtain a second position relative to the ground coordinate system based on the target transformation relationship, and determines the standing position of a person on the ground based on the second position; and determines the crowd density based on the first position and the standing position. Therefore, based on the target transformation relationship between the image coordinate system and the ground coordinate system, the first position of the first pixel unit belonging to the ground in the target image in the ground coordinate system can be effectively determined, and the standing position of a person on the ground can be effectively determined based on the second pixel unit belonging to the center position of a person's head in the target image. Thus, the crowd density can be effectively determined based on the standing position and the first position.

[0032] To achieve the above objectives, a second aspect of the present invention provides a crowd density detection device, comprising:

[0033] The first acquisition module is used to acquire the target image captured by the camera device;

[0034] The recognition module is used to recognize the target image to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center position of a human head.

[0035] The conversion module is used to convert the position of the first pixel unit in the screen coordinate system to obtain the first position in the ground coordinate system according to the target conversion relationship between the screen coordinate system and the ground coordinate system of the target screen.

[0036] The first determining module is used to transform the position of the second pixel unit in the screen coordinate system according to the target transformation relationship to obtain a second position relative to the ground coordinate system, so as to determine the standing position of a person on the ground based on the second position;

[0037] The second determining module is used to determine the crowd density based on the first position and the standing position.

[0038] Alternatively, as a first possible implementation of the second aspect, the identification module is configured to:

[0039] An instance segmentation algorithm is used to identify the target image to determine the first pixel unit belonging to the ground from the target image;

[0040] Based on the crowd scene category to which the target image belongs, the corresponding crowd detection algorithm is used to determine the second pixel unit belonging to the center position of the human head in the target image.

[0041] Optionally, as a second possible implementation of the second aspect, the crowd scene categories include sparse crowd scenes and dense crowd scenes;

[0042] The crowd detection algorithm corresponding to the sparse crowd scene is used to extract features of at least two different scales from the target image, and to make predictions based on the features of at least two different scales to obtain the second pixel unit belonging to the center position of the human head.

[0043] The crowd detection algorithm for the dense crowd scene is used to extract head features from the target image based on a convolutional neural network, generate a crowd density map based on the head features, and determine the second pixel unit belonging to the center position of the head through the crowd density map.

[0044] Alternatively, as a third possible implementation of the second aspect, the apparatus further includes:

[0045] The second acquisition module is used to acquire the target transformation relationship based on the intrinsic and extrinsic parameters of the camera device;

[0046] The internal and external parameters include installation point information and focal length information.

[0047] Alternatively, as a fourth possible implementation of the second aspect, the first determining module is configured to:

[0048] Based on the target transformation relationship, a first transformation relationship is determined between the screen coordinate system P and the coordinate system H established based on the center position of the human head, wherein the coordinate system H is a two-dimensional plane that passes through the center position of the human head and is parallel to the plane R on which the ground is located;

[0049] Based on the first transformation relationship, the position of the second pixel unit in the image coordinate system P is transformed to obtain the second position of the second pixel unit in the coordinate system H;

[0050] Based on the offset between the center position of the human head and the standing position, a second transformation relationship between the coordinate system H and the ground coordinate system is determined;

[0051] Based on the second transformation relationship, the second position is transformed to determine the standing position.

[0052] Optionally, as a fifth possible implementation of the second aspect, the installation point information includes the installation height of the camera device and the angle between the optical axis of the camera device and the vertical line of the ground.

[0053] The crowd density detection device of this invention acquires a target image captured by a camera; identifies the target image to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center of a person's head; transforms the position of the first pixel unit in the image coordinate system to obtain a first position in the ground coordinate system based on the target transformation relationship between the image coordinate system and the ground coordinate system; transforms the position of the second pixel unit in the image coordinate system to obtain a second position relative to the ground coordinate system based on the target transformation relationship, and determines the standing position of a person on the ground based on the second position; and determines the crowd density based on the first position and the standing position. Therefore, based on the target transformation relationship between the image coordinate system and the ground coordinate system, the first position of the first pixel unit belonging to the ground in the target image in the ground coordinate system can be effectively determined, and the standing position of a person on the ground can be effectively determined based on the second pixel unit belonging to the center of a person's head in the target image, thus effectively determining the crowd density based on the standing position and the first position.

[0054] To achieve the above objectives, a third aspect of the present invention provides an electronic device comprising:

[0055] At least one processor; and

[0056] A memory communicatively connected to the at least one processor; wherein,

[0057] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the crowd density detection method described in the first aspect.

[0058] To achieve the above objectives, a fourth aspect of the present invention provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the crowd density detection method described in the first aspect.

[0059] To achieve the above objectives, a fifth aspect of the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the crowd density detection method as described in the first aspect.

[0060] The technical solutions provided in this application have the following beneficial effects:

[0061] The system acquires a target image captured by a camera; it identifies the target image to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center of a person's head; based on the target transformation relationship between the image coordinate system and the ground coordinate system, it transforms the position of the first pixel unit in the image coordinate system to obtain a first position in the ground coordinate system; based on the target transformation relationship, it transforms the position of the second pixel unit in the image coordinate system to obtain a second position relative to the ground coordinate system, and based on the second position, it determines the person's standing position on the ground; based on the first position and the standing position, it determines the crowd density. Therefore, based on the target transformation relationship between the image coordinate system and the ground coordinate system, the first position of the first pixel unit belonging to the ground in the target image can be effectively determined in the ground coordinate system, and based on the second pixel unit belonging to the center of a person's head in the target image, the person's standing position on the ground can be effectively determined, and thus, based on the person's standing position and the first position, the crowd density can be effectively determined.

[0062] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0063] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0064] Figure 1The video image with an invalid background provided by this invention;

[0065] Figure 2 This is a flowchart illustrating a data processing method provided in an embodiment of the present invention;

[0066] Figure 3 The target image provided by this invention;

[0067] Figure 4 The irregular region formed by the enclosure of the first position provided by the present invention;

[0068] Figure 5 This is a flowchart illustrating another data processing method provided in an embodiment of the present invention;

[0069] Figure 6 A diagram showing the relationship between the ground coordinate system and the screen coordinate system provided by this invention;

[0070] Figure 7 This is a diagram showing the relationship between the coordinate system H and the screen coordinate system provided by the present invention;

[0071] Figure 8 A human head illustration of the standard sphere provided by this invention;

[0072] Figure 9 This is a schematic flowchart of the crowd density detection method provided by the present invention;

[0073] Figure 10 This is a schematic diagram of the structure of a crowd density detection device provided in an embodiment of the present invention;

[0074] Figure 11 This is a schematic diagram of another crowd density detection device provided in an embodiment of the present invention;

[0075] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0076] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0077] The crowd density detection method and apparatus of the present invention are described below with reference to the accompanying drawings.

[0078] In related technologies, there are two methods for calculating crowd density:

[0079] 1. Method for calculating crowd density via mobile phone signaling

[0080] This method is easy to implement, but it can only characterize the average crowd density within a scene, with a typical accuracy of 100m*100m. When applied to calculating crowd density in scenic areas and determining crowd distribution, this method lacks more precise sensing methods.

[0081] 2. Using computer vision to calculate the precise location of crowds

[0082] In this method, the front-end device needs to use a dual-light camera to determine the depth information of a person, and then determine the person's standing position. This method can accurately calculate crowd density, but it has special requirements for front-end monitoring.

[0083] Crowd density monitoring based on camera equipment has two main characteristics:

[0084] 1. Within the monitoring field of view of the camera equipment, the video images captured by the camera equipment contain a large amount of invalid background (such as buildings, green belts, etc.), for example, Figure 1 The image shows a video frame with an invalid background. In the image, area 11 is a building, area 12 is a green belt, and areas 13 and 14 are areas where people are active.

[0085] 2. The camera equipment is mostly shot from non-direct angles. Due to the height of the person, the person in the video image will be obscured, and in many cases it is impossible to detect the position of the person's feet. However, the position of the head in the video image is the least likely to be obscured.

[0086] In response to at least one of the aforementioned problems, this disclosure proposes a method, apparatus, electronic device, and medium for detecting population density.

[0087] Figure 2 This is a schematic flowchart of a crowd density detection method provided in an embodiment of the present invention.

[0088] This invention is illustrated by exemplifying the method for detecting crowd density in a crowd density detection device. This crowd density detection device can be applied to any electronic device so that the electronic device can perform the crowd density detection function.

[0089] Among them, electronic devices can be any device with computing capabilities, such as PCs (Personal Computers), mobile terminals, servers, etc. Mobile terminals can be hardware devices with various operating systems, touch screens and / or displays, such as in-vehicle devices, mobile phones, tablets, personal digital assistants, wearable devices, etc.

[0090] like Figure 2 As shown, the crowd density detection method includes the following steps:

[0091] Step 201: Acquire the target image captured by the camera device.

[0092] In this embodiment, the target image can be a video image containing a person.

[0093] In this embodiment, the target image can be captured by a camera device to obtain the target image captured by the camera device.

[0094] Step 202: Recognize the target image to determine the first pixel unit belonging to the ground and the second pixel unit belonging to the center of the head.

[0095] In this embodiment, the first pixel unit can be a pixel belonging to the ground, and the first pixel unit can be multiple pixels.

[0096] In this embodiment, the second pixel unit can be a pixel belonging to the center position of the human head, and the second pixel unit can be, but is not limited to, one; this disclosure does not impose any restrictions on this.

[0097] In this embodiment, the target image can be identified to determine the first pixel unit belonging to the ground from the target image.

[0098] As one possible implementation, an instance segmentation algorithm can be used to identify the target image in order to determine the first pixel unit belonging to the ground from the target image.

[0099] In this embodiment, the instance segmentation algorithm may include, for example, Mask R-CNN (Mask Region-Cable News Network), SOLO (Segmenting Objects by Locations), BlendMask, etc., and the present invention does not limit it.

[0100] As an example, the BlendMask algorithm is used to identify the target image, and the first pixel unit belonging to the ground can be determined from the target image based on the identification results.

[0101] In this embodiment, the target image can be identified to determine the second pixel unit belonging to the center position of the human head from the target image.

[0102] As one possible implementation, based on the crowd scene category to which the target image belongs, a corresponding crowd detection algorithm is used to determine the second pixel unit belonging to the center position of the human head in the target image.

[0103] In this embodiment, the crowd scene category may include, for example, sparse crowd scenes and dense crowd scenes.

[0104] In this embodiment, crowd detection algorithms for sparse crowd scenes may include Faster R-CNN (Faster Region-Cable News Network), SSD (Single Shot MultiBox Detector), YOLO (You Look Only Once), etc., and this invention does not limit them.

[0105] In this embodiment, crowd detection algorithms for densely populated scenes may include MCNN (Multi-column Convolutional Neural Network), FIDTM (Focal Inverse Distance Transform Maps), P2PNet (Point-to-Point Network), etc., and this invention does not limit them.

[0106] In this embodiment, a corresponding crowd detection algorithm can be used to determine the second pixel unit belonging to the center position of a person's head in the target image, based on the crowd scene category to which the target image belongs.

[0107] In one possible implementation of this embodiment, the crowd detection algorithm for sparse crowd scenes can be used to extract features at least two different scales from the target image, and to make predictions based on the features at least two different scales to obtain the second pixel unit belonging to the center position of the human head.

[0108] As an example, the crowd detection algorithm for sparsely populated scenes is the YOLO V5 algorithm. In the YOLO V5 algorithm, CSPDarknet53 (Cross Stage Partial Networks Darknet53) serves as the backbone network, while PAN (Path Aggregation Network) and FPN (Feature Pyramid Network) act as the neck. After extracting features from the target image, FPN and PAN can extract features at three different scales. The YOLO Head can predict these features at three different scales to obtain three head detection boxes at different scales. Based on these three head detection boxes, the center position of the head can be determined. For example, the center position of the head detection box at any scale can be determined separately, and the three center positions can be weighted and averaged to obtain the target position. This target position is then used as the head center position. Furthermore, based on this head center position, the second pixel unit in the target image belonging to the head center position can be determined.

[0109] In one possible implementation of this embodiment, the crowd detection algorithm for densely populated scenes can be used to extract head features from the target image based on a convolutional neural network, generate a crowd density map based on the head features, and determine the second pixel unit belonging to the center position of the head through the crowd density map.

[0110] In this embodiment, the crowd density map can be a location map showing the location of people.

[0111] As an example, the crowd detection algorithm for dense crowd scenes is the FIDTM algorithm, which can be used to extract head features from the target image based on convolutional neural networks and generate a crowd density map based on the head features. Based on the crowd density map, the KNN (K-Nearest Neighbor) algorithm strategy can be used to obtain the bounding box of the head, so that the second pixel unit belonging to the center position of the head can be determined according to the center position of the bounding box of the head.

[0112] Step 203: Based on the target transformation relationship between the screen coordinate system and the ground coordinate system of the target image, transform the position of the first pixel unit in the screen coordinate system to obtain the first position in the ground coordinate system.

[0113] In this embodiment, there can be a corresponding target transformation relationship between the screen coordinate system and the ground coordinate system.

[0114] In this embodiment, the position of the first pixel unit in the screen coordinate system can be transformed to obtain the first position in the ground coordinate system based on the target transformation relationship between the screen coordinate system and the ground coordinate system of the target screen.

[0115] Specifically, for any first pixel unit, the position of any first pixel unit in the screen coordinate system can be transformed according to the target transformation relationship between the screen coordinate system and the ground coordinate system to obtain the first position in the ground coordinate system.

[0116] Step 204: Based on the target transformation relationship, the position of the second pixel unit in the screen coordinate system is transformed to obtain the second position relative to the ground coordinate system, so as to determine the standing position of the person on the ground based on the second position.

[0117] In this embodiment, the relative ground coordinate system can be a coordinate system of a two-dimensional plane parallel to the plane on which the ground is located.

[0118] In this embodiment, the position of the second pixel unit in the image coordinate system can be transformed according to the target transformation relationship to obtain the second position relative to the ground coordinate system, thereby determining the standing position of a person on the ground based on the second position.

[0119] Step 205: Determine the crowd density based on the first position and the standing position.

[0120] In this embodiment, the crowd density can be determined based on the first position and the standing position.

[0121] As one possible approach, population density can be determined through the following steps:

[0122] 1. Use statistical methods to obtain the number of standing positions to determine the number of people.

[0123] For example, assuming that the number of standing positions is n using statistical methods, then the number of people is n.

[0124] 2. Obtain the target ground area enclosing the first position.

[0125] As one possible implementation, for a first position, multiple vertices that enclose the irregular region formed by the first position can be determined from the first position; based on the coordinates corresponding to each vertex, the first ground area of ​​the irregular region can be obtained; and the first ground area can be used as the target ground area enclosing the first position.

[0126] As an example, suppose the target image is as follows: Figure 3As shown, regions 31 and 33 in the target image are invalid backgrounds, and region 32 is the area where people are active. The first pixel unit belonging to the ground in target region 32 can be obtained. Based on the target transformation relationship between the screen coordinate system and the ground coordinate system, the position of the first pixel unit in the screen coordinate system can be transformed to obtain the first position in the ground coordinate system. For the aforementioned first position, multiple vertices R1, R2, ..., Rn enclosing the irregular region formed by the first position can be determined, where n is the number of vertices. For example, for a region like... Figure 4 The irregular region enclosed by the first position is shown, and the multiple vertices of the irregular region include R1, R2, ..., R10.

[0127] Assume the coordinates of each vertex are (x1, y1, 0), (x2, y2, 0), ..., (x... n ,y n If the coordinates of each vertex are 0, then the first ground area S of the irregular region is determined as follows:

[0128]

[0129] Where, x n+1 =x1,y n+1 =y1, where n is the number of vertices.

[0130] Therefore, the first ground area of ​​the irregular region can be used as the target ground area for enclosing the first position.

[0131] 3. Determine the crowd density based on the ratio of the number of people to the target ground area surrounding the first location.

[0132] For example, assuming the number of people is n and the area of ​​the ground surrounding the first location is S, the crowd density can be n / S.

[0133] As another possible implementation, a clustering algorithm can be used to divide multiple first positions into at least one subgroup, where each subgroup may include multiple first positions. For any subgroup, the first position belonging to that subgroup is designated as the third position. Based on the third position, the vertices of the irregular region enclosing the third positions in that subgroup are obtained from the third positions. Based on the coordinates of each vertex of the irregular region, the second ground area of ​​the corresponding irregular region is determined. Based on the sum of the second ground areas, the target ground area is determined. The number of standing positions is obtained using statistical methods to obtain the number of people, and the crowd density can be determined based on the ratio of the number of people to the target ground area.

[0134] For example, suppose the target image is like Figure 1As shown, regions 11 and 12 in the figure are invalid backgrounds, while regions 13 and 14 are areas of human activity. A clustering algorithm can be used to divide the multiple first positions into two subgroups: subgroup 1 and subgroup 2. Subgroup 1 includes multiple first positions corresponding to region 13, and subgroup 2 includes multiple first positions corresponding to region 14. For any subgroup, the first position belonging to that subgroup is taken as the third position. Based on the third position, the vertices of the irregular region enclosing the third position are obtained. Then, based on the vertices of the irregular region... The coordinates are used to determine the second ground area of ​​the irregular region; thus, the second ground area S1 of the irregular region formed by the third position in enclosing subgroup 1 and the second ground area S2 of the irregular region formed by the third position in enclosing subgroup 2 can be determined; based on the sum of the second ground areas (S1+S2), the target ground area S is determined, that is, the target ground surface is (S1+S2); and the number m of standing positions is obtained by statistical methods to obtain the number of people, that is, the number of people is m; finally, the crowd density can be obtained based on the ratio of the number of people to the ground area m / S.

[0135] The crowd density detection method in this embodiment acquires a target image captured by a camera device; identifies the target image to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center of a person's head; transforms the position of the first pixel unit in the image coordinate system to obtain a first position in the ground coordinate system based on the target transformation relationship between the image coordinate system and the ground coordinate system; transforms the position of the second pixel unit in the image coordinate system to obtain a second position relative to the ground coordinate system based on the target transformation relationship, and determines the standing position of a person on the ground based on the second position; and determines the crowd density based on the first position and the standing position. Therefore, based on the target transformation relationship between the image coordinate system and the ground coordinate system, the first position of the first pixel unit belonging to the ground in the target image in the ground coordinate system can be effectively determined, and the standing position of a person on the ground can be effectively determined based on the second pixel unit belonging to the center of a person's head in the target image. Thus, the crowd density can be effectively determined based on the standing position and the first position.

[0136] To clearly illustrate how the target transformation relationship between the image coordinate system and the ground coordinate system of the target image was obtained in the previous embodiment, this embodiment provides another method for detecting crowd density.

[0137] Figure 5 This is a schematic flowchart of another crowd density detection method provided in an embodiment of the present invention.

[0138] Step 501: Acquire the target image captured by the camera device.

[0139] Step 502: Recognize the target image to determine the first pixel unit belonging to the ground and the second pixel unit belonging to the center position of the human head.

[0140] The execution process of steps 501 to 502 can be referred to the execution process of steps 201-202 in the previous embodiment. The principle is the same, so it will not be repeated here.

[0141] Step 503: Obtain the target transformation relationship based on the intrinsic and extrinsic parameters of the camera device; wherein, the intrinsic and extrinsic parameters include installation point information and focal length information.

[0142] In some embodiments, the installation point information may include the installation height of the camera equipment and the angle between the optical axis of the camera equipment and the vertical line of the ground.

[0143] In this embodiment, the target transformation relationship is obtained based on the intrinsic and extrinsic parameters of the camera device, namely the installation point information and focal length information. For example, the target transformation relationship can be determined based on the calibrated installation point information and focal length information.

[0144] Step 504: Based on the target transformation relationship between the screen coordinate system and the ground coordinate system of the target image, the position of the first pixel unit in the screen coordinate system is transformed to obtain the first position in the ground coordinate system.

[0145] The execution process of step 504 can be referred to the execution process of step 203 in the previous embodiment. The principle is the same, so it will not be repeated here.

[0146] For example, Figure 6 The diagram shows the relationship between the ground coordinate system and the screen coordinate system. The ground coordinate system (the ground in the ground coordinate system is denoted as R in this disclosure) has its origin O at the intersection of the vertical line of the ground where the camera is located and the ground, the vertical line of the ground as the Z-axis, the projection of the optical axis of the camera on the ground as the horizontal axis X-axis, and the direction perpendicular to both the X-axis and the Z-axis as the vertical axis Y-axis.

[0147] The screen coordinate system (denoted as P in this disclosure) is as follows: Figure 6 As shown in the diagram, the horizontal direction is the x-axis, and the vertical direction is the y-axis;

[0148] The field-of-view section coordinate system (i.e., the ideal projected image perpendicular to the optical axis of the camera device) takes the projection of the origin of the image coordinate system onto the section as the origin and the projection of the vertical axis Y of the image coordinate system onto the section (M plane) as the vertical axis.

[0149] Assuming the camera's installation height is H', the angle between the camera's optical axis and the vertical line to the ground is α, the camera's vertical field of view is Camera.V, and the camera's horizontal field of view is Camera.H, and assuming the image coordinates of the first pixel unit a belonging to the ground in the target image are (Px, Py), firstly, calculate the coordinates (Mx, My) of the first pixel unit a in the field of view section coordinate system using the following formula:

[0150]

[0151]

[0152]

[0153] Where img.w is the width of the target image and img.h is the height of the target image.

[0154] Secondly, the coordinates (Rx, Ry, 0) of the first pixel unit a in the ground coordinate system can be calculated using the following formula:

[0155]

[0156]

[0157] Therefore, by using formulas (1)-(5), the target transformation relationship between the ground coordinate system and the screen coordinate system can be determined, and based on this target transformation relationship, the position of the first pixel unit in the screen coordinate system can be transformed to obtain the first position in the ground coordinate system.

[0158] Step 505: Based on the target transformation relationship, the position of the second pixel unit in the screen coordinate system is transformed to obtain the second position relative to the ground coordinate system, so as to determine the standing position of the person on the ground based on the second position.

[0159] The execution process of step 505 can be referred to the execution process of step 204 in the previous embodiment. The principle is the same, so it will not be repeated here.

[0160] In one possible implementation of this embodiment, the standing position of a person on the ground can be determined according to the following steps:

[0161] 1. Based on the target transformation relationship, determine the first transformation relationship between the screen coordinate system P and the coordinate system H established based on the center position of the human head.

[0162] The coordinate system H can be a two-dimensional plane that passes through the center of the head and is parallel to the plane R on the ground.

[0163] In this embodiment, the first transformation relationship between the screen coordinate system P and the coordinate system H established based on the center position of the human head can be determined according to the target transformation relationship.

[0164] 2. Based on the first transformation relationship, the position of the second pixel unit in the screen coordinate system P is transformed to obtain the second position of the second pixel unit in the coordinate system H.

[0165] In this embodiment, the position of the second pixel unit in the screen coordinate system P can be transformed based on the first transformation relationship to obtain the second position of the second pixel unit in the coordinate system H.

[0166] Using the example above as an illustration, Figure 7 This diagram illustrates the relationship between coordinate system H and the screen coordinate system. In coordinate system H, the origin O' is the intersection of the vertical line from the ground where the camera is located and the plane where coordinate system H is located. The projection of the optical axis of the camera onto the plane where coordinate system H is located is the horizontal axis X', and the direction perpendicular to the X' axis is the vertical axis Y'.

[0167] Assuming the distance between coordinate system H and the plane R containing the ground is h, and the distance between the camera and the ground is H', then the distance between the camera and coordinate system H is H'-h; the angle between the optical axis of the camera and the perpendicular line to the ground is α, the vertical field of view of the camera is Camera.V, and the horizontal field of view of the camera is Camera.V. Assuming the coordinates of the second pixel unit b of the target image in the image coordinate system are (P... b.x P b.y First, the coordinates (M) of the second pixel unit b in the field-of-view section coordinate system are calculated using the following formula. b.x M b.y ):

[0168]

[0169]

[0170]

[0171] Where img.w is the width of the target image and img.h is the height of the target image.

[0172] Secondly, the coordinates (R, R) of the second pixel unit b in coordinate system H can be calculated using the following formula. b.x R b.y ):

[0173]

[0174]

[0175] Therefore, the position of the second pixel unit in the image coordinate system P is transformed to obtain the second position of the second pixel unit in the coordinate system H.

[0176] 3. Based on the offset between the center position of the human head and the standing position, determine the second transformation relationship between coordinate system H and the ground coordinate system.

[0177] In this embodiment, the second transformation relationship between coordinate system H and ground coordinate system is determined based on the offset between the center position of the human head and the standing position.

[0178] For example, let's assume that a person's height is represented by the average height h. a This indicates that the human head is a standard sphere. Figure 8 This is a diagram illustrating a human head shaped like a standard sphere, with a head radius of R. a The offset between the center of the head and the standing position, that is, the distance h between coordinate system H and the plane containing the ground, is:

[0179] h = h a -R a (11)

[0180] Assume the coordinates of point A in coordinate system H are (A x A y If the coordinates of this point in the ground coordinate system are (Bx, By, Bz), then the second transformation relationship between coordinate system H and the ground coordinate system is:

[0181]

[0182] Therefore, the second transformation relationship between coordinate system H and the ground coordinate system can be determined.

[0183] 4. Based on the second transformation relationship, transform the second position to determine the standing position.

[0184] In this embodiment, the second position can be transformed based on the second transformation relationship to determine the standing position.

[0185] Using the example above, the coordinates (R) of the second pixel unit b, which belongs to the center of the human head, in the second position of coordinate system H are... b.x R b.y By transforming this second position, the person's standing position can be determined as (R). b.x R b.y ,0).

[0186] Step 506: Determine the crowd density based on the first position and the standing position.

[0187] The execution process of steps 502 to 506 can be referred to the execution process of steps 203-205 in the previous embodiment. The principle is the same, so it will not be repeated here.

[0188] The crowd density detection method in this embodiment determines a first transformation relationship between the screen coordinate system P and a coordinate system H established based on the center position of a person's head, according to a target transformation relationship. Coordinate system H is a two-dimensional plane that passes through the center position of the person's head and is parallel to the ground plane R. Based on the first transformation relationship, the position of a second pixel unit in the screen coordinate system P is transformed to obtain a second position of the second pixel unit in the coordinate system H. A second transformation relationship between coordinate system H and the ground coordinate system is determined based on the offset between the center position of the person's head and the standing position. Based on the second transformation relationship, the second position is transformed to determine the standing position. Therefore, based on the target transformation relationship, the standing position of a person on the ground can be effectively determined.

[0189] The crowd density detection method of the present invention will be explained below with reference to examples.

[0190] As an example, Figure 9 This is a schematic flowchart of the crowd density detection method provided by the present invention. The crowd density detection method of the present invention may include the following steps:

[0191] Step S1. Acquire the video image at time t0 (referred to as the target image in this disclosure);

[0192] Step S2. Reduce invalid background information (such as building background) by using an instance segmentation algorithm to obtain ground pixel information (referred to as the first pixel unit in this disclosure) that belongs to the ground in the video image.

[0193] The ground pixel information of the monitored scene is obtained by using an instance segmentation algorithm on the video image at time t0. Instance segmentation algorithms can include, for example, Mask R-CNN, SOLO, and BlendMask.

[0194] The purpose of instance segmentation is to detect targets in an input image and assign a category label to each pixel of the target. Therefore, in this disclosure, when segmenting video images using an instance segmentation algorithm, only the ground scene category needs to be considered to obtain ground pixel information.

[0195] Step S3. Detect the position of human heads in the video image using a crowd detection algorithm, and obtain the pixel information belonging to the center position of the human head (referred to as the second pixel unit in this disclosure).

[0196] Crowd detection algorithms can be categorized into those applicable to sparse crowd scenarios and those applicable to dense crowd scenarios. Dense crowd scenarios are characterized by a large number of people distributed in a relatively dense manner, while sparse crowd scenarios are characterized by a smaller number of people distributed sparsely.

[0197] Crowd detection algorithms for sparsely populated scenes, such as the YOLOv5 algorithm, can extract features at least two different scales from the target image to make predictions based on these features, thereby obtaining the second pixel unit belonging to the center position of the human head.

[0198] Crowd detection algorithms for densely populated scenes, such as the FIDTM algorithm, can be used to extract head features from the target image based on convolutional neural networks, generate a crowd density map based on the head features, and determine the second pixel unit belonging to the center position of the head through the crowd density map.

[0199] Step S4. Given the installation point information and focal length information of the camera equipment, the coordinate transformation relationship between the screen coordinate system where the video image is located and the ground coordinate system of the real ground can be calculated (referred to as the target transformation relationship in this disclosure); based on the coordinate transformation relationship, the ground pixel information is transformed to obtain the ground coordinates in the ground coordinate system of the real ground (referred to as the first position in this disclosure), and the pixel information of the center position of the head is transformed to obtain the second position relative to the ground coordinate system of the real ground, so as to determine the standing position of the person on the ground based on the second position.

[0200] Specifically, step S4 can be described in detail through the following steps:

[0201] S41. By using the installation point information and focal length information of the camera equipment, the coordinate transformation relationship between the screen coordinate system where the video image is located and the ground coordinate system of the real ground can be calculated, and the ground pixel information belonging to the ground in the video image can be transformed to obtain the ground coordinates in the ground coordinate system of the real ground.

[0202] Figure 6 The diagram shows the relationship between the ground coordinate system and the screen coordinate system. The ground coordinate system (the ground on which the ground coordinate system is located, denoted as R in this disclosure) has its origin O at the intersection of the vertical line of the ground where the camera is located and the ground, the vertical line of the ground as the Z-axis, the projection of the optical axis of the camera on the ground as the horizontal axis X-axis, and the direction perpendicular to both the X-axis and the Z-axis as the vertical axis Y-axis.

[0203] The screen coordinate system (denoted as P in this disclosure) is as follows: Figure 6 As shown in the diagram, the horizontal direction is the x-axis, and the vertical direction is the y-axis;

[0204] The field-of-view section coordinate system (i.e., the ideal projected image perpendicular to the optical axis of the camera device) takes the projection of the origin of the image coordinate system onto the section as the origin and the projection of the vertical axis Y of the image coordinate system onto the section (M plane) as the vertical axis.

[0205] Assuming the installation height of the camera is H', the angle between the optical axis of the camera and the vertical line of the ground is α, the vertical field of view of the camera is Camera.V, and the horizontal field of view of the camera is Camera.H, and assuming the screen coordinates of the ground pixel information belonging to the ground in the video image are (Px, Py), firstly, the ground coordinates (Rx, Ry, 0) of the ground pixel information in the ground coordinate system are calculated by formulas (1)-(5).

[0206] S42. A simplified plane for the center position of the human head (denoted as coordinate system H in this disclosure) is proposed. Based on the camera installation angle and focal length information, that is, based on the coordinate transformation relationship between the screen coordinate system where the video image is located and the ground coordinate system of the real ground, a first transformation relationship between the simplified plane for the center position of the human head and the screen coordinate system of the video image can be obtained. Based on the first transformation relationship, the position of the pixel information of the center position of the human head in the screen coordinate system is transformed to obtain the second position of the pixel information of the center position of the human head in the simplified plane for the center position of the human head. According to the offset between the center position of the human head and the standing position, a second transformation relationship between the simplified plane for the center position of the human head and the plane where the ground is located is determined. Based on the second transformation relationship, the second position is transformed to determine the standing position.

[0207] The inventors discovered that when no human feet are detected in the video footage, it is difficult to determine a person's standing position on the ground using the video footage information. Therefore, the inventors proposed a method to determine a person's standing position on the ground by using a simplified plane (denoted as coordinate system H in this disclosure) based on the center position of the human head.

[0208] Specifically, based on the target transformation relationship between the screen coordinate system and the ground coordinate system, a first transformation relationship is determined between the screen coordinate system P and the coordinate system H established based on the center position of the human head. The coordinate system H is a two-dimensional plane that passes through the center position of the human head and is parallel to the plane R on which the ground is located. Based on the first transformation relationship, the position of the second pixel unit in the screen coordinate system P is transformed to obtain the second position of the second pixel unit in the coordinate system H. Based on the offset between the center position of the human head and the standing position, a second transformation relationship between the coordinate system H and the ground coordinate system is determined. Based on the second transformation relationship, the second position is transformed to determine the standing position.

[0209] Step S5. Determine the crowd density on the ground at time t0 based on the person's standing position and ground coordinates.

[0210] Statistical methods can be used to obtain the number of standing positions to determine the number of people; the target ground area surrounding the first position can be obtained; and the crowd density can be determined based on the ratio of the number of people to the target ground area surrounding the first position.

[0211] The population density detection method of the present invention has at least the following advantages:

[0212] 1. By using an instance segmentation algorithm, invalid background areas that are not ground can be removed to obtain the effective ground area.

[0213] 2. This invention employs a simplified plane representing the center position of the human head (denoted as coordinate system H in this disclosure), converting the coordinate transformation relationship between the ground coordinate system where the person is located and the coordinate system where the image is located (i.e., the image coordinate system) into a relationship between the simplified plane representing the center position of the human head and the image coordinate system. Furthermore, since the simplified plane representing the center position of the human head is parallel to the ground, there is a coordinate transformation relationship between the simplified plane representing the center position of the human head and the ground coordinate system, thereby allowing the determination of the person's standing position on the ground.

[0214] In summary, the crowd density detection algorithm of this invention proposes a simplified plane for the center position of the human head, which can effectively solve the problem of difficulty in locating the standing position of a person on the ground due to the inability of ordinary camera equipment to locate the foot information caused by the limited field of view and obstruction. This invention can achieve effective human positioning without changing the camera equipment. Furthermore, this invention can effectively remove background information through visual algorithms to obtain the effective ground area, making the calculation of crowd density more accurate.

[0215] With the above Figures 2 to 5 Corresponding to the crowd density detection method provided in the embodiments, the present invention also provides a crowd density detection device. Because the crowd density detection device provided in the embodiments of the present invention is similar to the one described above... Figures 2 to 5 The crowd density detection method provided in the embodiments corresponds to the crowd density detection method provided in the embodiments of the present invention, and therefore the implementation of the crowd density detection method is also applicable to the crowd density detection device provided in the embodiments of the present invention, and will not be described in detail in the embodiments of the present invention.

[0216] Figure 10 This is a schematic diagram of a crowd density detection device provided in an embodiment of the present invention.

[0217] like Figure 10 As shown, the crowd density detection device 1000 includes: a first acquisition module 1001, an identification module 1002, a conversion module 1003, a first determination module 1004, and a second determination module 1005.

[0218] The first acquisition module 1001 is used to acquire the target image captured by the camera device.

[0219] The recognition module 1002 is used to recognize the target image to determine the first pixel unit belonging to the ground and the second pixel unit belonging to the center position of the human head.

[0220] The conversion module 1003 is used to convert the position of the first pixel unit in the screen coordinate system to obtain the first position in the ground coordinate system according to the target conversion relationship between the screen coordinate system and the ground coordinate system of the target screen.

[0221] The first determining module 1004 is used to transform the position of the second pixel unit in the screen coordinate system according to the target transformation relationship to obtain the second position relative to the ground coordinate system, so as to determine the standing position of the person on the ground based on the second position.

[0222] The second determining module 1005 is used to determine the crowd density based on the first position and the standing position.

[0223] Furthermore, in one possible implementation of this invention, the identification module 1002 is further configured to:

[0224] An instance segmentation algorithm is used to identify the target image in order to determine the first pixel unit belonging to the ground from the target image;

[0225] Based on the crowd scene category to which the target image belongs, the corresponding crowd detection algorithm is used to determine the second pixel unit belonging to the center position of the human head in the target image.

[0226] Furthermore, in one possible implementation of this invention, the crowd scene category includes sparse crowd scenes and dense crowd scenes;

[0227] Among them, the crowd detection algorithm for sparse crowd scenes is used to extract features of at least two different scales from the target image, and to make predictions based on features of at least two different scales to obtain the second pixel unit belonging to the center position of the human head.

[0228] The crowd detection algorithm for densely populated scenes is used to extract head features from the target image based on convolutional neural networks, generate a crowd density map based on the head features, and determine the second pixel unit belonging to the center position of the head through the crowd density map.

[0229] Based on the above embodiment, this invention also provides a possible implementation of a crowd density detection device. Figure 11 This is a schematic diagram of another crowd density detection device 1000 provided in an embodiment of the present invention. Based on the previous embodiment, the crowd density detection device further includes: a second acquisition module 1006.

[0230] The second acquisition module 1006 is used to acquire the target transformation relationship based on the internal and external parameters of the camera device; wherein, the internal and external parameters include installation point information and focal length information.

[0231] Furthermore, in one possible implementation of this invention, the first determining module 1004 is further configured to:

[0232] Based on the target transformation relationship, the first transformation relationship between the screen coordinate system P and the coordinate system H established based on the center position of the human head is determined. The coordinate system H is a two-dimensional plane that passes through the center position of the human head and is parallel to the plane R where the ground is located.

[0233] Based on the first transformation relationship, the position of the second pixel unit in the image coordinate system P is transformed to obtain the second position of the second pixel unit in the coordinate system H;

[0234] Based on the offset between the center position of the human head and the standing position, determine the second transformation relationship between coordinate system H and the ground coordinate system;

[0235] Based on the second transformation relationship, the second position is transformed to determine the standing position.

[0236] Furthermore, in one possible implementation of this invention, the installation point information includes the installation height of the camera device and the angle between the optical axis of the camera device and the vertical line of the ground.

[0237] The crowd density detection device in this embodiment acquires a target image captured by a camera; identifies the target image to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center of a person's head; transforms the position of the first pixel unit in the image coordinate system to obtain a first position in the ground coordinate system based on the target transformation relationship between the image coordinate system and the ground coordinate system; transforms the position of the second pixel unit in the image coordinate system to obtain a second position relative to the ground coordinate system based on the target transformation relationship, and determines the standing position of a person on the ground based on the second position; and determines the crowd density based on the first position and the standing position. Therefore, based on the target transformation relationship between the image coordinate system and the ground coordinate system, the first position of the first pixel unit belonging to the ground in the target image in the ground coordinate system can be effectively determined, and the standing position of a person on the ground can be effectively determined based on the second pixel unit belonging to the center of a person's head in the target image. Thus, the crowd density can be effectively determined based on the standing position and the first position.

[0238] To implement the above embodiments, the present invention also provides an electronic device, which may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the crowd density detection method proposed in any of the above embodiments of the present invention.

[0239] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 12 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0240] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, which can realize the present invention. Figure 2-5 The flow of the method embodiment shown is as follows: Figure 12 As shown, the electronic device may include: a housing 1201, a processor 1202, a memory 1203, a circuit board 1204, and a power supply circuit 1205. The circuit board 1204 is disposed inside the space enclosed by the housing 1201, and the processor 1202 and the memory 1203 are disposed on the circuit board 1204. The power supply circuit 1205 is used to supply power to the various circuits or devices of the electronic device. The memory 1203 is used to store executable program code. The processor 1202 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 1203, for executing the crowd density detection method described in any of the foregoing embodiments.

[0241] For details on the specific execution process of the above steps by the processor 1202, and the steps further executed by the processor 1202 through running executable program code, please refer to the present invention. Figure 2-5 The description of the method embodiments shown will not be repeated here.

[0242] To implement the above embodiments, the present invention also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the data processing method proposed in any of the above embodiments of the present invention. In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0243] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0244] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0245] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0246] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0247] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0248] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0249] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting crowd density, characterized in that, include: Acquire the target image captured by the camera device; The target image is identified to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center of a human head. Based on the target transformation relationship between the screen coordinate system and the ground coordinate system of the target image, the position of the first pixel unit in the screen coordinate system is transformed to obtain the first position in the ground coordinate system; According to the target transformation relationship, the position of the second pixel unit in the image coordinate system is transformed to obtain a second position relative to the ground coordinate system, so as to determine the standing position of the person on the ground based on the second position; Based on the first location and the standing location, the crowd density is determined; The method further includes: Obtain the target transformation relationship based on the intrinsic and extrinsic parameters of the camera device; The internal and external parameters include installation point information and focal length information; The step of transforming the position of the second pixel unit in the image coordinate system according to the target transformation relationship to obtain a second position relative to the ground coordinate system, and determining the standing position of the person on the ground based on the second position, includes: Based on the target transformation relationship, a first transformation relationship is determined between the screen coordinate system P and the coordinate system H established based on the center position of the human head, wherein the coordinate system H is a two-dimensional plane that passes through the center position of the human head and is parallel to the plane R on which the ground is located; Based on the first transformation relationship, the position of the second pixel unit in the image coordinate system P is transformed to obtain the second position of the second pixel unit in the coordinate system H; Based on the offset between the center position of the human head and the standing position, a second transformation relationship between the coordinate system H and the ground coordinate system is determined; Based on the second transformation relationship, the second position is transformed to determine the standing position.

2. The method according to claim 1, characterized in that, The step of identifying the target image to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center position of a human head includes: An instance segmentation algorithm is used to identify the target image to determine the first pixel unit belonging to the ground from the target image; Based on the crowd scene category to which the target image belongs, the corresponding crowd detection algorithm is used to determine the second pixel unit belonging to the center position of the human head in the target image.

3. The method according to claim 2, characterized in that, The crowd scene categories include sparse crowd scenes and dense crowd scenes; The crowd detection algorithm corresponding to the sparse crowd scene is used to extract features of at least two different scales from the target image, and to make predictions based on the features of at least two different scales to obtain the second pixel unit belonging to the center position of the human head. The crowd detection algorithm for the dense crowd scene is used to extract head features from the target image based on a convolutional neural network, generate a crowd density map based on the head features, and determine the second pixel unit belonging to the center position of the head through the crowd density map.

4. The method according to claim 1, characterized in that, The installation location information includes the installation height of the camera equipment and the angle between the optical axis of the camera equipment and the vertical line of the ground.

5. A crowd density detection device, employing the crowd density detection method according to any one of claims 1-4, characterized in that, include: The acquisition module is used to acquire the target image captured by the camera device; The recognition module is used to recognize the target image to determine a first pixel unit belonging to the ground and a second pixel unit belonging to the center position of a human head. The conversion module is used to convert the position of the first pixel unit in the screen coordinate system to obtain the first position in the ground coordinate system according to the target conversion relationship between the screen coordinate system and the ground coordinate system of the target screen. The first determining module is used to transform the position of the second pixel unit in the screen coordinate system according to the target transformation relationship to obtain a second position relative to the ground coordinate system, so as to determine the standing position of a person on the ground based on the second position; The second determining module is used to determine the crowd density based on the first position and the standing position.

6. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to cause the at least one processor to perform the crowd density detection method according to any one of claims 1-4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the crowd density detection method according to any one of claims 1-4.

8. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the crowd density detection method as described in any one of claims 1-4.

Images