Pedestrian counting method and device, electronic equipment and storage medium

Abstract

The invention discloses a pedestrian counting method and device, electronic equipment and a storage medium, the calculation process is relatively simple and efficient, and a relatively accurate counting result can be taken into account. The pedestrian counting method comprises the following steps: inputting an acquired image into a target part detector to obtain a target part detection frame of the image; wherein the target part detection frame is used for selecting a target part of the head of a target pedestrian in the image; tracking the target part detection frame; based on the proportion of the target part to the human shape and the tracked target part detection frame, obtaining a human shape movement track of the target pedestrian; and when the human shape motion trail of the target pedestrian satisfies a preset condition, counting the target pedestrian.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a pedestrian counting method, device, electronic device and storage medium. Background Art

[0002] Pedestrian detection technology is a hot topic in the field of computer vision. The problem to be solved is to find the position and / or size of pedestrians in images or videos. Pedestrian detection technology can be combined with other technologies such as pedestrian tracking and applied to unmanned driving, video surveillance, passenger flow statistics and other fields.

[0003] In the related art, when detecting pedestrians, MobileNetV1 is usually used as the backbone network to construct a pedestrian detector and a head detector, and the pedestrian target and the head target of the input image are predicted. The pedestrian target and the head target are traversed to calculate the IOU (Intersection-over-Union). If the IOU is greater than a certain threshold, it is determined that the current head and the pedestrian belong to the same target. If the IOU is less than a certain threshold, it is determined that they are not matched. The Euclidean distance is calculated for the unmatched head target and the matched head target. If the distance between the two is less than a certain distance threshold, it is determined that the pedestrian corresponding to the head of the target is blocked, and the head target is bound to the pedestrian target. If there are multiple distance values ​​less than the threshold, the pedestrian target with the smallest distance is selected for binding; if the distance is greater than the threshold, it is discarded.

[0004] However, when applied to scenes with high-density crowd images, the above method needs to traverse and calculate all pedestrian targets and head targets in order to determine the correspondence between head targets and pedestrian targets, resulting in a large amount of calculation. Summary of the invention

[0005] The present application aims to solve the technical problems existing in the above-mentioned related technologies. To this end, the present application provides a pedestrian counting method, device, electronic device and storage medium. The calculation process of the present application is relatively simple, efficient and accurate in counting results.

[0006] In a first aspect, the present application provides a pedestrian counting method, comprising:

[0007] Inputting the acquired image into a target part detector to obtain a target part detection frame of the image; wherein the target part detection frame is used to frame a target part of a target pedestrian in the image;

[0008] Tracking the target part detection frame;

[0009] Based on the ratio of the target part to the human figure and the tracked detection frame of the target part, a human figure motion trajectory of the target pedestrian is obtained;

[0010] When the human-shaped motion trajectory of the target pedestrian meets a preset condition, the target pedestrian is counted.

[0011] In some embodiments, obtaining the human-shaped motion trajectory of the target pedestrian based on the ratio of the target part to the human shape and the tracked target part detection frame includes:

[0012] Based on the tracked target part detection frame, obtaining the target part detection frame in the image at the current moment and the target part detection frame in the image at the previous moment;

[0013] Based on the ratio of the target part to the human figure, the target part detection frame in the image at the current moment and the target part detection frame in the image at the previous moment, a human figure detection frame in the image at the current moment and a human figure detection frame in the image at the previous moment are fitted; wherein the human figure detection frame is used to select the target pedestrian in the image;

[0014] Based on the target point of the human figure detection frame in the image at the current moment and the target point of the human figure detection frame in the image at the previous moment, the motion segment of the target pedestrian is obtained; wherein, the motion segment of the target pedestrian is used to constitute at least a part of the human figure motion trajectory of the target pedestrian.

[0015] In some embodiments, the pedestrian counting method further includes:

[0016] Obtaining a foreground mask in the image;

[0017] When the outline of the foreground mask meets a preset condition, the corresponding human figure detection frame is corrected based on the outline of the foreground mask.

[0018] In some embodiments, the contour of the foreground mask satisfies a preset condition, including:

[0019] The perimeter of the outline of the foreground mask is greater than a preset perimeter threshold;

[0020] Alternatively, the aspect ratio of the outline of the foreground mask is within a preset ratio range.

[0021] In some embodiments, when the humanoid motion trajectory of the target pedestrian meets a preset condition, counting the target pedestrian includes:

[0022] When the moving line segment of the target pedestrian intersects with a preset boundary line, it is determined that the target pedestrian has crossed the boundary, and the target pedestrian is counted.

[0023] In some embodiments, when the humanoid motion trajectory of the target pedestrian meets a preset condition, counting the target pedestrian includes:

[0024] When the target pedestrian's motion line segment intersects with a preset boundary line, determining that the target pedestrian has crossed the boundary;

[0025] When the tracking start point and the tracking end point are located at both sides of the boundary line respectively, the target pedestrian is counted; wherein the tracking start point and the tracking end point are determined according to the human-shaped motion trajectory of the target pedestrian.

[0026] In some embodiments, the target point of the human figure detection frame includes: the midpoint of the lower line segment of the human figure detection frame.

[0027] In some embodiments, the target part is the pedestrian's head; or, the target part includes: the pedestrian's head and shoulders.

[0028] In a second aspect, the present application further provides a pedestrian counting device, comprising:

[0029] A first processing unit is used to input the acquired image into a target part detector to obtain a target part detection frame of the image; wherein the target part detection frame is used to frame a target part of a target pedestrian's head in the image;

[0030] A tracking unit, used for tracking the target part detection frame;

[0031] A second processing unit, configured to obtain a human-shaped motion trajectory of the target pedestrian based on a ratio between the target part and the human-shaped figure and a tracked detection frame of the target part;

[0032] The counting unit counts the target pedestrian when the human-shaped motion trajectory of the target pedestrian meets a preset condition.

[0033] In a third aspect, the present application further provides an electronic device, including:

[0034] Processor; and

[0035] A memory, configured to store executable instructions of the processor;

[0036] The processor is configured to perform any of the methods described above by executing the executable instructions.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method as described in any one of the above items is implemented.

[0038] The pedestrian counting method provided by the present application obtains a target part detection frame of the image by inputting the acquired image into a target part detector; wherein the target part detection frame is used to select the target part of the target pedestrian in the image; the target part detection frame of the target pedestrian is tracked to obtain the target part motion trajectory; based on the ratio of the target part to the human figure and the tracked target part detection frame, the human-shaped motion trajectory of the target pedestrian is obtained; when the human-shaped motion trajectory of the target pedestrian meets a preset condition, the target pedestrian is counted. Therefore, the present embodiment does not need to traverse all pedestrian targets and target parts, but can obtain the human-shaped motion trajectory of the target pedestrian based on the tracked target part detection frame and the ratio of the target part to the human figure, thereby realizing pedestrian counting. The calculation process of the present embodiment is relatively simple and efficient, and can take into account relatively accurate counting results.

[0039] Additional aspects and advantages of the present application will be given in part in the description below, and in part will be highlighted from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flowchart of a pedestrian counting method provided by an exemplary embodiment;

[0041] Figure 2 is a flowchart of a pedestrian counting method provided by another exemplary embodiment;

[0042] Figure 3 is a structural schematic diagram of a pedestrian counting device provided by an exemplary embodiment;

[0043] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0044] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0045] In the related technology, when pedestrian detection technology is applied to the scene of abnormal behavior identification of pedestrians crossing the boundary, the head information in the current video frame can also be detected through the Adaboost cascade classifier based on the multi-scale wavelet change motion foreground detection method of adjacent frames and the difference of individual head-like Haar features; the head in the video scene is tracked through a tracking chain based on a particle filter, and the tracking is dynamically updated according to the appearance model of the head; based on the head detection and tracking results, a three-dimensional plane equation is established according to the invariance of human body height to estimate the plane above the pedestrian's head; based on the plane above the pedestrian's head, a 3D space virtual warning wall is established according to the two-dimensional scene warning line manually set on the ground; the vertical distance and distance direction between the ground warning line of the 3D space virtual warning wall and the projection point of the head on the ground are calculated, and the change of the vertical distance and distance direction in the video determines whether the pedestrian has crossed the boundary.

[0046] However, when the above method is applied to a scene with a large number of people in the picture, errors are prone to occur in the construction of the plane above the people's heads, which reduces the accuracy of the out-of-bounds judgment.

[0047] The present embodiment provides a pedestrian counting method, which detects the target part of the target pedestrian and tracks the target part using a tracking algorithm. It can expand a human detection frame by combining the proportion fitting of the human figure and the target part, and obtain the human motion trajectory based on the human detection frame at multiple moments. If the human motion trajectory meets the matching result of preset conditions, pedestrian counting is performed without traversing all pedestrian targets and target heads, thereby making the calculation process of the present embodiment simpler and more efficient, and taking into account the accuracy of the detection results.

[0048] The implementation process of the pedestrian counting method according to the embodiment of the present application is described below with reference to the accompanying drawings.

[0049] Figure 1 is a flowchart of a pedestrian counting method provided by an exemplary embodiment. Figure 1 The pedestrian counting method provided in this embodiment includes the following steps S110 to S130:

[0050] Step S110, inputting the acquired image into a target part detector to obtain a target part detection frame of the image; wherein the target part detection frame is used to frame a target part of a target pedestrian in the image;

[0051] Step S120, tracking the target part detection frame;

[0052] Step S130, obtaining a humanoid motion trajectory of the target pedestrian based on the ratio of the target part to the humanoid and the tracked target part detection frame;

[0053] Step S140: When the humanoid motion trajectory of the target pedestrian meets a preset condition, count the target pedestrian.

[0054] The execution subject of this embodiment may be a terminal, which may be used to display a graphical user interface. The terminal is used to interact with the user through the graphical user interface, such as downloading and installing a corresponding client through the terminal and running it, such as calling a corresponding applet and running it, such as logging into a website and presenting a corresponding graphical user interface, etc. In other embodiments, the execution subject may also be a server with data processing capabilities.

[0055] This embodiment can be applied to a monocular vision scene, which refers to a scene in which an image is captured using one visual sensor. Of course, if necessary, this embodiment can also be applied to a multi-eye vision scene.

[0056] In step S110, the image captured by the visual sensor is obtained. The captured information can be directly obtained from the visual sensor, or the information captured by the visual sensor can be obtained from other devices. If the acquired information is a video screen such as a surveillance video or a video recording, the video screen can be converted into a sequence image, which refers to a plurality of continuous frames of images. If the acquired information is an image, the image can be arranged according to the capture time.

[0057] The above image is input to the target part detector, which identifies the target part and its position of the target pedestrian in the image and generates a target part detection frame for selecting the target part. If necessary, the target part detection frame can be displayed in the image.

[0058] The target part can be a human head or a head and shoulders, which can be selected according to actual needs. When the target part is a human head, the target part detector can be a human head detector; when the target part is a head and shoulders, the target part detector can be a head and shoulders detector.

[0059] The target part detection frame may refer to a bounding box of the target part. The position of the target part detection frame may be reflected by coordinates. Optionally, the target part detection frame may be rectangular. The coordinates of the target part detection frame may include: center point coordinates, upper left corner coordinates, and lower right corner coordinates. In other examples, the coordinates of the target part detection frame may also include center point coordinates, upper right corner coordinates, and lower left corner coordinates.

[0060] Of course, the target part detection frame may also be in other shapes, for example, the target part detection frame is an oblong shape; for another example, the target part detection frame may be formed by combining a circular frame and a rectangular frame. The specific shape of the target part detection frame may be set according to actual needs. The coordinates of the target part detection frame may include: the coordinates of the center point, and the coordinates of multiple other points that can reflect the length and width of the target part detection frame.

[0061] In addition, the target pedestrian can be one pedestrian or multiple pedestrians, which can be set according to actual needs. Each target pedestrian corresponds to a target part detection frame. Optionally, an identity document (ID) can be set for each target part detection frame. Among them, one ID corresponds to one target pedestrian.

[0062] In step S120, a target tracking algorithm is used to track the target part detection frame detected in step S110. The target tracking algorithm can be a conventional algorithm in the field of computer vision, as long as it can detect and track the target of interest from the image sequence. It is understandable that when there are multiple target pedestrians, a multi-target tracking algorithm can be used.

[0063] Exemplarily, the similarity of the target parts in the target part detection frames in multiple images can be determined, and the target part detection frames whose similarity reaches a similarity threshold are associated to achieve tracking of the target part detection frames. According to actual needs, this embodiment can also obtain the target part motion trajectory according to the coordinates of the associated target part detection frames.

[0064] In step S130, taking the target part as the head as an example, the ratio of the head to the human figure can be obtained in advance. The human figure refers to a model of the entire human body. In some examples, in order to improve the detection efficiency, the predetermined and stored ratio of the head to the human figure can be obtained. Specifically, the ratio of the head to the human figure can be determined and stored based on the size of the head and the size of the human figure in the historical image. Generally speaking, along the height direction of the human figure (that is, the up and down direction of the pedestrian), the human figure is about 7.5 times the size of the head; along the lateral direction of the human figure (that is, the left and right direction of the pedestrian), the human figure is about 2 times the size of the head. In other examples, the image in which the target pedestrian first displays a complete human figure can be selected in the currently input image, and the ratio of the head to the human figure can be determined based on the size of the head and the size of the human figure in the image.

[0065] After obtaining the ratio of the head to the human figure, the human figure detection frame can be fitted and expanded based on the tracked target part detection frame and the ratio of the head to the human figure, and the human figure motion trajectory of the target pedestrian can be obtained based on the human figure detection frames at multiple moments.

[0066] Exemplarily, the human figure detection frame may be rectangular, and the coordinates of the human figure detection frame may include: center point coordinates, upper left corner coordinates, and lower right corner coordinates. In other examples, the coordinates of the human figure detection frame may also include center point coordinates, upper right corner coordinates, and lower left corner coordinates. Optionally, the target part detection frame and the human figure detection block are both rectangular, and their coordinates include: center point coordinates, upper left corner coordinates, and lower right corner coordinates.

[0067] In addition, it can be understood that during the movement of the target pedestrian, at the same time, the movement trajectory of the human figure of the target pedestrian and the movement direction corresponding to the movement trajectory of the target part are adapted.

[0068] Optionally, if there are discontinuous images showing the same pedestrian, the same pedestrian shown in the previous multiple continuous images can be counted as one target pedestrian; if the pedestrian is shown again in multiple continuous images after a preset number of images, it is still counted as the target pedestrian and the human-shaped motion trajectory of the target pedestrian includes the information of the target pedestrian at different time points, or the pedestrian can be counted as a new target pedestrian, and each target pedestrian corresponds to a human-shaped motion trajectory. The preset number can be set according to actual needs, and the specific value is not limited in this embodiment.

[0069] After obtaining the humanoid motion trajectory of the target pedestrian, a more accurate analysis and judgment can be made based on the humanoid motion trajectory as to whether the target pedestrian has crossed the boundary, the target pedestrian's motion range, etc.

[0070] In step S140, it is determined whether the obtained humanoid motion trajectory meets the preset conditions; if so, the target pedestrian is counted; if not, the target pedestrian is not counted. Exemplarily, the preset conditions may include: the humanoid motion trajectory of the target pedestrian intersects with a preset boundary line; or, the starting point and the end point of the humanoid motion trajectory are respectively located on both sides of the preset boundary line; or, the preset conditions may include: the humanoid motion trajectory of the target pedestrian intersects with the preset boundary line, and the starting point and the end point of the humanoid motion trajectory are respectively located on both sides of the preset boundary line.

[0071] The pedestrian counting method provided by the present embodiment obtains a target part detection frame of the image by inputting the acquired image into a target part detector; wherein the target part detection frame is used to frame the target part of the target pedestrian in the image; the target part detection frame of the target pedestrian is tracked to obtain the target part motion trajectory; based on the ratio of the target part to the human figure and the tracked target part detection frame, the human-shaped motion trajectory of the target pedestrian is obtained; when the human-shaped motion trajectory of the target pedestrian meets the preset conditions, the target pedestrian is counted. Therefore, the present embodiment does not need to traverse all pedestrian targets and target heads, but obtains the human-shaped motion trajectory of the target pedestrian based on the tracked target part detection frame and the ratio of the target part to the human figure to achieve pedestrian counting. The calculation process of the present embodiment is simpler and more efficient, and can take into account relatively accurate counting results.

[0072] In some embodiments, step S130, based on the ratio of the target part to the human figure and the tracked target part detection frame, obtains the human figure motion trajectory of the target pedestrian, which may specifically include the following steps a1 to a3:

[0073] Step a1, based on the tracked target part detection frame, obtaining the target part detection frame in the image at the current moment and the target part detection frame in the image at the previous moment;

[0074] Step a2: based on the ratio of the target part to the human figure, the target part detection frame in the image at the current moment and the target part detection frame in the image at the previous moment, a human figure detection frame in the image at the current moment and a human figure detection frame in the image at the previous moment are fitted; wherein the human figure detection frame is used to frame the target pedestrian in the image;

[0075] Step a3: obtaining the motion line segment of the target pedestrian based on the target point of the human figure detection frame in the image at the current moment and the target point of the human figure detection frame in the image at the previous moment; wherein the motion line segment of the target pedestrian is used to constitute at least part of the human figure motion trajectory of the target pedestrian.

[0076] In step a2, the ratio of the target part to the human figure and the target part detection frame in the current image are fitted and expanded to obtain the human figure detection frame in the current image. The ratio of the target part to the human figure and the target part detection frame in the previous image are fitted and expanded to obtain the human figure detection frame in the previous image.

[0077] Exemplarily, taking the target part detection frame as a head detection frame, along the height direction of the human figure (that is, the up and down direction of the pedestrian), the human figure is approximately 7.5 times the size of the head, and along the lateral direction of the human figure (that is, the left and right direction of the pedestrian), the human figure is approximately twice the size of the head. As an example: along the height direction of the human figure, the target part detection frame is extended downward until the size of the extended detection frame along the height direction is 7.5 times the size of the target part detection frame; along the lateral direction of the human figure, the target part detection frame is extended to the left and right directions respectively until the size of the extended detection frame along the height direction is twice the size of the target part detection frame, to obtain a human figure detection frame, wherein the center line of the human figure detection frame along the height direction coincides with the center line of the target part detection frame along the height direction.

[0078] In step a3, the target point of the human figure detection frame in the image at the current moment and the target point of the human figure detection frame in the image at the previous moment are connected to form a line segment to obtain the moving line segment of the target pedestrian. Optionally, in order to be closer to the viewing angle of the monitoring screen, the target point of the human figure detection frame can be the midpoint of the lower line segment of the human figure detection frame, or the left vertex of the lower line segment, or the right vertex of the lower line segment, which can be set according to actual needs.

[0079] After obtaining the line segments composed of the target points of the human figure detection frames at multiple adjacent moments, that is, after obtaining multiple motion line segments, the motion line segments can be connected according to time to obtain the human figure motion trajectory of the target pedestrian.

[0080] In this embodiment, the above configuration is helpful to reduce the false detection rate and achieve a better pedestrian detection effect.

[0081] In some embodiments, the pedestrian counting method of this embodiment may further include the following steps b1 to b2 before step a3:

[0082] Step b1, obtaining a foreground mask in the image;

[0083] Step b2: when the outline of the foreground mask meets the preset conditions, the corresponding human detection frame is corrected based on the outline of the foreground mask.

[0084] In step b1, for the input image, a background model can be established by using a Gaussian mixture model (GMM) or other method, the foreground in the image can be analyzed, and morphological operations can be used for denoising to obtain a foreground mask in the image. The foreground mask refers to a binary image of a moving object in the image. For each input image, the foreground mask can be obtained by using the above method. In addition, the implementation process of obtaining the foreground mask is not limited to this. This embodiment is only used as an example, as long as the function of obtaining the foreground mask in the image can be achieved.

[0085] In step b2, after obtaining the foreground mask, the contour in the foreground mask is obtained using the image contour search function. When the contour of the foreground mask meets the preset conditions, it is determined that the contour represents a human figure. Then, the circumscribed rectangle of the contour is determined, and the corresponding human detection frame is corrected based on the circumscribed rectangle, thereby further improving the accuracy of pedestrian detection.

[0086] Optionally, the contour of the foreground mask satisfies a preset condition, including: the perimeter of the contour of the foreground mask is greater than a preset perimeter threshold; or the aspect ratio of the contour of the foreground mask is within a preset ratio range. In specific implementation, when the contour of the foreground mask satisfies one of the perimeter being greater than the preset perimeter threshold and the aspect ratio being within a preset ratio range, it can be determined that the contour represents a human figure; or when the contour of the foreground mask satisfies both the perimeter being greater than the preset perimeter threshold and the aspect ratio being within a preset ratio range, it can be determined that the contour represents a human figure.

[0087] Among them, in the case where there are multiple target pedestrians in the image, multiple foreground masks can be obtained from the image, and the offset distance between the circumscribed rectangle of the foreground mask and the human figure detection frame of each target pedestrian is determined. For example, the offset distance between the midpoint of the lower edge segment of the circumscribed rectangle of the foreground mask and the midpoint of the lower edge segment of the human figure detection frame of each target pedestrian is determined, and the human figure detection frame of the target pedestrian with the smallest offset distance is used as the human figure detection frame corresponding to the foreground mask.

[0088] In the correction process, in some examples, the aspect ratio of the human detection frame can be corrected based on the aspect ratio of the outline of the foreground mask until the deviation between the aspect ratio of the human detection frame and the aspect ratio of the outline of the foreground mask is less than a preset error amount. In still other examples, when the above-mentioned offset distance is greater than a preset distance threshold, the human detection frame can be moved toward the circumscribed rectangle of the foreground mask until the offset distance between the two is less than or equal to the distance threshold. Of course, the specific correction process is not limited to this, and this embodiment is only used as an example here.

[0089] In addition, when the number of foreground mask contours in the image is less than the number of human detection frames, the probability that the human detection frames without corresponding foreground mask contours are false detections is relatively high. Corresponding prompt information can be generated based on the human detection frames for further judgment, or the human detection frames without corresponding foreground mask contours can be directly deleted. The specific settings can be made according to actual needs.

[0090] When the number of contours of the foreground mask in the image is greater than the number of human detection frames, there may be a missed detection. In this case, the target part detection frame can be re-acquired and the human detection frame can be further obtained until the number of contours of the foreground mask is equal to the number of human detection frames. If the number of contours of the foreground mask is still greater than the number of human detection frames after reprocessing, a corresponding prompt message is generated.

[0091] In this embodiment, through the above settings, the false detection rate can be further reduced and the accuracy of pedestrian detection can be improved.

[0092] Figure 2 is a flowchart of a pedestrian counting method provided by another exemplary embodiment. Figure 2 In some embodiments, the human figure detection method of this embodiment includes the following steps S210 to S280:

[0093] Step S210: input the acquired image into a target part detector to obtain a target part detection frame of the image; wherein the target part detection frame is used to frame a target part of a target pedestrian in the image;

[0094] Step S220, tracking the target part detection frame of the target pedestrian;

[0095] Step S230, based on the tracked target part detection frame, obtaining the target part detection frame in the image at the current moment and the target part detection frame in the image at the previous moment;

[0096] Step S240, based on the ratio of the target part to the human figure, the target part detection frame in the image at the current moment and the target part detection frame in the image at the previous moment, a human figure detection frame in the image at the current moment and a human figure detection frame in the image at the previous moment are fitted; wherein the human figure detection frame is used to frame the target pedestrian in the image;

[0097] Step S250, obtaining a motion line segment of the target pedestrian based on the target point of the human figure detection frame in the image at the current moment and the target point of the human figure detection frame in the image at the previous moment; wherein the motion line segment of the target pedestrian is used to constitute at least a part of the human figure motion trajectory of the target pedestrian;

[0098] Step S260: when the target pedestrian's moving line segment intersects with a preset boundary line, it is determined that the target pedestrian has crossed the boundary;

[0099] Step S270: When the tracking start point and the tracking end point are located on both sides of the boundary line respectively, the target pedestrian is counted; wherein the tracking start point and the tracking end point are determined according to the human motion trajectory of the target pedestrian.

[0100] Among them, the implementation process of step S210 to step S220 is the same as the implementation process of the aforementioned step S110 to step S120; the implementation process of step S230 to step S250 is the same as the implementation process of the aforementioned step a1 to step a3.

[0101] In step S260, it is determined whether the target pedestrian's moving line segment intersects with a preset boundary line. If the target pedestrian's moving line segment intersects with the preset boundary line, it is determined that the target pedestrian has crossed the boundary. The boundary line may be a straight line segment, a curve segment, or a closed curve, and the boundary line may be set according to actual needs.

[0102] If the motion line segment of the target pedestrian does not intersect with the preset boundary line, continue to obtain the motion line segment composed of the target point of the human figure detection frame in the image at the next moment and the target point of the human figure detection frame in the image at the current moment, and judge whether it intersects with the preset boundary line. Repeat this cycle until the motion line segments of the target pedestrian at each moment are judged.

[0103] Optionally, when multiple motion segments of the target pedestrian intersect with a preset boundary line, the number, time, or position of the intersections may be recorded and saved according to actual needs, that is, the number, time, or position of the boundary crossing behavior may be recorded and saved.

[0104] In step S270, multiple motion line segments of the target pedestrian are connected in chronological order to obtain the humanoid motion trajectory of the target pedestrian, and the motion starting point (i.e., the starting point of the curved segment used to illustrate the humanoid motion trajectory) and the motion ending point (i.e., the ending point of the curved segment used to illustrate the humanoid motion trajectory) of the humanoid motion trajectory can be obtained. The motion starting point can be used as the tracking starting point, and the motion ending point can be used as the tracking ending point.

[0105] In other examples, a point after a preset time interval from a movement start point may be used as a tracking start point, and a point before a preset time interval from a movement end point may be used as a tracking end point.

[0106] After determining the tracking start point and the tracking end point, the relative positions of the tracking start point and the tracking end point to the boundary line are determined. Generally speaking, when the target pedestrian moves within the range defined by the boundary line, the tracking start point and the tracking end point are located on the same side of the boundary line. If the tracking start point and the tracking end point are located on both sides of the boundary line, respectively, and the target pedestrian's moving line segment intersects with the preset boundary line, it means that the target pedestrian is outside the boundary line at the end of tracking, and the target pedestrian is counted, for example, the target pedestrian is counted in the number of pedestrians crossing the boundary.

[0107] If the tracking start point and the tracking end point are on the same side of the boundary line, it means that the target pedestrian is within the range defined by the boundary line at the end of tracking, and the target pedestrian is not counted. In other examples, if the tracking start point and the tracking end point are on the same side of the boundary line, the target pedestrian can also be counted, specifically, it can be counted in the number of pedestrians who have not crossed the boundary.

[0108] In other embodiments, according to actual needs, in step S260, when the target pedestrian's moving line segment intersects with a preset boundary line, it is determined that the target pedestrian has crossed the boundary and the target pedestrian is counted. In this case, there is no need to perform steps S270 and S280.

[0109] In some other embodiments, according to actual needs, in step S250, after obtaining the human-shaped motion trajectory of the target pedestrian, steps S270 and S280 can be directly executed to count the target pedestrian when the tracking start point and the tracking end point are respectively located on both sides of the boundary line.

[0110] It can be understood that: this embodiment can be applicable to unmanned driving, video surveillance, passenger flow statistics, and game scenarios, etc.

[0111] In order to better implement the pedestrian counting method of the embodiment of the present application, the embodiment of the present application also provides a pedestrian counting device. Figure 3 For a schematic diagram of the structure of a pedestrian counting device provided in an embodiment of the present application, please refer to Figure 3, the device includes the following units:

[0112] The first processing unit 301 is used to input the acquired image into the target part detector to obtain a target part detection frame of the image; wherein the target part detection frame is used to frame the target part of the target pedestrian in the image;

[0113] A tracking unit 302, used to track the target part detection frame;

[0114] The second processing unit 303 is used to obtain the human-shaped motion trajectory of the target pedestrian based on the ratio of the target part to the human figure and the tracked target part detection frame;

[0115] The counting unit 304 counts the target pedestrian when the human-shaped motion trajectory of the target pedestrian meets a preset condition.

[0116] In some embodiments, the second processing unit is specifically configured to:

[0117] Based on the tracked target part detection frame, obtain the target part detection frame in the image at the current moment and the target part detection frame in the image at the previous moment;

[0118] Based on the ratio of the target part to the human figure, the target part detection frame in the current image and the target part detection frame in the previous image, a human figure detection frame in the current image and a human figure detection frame in the previous image are fitted; wherein the human figure detection frame is used to select the target pedestrian in the image;

[0119] Based on the target points of the human figure detection frame in the image at the current moment and the target points of the human figure detection frame in the image at the previous moment, the motion line segments of the target pedestrian are obtained; wherein the motion line segments of the target pedestrian are used to constitute at least part of the human figure motion trajectory of the target pedestrian.

[0120] In some embodiments, the pedestrian counting device further comprises:

[0121] An acquisition unit, used for acquiring a foreground mask in an image;

[0122] The correction unit is used to correct the corresponding human detection frame based on the outline of the foreground mask when the outline of the foreground mask meets the preset condition.

[0123] In some embodiments, the contour of the foreground mask satisfies a preset condition, including: the perimeter of the contour of the foreground mask is greater than a preset perimeter threshold; or the aspect ratio of the contour of the foreground mask is within a preset ratio range.

[0124] In some embodiments, the counting unit is specifically used for:

[0125] When the target pedestrian's moving line segment intersects with a preset boundary line, it is determined that the target pedestrian has crossed the boundary and the target pedestrian is counted.

[0126] In some embodiments, the counting unit is specifically used for:

[0127] When the target pedestrian's moving line segment intersects with a preset boundary line, it is determined that the target pedestrian has crossed the boundary;

[0128] When the tracking start point and the tracking end point are located at both sides of the boundary line respectively, the target pedestrian is counted; wherein the tracking start point and the tracking end point are determined according to the human-shaped motion trajectory of the target pedestrian.

[0129] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0130] The pedestrian counting device of this embodiment may be integrated into a terminal or server that has a storage device and a processor installed therein and has computing capability, or the pedestrian counting device is the terminal or the server.

[0131] Figure 4 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 4 As shown, in some embodiments, the present application further provides an electronic device, including: a communication interface 401, a memory 402 and a processor 403, wherein the communication interface 401 is used to communicate with an external device, and the memory 402 is used to store executable instructions of the processor 403, and the processor 402 is configured to execute the steps in the above-mentioned method embodiments by executing the executable instructions. In addition, the electronic device may also include a communication bus 404 for connecting the communication interface 401, the memory 402 and the processor 403.

[0132] The processor may include one or more processing cores. The memory may include one or more computer-readable storage media. The memory stores a computer program, which can be run on the processor. The processor is electrically connected to the memory. Those skilled in the art will appreciate that the computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0133] The processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiment of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0134] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0135] The above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiment of the present application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM RAM (DR RAM), etc. That is to say, the memory in the embodiment of the present application is intended to include but not limited to these and any other suitable types of memory.

[0136] although Figure 4 Not shown, the electronic device may further include a power module, a wireless fidelity module, a Bluetooth module, an input module, etc., which will not be described in detail here.

[0137] In some embodiments, the present application further provides a computer-readable storage medium, which stores a computer program. The computer-readable storage medium can be applied to an electronic device, and the computer program enables the electronic device to execute the corresponding process in the method in the embodiment of the present application, which will not be described here for the sake of brevity.

[0138] In some embodiments, the present application further provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of the electronic device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the electronic device executes the corresponding process in the method in the embodiment of the present application, which will not be described here for the sake of brevity.

[0139] In some embodiments, the present application further provides a computer program, which includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the electronic device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the electronic device executes the corresponding process in the method in the embodiment of the present application, which will not be repeated here for the sake of brevity.

[0140] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0141] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0142] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0143] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] In addition, each functional unit in the embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0145] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for an electronic device (which can be a personal computer, a server) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk.

[0146] It should be noted that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or equipment (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or equipment and execute instructions), or used in combination with these instruction execution systems, devices or equipment.

[0147] For the purposes of this specification, a "computer readable medium" may be any device that can contain, store, communicate, propagate or transport a program for use with or in conjunction with an instruction execution system, apparatus or device. More specific examples (a non-exhaustive list) of computer readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM).

[0148] In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0149] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A pedestrian counting method, characterized in that: include: Inputting the acquired image into a target part detector to obtain a target part detection frame of the image; wherein the target part detection frame is used to frame a target part of a target pedestrian in the image; Tracking the target part detection frame; Based on the ratio of the target part to the human figure and the tracked detection frame of the target part, a human figure motion trajectory of the target pedestrian is obtained; When the human-shaped motion trajectory of the target pedestrian meets a preset condition, the target pedestrian is counted.

2. The pedestrian counting method according to claim 1, characterized in that: The method of obtaining the human motion trajectory of the target pedestrian based on the ratio of the target part to the human figure and the tracked target part detection frame includes: Based on the tracked target part detection frame, obtaining the target part detection frame in the image at the current moment and the target part detection frame in the image at the previous moment; Based on the ratio of the target part to the human figure, the target part detection frame in the image at the current moment and the target part detection frame in the image at the previous moment, a human figure detection frame in the image at the current moment and a human figure detection frame in the image at the previous moment are obtained; wherein the human figure detection frame is used to frame the target pedestrian in the image; Based on the target point of the human figure detection frame in the image at the current moment and the target point of the human figure detection frame in the image at the previous moment, the motion segment of the target pedestrian is obtained; wherein, the motion segment of the target pedestrian is used to constitute at least a part of the human figure motion trajectory of the target pedestrian.

3. The pedestrian counting method according to claim 2, characterized in that: Also includes: Obtaining a foreground mask in the image; When the outline of the foreground mask meets a preset condition, the corresponding human figure detection frame is corrected based on the outline of the foreground mask.

4. The pedestrian counting method according to claim 3, characterized in that: The contour of the foreground mask meets the preset conditions, including: The perimeter of the outline of the foreground mask is greater than a preset perimeter threshold; Alternatively, the aspect ratio of the outline of the foreground mask is within a preset ratio range.

5. The pedestrian counting method according to claim 2, characterized in that: When the human-shaped motion trajectory of the target pedestrian meets a preset condition, counting the target pedestrian includes: When the moving line segment of the target pedestrian intersects with a preset boundary line, it is determined that the target pedestrian has crossed the boundary, and the target pedestrian is counted.

6. The pedestrian counting method according to claim 2, characterized in that: When the human-shaped motion trajectory of the target pedestrian meets a preset condition, counting the target pedestrian includes: When the target pedestrian's motion line segment intersects with a preset boundary line, determining that the target pedestrian has crossed the boundary; When the tracking start point and the tracking end point are located at both sides of the boundary line respectively, the target pedestrian is counted; wherein the tracking start point and the tracking end point are determined according to the human-shaped motion trajectory of the target pedestrian.

7. The pedestrian counting method according to claim 2, characterized in that: The target point of the human figure detection frame includes: the midpoint of the lower line segment of the human figure detection frame; The target part is the pedestrian's head; or, the target part includes: the pedestrian's head and shoulders.

8. A pedestrian counting device, characterized in that: include: A first processing unit is used to input the acquired image into a target part detector to obtain a target part detection frame of the image; wherein the target part detection frame is used to frame a target part of a target pedestrian in the image; A tracking unit, used for tracking a target part detection frame; A second processing unit, configured to obtain a human-shaped motion trajectory of the target pedestrian based on a ratio between the target part and the human-shaped figure and a tracked detection frame of the target part; The counting unit counts the target pedestrian when the human-shaped motion trajectory of the target pedestrian meets a preset condition.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images