Method and system for counting people in a region based on an adaptive ensemble model

Abstract

The application discloses a regional population statistics method and system based on an adaptive integrated model, and belongs to the technical field of artificial intelligence. The technical problem to be solved by the application is that the prior art is difficult to adapt to different shooting angles and distance scenarios for regional population statistics. The technical solution is as follows: the method is as follows: video data is acquired through a camera in a public area of a city, an image after frame extraction is acquired from the video data, the number and position information of human body targets in the image are acquired through a human body feature detection algorithm; whether the human body targets appear is judged according to the number and position information of the human body targets; the height-width ratio AR of the human body targets is compared with a threshold value AR th ; a shooting distance quantization coefficient DR is calculated according to the average target size; and regional population information and position information are acquired according to all head position information, human body feature position information, human body skeleton position information and face skeleton position information, and based on a weighted non-maximum suppression algorithm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a regional population statistics method and system based on an adaptive integrated model. BACKGROUND

[0002] With the gradual lifting of control over population flow, the process of urbanization is accelerating, and the population density of developed coastal cities is increasing. In order to prevent infectious diseases and prevent social security problems, a large number of cameras have been installed on urban roads and key places for real-time detection of personnel and behavior in specific areas, so as to quickly locate special personnel.

[0003] In view of the above problems, it is necessary to use the cameras in urban public areas to view video data, based on intelligent video analysis, to count the number of personnel in the delineated area, and when the number of personnel in the area is detected to exceed the set threshold, to automatically capture and generate an alarm, reducing the cost of human supervision.

[0004] The prior art mainly falls into two categories, as follows:

[0005] One is population statistics based on infrared thermal imaging principle, using instance segmentation technology to analyze the number of living bodies appearing in thermal imaging images. This method is only suitable for subway, shopping mall and office building entrance scenes, and the device cost is high. At the same time, in open areas, it is easy to cause missed statistics due to long distance between personnel.

[0006] The other is population statistics based on visible light imaging principle, which counts the number of personnel through neural network algorithm. This method has high accuracy, but it is difficult to adapt to different shooting angles and distances. SUMMARY

[0007] The technical task of the present application is to provide a regional population statistics method and system based on an adaptive integrated model, to solve the problem of existing technology in identifying regional population statistics in different shooting angle and distance scenes.

[0008] The technical task of the present application is achieved in the following way, a regional population statistics method based on an adaptive integrated model, which is as follows:

[0009] Obtain video data through the cameras in urban public areas, obtain images after frame extraction from the video data, and obtain the number and position information of human body targets in the images through a human body feature detection algorithm;

[0010] Determine whether a human body target appears according to the number and position information of the human body target:

[0011] If a human body target appears, calculate the average height-width ratio AR of the human body target;

[0012] If no human target appears, a face feature detection algorithm is called to obtain face skeleton position information;

[0013] The human target aspect ratio AR is compared with a threshold value AR th

[0014] When AR < AR th , the corresponding urban public area camera is a high-altitude overhead camera, and a head feature detection algorithm is called to obtain head position information;

[0015] When AR > AR th , the corresponding urban public area camera is not a high-altitude overhead camera, and the average target size

[0016] According to the average target size , a shooting distance quantization coefficient DR is calculated; the specific cases are as follows:

[0017] When DR < DR min , it is determined to be close-range shooting, and a face feature detection algorithm and a skeleton key point detection algorithm are called to obtain face skeleton position information and human skeleton position information;

[0018] When DR min < DR < DR max , it is determined to be medium-range shooting, and a skeleton key point detection algorithm and a human feature detection algorithm are called to obtain human skeleton position information and human feature position information;

[0019] When DR > DR max , it is determined to be long-range shooting, and a human feature detection algorithm and a head feature detection algorithm are called to obtain human feature position information and head position information;

[0020] Wherein, DR min and DR max are shooting distance thresholds, and the optimal values according to historical experience are DR min = 0.5 and DR max = 2, which can be adjusted according to the actual situation of the data set;

[0021] According to all the head position information, human feature position information, human skeleton position information and face skeleton position information, and based on a weighted non-maximum suppression algorithm, regional population information and position information are obtained.

[0022] As a preferred, the calculation formula of the human target average aspect ratio AR is as follows:

[0023]

[0024] Wherein, N represents the number of human targets identified; X​B1,i ,X B2,i ,Y B1,i ,Y B2,i represents the four coordinate positions of the i-th human body target;

[0025] average target size The calculation formula is as follows:

[0026]

[0027] wherein N represents the number of human body targets identified; X B1,i ,X B2,i ,Y B1,i ,Y B2,i represents the four coordinate positions of the i-th human body target; W represents the image width; and H represents the image height;

[0028] The calculation formula of the shooting distance quantization coefficient DR is as follows:

[0029]

[0030] wherein S st represents the standard value of the target size, which is set by the user or calculated based on all human body targets in the data set as S st .

[0031] More preferably, the head feature detection algorithm adopts the YOLO series algorithm in the single-stage target detection, and the model is trained through the head feature detection algorithm training data set. After training, the obtained model detects the two-dimensional coordinates (X H1,i ,X H2,i ,Y H1,i ,Y H2,i ) and the confidence C Hi of all head external rectangles in the image.

[0032] When constructing the head feature detection algorithm training data set, the image data obtained by the overhead camera is adopted. The image data obtained by the overhead camera includes a plurality of human head targets.

[0033] More preferably, the face feature detection algorithm adopts the MTCNN algorithm, and the model is trained through the network open source face feature data set. After training, the obtained model detects the two-dimensional coordinates (X F1,i ,X F2,i ,Y F1,i ,Y F2,i ) and the confidence C Fi of all face external rectangles in the image.

[0034] The skeleton key point detection algorithm adopts the OpenPose algorithm, and the model is trained through the open source skeleton key point data set. After training, the model detects the two-dimensional coordinates (X S1,i , S2,i , S1,i , S2,i ) and confidence C Si of the external rectangle of all human skeletons in the image.

[0035] More preferably, the human feature detection algorithm adopts the YOLO series algorithm in the single-stage target detection, and the model is trained through the self-built human feature detection algorithm training data set. After training, the model detects the two-dimensional coordinates (X B1,i , B2,i , B1,i , B2,i ) and confidence C Bi of the external rectangle of all human targets in the image.

[0036] When constructing the human feature detection algorithm training data set, a mixed image data set with multiple shooting angles and distances is adopted, which includes head target from the top, face target from the front, and full body and half body targets of human body at various distances.

[0037] More preferably, the weighted non-maximum suppression algorithm introduces a weight parameter to correct the confidence of the target candidate frame. The corrected confidence is as follows

[0038] C' Fi =W CF ×C Fi ;

[0039] C' Si =W CS ×C Si ;

[0040] C' Bi =W CB ×C Bi ;

[0041] C' Hi =W CH ×C Hi ;

[0042] Wherein, W CF , W CS , W CB and W CH are the weight of the face feature detection algorithm, the weight of the skeleton key point detection algorithm, the weight of the human feature detection algorithm and the weight of the head feature detection algorithm, respectively.

[0043] The weight of the face feature detection algorithm W CFThe weight W of the human body feature detection algorithm CS The weight W of the human body feature detection algorithm CB The weight W of the head feature detection algorithm CH The calculation method is as follows:

[0044]

[0045]

[0046]

[0047] The specific implementation of the weighted non-maximum suppression algorithm is as follows:

[0048] (1) Construct an initialization candidate box set H, and the initialization candidate box set H contains all N candidate boxes;

[0049] (2) Construct a set M for storing the optimal box and initialize it as an empty set;

[0050] (3) Search all candidate boxes, find the candidate box m with the highest correction confidence, and move it from the initialization candidate box set H to the optimal box set M. Traverse all candidate boxes in the initialization candidate box set H, calculate the intersection ratio IoU with the candidate box m with the highest correction confidence, and judge whether it exceeds the threshold value:

[0051] If it exceeds the threshold value, it is considered that the corresponding candidate box overlaps with the candidate box m with the highest correction confidence, and the candidate box is removed from the initialization candidate box set H;

[0052] (4) Repeat step (3) until the initialization candidate box set H is empty, and the candidate boxes in the initialization candidate box set H are the final returned targets.

[0053] A regional people counting system based on an adaptive integrated model, the system comprises,

[0054] A video frame extraction module for obtaining video data through a camera in a public area of a city, obtaining an image after frame extraction from the video data, and obtaining human target quantity and position information in the image through a human feature detection algorithm;

[0055] A human target judgment module for judging whether a human target appears according to the human target quantity and position information:

[0056] If a human target appears, calculate the average height-width ratio AR of the human target;

[0057] If no human target appears, call a face feature detection algorithm to obtain face skeleton position information;

[0058] a threshold comparison module configured to compare the human target aspect ratio AR with a threshold AR th The comparison is as follows:

[0059] When AR < AR th , the corresponding camera of the urban public area is a high-altitude overhead camera, and a head feature detection algorithm is called to obtain head position information.

[0060] When AR > AR th , the corresponding camera of the urban public area is not a high-altitude overhead camera, and the average target size

[0061] A shooting distance determination module is configured to calculate a shooting distance quantization coefficient DR according to the average target size The specific conditions are as follows:

[0062] When DR < DR min , it is determined to be close-range shooting, and a face feature detection algorithm and a skeleton key point detection algorithm are called to obtain face skeleton position information and human skeleton position information.

[0063] When DR min < DR < DR max , it is determined to be medium-range shooting, and a skeleton key point detection algorithm and a human feature detection algorithm are called to obtain human skeleton position information and human feature position information.

[0064] When DR > DR max , it is determined to be long-range shooting, and a human feature detection algorithm and a head feature detection algorithm are called to obtain human feature position information and head position information.

[0065] Wherein, DR min and DR max are shooting distance thresholds, and the optimal values are DR min = 0.5 and DR max = 2, which are adjusted according to the actual situation of the data set.

[0066] A regional population and position information acquisition module is configured to obtain regional population information and position information according to all head position information, human feature position information, human skeleton position information and face skeleton position information, and based on a weighted non-maximum suppression algorithm.

[0067] As a preferred, the calculation formula of the human target average aspect ratio AR is as follows:

[0068]

[0069] Wherein, N represents the number of human targets identified; X B1,i , X B2,iY B1,i Y B2,i represents the four coordinate positions of the i-th human body target;

[0070] Average target size The calculation formula is as follows:

[0071]

[0072] Wherein, N represents the number of human body targets identified; X B1,i X B2,i Y B1,i Y B2,i represents the four coordinate positions of the i-th human body target; W represents the image width; H represents the image height;

[0073] The calculation formula of the shooting distance quantization coefficient DR is as follows:

[0074]

[0075] Wherein, S st represents the standard value of the target size, which is set by itself or calculated from all human body targets in the data set as S st .

[0076] More preferably, the head feature detection algorithm adopts the YOLO series algorithm in the single-stage target detection, and the model is trained through the head feature detection algorithm training data set. After training, the obtained model detects the two-dimensional coordinates (X H1,i X H2,i Y H1,i Y H2,i ) and confidence C Hi of all head external rectangles in the image;

[0077] When constructing the head feature detection algorithm training data set, the image data obtained by the overhead camera is adopted. The image data obtained by the overhead camera includes a plurality of human head targets;

[0078] The face feature detection algorithm adopts the MTCNN algorithm, and the model is trained through the network open source face feature data set. After training, the obtained model detects the two-dimensional coordinates (X F1,i X F2,i Y F1,i Y F2,i ) and confidence C Fi of all face external rectangles in the image;

[0079] The skeleton key point detection algorithm adopts the OpenPose algorithm, and a network open source skeleton key point data set is used for model training, and a model obtained after training detects two-dimensional coordinates (X S1,i , S2,i , Y S1,i , Y S2,i ) and confidence C Si of all human skeleton external rectangles in an image.

[0080] The human feature detection algorithm adopts the YOLO series algorithm in the single-stage target detection, and a self-built human feature detection algorithm training data set is used for model training, and a model obtained after training detects two-dimensional coordinates (X B1,i , B2,i , Y B1,i , Y B2,i ) and confidence C Bi of all human target external rectangles in an image.

[0081] When the human feature detection algorithm training data set is constructed, a mixed image data set of multiple shooting angles and distances is adopted, and the mixed image data set includes head target in a downward shot, face target in a normal shot, and human full body and half body targets at various distances.

[0082] The weighted non-maximum suppression algorithm introduces a weight parameter to correct the confidence of the target candidate frame, and the corrected confidence is as follows:

[0083] C' Fi =W CF ×C Fi ;

[0084] C' Si =W CS ×C Si ;

[0085] C' Bi =W CB ×C Bi ;

[0086] C' Hi =W CH ×C Hi ;

[0087] Wherein, W CF , W CS , W CB and W CH are the weight of the face feature detection algorithm, the weight of the skeleton key point detection algorithm, the weight of the human feature detection algorithm and the weight of the head feature detection algorithm respectively.

[0088] The weight of the face feature detection algorithm W CF, the weight W of the bone key point detection algorithm CS , the weight W of the human feature detection algorithm CB , and the weight W of the head feature detection algorithm CH The calculation method is as follows:

[0089]

[0090]

[0091]

[0092] The specific implementation of the weighted non-maximum suppression algorithm is as follows:

[0093] (1) Construct an initialization candidate box set H, and the initialization candidate box set H contains all N candidate boxes;

[0094] (2) Construct a set M for storing the optimal box and initialize it as an empty set;

[0095] (3) Search all candidate boxes, find the candidate box m with the highest correction confidence, and move it from the initialization candidate box set H to the optimal box set M. Traverse all candidate boxes in the initialization candidate box set H, calculate the intersection ratio IoU with the candidate box m with the highest correction confidence, and judge whether it exceeds the threshold value:

[0096] If it exceeds the threshold value, it is considered that the corresponding candidate box overlaps with the candidate box m with the highest correction confidence, and the candidate box is removed from the initialization candidate box set H;

[0097] (4) Repeat step (3) until the initialization candidate box set H is empty, and the candidate boxes in the initialization candidate box set H are the final returned targets.

[0098] A computer readable storage medium, the computer readable storage medium stores a computer program, the computer program can be executed by a processor to realize the area people counting method based on adaptive integrated model as above.

[0099] The area people counting method and system based on adaptive integrated model of the application have the following advantages:

[0100] (I) The human feature detection algorithm is adopted in the application, the shooting angle and shooting distance of the camera can be calculated, the model fusion is performed according to the shooting angle and distance, the integrated model obtained has better compatibility, and has more advantages in accuracy;

[0101] (II) The method of weighted non-maximum suppression is adopted in the application to integrate the model, the problem that a single model is prone to unstable performance caused by changes in shooting angle and distance is solved, and the area people counting demand in multiple scenes can be met;

[0102] (III) This invention solves the problem that a single algorithm model cannot adapt to different angles and shooting distances. By integrating models, the robustness of the system is improved. The integrated algorithm model is much better than the single model in terms of recognition accuracy. Attached Figure Description

[0103] The invention will be further described below with reference to the accompanying drawings.

[0104] Appendix Figure 1 The flowchart shows a regional population statistics method based on an adaptive ensemble model.

[0105] Appendix Figure 2 This is a structural diagram of regional population statistics based on an adaptive ensemble model. Detailed Implementation

[0106] The regional population statistics method and system based on the adaptive ensemble model of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0107] Example 1:

[0108] As attached Figure 1 As shown in the figure, this embodiment provides a regional population statistics method based on an adaptive ensemble model, which is as follows:

[0109] S1. Obtain video data through cameras in urban public areas, extract frames from the video data, and obtain the number and location information of human targets in the image through a human feature detection algorithm;

[0110] S2. Determine whether a human target has appeared based on the number and location information of human targets:

[0111] ① If a human target is detected, proceed to step S3;

[0112] ② If no human target is found, the facial feature detection algorithm is called to obtain the facial bone position information;

[0113] S3. Calculate the average aspect ratio (AR) of the human target and infer the shooting angle based on the aspect ratio (AR) of the human target.

[0114] S4. Combine the aspect ratio AR of the human target with the threshold AR. th Comparison:

[0115] When AR <AR th In this case, the corresponding camera in the urban public area is a high-altitude overhead camera, and the head feature detection algorithm is called to obtain the head position information;

[0116] When AR>AR thWhen the average target size is greater than the average target size threshold, the corresponding camera of the urban public area is not a high-altitude overhead camera, and the average target size is calculated

[0117] S5、According to the average target size The shooting distance quantization coefficient DR is calculated; the specific case is as follows:

[0118] When DR < DR min , it is determined to be close-range shooting, and the face feature detection algorithm and the skeleton key point detection algorithm are called to obtain the face skeleton position information and the human skeleton position information;

[0119] When DR min < DR < DR max , it is determined to be medium-range shooting, and the skeleton key point detection algorithm and the human feature detection algorithm are called to obtain the human skeleton position information and the human feature position information;

[0120] When DR > DR max , it is determined to be long-range shooting, and the human feature detection algorithm and the head feature detection algorithm are called to obtain the human feature position information and the head position information;

[0121] Wherein, DR min and DR max are the shooting distance thresholds, and the optimal values are DR min = 0.5 and DR max = 2, which are adjusted according to the actual situation of the data set;

[0122] S6, According to all the head position information, human feature position information, human skeleton position information and face skeleton position information, and based on the weighted non-maximum suppression algorithm, the area number information and position information are obtained.

[0123] The calculation formula of the human target average height-width ratio AR in step S3 of the embodiment is as follows:

[0124]

[0125] Wherein, N represents the number of human targets identified; X B1,i , X B2,i , Y B1,i , Y B2,i represent the four coordinate positions of the i-th human target;

[0126] The calculation formula of the average target size in step S4 of the embodiment is as follows:

[0127]

[0128] Wherein, N represents the number of human targets identified; XB1,i ,X B2,i ,Y B1,i ,Y B2,i represents four coordinate positions of the ith human body target; W represents the image width; and H represents the image height;

[0129] The shooting distance quantization coefficient DR in step S5 of the embodiment is calculated according to the following formula:

[0130]

[0131] wherein S st represents the standard value of the target size, which is set by the user or calculated based on all human body targets in the data set S st .

[0132] The head feature detection algorithm in the embodiment adopts the YOLO series algorithm in the single-stage target detection, and the model is trained through the head feature detection algorithm training data set. After training, the obtained model detects the two-dimensional coordinates (X H1,i ,X H2,i ,Y H1,i ,Y H2,i ) and the confidence C Hi of all head external rectangles in the image.

[0133] When constructing the head feature detection algorithm training data set, the image data obtained by the overhead camera is used, and the image data obtained by the overhead camera includes a plurality of human head targets.

[0134] The face feature detection algorithm in the embodiment adopts the MTCNN algorithm, and the model is trained through the network open source face feature data set. After training, the obtained model detects the two-dimensional coordinates (X F1,i ,X F2,i ,Y F1,i ,Y F2,i ) and the confidence C Fi of all face external rectangles in the image.

[0135] The skeleton key point detection algorithm in the embodiment adopts the OpenPose algorithm, and the model is trained through the network open source skeleton key point data set. After training, the obtained model detects the two-dimensional coordinates (X S1,i ,X S2,i ,Y S1,i ,Y S2,i ) and the confidence C Si of all human body skeleton external rectangles in the image.

[0136] The human feature detection algorithm in the embodiment adopts the YOLO series algorithm in the single-stage target detection, and the model is trained through a self-built human feature detection algorithm training data set. The model obtained after training detects the two-dimensional coordinates (X B1,i , B2,i , B1,i , B2,i ) and the confidence C Bi of the outer rectangle of all human targets in the image.

[0137] When constructing the human feature detection algorithm training data set, a mixed image data set with multiple shooting angles and distances is adopted, and the mixed image data set includes head target in a downward shot, face target in a normal shot, and full-body and half-body targets of human body at various distances.

[0138] The weighted non-maximum suppression algorithm in the embodiment introduces a weight parameter to correct the confidence of the target candidate box. The corrected confidence is as follows

[0139] C' Fi =W CF ×C Fi ;

[0140] C' Si =W CS ×C Si ;

[0141] C' Bi =W CB ×C Bi ;

[0142] C' Hi =W CH ×C Hi ;

[0143] Wherein, W CF , W CS , W CB and W CH are the weight of the face feature detection algorithm, the weight of the skeleton key point detection algorithm, the weight of the human feature detection algorithm, and the weight of the head feature detection algorithm, respectively.

[0144] The calculation method of the weight W CF of the face feature detection algorithm, the weight W CS of the skeleton key point detection algorithm, the weight W CB of the human feature detection algorithm, and the weight W CH of the head feature detection algorithm is as follows:

[0145]

[0146]

[0147]

[0148] The specific implementation of the weighted non-maximum suppression algorithm in this embodiment is as follows:

[0149] (1) Construct an initialization candidate box set H, and the initialization candidate box set H contains all N candidate boxes;

[0150] (2) Construct a set M for storing the optimal box and initialize it as an empty set;

[0151] (3) Search all candidate boxes, find the candidate box m with the highest correction confidence, and move it from the initialization candidate box set H to the set M for storing the optimal box. Traverse all candidate boxes in the initialization candidate box set H, calculate the intersection ratio IoU with the candidate box m with the highest correction confidence, and judge whether it exceeds the threshold value:

[0152] If it exceeds the threshold value, it is considered that the corresponding candidate box overlaps with the candidate box m with the highest correction confidence, and the candidate box is removed from the initialization candidate box set H;

[0153] (4) Repeat step (3) until the initialization candidate box set H is empty. The candidate boxes in the initialization candidate box set H are the final returned targets.

[0154] Embodiment 2:

[0155] As shown in the accompanying Figure 2 , the embodiment provides a regional people counting system based on an adaptive integrated model, which comprises,

[0156] a video frame extraction module, configured to acquire video data through a camera in a city public area, acquire an image after frame extraction from the video data, and obtain human target quantity and position information in the image through a human feature detection algorithm;

[0157] a human target judgment module, configured to judge whether a human target appears according to the human target quantity and position information:

[0158] if the human target appears, calculate a human target average height-width ratio AR;

[0159] if the human target does not appear, call a face feature detection algorithm to acquire face skeleton position information;

[0160] a threshold comparison module, configured to compare the human target height-width ratio AR with a threshold value AR th :

[0161] when AR < AR thWhen AR>AR

[0162] When AR>AR th , the corresponding camera of the urban public area is not a high-altitude overhead camera, and the average target size is calculated

[0163] The shooting distance determination module is configured to calculate a shooting distance quantization coefficient DR according to the average target size The specific case is as follows:

[0164] When DR<DR min , it is determined to be close-range shooting, and the face feature detection algorithm and the skeleton key point detection algorithm are called to obtain the face skeleton position information and the human body skeleton position information;

[0165] When DR min <DR<DR max , it is determined to be medium-range shooting, and the skeleton key point detection algorithm and the human body feature detection algorithm are called to obtain the human body skeleton position information and the human body feature position information;

[0166] When DR>DR max , it is determined to be long-range shooting, and the human body feature detection algorithm and the head feature detection algorithm are called to obtain the human body feature position information and the head position information;

[0167] Wherein, DR min and DR max are shooting distance thresholds, and the optimal values are DR min =0.5 and DR max =2, which are adjusted according to the actual situation of the data set;

[0168] The area number and position information acquisition module is configured to obtain the area number information and the position information according to all the head position information, the human body feature position information, the human body skeleton position information and the face skeleton position information, and based on the weighted non-maximum suppression algorithm.

[0169] The calculation formula of the average height-width ratio AR of the human target in the embodiment is as follows:

[0170]

[0171] Wherein, N represents the number of recognized human targets; X B1,i , X B2,i , Y B1,i , and Y B2,i represent the four coordinate positions of the i-th human target.

[0172] The average target size The calculation formula is as follows:

[0173]

[0174] Wherein, N represents the number of human body targets identified; X B1,i ,X B2,i ,Y B1,i ,Y B2,i represents the four coordinate positions of the ith human body target; W represents the image width; H represents the image height;

[0175] The calculation formula of the shooting distance quantization coefficient DR in the embodiment is as follows:

[0176]

[0177] Wherein, S st represents the standard value of the target size, which is set by itself or calculated from all human body targets in the data set S st .

[0178] The head feature detection algorithm in the embodiment adopts the YOLO series algorithm in the single-stage target detection, and the model is trained through the head feature detection algorithm training data set. After training, the obtained model detects the two-dimensional coordinates (X H1,i ,X H2,i ,Y H1,i ,Y H2,i ) and confidence C Hi of all head external rectangles in the image;

[0179] When constructing the head feature detection algorithm training data set, the image data obtained by the overhead camera is used, and the image data obtained by the overhead camera includes a plurality of human head targets;

[0180] The face feature detection algorithm in the embodiment adopts the MTCNN algorithm, and the model is trained through the network open source face feature data set. After training, the obtained model detects the two-dimensional coordinates (X F1,i ,X F2,i ,Y F1,i ,Y F2,i ) and confidence C Fi of all face external rectangles in the image;

[0181] The skeleton key point detection algorithm in the embodiment adopts the OpenPose algorithm, and the model is trained through the network open source skeleton key point data set. After training, the obtained model detects the two-dimensional coordinates (X S1,i ,X S2,i ,Y S1,i ,YS2,i ) and confidence C Si ;

[0182] The human feature detection algorithm in the embodiment adopts the YOLO series algorithm in the single-stage target detection, and the model is trained through the self-built human feature detection algorithm training data set. The model obtained after training detects the two-dimensional coordinates (X B1,i , B2,i , B1,i , B2,i ) and confidence C Bi of the outer rectangle of all human targets in the image.

[0183] When constructing the human feature detection algorithm training data set, a mixed image data set with multiple shooting angles and distances is adopted, and the mixed image data set includes head target in a downward shot, face target in a normal shot, and full body and half body targets of human body at various distances.

[0184] The weighted non-maximum suppression algorithm in the embodiment introduces a weight parameter to correct the confidence of the target candidate box. The corrected confidence is as follows:

[0185] C' Fi = W CF * C Fi ;

[0186] C' Si = W CS * C Si ;

[0187] C' Bi = W CB * C Bi ;

[0188] C' Hi = W CH * C Hi ;

[0189] Wherein, W CF , W CS , W CB and W CH are the weight of the face feature detection algorithm, the weight of the skeleton key point detection algorithm, the weight of the human feature detection algorithm, and the weight of the head feature detection algorithm, respectively.

[0190] The calculation method of the weight W CF of the face feature detection algorithm, the weight W CS of the skeleton key point detection algorithm, the weight W CB of the human feature detection algorithm, and the weight W CH of the head feature detection algorithm is as follows:

[0191]

[0192]

[0193]

[0194] The weighted non-maximum suppression algorithm in the embodiment is implemented as follows:

[0195] (1) Construct an initialization candidate box set H, and the initialization candidate box set H contains all N candidate boxes;

[0196] (2) Construct a set M for storing optimal boxes and initialize the set M as an empty set;

[0197] (3) Search all candidate boxes, find a candidate box m with the highest correction confidence, and move the candidate box m from the initialization candidate box set H to the set M for storing optimal boxes. Traverse all candidate boxes in the initialization candidate box set H, calculate the intersection over union IoU with the candidate box m, and determine whether the IoU exceeds a threshold value:

[0198] If the IoU exceeds the threshold value, it is considered that the corresponding candidate box overlaps with the candidate box m with the highest correction confidence, and the candidate box is removed from the initialization candidate box set H;

[0199] (4) Repeat step (3) until the initialization candidate box set H is empty. The candidate boxes in the initialization candidate box set H are the final returned targets.

[0200] Embodiment 3

[0201] The embodiment also provides a computer readable storage medium, in which a plurality of instructions are stored. The instructions are loaded by a processor to make the processor execute the region people counting method based on the adaptive integrated model in any embodiment of the application. Specifically, a system or device provided with a storage medium can be provided. The storage medium stores software program codes for implementing the functions of any one of the above embodiments, and the computer (or CPU or MPU) of the system or device reads and executes the program codes stored in the storage medium.

[0202] In this case, the program codes read from the storage medium can implement the functions of any one of the above embodiments, and thus the program codes and the storage medium storing the program codes constitute a part of the application.

[0203] The storage medium for providing the program code includes floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), magnetic tapes, nonvolatile memory cards and ROM. Alternatively, the program code can be downloaded from a server computer through a communication network.

[0204] In addition, it should be clear that not only the program code read by the computer can be executed, but also the operating system and the like operating on the computer can be caused to perform part or all of the actual operations based on the instructions of the program code, thereby realizing the functions of any one of the above embodiments.

[0205] In addition, it should be understood that the program code read by the storage medium is written into the memory provided in the expansion board inserted into the computer or the memory provided in the expansion unit connected to the computer, and then the CPU and the like installed on the expansion board or the expansion unit are caused to perform part or all of the actual operations based on the instructions of the program code, thereby realizing the functions of any one of the above embodiments.

[0206] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can still be modified, or part or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for counting the number of people in a region based on an adaptive ensemble model, characterized by, The method specifically comprises the following steps: Obtain video data through a camera in a public area of a city, obtain images after frame extraction from the video data, and obtain the number and position information of human targets in the images through a human feature detection algorithm; Determine whether a human target appears according to the number and position information of the human target: If a human target appears, calculate the average height-width ratio AR of the human target; If no human target appears, call a face feature detection algorithm to obtain face skeleton position information; comparing the human target aspect ratio AR to a threshold AR th compare: When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR When AR > AR th then the corresponding city public area camera is not a high-altitude overhead camera, and the average target size is calculated According to the average target size A shooting distance quantization coefficient DR is calculated; the details are as follows: When DR < DR min When DR < DR min When DR < DR min When DR < DR min When DR < DR min When DR < DR min When DR < DR min When DR < DR min When DR < DR min When DR < DR min When DR < DR min When DR min <DR<DR max is less than 1.5, it is determined to be a medium distance shooting, and a skeleton key point detection algorithm and a human feature detection algorithm are called to obtain human skeleton position information and human feature position information. When DR > DR max If so, it is determined to be a long-distance shooting, and a human feature detection algorithm and a head feature detection algorithm are called to obtain human feature position information and head position information. Wherein, DR min and DR max are the shooting distance threshold values, the optimal values are DR min = 0.5 and DR max = 2, which are adjusted according to the actual situation of the data set. Obtain regional population information and position information according to all head position information, human feature position information, human skeleton position information and face skeleton position information, and based on a weighted non-maximum suppression algorithm; The head feature detection algorithm employs the YOLO series of algorithms in single-stage object detection. The model is trained using a training dataset from the head feature detection algorithm. The trained model then detects the two-dimensional coordinates (X, Y, X) of the bounding rectangles of all heads in the image. H1,i ,X H2,i ,Y H1,i ,Y H2,i ) and confidence level C Hi ; When constructing a training data set of the head feature detection algorithm, image data obtained by a high-altitude camera is used, and the image data obtained by the high-altitude camera includes a plurality of human head targets; The face feature detection algorithm adopts MTCNN algorithm, and a model is trained through a network open-source face feature data set. After the training, the model detects all the two-dimensional coordinates (X F1,i , F2,i , F1,i , F2,i ) and confidence C Fi of the outer rectangle of the face in the image. The skeleton key point detection algorithm adopts the OpenPose algorithm, and a model is trained through an open source skeleton key point data set. After training, the model detects the two-dimensional coordinates (X S1,i , S2,i , S1,i , S2,i ) and confidence C Si of all human skeleton external rectangles in an image. The human feature detection algorithm adopts a YOLO series algorithm in a single-stage target detection. A model is trained through a self-built human feature detection algorithm training data set. After the training, the model detects two-dimensional coordinates (X B1,i , B2,i Y B1,i , B2,i ) and a confidence C Bi of an outer rectangle of all human targets in an image. When constructing a training data set of the human feature detection algorithm, a mixed image data set of multiple shooting angles and distances is used, and the mixed image data set includes head targets taken from a low angle, face targets taken from a front angle, and full-body and half-body targets of a human body taken from various distances; The weighted non-maximum suppression algorithm introduces a weight parameter to correct the confidence of the target candidate box, and the correction of the confidence is specifically as follows C′ Fi = W CF x C Fi ; C′ Si = W CS x C Si ; C′ Bi = W CB x C Bi ; C' Hi = W CH x C Hi ; wherein W CF , W CS , W CB and W CH are the weights of the face feature detection algorithm, the skeleton key point detection algorithm, the human feature detection algorithm and the head feature detection algorithm, respectively. The weight W of the face feature detection algorithm CF The weight W of the skeleton key point detection algorithm CS The weight W of the human feature detection algorithm CB The weight W of the head feature detection algorithm CH is calculated as follows: The specific implementation of the weighted non-maximum suppression algorithm is as follows: (1) Construct an initialization candidate box set H, and the initialization candidate box set H contains all N candidate boxes; (2) Construct a set M for storing optimal boxes and initialize the set M as an empty set; (3) Search all candidate boxes, find a candidate box m with the highest corrected confidence, and move the candidate box m from the initialization candidate box set H to the set M for storing optimal boxes; traverse all candidate boxes in the initialization candidate box set H, calculate the intersection-over-union IoU with the candidate box m with the highest corrected confidence, and determine whether the IoU exceeds a threshold value: If the threshold value is exceeded, it is considered that the corresponding candidate box overlaps with the candidate box m with the highest corrected confidence, and the candidate box is removed from the initialization candidate box set H; (4) Repeat step (3) until the initialization candidate box set H is empty, and the candidate boxes in the initialization candidate box set H are the final returned targets.

2. The adaptive ensemble model based regional people counting method of claim 1, wherein, The calculation formula of the average height-width ratio AR of the human target is as follows: wherein N represents the number of human object identified; X B1,i ,X B2,i ,Y B1,i ,Y B2,i represents the four coordinate positions of the i-th human object; Average target size The formula for the calculation is as follows: wherein N represents the number of human object identified; X B1,i ,X B2,i ,Y B1,i ,Y B2,i represents the four coordinate positions of the i-th human object; W represents the image width; and H represents the image height. The calculation formula of the distance quantization coefficient DR is as follows: where S st represents a standard value of the target size, and is set by the user or calculated from all human targets in the data set S st .

3. A system for counting people in a region based on an adaptive ensemble model, characterized by, The system comprises, a video frame extraction module, configured to obtain video data through a camera in a public area of a city, obtain images after frame extraction from the video data, and obtain the number and position information of human targets in the images through a human feature detection algorithm; a human target judgment module, configured to determine whether a human target appears according to the number and position information of the human target: If a human target appears, calculate the average height-width ratio AR of the human target; If no human target appears, call a face feature detection algorithm to obtain face skeleton position information; a threshold comparison module to compare the human target aspect ratio AR to a threshold AR th comparison is made: When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR th When AR < AR When AR > AR th then the corresponding city public area camera is not a high-altitude overhead camera, and the average target size is calculated The shooting distance determination module is configured to determine the shooting distance according to the average target size S obj The shooting distance quantization coefficient DR is calculated, and the specific case is as follows: When DR < DR min If so, it is determined that the photographing is close-up, and a face feature detection algorithm and a skeleton key point detection algorithm are called to obtain face skeleton position information and human skeleton position information. When DR min <DR<DR max is less than 1.5, it is determined to be a medium distance shooting, and a skeletal key point detection algorithm and a human feature detection algorithm are called to obtain human skeletal position information and human feature position information. When DR > DR max If so, it is determined that the photograph is taken at a long distance, and a human body feature detection algorithm and a head feature detection algorithm are called to obtain human body feature position information and head position information. DR min and DR max are the shooting distance threshold values, the optimal values are DR min = 0.5, DR max = 2, adjusted according to the actual situation of the data set; The regional population and position information acquisition module is configured to acquire regional population information and position information according to all head position information, human feature position information, human skeleton position information and face skeleton position information, and based on a weighted non-maximum suppression algorithm; wherein the head feature detection algorithm adopts a YOLO series algorithm in a single-stage target detection, a model is trained through a head feature detection algorithm training data set, and a model obtained after training detects two-dimensional coordinates (X H1,i , X H2,i , Y H1,i , Y H2,i ) and a confidence C Hi of all head external rectangles in an image. When constructing a training data set of the head feature detection algorithm, image data obtained by a high-altitude camera is used, and the image data obtained by the high-altitude camera includes a plurality of human head targets; The face feature detection algorithm adopts MTCNN algorithm, and a model is trained through a network open-source face feature data set. After training, the obtained model detects two-dimensional coordinates (X F1,i , X F2,i , Y F1,i , Y F2,i ) and confidence C Fi of all face bounding rectangles in an image. The skeleton key point detection algorithm adopts the OpenPose algorithm, and a model is trained through an open source skeleton key point data set. S1,i S2,i S1,i S2,i Si ;​​​​ The human feature detection algorithm adopts a YOLO series algorithm in a single-stage target detection, a model is trained through a self-built human feature detection algorithm training data set, and a two-dimensional coordinate (X B1,i , X B2,i , Y B1,i , Y B2,i ) and a confidence C Bi of an outer rectangle of all human targets in an image are detected by the trained model. In constructing the human feature detection algorithm training data set, a mixed image data set of multiple shooting angles and distances is adopted, and the mixed image data set includes head target shot from above, face target shot from front, and full body and half body targets of human body at various distances; The weighted non-maximum suppression algorithm introduces a weight parameter to correct the confidence of the target candidate box, and the corrected confidence is as follows: C′ Fi = W CF x C Fi ; C′ Si = W CS x C Si ; C' Bi = W CB x C Bi ; C′ Hi = W CH x C Hi ; wherein W CF , W CS , W CB and W CH are the weights of the face feature detection algorithm, the skeleton key point detection algorithm, the human feature detection algorithm and the head feature detection algorithm, respectively. The weight W of the face feature detection algorithm CF The weight W of the skeleton key point detection algorithm CS The weight W of the human feature detection algorithm CB The weight W of the head feature detection algorithm CH is calculated as follows: The specific implementation of the weighted non-maximum suppression algorithm is as follows: (1) Construct an initialization candidate box set H, and the initialization candidate box set H contains all N candidate boxes; (2) Construct a set M for storing the optimal box and initialize the set M as an empty set; (3) Search all candidate boxes, find a candidate box m with the highest corrected confidence, and move the candidate box m from the initialization candidate box set H to the set M for storing the optimal box; traverse all candidate boxes in the initialization candidate box set H, calculate the intersection ratio IoU of the candidate boxes with the candidate box m with the highest corrected confidence, and judge whether the intersection ratio IoU exceeds a threshold value: If the threshold value is exceeded, it is considered that the corresponding candidate box overlaps with the candidate box m with the highest corrected confidence, and the candidate box is removed from the initialization candidate box set H; (4) Repeat step (3) until the initialization candidate box set H is empty, and the candidate boxes in the initialization candidate box set H are the final returned targets.

4. The adaptive ensemble model based regional people counting system of claim 3, wherein, The calculation formula of the average height-width ratio AR of the human body target is as follows: wherein N represents the number of human object identified; X B1,i ,X B2,i ,Y B1,i ,Y B2,i represents the four coordinate positions of the i-th human object; Average target size The formula for the calculation is as follows: wherein N represents the number of human object identified; X B1,i ,X B2,i ,Y B1,i ,Y B2,i represents the four coordinate positions of the i-th human object; W represents the image width; and H represents the image height. The calculation formula of the shooting distance quantization coefficient DR is as follows: where S st represents a standard value of the target size, and is set by the user or calculated from all human targets in the data set S st .

5. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program can be executed by the processor to implement the region people counting method based on the adaptive integrated model according to claim 1 or 2.

Images