A database building method and device based on pedestrian detection, equipment and storage medium

Abstract

The application discloses a kind of based on pedestrian detection's building library method, device, equipment and storage medium, the method includes: obtaining the monitoring video of target camera, and according to preset frame number from the monitoring video intercepts multiple detection picture frames;In turn from each detection picture frame determines pedestrian detection frame, and all detection picture frames in the same pedestrian of pedestrian detection frame clustering is a pedestrian group;Wherein, the pedestrian detection frame is used to represent the actual position of pedestrian in detection picture frame;It is judged whether there is the pedestrian file matched with the pedestrian group in the preset pedestrian library;Wherein, each pedestrian file in the pedestrian library is obtained based on the monitoring video of other camera;If yes, then the pedestrian group is stored in the matched pedestrian file, if no, then the pedestrian group is stored in the pedestrian file newly built in the pedestrian library;The application can be based on the detection of automatic building library to pedestrian.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a database building method and device based on pedestrian detection, equipment and storage medium. BACKGROUND

[0002] In a construction site, a monitoring device needs to be installed to manage the pedestrians entering and leaving the construction site. The identity information of the pedestrians can be tracked and identified using the monitoring device to establish a pedestrian database containing the identity information of the pedestrians. The most commonly used method is to track and identify the facial feature information of the pedestrians through the function of face recognition. However, in an actual construction site, the monitoring device is installed at a high position, and the pixel of the imaged face of the pedestrian is small, so that the face recognition cannot be accurately performed. In addition, due to the influence of light, the color of the face of the pedestrian changes greatly, so that the face image of the pedestrian cannot be accurately identified, and thus the images of the same pedestrian are distributed in multiple different pedestrian files in the pedestrian database.

[0003] Therefore, how to accurately track and identify the pedestrians and establish a pure pedestrian file for each pedestrian in the pedestrian database has become a technical problem to be solved by those skilled in the art. SUMMARY

[0004] The present application aims to provide a database building method and device based on pedestrian detection, equipment and storage medium, which can accurately identify pedestrians and efficiently establish a pedestrian database.

[0005] According to one aspect of the present application, a database building method based on pedestrian detection is provided, which comprises:

[0006] acquiring a monitoring video of a target camera, and extracting a plurality of detection picture frames from the monitoring video according to a preset frame number;

[0007] determining a pedestrian detection frame from each detection picture frame in sequence, and clustering the pedestrian detection frames belonging to the same pedestrian in all detection picture frames into a pedestrian group; wherein the pedestrian detection frame is used to represent the actual position of the pedestrian in the detection picture frame;

[0008] judging whether there is a pedestrian file matched with the pedestrian group in a preset pedestrian database; wherein each pedestrian file in the pedestrian database is obtained based on the monitoring video of other cameras;

[0009] if yes, storing the pedestrian group into the matched pedestrian file, and if no, storing the pedestrian group into a newly created pedestrian file in the pedestrian database.

[0010] Optionally, the pedestrian detection frame is determined from each detection picture frame in sequence, and the pedestrian detection frames belonging to the same pedestrian in all detection picture frames are clustered into a pedestrian group, which comprises:

[0011] obtaining two adjacent detection image frames from all the intercepted detection image frames, and determining monitoring time of the two adjacent detection image frames;

[0012] determining a first detection box for identifying a pedestrian from a detection image frame with earlier monitoring time among the two adjacent detection image frames, and performing pedestrian prediction based on the first detection box to obtain a prediction box; wherein the prediction box is used to represent a predicted position of a pedestrian appearing in a previous image frame of the target image frame in the target image frame;

[0013] determining a second detection box from a detection image frame with later monitoring time among the two adjacent detection image frames;

[0014] calculating a cosine distance of the prediction box and the second detection box, and obtaining a cost matrix based on the cosine distance;

[0015] inputting the cost matrix into a Hungarian matching algorithm to obtain a matched prediction box and a second detection box;

[0016] calculating whether a center distance of the matched prediction box and the second detection box is less than a preset distance, and if yes, confirming that the first detection box and the second detection box are the same target pedestrian and adding the first detection box and the second detection box to the same pedestrian group.

[0017] Optionally, the judging whether there is a pedestrian file matched with the pedestrian group in a preset pedestrian library comprises:

[0018] calculating an average feature vector of the pedestrian group according to feature vectors of pedestrian detection boxes in the pedestrian group;

[0019] judging whether there is a comprehensive feature vector matched with the average feature vector of the pedestrian group in the pedestrian library;

[0020] if yes, there is a pedestrian file matched with the pedestrian group in the pedestrian library, and if no, creating a pedestrian file for the pedestrian group.

[0021] Optionally, the judging whether there is a comprehensive feature vector matched with the average feature vector of the pedestrian group in the pedestrian library comprises:

[0022] when a cosine distance of the average feature vector and the comprehensive feature vector is greater than or equal to a preset distance threshold, obtaining a pedestrian file corresponding to the comprehensive feature vector, and adding a pedestrian ID corresponding to the pedestrian file in the pedestrian detection box of the pedestrian group;

[0023] create a pedestrian file and a pedestrian ID for the pedestrian group when a cosine distance between the average feature vector and the comprehensive feature vector is less than a preset distance threshold, and add the pedestrian ID in a pedestrian detection frame of the pedestrian group.

[0024] Optionally, the storing the pedestrian group into the matched pedestrian file comprises:

[0025] storing the pedestrian detection frame and the feature vector of the pedestrian detection frame in the pedestrian group into the matched pedestrian file;

[0026] calculating an average value of the feature vectors of all pedestrian detection frames in the pedestrian file, and taking the calculation result as a comprehensive feature vector of the pedestrian file.

[0027] Optionally, after the storing the pedestrian group into the matched pedestrian file, the method further comprises:

[0028] determining a monitoring time of the pedestrian detection frame when monitoring the storing of the pedestrian detection frame into the pedestrian file;

[0029] obtaining a monitoring video corresponding to the pedestrian detection frame and determining a target camera and a target camera position corresponding to the monitoring video data;

[0030] adding the monitoring time, the target camera and the target camera position of the pedestrian detection frame into the pedestrian file.

[0031] In order to achieve the above object, the application further provides a database building device based on pedestrian detection, which comprises:

[0032] a cutting module, configured to obtain a monitoring video of a target camera, and cut a plurality of detection picture frames from the monitoring video according to a preset frame number;

[0033] a clustering module, configured to determine pedestrian detection frames from each detection picture frame in sequence, and cluster pedestrian detection frames belonging to the same pedestrian in all detection picture frames into a pedestrian group; wherein the pedestrian detection frame is used to represent an actual position of a pedestrian in the detection picture frame;

[0034] a judging module, configured to judge whether there is a pedestrian file matched with the pedestrian group in a preset pedestrian library; wherein each pedestrian file in the pedestrian library is obtained based on a monitoring video of other cameras;

[0035] a storage module, configured to store the pedestrian group into the matched pedestrian file if yes, and store the pedestrian group into a newly created pedestrian file in the pedestrian library if no.

[0036] Optionally, the device further comprises:

[0037] a frame taking module, configured to store the pedestrian detection frame and the feature vector of the pedestrian detection frame in the matched pedestrian file;

[0038] a calculating module, configured to calculate the average value of the feature vectors of all the pedestrian detection frames in the pedestrian file, and take the calculation result as the comprehensive feature vector of the pedestrian file.

[0039] In order to achieve the above-mentioned purpose, the present application further provides a computer device, which specifically comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the pedestrian detection-based library building method introduced above when executing the computer program.

[0040] In order to achieve the above-mentioned purpose, the present application further provides a computer storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the pedestrian detection-based library building method introduced above.

[0041] The pedestrian detection-based library building method, device, equipment and storage medium provided by the present application detect the pedestrian detection frame in the intercepted monitoring video, cluster all the pedestrian detection frames of the same pedestrian to obtain a pedestrian group, match the pedestrian group with the pedestrian library, and store the pedestrian group in the pedestrian file of the pedestrian library according to the matching result, so that the pedestrian detection algorithm provided by the present application does not need to identify the facial features of the pedestrian, but only needs to determine the same pedestrian in the monitoring video through pedestrian prediction and pedestrian identification, thereby eliminating the influence of light and environment on the pedestrian face recognition, reducing the identification difficulty, overcoming the problems of disordered pedestrian trajectory and overlapped pedestrian detection frame, improving the identification accuracy and efficiency, and further comparing and updating the identification result with the pedestrian library in real time to ensure the purity of the pedestrian file in the pedestrian library. BRIEF DESCRIPTION OF DRAWINGS

[0042] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the application. In the drawings:

[0043] Figure 1 An optional flowchart of the pedestrian detection-based library building provided for embodiment one;

[0044] Figure 2 A schematic diagram of a pedestrian file provided for embodiment one;

[0045] Figure 3 A schematic diagram of the pedestrian detection frame stored in the pedestrian file provided for embodiment one;

[0046] Figure 4 An effect diagram of the pedestrian detection provided for the embodiment one;

[0047] Figure 5 An optional component structure schematic diagram of the database building based on the pedestrian detection provided for the embodiment three;

[0048] Figure 6 An optional hardware architecture schematic diagram of the computer device provided for the embodiment four. DETAILED DESCRIPTION

[0049] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0050] Embodiment one

[0051] The embodiment of the present application provides a database building method based on pedestrian detection, as shown in the figure, the method specifically comprises the following steps: Figure 1

[0052] Step S101: acquiring a monitoring video of a target camera, and cutting a plurality of detection picture frames from the monitoring video according to a preset frame number.

[0053] Preferably, the preset frame number is 12 frames.

[0054] In actual construction sites, there are a large number of cameras used to monitor the pedestrians entering and leaving the construction site in real time. In order to facilitate the management of pedestrians, it is necessary to detect the pedestrians in the monitoring video captured by the camera to identify the identity of the pedestrians, and to establish a pedestrian database based on the detection result. On the one hand, the effect of pedestrian detection can be improved by establishing the pedestrian database, and on the other hand, the pedestrian features can be improved by constantly storing the pedestrian pictures in the pedestrian database to achieve more accurate identification of pedestrians.

[0055] ​In the embodiment, the conventional pedestrian recognition is frame-by-frame decomposition of the monitoring video, obtaining the pedestrian detection frame of each frame, and using Kalman filtering algorithm to predict the pedestrian track of the pedestrian detection frame. However, in actual application, due to the flexible action and variable action track of the pedestrian, the monitoring video is intercepted according to the preset frame number in the embodiment, which is different from the conventional frame-by-frame pedestrian detection method. The detection picture frame obtained by the interception method can not only avoid the overlapping of the pedestrian pictures of multiple pedestrians in adjacent detection picture frames to affect the pedestrian detection result, but also can save computer resources, improve the calculation speed and detection efficiency without affecting the accuracy of the pedestrian detection.

[0056] Step S102: determining the pedestrian detection frame from each detection picture frame in sequence, and clustering the pedestrian detection frames belonging to the same pedestrian in all detection picture frames into a pedestrian group; wherein the pedestrian detection frame is used to represent the actual position of the pedestrian in the detection picture frame.

[0057] The pedestrian detection frame is obtained by target detection algorithm for target detection of the detection picture frame. The pedestrian detection frame frames the actual position of each pedestrian in the detection picture frame. In addition, the size of the pedestrian detection frame is determined by the actual size of the pedestrian in the detection picture frame.

[0058] Since the motion track of the pedestrian in the monitoring video is continuous, the same pedestrian appears in multiple detection picture frames of a segment of the monitoring video. Clustering the pedestrian detection frames of the same pedestrian in different detection picture frames into a pedestrian group can obtain multiple pedestrian pictures representing different actions and postures of the pedestrian.

[0059] Specifically, the step S102 includes the following steps:

[0060] Step A1: obtaining two adjacent detection picture frames from all the detection picture frames, and determining the monitoring time of the two adjacent detection picture frames.

[0061] Step A2: determining the first detection frame for identifying the pedestrian from the detection picture frame with earlier monitoring time in the two adjacent detection picture frames, and performing pedestrian prediction based on the first detection frame to obtain a prediction frame; wherein the prediction frame is used to represent the predicted position of the pedestrian appearing in the previous image frame of the target image frame in the target image frame.

[0062] The first detection frame is obtained by detecting each pedestrian in the detection picture frame with a motion target detector in a Deepsort multi-target tracking algorithm.

[0063] Step A3: determining a second detection frame from the detection picture frame with a later monitoring time.

[0064] The second detection frame is obtained by detecting each pedestrian in the detection picture frame with a motion target detector in a Deepsort multi-target tracking algorithm.

[0065] Step A4: calculating a cosine distance between the predicted frame and the second detection frame, and obtaining a cost matrix based on the cosine distance.

[0066] The cosine distance is used to represent the similarity of the pedestrian features between the predicted frame and the second detection frame. Since there can be multiple pedestrians in a detection picture frame, i.e., multiple first detection frames, a predicted frame is obtained for each first detection frame through the Kalman filtering algorithm. Therefore, the cosine distance between any predicted frame and any second detection frame is calculated to obtain the cost matrix.

[0067] Step A5: inputting the cost matrix into a Hungarian matching algorithm to obtain the matched predicted frame and second detection frame.

[0068] The Hungarian matching algorithm can match the cost matrix generated according to the cosine distance to match the predicted frame to the second detection frame, so as to preliminarily identify the pedestrians in the matched predicted frame and second detection frame as the same pedestrian.

[0069] Step A6: calculating whether the center distance between the matched predicted frame and second detection frame is less than a preset distance, and if so, confirming that the first detection frame and the second detection frame are the same target pedestrian and adding the first detection frame and the second detection frame to the same pedestrian group.

[0070] Preferably, the preset distance is 50 pixels.

[0071] The center distance is the pixel distance between the predicted bounding box and the detection bounding box in the target image frame. Adding the first and second detection bounding boxes to the same pedestrian group means adding the pedestrian images bounded by the first and second detection bounding boxes to the same pedestrian group. After processing the acquired detection image frames using the algorithm in steps A1-A5, the output results are post-processed, i.e., the center distance is calculated. This further confirms that the pedestrian images bounded by the first and second detection bounding boxes with a center distance that is at a preset distance are the same pedestrians, improving the accuracy of pedestrian re-identification and detection, and providing rich pedestrian material for subsequently storing the pedestrian images corresponding to the detection bounding boxes in the pedestrian database.

[0072] In this embodiment, the original Deepsort multi-object tracking algorithm is improved. It eliminates the need to calculate Mahalanobis distance for the predicted bounding boxes obtained from Kalman filtering. Since the surveillance video is captured according to a preset frame rate, the motion trajectory of the same pedestrian in two detection image frames spaced 12 frames apart no longer conforms to a Gaussian distribution. Therefore, the process of calculating the Mahalanobis distance of the predicted bounding boxes is eliminated. Only the cosine distance between the predicted bounding box and the second detection bounding box needs to be calculated. The cost matrix is ​​then generated using the cosine distance and input into the Hungarian matching algorithm to obtain matching predicted and second detection bounding boxes. This simplifies the matching process for pedestrian detection bounding boxes of the same pedestrian in different detection image frames and saves computer resources. Simultaneously, logic for calculating the center distance between the predicted and second detection bounding boxes is added, making the detection results for different pedestrian detection bounding boxes of the same pedestrian more accurate.

[0073] Step S103: Determine whether there is a pedestrian file matching the pedestrian group in the preset pedestrian database; wherein, each pedestrian file in the pedestrian database is obtained based on the surveillance video of other cameras.

[0074] Among them, such as Figure 2 As shown, each pedestrian file in the pedestrian database corresponds to one pedestrian, such as... Figure 3 As shown, the pedestrian file stores images of the pedestrian in different poses and positions captured from surveillance videos of various cameras.

[0075] Specifically, step S103 includes the following steps:

[0076] Step B1: Calculate the average feature vector of the pedestrian group based on the feature vectors of the pedestrian detection boxes in the pedestrian group.

[0077] Specifically, by aggregating all pedestrian detection boxes obtained from pedestrian detection for the same pedestrian, and calculating the mean of the feature vectors of all pedestrian detection boxes, an average feature vector representing the pedestrian feature information in the detection image frame captured from the surveillance video is obtained.

[0078] Step B2: judging whether there is a comprehensive feature vector matching the average feature vector of the pedestrian group in the pedestrian library.

[0079] The comprehensive feature vector is obtained by calculating the average of the feature vectors of all pedestrian pictures in a pedestrian file. The comprehensive feature vector is used to represent the feature information of a pedestrian corresponding to the pedestrian file. The more the number of pedestrian pictures in the pedestrian file and the richer the pedestrian posture in the pedestrian pictures, the more accurately the calculated comprehensive feature vector can represent the pedestrian feature information of a pedestrian.

[0080] Further, the step B2 includes the following steps:

[0081] Step B21: when the cosine distance between the average feature vector and the comprehensive feature vector is greater than or equal to a preset distance threshold, obtaining the pedestrian file corresponding to the comprehensive feature vector, and adding the pedestrian ID corresponding to the pedestrian file in the pedestrian detection frame of the pedestrian group.

[0082] Wherein, one pedestrian corresponds to one pedestrian file, and one pedestrian file corresponds to a unique pedestrian ID. By calculating the cosine distance between the average feature vector and the comprehensive feature vector, it is judged whether there is a comprehensive feature vector similar to the average feature vector obtained by clustering the pedestrian group in the pedestrian library, that is, whether the pedestrian corresponding to the pedestrian group is stored in the pedestrian library.

[0083] It should be noted that, as shown in Figure 4 Adding pedestrian ID to pedestrian detection frame is to add pedestrian ID in the pedestrian detection frame of the pedestrian group, and also need to add pedestrian ID in the pedestrian detection frame of the detection picture frame.

[0084] Step B22: when the cosine distance between the average feature vector and the comprehensive feature vector is less than the preset distance threshold, creating a pedestrian file and a pedestrian ID for the pedestrian group, and adding the pedestrian ID in the pedestrian detection frame of the pedestrian group.

[0085] When the pedestrian corresponding to the pedestrian group is not stored in the pedestrian library, the value of the largest pedestrian ID in the pedestrian library is added by 1 to obtain the pedestrian ID of the pedestrian group, and the pedestrian file corresponding to the pedestrian ID is created to avoid the repetition of pedestrian ID.

[0086] Step B3: if yes, there is a pedestrian file matching the pedestrian group in the pedestrian library, if no, creating a pedestrian file for the pedestrian group.

[0087] Further, the step B3 includes the following steps:

[0088] Step B31: Store the pedestrian detection boxes and feature vectors of the pedestrian detection boxes in the pedestrian group into the matching pedestrian file.

[0089] Specifically, the system searches for the corresponding pedestrian file in the pedestrian database based on the pedestrian ID of the pedestrian detection box in the pedestrian group. It should be noted that, in this embodiment, storing the pedestrian detection boxes in the pedestrian group to the matching pedestrian file is equivalent to storing the pedestrian image defined by the detection box.

[0090] Step B32: Calculate the average value of the feature vectors of all pedestrian detection boxes in the pedestrian file, and use the calculation result as the comprehensive feature vector of the pedestrian file.

[0091] The comprehensive feature vector is used to represent the feature information of a pedestrian corresponding to a pedestrian file. The more pedestrian images there are in the pedestrian file and the richer the pedestrian poses in the images, the more accurately the calculated comprehensive feature vector can represent the pedestrian feature information of a pedestrian.

[0092] Step S104: If yes, store the pedestrian group in the matching pedestrian file; otherwise, store the pedestrian group in a newly created pedestrian file in the pedestrian database.

[0093] Specifically, storing the pedestrian group into the matching pedestrian file involves storing the pedestrian images captured by the pedestrian detection boxes in the pedestrian group, along with the feature vectors of those images, together into the pedestrian file.

[0094] In this embodiment, by continuously detecting pedestrians in the surveillance videos from each camera and storing the pedestrian images captured by the detection boxes into pedestrian files in the pedestrian database, the number of pedestrian images for each pedestrian in the database becomes increasingly rich. This leads to a more accurate calculation of the comprehensive feature vector for each pedestrian file, making the determination of whether a pedestrian file corresponding to a pedestrian group exists in the database more precise. Furthermore, the detection and judgment logic provided in this embodiment ensures the purity of each pedestrian file in the pedestrian database, greatly avoiding the situation where different detection boxes for a single pedestrian are stored in different pedestrian files.

[0095] Specifically, step S104 includes the following steps:

[0096] Step C1: When it is detected that the pedestrian detection box is stored in the pedestrian file, determine the monitoring time of the pedestrian detection box corresponding to the pedestrian detection box.

[0097] The monitoring time refers to the monitoring time displayed in the detection image frame to which the pedestrian detection box belongs.

[0098] Step C2: Obtain the monitoring video corresponding to the pedestrian detection frame and determine the target camera and target camera position corresponding to the monitoring video data.

[0099] Step C3: Add the association of the monitoring time of the pedestrian detection frame, the target camera and the target camera position to the pedestrian file.

[0100] In the embodiment, by detecting the pedestrian detection frame in the intercepted monitoring video, clustering all pedestrian detection frames of the same pedestrian to obtain a pedestrian group, and matching the pedestrian group with the pedestrian library, and storing the pedestrian group into the pedestrian file of the pedestrian library according to the matching result, the pedestrian detection algorithm provided by the application does not need to identify the facial features of the pedestrian, and only determines the same pedestrian in the monitoring video through pedestrian prediction and pedestrian identification, which eliminates the influence of light and environment on pedestrian face recognition, reduces the identification difficulty, overcomes the problem of disordered pedestrian trajectory and overlapping pedestrian detection frame, improves the identification accuracy and identification efficiency, and updates the identification result with the pedestrian library in real time, ensures the purity of the pedestrian file in the pedestrian library. In addition, by obtaining the monitoring time, the target camera and the target camera position corresponding to the pedestrian detection frame for storage, it can help to track and trace the behavior of the pedestrian, improve the safety protection, and facilitate the management of the personnel in the construction site.

[0101] Embodiment two

[0102] The embodiment of the application further provides a library building method based on pedestrian detection, which specifically comprises the following steps:

[0103] Step S1: Obtain the monitoring video of the target camera, and intercept a plurality of detection picture frames from the monitoring video according to a preset frame number.

[0104] Step S2: Obtain two adjacent detection picture frames from all the intercepted detection picture frames, and determine the monitoring time of the two adjacent detection picture frames.

[0105] Step S3: Determine a first detection frame for identifying a pedestrian from a detection picture frame earlier in monitoring time among the two adjacent detection picture frames, and perform pedestrian prediction based on the first detection frame to obtain a prediction frame; wherein the prediction frame is used to represent the predicted position of the pedestrian appearing in a previous image frame of the target image frame in the target image frame.

[0106] Step S4: Determine a second detection frame from a detection picture frame later in monitoring time among the two adjacent detection picture frames.

[0107] Step S5: Calculate the cosine distance of the prediction frame and the second detection frame, and obtain a cost matrix based on the cosine distance.

[0108] Step S6: inputting the cost matrix into a Hungarian matching algorithm to obtain matched prediction boxes and second detection boxes.

[0109] Step S7: calculating whether a center distance of the matched prediction boxes and second detection boxes is less than a preset distance, and if yes, confirming that the first detection box and the second detection box are the same target pedestrian and adding the first detection box and the second detection box to the same pedestrian group.

[0110] Repeating steps S2 to S7 for all detection picture frames until all clustering completed pedestrian groups are obtained.

[0111] Step S8: calculating an average feature vector of the pedestrian group according to feature vectors of pedestrian detection boxes in the pedestrian group.

[0112] Step S9: judging whether there is a comprehensive feature vector matching the average feature vector of the pedestrian group in a pedestrian library; if yes, there is a pedestrian file matching the pedestrian group in the pedestrian library, and if no, creating a pedestrian file for the pedestrian group.

[0113] Step S901: when a cosine distance between the average feature vector and the comprehensive feature vector is greater than or equal to a preset distance threshold, obtaining a pedestrian file corresponding to the comprehensive feature vector, and adding a pedestrian ID corresponding to the pedestrian file in the pedestrian detection box of the pedestrian group.

[0114] Step S902: when the cosine distance between the average feature vector and the comprehensive feature vector is less than the preset distance threshold, creating a pedestrian file and a pedestrian ID for the pedestrian group, and adding the pedestrian ID in the pedestrian detection box of the pedestrian group.

[0115] Step S10: storing the pedestrian detection box in the pedestrian group and the feature vector of the pedestrian detection box in the matched pedestrian file.

[0116] Step S11: calculating an average value of the feature vectors of all pedestrian detection boxes in the pedestrian file, and taking the calculation result as a comprehensive feature vector of the pedestrian file.

[0117] Step S12: when monitoring the pedestrian detection box stored in the pedestrian file, determining a monitoring time of the pedestrian detection box.

[0118] Step S13: obtaining a monitoring video corresponding to the pedestrian detection box and determining a target camera and a target camera position corresponding to the monitoring video data.

[0119] Step S14: adding the monitoring time, the target camera and the target camera position associated with the pedestrian detection box to the pedestrian file.

[0120] In the embodiment, by detecting the pedestrian detection frame in the intercepted monitoring video, clustering all pedestrian detection frames of the same pedestrian to obtain a pedestrian group, and matching the pedestrian group with the pedestrian library, and storing the pedestrian group into the pedestrian file of the pedestrian library according to the matching result, the pedestrian detection algorithm provided by the application can determine the same pedestrian in the monitoring video only through pedestrian prediction and pedestrian recognition without recognizing the facial features of the pedestrian, eliminate the influence of light and environment on the pedestrian face recognition, reduce the recognition difficulty, overcome the problem of disordered pedestrian trajectory and overlapping pedestrian detection frame, improve the recognition accuracy and recognition efficiency, and update the recognition result with the pedestrian library in real time to ensure the purity of the pedestrian file in the pedestrian library. In addition, by storing the monitoring time corresponding to the pedestrian picture, the target camera and the target camera position together, the tracking and tracing of the pedestrian behavior can be facilitated, the safety prevention can be improved, and the management of personnel in the construction site can be facilitated.

[0121] Embodiment three

[0122] The embodiment of the application provides a library building device based on pedestrian detection, as shown in the figure, the device specifically comprises the following components: Figure 5

[0123] The intercepting module 501 is used for acquiring the monitoring video of the target camera, and intercepting a plurality of detection picture frames from the monitoring video according to a preset frame number;

[0124] The clustering module 502 is used for sequentially determining the pedestrian detection frame from each detection picture frame, and clustering the pedestrian detection frames belonging to the same pedestrian in all detection picture frames into a pedestrian group; wherein the pedestrian detection frame is used to represent the actual position of the pedestrian in the detection picture frame;

[0125] The judgment module 503 is used for judging whether there is a pedestrian file matched with the pedestrian group in the preset pedestrian library; wherein each pedestrian file in the pedestrian library is obtained based on the monitoring video of other cameras;

[0126] The storage module 504 is used for storing the pedestrian group into the matched pedestrian file if yes, and storing the pedestrian group into the newly created pedestrian file in the pedestrian library if no.

[0127] The clustering module 502 is used for:

[0128] The adjacent two detection picture frames are acquired from all intercepted detection picture frames, and the monitoring time of the adjacent two detection picture frames is determined;

[0129] ​determine a first detection box for identifying a pedestrian from a detection picture frame earlier in time among the two adjacent detection picture frames, and perform pedestrian prediction based on the first detection box to obtain a prediction box; wherein the prediction box is used to represent a predicted position of a pedestrian appearing in a previous image frame of the target image frame in the target image frame;

[0130] determine a second detection box from a detection picture frame later in time among the two adjacent detection picture frames;

[0131] calculate a cosine distance of the prediction box and the second detection box, and obtain a cost matrix based on the cosine distance;

[0132] input the cost matrix into a Hungarian matching algorithm to obtain a matched prediction box and second detection box;

[0133] calculate whether a center distance of the matched prediction box and second detection box is less than a preset distance, and if yes, confirm that the first detection box and the second detection box are the same target pedestrian and add the first detection box and the second detection box to the same pedestrian group.

[0134] Specifically, the judging module 503 comprises:

[0135] a calculating unit configured to calculate an average feature vector of the pedestrian group according to feature vectors of pedestrian detection boxes in the pedestrian group;

[0136] a judging unit configured to judge whether there is a comprehensive feature vector matching the average feature vector of the pedestrian group in a pedestrian library;

[0137] a matching unit configured to, if yes, there is a pedestrian file matching the pedestrian group in the pedestrian library, and if no, create a pedestrian file for the pedestrian group.

[0138] Further, the judging unit is configured to:

[0139] when a cosine distance of the average feature vector and the comprehensive feature vector is greater than or equal to a preset distance threshold, obtain a pedestrian file corresponding to the comprehensive feature vector, and add a pedestrian ID corresponding to the pedestrian file in pedestrian detection boxes of the pedestrian group;

[0140] when the cosine distance of the average feature vector and the comprehensive feature vector is less than the preset distance threshold, create a pedestrian file and a pedestrian ID for the pedestrian group, and add the pedestrian ID in the pedestrian detection boxes of the pedestrian group.

[0141] Still further, the matching unit is configured to:

[0142] store the pedestrian detection frame in the pedestrian group and the feature vector of the pedestrian detection frame into a matched pedestrian file;

[0143] calculate the average of the feature vectors of all pedestrian detection frames in the pedestrian file, and take the result as the comprehensive feature vector of the pedestrian file.

[0144] Further, the storage module 504 is configured to:

[0145] when the pedestrian detection frame is stored in the pedestrian file, determine the monitoring time of the pedestrian detection frame corresponding to the pedestrian detection frame;

[0146] obtain the monitoring video corresponding to the pedestrian detection frame and determine the target camera and the target camera position corresponding to the monitoring video data;

[0147] add the monitoring time of the pedestrian detection frame, the target camera and the target camera position corresponding to the pedestrian detection frame into the pedestrian file.

[0148] Embodiment Four

[0149] The embodiment also provides a computer device, such as a smartphone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a cabinet server (including a single server or a server cluster composed of multiple servers), which can execute programs. As shown in the figure, the computer device 60 of the embodiment at least includes but is not limited to a memory 601 and a processor 602 which can be connected to each other through a system bus. It should be noted that, Figure 6 only the computer device 60 with components 601-602 is shown, but it should be understood that all the shown components are not required to be implemented, and more or less components can be alternatively implemented. Figure 6

[0150] ​The memory 601 (i.e., a storage medium) in this embodiment includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 601 can be an internal storage unit of the computer device 60, such as a hard disk or a memory of the computer device 60. In other embodiments, the memory 601 can also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 60. Of course, the memory 601 can also include both the internal storage unit and the external storage device of the computer device 60. In this embodiment, the memory 601 is generally used to store an operating system and various application software installed on the computer device 60. In addition, the memory 601 can also be used to temporarily store various data that has been output or will be output.

[0151] The processor 602 in some embodiments can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other human-detection-based library building chip. The processor 602 is generally used to control the overall operation of the computer device 60.

[0152] Specifically, in this embodiment, the processor 602 is configured to execute a human-detection-based library building method program stored in the memory 601, and the human-detection-based library building method program, when executed, implements the following steps:

[0153] obtaining a monitoring video of a target camera, and extracting a plurality of detection picture frames from the monitoring video according to a preset frame number;

[0154] determining a human detection box from each detection picture frame in sequence, and clustering human detection boxes belonging to the same human in all detection picture frames into a human group; wherein the human detection box is used to represent the actual position of the human in the detection picture frame;

[0155] determining whether there is a human file matched with the human group in a preset human library; wherein each human file in the human library is obtained based on the monitoring video of another camera;

[0156] if yes, storing the human group into the matched human file, and if no, storing the human group into a newly created human file in the human library.

[0157] For a detailed description of the above method steps, please refer to Example 1. This example will not be repeated here.

[0158] Example 5

[0159] This embodiment also provides a computer storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, app store, etc., which stores a computer program. When the computer program is executed by a processor, it implements the following method steps:

[0160] Acquire the surveillance video from the target camera, and extract multiple detection image frames from the surveillance video according to a preset frame rate;

[0161] Pedestrian detection boxes are sequentially determined from each detected image frame, and pedestrian detection boxes belonging to the same pedestrian in all detected image frames are clustered into a pedestrian group; wherein, the pedestrian detection box is used to represent the actual position of the pedestrian in the detected image frame;

[0162] Determine whether there is a pedestrian file matching the pedestrian group in a preset pedestrian database; wherein, each pedestrian file in the pedestrian database is obtained based on surveillance video from other cameras;

[0163] If yes, the pedestrian group is stored in the matching pedestrian file; otherwise, the pedestrian group is stored in a newly created pedestrian file in the pedestrian database.

[0164] For a detailed description of the above method steps, please refer to Example 1. This example will not be repeated here.

[0165] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0166] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0167] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and the necessary general hardware platform, of course, can also be realized by hardware, but in many cases, the former is a better embodiment.

[0168] The above are only preferred embodiments of the present application, and do not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for building a database based on pedestrian detection, characterized in that, The method comprises: acquiring a monitoring video of a target camera, and extracting a plurality of detection picture frames from the monitoring video according to a preset frame number; determining a pedestrian detection box from each detection picture frame in turn, and clustering pedestrian detection boxes belonging to the same pedestrian in all detection picture frames into a pedestrian group; wherein the pedestrian detection box is used to represent the actual position of the pedestrian in the detection picture frame; judging whether there is a pedestrian file matched with the pedestrian group in a preset pedestrian library; wherein each pedestrian file in the pedestrian library is obtained based on the monitoring video of other cameras; if yes, storing the pedestrian group into the matched pedestrian file, and if no, storing the pedestrian group into a newly created pedestrian file in the pedestrian library; wherein the determination of the pedestrian detection box from each detection picture frame in turn and the clustering of the pedestrian detection boxes belonging to the same pedestrian in all detection picture frames into a pedestrian group comprises: acquiring two adjacent detection picture frames from all extracted detection picture frames; determining a first detection box for identifying a pedestrian from a detection picture frame earlier in time among the two adjacent detection picture frames, and performing pedestrian prediction based on the first detection box to obtain a prediction box; wherein the prediction box is used to represent the predicted position of the pedestrian appearing in a previous image frame of a target image frame in the target image frame; determining a second detection box from a detection picture frame later in time among the two adjacent detection picture frames; calculating the cosine distance of the prediction box and the second detection box, and obtaining a cost matrix based on the cosine distance; inputting the cost matrix into a Hungarian matching algorithm to obtain matched prediction box and second detection box; calculating whether the center distance of the matched prediction box and second detection box is less than a preset distance, if yes, confirming that the first detection box and the second detection box are the same target pedestrian and adding the first detection box and the second detection box to the same pedestrian group; the judgment of whether there is a pedestrian file matched with the pedestrian group in a preset pedestrian library comprises: calculating the average feature vector of the pedestrian group according to the feature vector of the pedestrian detection box in the pedestrian group; judging whether there is a comprehensive feature vector matched with the average feature vector of the pedestrian group in the pedestrian library; if yes, the pedestrian library has a pedestrian file matched with the pedestrian group, and if no, creating a pedestrian file for the pedestrian group; the storage of the pedestrian group into the matched pedestrian file comprises: storing the pedestrian detection box in the pedestrian group and the feature vector of the pedestrian detection box into the matched pedestrian file; calculating the average value of the feature vectors of all pedestrian detection boxes in the pedestrian file, and taking the calculation result as the comprehensive feature vector of the pedestrian file.

2. The method for building a database based on pedestrian detection according to claim 1, wherein, the judgment of whether there is a comprehensive feature vector matched with the average feature vector of the pedestrian group in the pedestrian library comprises: When the cosine distance between the average feature vector and the comprehensive feature vector is greater than or equal to a preset distance threshold, a pedestrian file corresponding to the comprehensive feature vector is obtained, and a pedestrian ID corresponding to the pedestrian file is added to a pedestrian detection frame of the pedestrian group; When the cosine distance between the average feature vector and the comprehensive feature vector is less than the preset distance threshold, a pedestrian file and a pedestrian ID are created for the pedestrian group, and the pedestrian ID is added to the pedestrian detection frame of the pedestrian group. 3.The method of claim 1, wherein, After the pedestrian group is stored in the matched pedestrian file, the following steps are included: When it is monitored that the pedestrian detection frame is stored in the pedestrian file, a monitoring time of the pedestrian detection frame is determined; A monitoring video corresponding to the pedestrian detection frame is obtained, and a target camera and a target camera position corresponding to monitoring video data are determined; The monitoring time, the target camera, and the target camera position of the pedestrian detection frame are associated and added to the pedestrian file.

4. A device for building a database based on pedestrian detection, characterized by The device includes: An intercepting module is configured to obtain a monitoring video of a target camera, and intercept a plurality of detection picture frames from the monitoring video according to a preset frame number; A clustering module is configured to sequentially determine pedestrian detection frames from each detection picture frame, and cluster pedestrian detection frames belonging to the same pedestrian in all detection picture frames into a pedestrian group; wherein the pedestrian detection frame is used to represent the actual position of the pedestrian in the detection picture frame; A judging module is configured to judge whether there is a pedestrian file matched with the pedestrian group in a preset pedestrian library; wherein each pedestrian file in the pedestrian library is obtained based on the monitoring video of other cameras; A storage module is configured to, if yes, store the pedestrian group in the matched pedestrian file, and if no, store the pedestrian group in a newly created pedestrian file in the pedestrian library; The clustering module is configured to: Obtain two adjacent detection picture frames from all intercepted detection picture frames; Determine a first detection frame for identifying a pedestrian from a detection picture frame with an earlier monitoring time among the two adjacent detection picture frames, and perform pedestrian prediction based on the first detection frame to obtain a prediction frame; wherein the prediction frame is used to represent the predicted position of the pedestrian appearing in a previous image frame in a target image frame; Determine a second detection frame from a detection picture frame with a later monitoring time among the two adjacent detection picture frames; Calculate the cosine distance between the prediction frame and the second detection frame, and obtain a cost matrix based on the cosine distance; Input the cost matrix into a Hungarian matching algorithm to obtain matched prediction frame and second detection frame; Calculate whether the center distance between the matched prediction frame and the second detection frame is less than a preset distance, if yes, confirm that the first detection frame and the second detection frame are the same target pedestrian, and add the first detection frame and the second detection frame to the same pedestrian group; The judging module is configured to: Calculate an average feature vector of the pedestrian group according to the feature vectors of the pedestrian detection frames in the pedestrian group; Judge whether there is a comprehensive feature vector matched with the average feature vector of the pedestrian group in the pedestrian library; If yes, there is a pedestrian file in the pedestrian library matching the pedestrian group, and if no, a pedestrian file is created for the pedestrian group; The device further comprises: a bounding box module, configured to store the pedestrian bounding boxes in the pedestrian group and the feature vectors of the pedestrian bounding boxes into the matching pedestrian file; a calculation module, configured to calculate the average of the feature vectors of all the pedestrian bounding boxes in the pedestrian file, and take the calculation result as the comprehensive feature vector of the pedestrian file.

5. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method in any one of claims 1 to 3 when executing the computer program.

6. A computer storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 3.

Images