Lurker detection method, apparatus and electronic device

Abstract

The application provides a loitering person detection method and device and electronic equipment. The application obtains a scene picture, extracts behavior features of each target in the scene picture, establishes a tracking chain for each target in the scene picture, tracks the target, saves the behavior features of the corresponding target in the tracking chain, performs occlusion matching, hierarchical matching and time sequence matching on the tracked target according to the behavior features, and identifies loitering persons in the scene picture according to the matching result. The application tracks nearby persons in batches based on various factors such as personnel features and personnel trends under the lens, time, position direction and loitering time of personnel disappearance under the lens, can guarantee superior matching ability of the personnel after disappearance or substantial occlusion, greatly weakens tracking errors caused by similar clothing or personnel occlusion, and can efficiently and accurately prewarn loitering persons.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision image processing, and in particular to a loitering person detection method and device and electronic equipment. BACKGROUND

[0002] The demand and protection of security work in places such as banks and jewelry stores are higher than in other scenes, in recent years, with the rapid development of intelligent cameras and artificial intelligence, security work has been converted from traditional security personnel monitoring, setting up roadblocks and other methods to artificial intelligence security, which can further determine the loitering time of the surrounding crowd by covering the surrounding environment information of the security shooting place, thereby saving the image of the loitering person and doing a good job in advance security work.

[0003] At present, the main methods for detecting personnel loitering in important places include detecting personnel multiple times in and out of the scene, and detecting loitering personnel in a low-people scene.

[0004] The personnel multiple times in and out of the scene algorithm mainly establishes a detection frame set of personnel by detecting personnel, judges whether the detection frame is constantly approaching the target scene, and if the number of approaches is greater than the determination threshold, the target is determined to be a loitering person. Since the crowd is dense, occlusion occurs, the detection frame is unstable, and the target person cannot be locked with high accuracy.

[0005] The algorithm for detecting loitering personnel in a low-people scene mainly uses feature extraction to match personnel, but the scene used is a low-people scene, and once occlusion or personnel concentration occurs, tracking errors, delayed tracking, and other situations occur, which do not have high stability. SUMMARY

[0006] The present application aims to provide a loitering person detection method, device and electronic equipment to solve the limitations of conventional deep learning target detection algorithms that can only identify known categories and the poor generalization of traditional machine learning background modeling methods.

[0007] In a first aspect, the present application provides a loitering person detection method, comprising:

[0008] Obtaining a scene picture;

[0009] Extracting the behavior features of each target in the scene picture;

[0010] Establishing a tracking chain for each target in the scene picture, tracking the target, and saving the behavior features of the corresponding target in the tracking chain;

[0011] In sequence, the tracking chain and the detection frame of the target are matched according to three matching mechanisms of occlusion matching, hierarchical matching and time sequence matching, to continuously dynamically track the target;

[0012] The occlusion matching comprises: calculating an occlusion rate of the currently tracked target and other targets in the scene picture respectively; filtering the detection frame of the occluded target when the occlusion rate is greater than an occlusion rate threshold; calculating a tracking threshold for the detection frame when the occluded target exposes the entire detection frame again; calculating a cosine similarity between a current behavior feature of the target and a last behavior feature in a tracking chain corresponding to the target; matching the detection frame with a cosine similarity less than the tracking threshold to the tracking chain of the target, and continuing to track the target;

[0013] The hierarchical matching comprises: hierarchizing the targets from high to low according to a length of a stay time of the target in the scene, and assigning a tracking chain with a higher level a more preferential tracking order and a tracking threshold with a wider range according to the hierarchical condition;

[0014] The time sequence matching comprises: recording a last time and a coordinate position of a tracking chain of a target whose stay time in the scene exceeds a time threshold; taking the position where the tracking chain disappears as a starting point, determining a matching range and a matching threshold according to a time when the tracking chain disappears, matching the detection frame in the range to the tracking chain; after a time interval, continuing to expand the matching range according to the disappearance time, and increasing the matching threshold simultaneously until a target detection frame that appears again after disappearing is matched or all tracking chains are traversed.

[0015] According to the matching result, the lingering personnel in the scene picture are identified.

[0016] Further, in the step of extracting the behavior feature of each target in the scene picture, the behavior feature comprises a target position, a target appearance time and a target feature.

[0017] Further, in the step of sequentially matching the tracking chain of the target to the detection frame according to the three matching mechanisms of occlusion matching, hierarchical matching and time sequence matching to continuously track the target dynamically, the occlusion matching specifically comprises:

[0018] The occlusion rate of the currently tracked target and other targets in the scene picture is calculated according to the following formula respectively: cover= (Area_AB) / (Area_A + Area_B); in the formula, cover represents the occlusion rate, Area_AB represents an overlapping area of the targets A and B, Area_A represents an area of the target A, and Area_B represents an area of the target B;

[0019] It is judged whether a maximum value of the occlusion rate is greater than a dynamic threshold;

[0020] If it is judged that the maximum value of the occlusion rate is greater than a dynamic threshold, the area of the detection frame of the current target is compared with the area of the detection frame of the target corresponding to the maximum value of the occlusion rate, and the detection frame with a smaller area is filtered out;

[0021] According to the current system time and the time of the last behavior feature record of the tracking chain, the occlusion time is calculated and obtained;

[0022] After the occlusion time elapses, when the occluded target exposes the entire detection frame again, the behavior feature of the target is extracted, and a tracking threshold value is calculated for the detection frame according to the following formula: cos_cover=D_default+(D_max-D_default)×min(T,Tmax) / Tmax, wherein the tracking threshold value is cos_cover, the default tracking threshold value D_default=0.25, the maximum tracking threshold value D_max=0.35, and the maximum occlusion time Tmax;

[0023] The cosine similarity between the current behavior feature of the target and the last behavior feature of the target in the tracking chain is calculated;

[0024] The detection frame with a cosine similarity less than the tracking threshold value is matched with the tracking chain of the target, and the tracking of the target is continued.

[0025] Further, in the step of sequentially matching the tracking chain of the target with the detection frame according to the occlusion matching, the hierarchical matching, and the time sequence matching to continuously track the target dynamically, the hierarchical matching specifically includes:

[0026] According to the length of the stay time of the target in the scene, the target is hierarchically classified from high to low, and the original loitering person is given a higher tracking level for subsequent matching;

[0027] When other persons appear in the scene, new tracking chains are generated, the occlusion occurs between the other persons and the original loitering person, according to the hierarchical situation, the tracking chain with a higher level is given a more preferential tracking order and a tracking threshold value with a wider range, the original loitering person is matched to the detection frame in advance and becomes a tracked state, and the other persons match the remaining detection frames and become tracked states;

[0028] After the matching is completed, the state is updated according to the stay time of the person, and the tracking level of the original loitering person and the other persons is updated.

[0029] Further, in the step of sequentially matching the tracking chain of the target with the detection frame according to the occlusion matching, the hierarchical matching, and the time sequence matching to continuously track the target dynamically, the time sequence matching specifically includes:

[0030] a higher time sequence matching level is given to a target whose staying duration exceeds a preset time threshold;

[0031] The matching range and the matching threshold are determined according to the time of disappearance of the tracking chain, and the bounding box in the range is matched with the tracking chain; the time matching range is according to the formula: r = min(50, min(T, 6) * 50), wherein r is the matching range, and T is the time of disappearance; the matching threshold calculation formula is: if (distance < r) cos_time = cos_person / r*distance, wherein cos_person is the artificial threshold, 0.05 when T is less than 3, and 0.1 when T is greater than 3, r is the matching range, distance is the distance between the target and the origin of the orange target, if, distance < r, indicates that the target must be in the matching range, and the matching threshold gradually decreases with the increase of the distance of the center point;

[0032] In the case of no successful matching, after a period of time, the matching range is expanded according to the time of disappearance, and the matching threshold is increased at the same time, and the occluded bounding box is continuously searched until the target bounding box that appears again after disappearance is matched or all tracking chains are traversed.

[0033] Further, according to the matching result, the loitering person in the scene picture is identified, including:

[0034] The loitering time of different targets in the scene is obtained according to the matching result;

[0035] The loitering time is sorted from large to small;

[0036] It is judged whether the maximum value of the loitering time is greater than a potential loitering person threshold value;

[0037] If the maximum value of the loitering time is not greater than the potential loitering person threshold value, there is no potential loitering person;

[0038] If the maximum value of the loitering time is greater than the potential loitering person threshold value, it is judged whether the loitering time of each target is greater than a time threshold value of the temporary picture;

[0039] If the loitering time of the target is greater than the time threshold value of the temporary picture, the picture of the target is temporarily stored in the tracking chain;

[0040] It is judged whether the loitering time of the target is greater than a loitering person threshold value;

[0041] If the loitering time of the target is greater than the loitering person threshold value, the picture of the target is stored in the tracking chain for a long time;

[0042] if the loitering time of the target is less than a loitering person threshold, determining whether the loitering time of the target is greater than a loitering person deletion threshold;

[0043] then deleting the picture of the target temporarily stored in the tracking chain.

[0044] In a second aspect, the present application provides a loitering person detection device, comprising:

[0045] an acquisition unit configured to acquire a scene picture;

[0046] an extraction unit configured to extract a behavior feature of each target in the scene picture;

[0047] a tracking unit configured to establish a tracking chain for each target in the scene picture, and track the target, wherein the tracking chain stores the behavior feature of the corresponding target;

[0048] a matching unit configured to match the tracking chain of the target with a detection frame in sequence according to three matching mechanisms, i.e., occlusion matching, hierarchical matching and time sequence matching, so as to continuously track the target dynamically;

[0049] wherein the occlusion matching comprises: calculating an occlusion rate of the currently tracked target and other targets in the scene picture respectively; filtering the detection frame of the occluded target when the occlusion rate is greater than an occlusion rate threshold; calculating a tracking threshold for the detection frame when the occluded target exposes the entire detection frame again; calculating a cosine similarity between the current behavior feature of the target and the last behavior feature in the tracking chain of the target; matching the detection frame with the tracking chain of the target when the cosine similarity is less than the tracking threshold, and continuing to track the target;

[0050] the hierarchical matching comprises: classifying the targets from high to low according to the length of the stay time of the target in the scene, and assigning the tracking chain with a higher level a more preferential tracking order and a tracking threshold with a wider range according to the classification;

[0051] the time sequence matching comprises: for a target with a stay time in the scene exceeding a time threshold, recording the last time and coordinate position of the tracking chain of the target; taking the position where the tracking chain disappears as the origin, determining a matching range and a matching threshold according to the time when the tracking chain disappears, matching the detection frame in the range with the tracking chain; after a period of time, continuing to expand the matching range according to the disappearance time, and increasing the matching threshold at the same time, until the target detection frame that appears again after disappearing is matched or all tracking chains are traversed;

[0052] a recognition unit configured to recognize a loitering person in the scene picture according to the matching result.

[0053] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to perform the steps of the above method.

[0054] The present application has the following beneficial effects: the loitering person detection method, device and electronic device provided by the present application acquire a scene picture, extract behavior features of each target in the scene picture, establish a tracking chain for each target in the scene picture, track the target, save the behavior features of the corresponding target in the tracking chain, perform occlusion matching, hierarchical matching and time sequence matching on the tracked target according to the behavior features, and identify the loitering person in the scene picture according to the matching result; the present application performs hierarchical and batch tracking on nearby persons based on multiple factors such as the features and movement of the persons under the lens, the time, position direction and loitering time of the persons disappearing under the lens, which can ensure that the persons have more superior matching ability after disappearing or being largely occluded, greatly weaken the tracking errors caused by similar clothing or person occlusion, and efficiently and accurately perform early warning on the loitering person. BRIEF DESCRIPTION OF DRAWINGS

[0055] For a further understanding of the features and technical contents of the present application, please refer to the following detailed description of the present application and the accompanying drawings, which are provided for reference and illustration only, and are not used to limit the present application.

[0056] In the drawings,

[0057] Figure 1 The flowchart of the loitering person detection method of the present application;

[0058] Figure 2 The schematic diagram of a dense crowd scene;

[0059] Figure 3 The schematic diagram of a filtering mechanism under an occlusion condition;

[0060] Figure 4 The schematic diagram of a matching mechanism under an occlusion condition;

[0061] Figure 5 The schematic diagram of a hierarchical mechanism under a loitering condition;

[0062] Figure 6 The schematic diagram of a hierarchical matching mechanism under a loitering condition;

[0063] Figure 7 The schematic diagram of loitering person tracking level updating;

[0064] Figure 8 The schematic diagram of loitering person time sequence matching screening;

[0065] Figure 9Low level timing matching schematic diagram;

[0066] Figure 10 High level timing matching schematic diagram;

[0067] Figure 11 Wanderer detection device block diagram of the present application;

[0068] Figure 12 Electronic device block diagram of the present application. DETAILED DESCRIPTION

[0069] In order to further clarify the technical means adopted by the present application and its effects, the following will describe in detail the technical solutions in the preferred embodiments of the present application and their accompanying drawings.

[0070] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0071] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application. In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0072] In this application, the word "exemplary" is used to mean "serving as an example, instance, or illustration." Any implementation described as "exemplary" in this application is not necessarily to be construed as preferred or advantageous over other implementations. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, for purposes of explanation, specific details are set forth. It is apparent to those skilled in the art that the present application can be practiced without using these specific details. In other instances, well-known structures and processes are not elaborated upon in order not to obscure the description of the present application with unnecessary details. Thus, the present application is not intended to be limited by the embodiments shown, but is to be accorded with the widest scope consistent with the principles and features disclosed.

[0073] Currently, the scene suitable for wandering detection is, for example, a bank entrance, a community, a jewelry store, a place with a large number of floating population, and a situation in which people block each other and have similar dress. It is difficult to rely on manual video screening or rely on the appearance features of personnel to track suspicious personnel, and it is impossible to detect and warn the surrounding environment in real time.

[0074] In the field of deep learning, based on personnel features and personnel time sequence positions, a new method for real-time detection of wandering personnel is proposed and applied to different density crowds. Real-time images in a detection area are collected by a test scene overhead camera, deep learning is used to extract human behavior features, and it is further determined whether the personnel continuously or intermittently appear in the area for a long time, and finally the information of the wandering personnel is fed back.

[0075] Referring to Figure 1 The embodiment of the present application provides a wandering personnel detection method, which comprises the following steps.

[0076] S101, a scene picture is acquired.

[0077] Specifically, the scene picture can be a scene picture collected by a monitoring video of a place with a large number of floating population, such as a bank entrance, a community, a jewelry store.

[0078] S102, behavior features of each target in the scene picture are extracted.

[0079] Specifically, the behavior features include target positions, target appearance times, and target features.

[0080] S103, a tracking chain of each target in the scene picture is established, and the target is tracked, and the behavior features of the corresponding target are saved in the tracking chain.

[0081] S104, the tracking chain of the target and the detection frame are matched in sequence according to three matching mechanisms of occlusion matching, hierarchical matching, and time sequence matching, so as to continuously dynamically track the target.

[0082] The occlusion matching comprises: calculating an occlusion rate of a current tracked target and other targets in a scene picture respectively; filtering a detection box of an occluded target when the occlusion rate is greater than an occlusion rate threshold; calculating a tracking threshold for the detection box when the occluded target exposes the whole detection box again; calculating a cosine similarity between a current behavior feature of the target and a last behavior feature in a tracking chain corresponding to the target; and matching the detection box with the cosine similarity less than the tracking threshold to the tracking chain of the target to continue tracking the target.

[0083] In the embodiment, the occlusion rate of the current tracked target and other targets in the scene picture is calculated respectively; it is judged whether a maximum value of the occlusion rate is greater than a dynamic threshold; if it is judged that the maximum value of the occlusion rate is greater than the dynamic threshold, the area of the detection box of the current target is compared with the area of the detection box of the target corresponding to the maximum value of the occlusion rate, and the detection box with a smaller area is filtered out; an occlusion time is calculated according to the current system time and the time recorded by the last behavior feature of the tracking chain; after the occlusion time elapses, when the occluded target exposes the whole detection box again, the behavior feature of the target is extracted, and a tracking threshold for the detection box is calculated; a cosine similarity between a current behavior feature of the target and a last behavior feature in a tracking chain corresponding to the target is calculated; and the detection box with the cosine similarity less than the tracking threshold is matched to the tracking chain of the target to continue tracking the target.

[0084] The hierarchical matching comprises: hierarchically classifying the targets from high to low according to the length of the staying time of the targets in the scene, and assigning the tracking chain with a higher level a more preferential tracking order and a tracking threshold with a wider range according to the hierarchical situation.

[0085] In the embodiment, the targets are hierarchically classified from high to low according to the length of the staying time of the targets in the scene, and the original loitering person is assigned a higher tracking level for subsequent matching; when other persons appear in the scene, new tracking chains are generated, and the occlusion occurs between the other persons and the original loitering person; the tracking chain with a higher level is assigned a more preferential tracking order and a tracking threshold with a wider range according to the hierarchical situation, the original loitering person is matched to the detection box in advance and becomes a tracked state, the other persons are matched to the remaining detection boxes and become tracked states, and the tracking level of the original loitering person and the other persons is updated according to the staying time of the persons after the matching is completed.

[0086] The time sequence matching comprises: for a target whose stay time in the scene exceeds a time threshold, recording the last time and coordinate position of the tracking chain of the target; taking the position where the tracking chain disappears as a starting point, determining a matching range and a matching threshold according to the time when the tracking chain disappears, and matching the detection frame in the range with the tracking chain; after a time interval, continuing to expand the matching range according to the disappearance time, and meanwhile increasing the matching threshold, until a target detection frame that appears again after disappearing is matched or all tracking chains are traversed.

[0087] In the embodiment, a target whose stay time exceeds a preset time threshold is given a higher time sequence matching level; a matching range and a matching threshold are determined according to the time when the tracking chain disappears, and the detection frame in the range is matched with the tracking chain; in the case of no successful matching, after a time interval, the matching range is continued to be expanded according to the disappearance time, and meanwhile the matching threshold is increased, and the occluded detection frame is continued to be searched, until a target detection frame that appears again after disappearing is matched or all tracking chains are traversed.

[0088] The present application takes dynamic tracking as the main research direction, and the main difficulty is that the feature representation of dense crowds and sparse crowds is different, a large number of personnel occlusions exist in dense crowds, once the detection frames of two people coincide, the extracted features of the two people also contain each other, the feature representation is low, and it is difficult to track the correct personnel. The present application proposes three matching mechanisms of occlusion matching, hierarchical matching and time sequence matching to resist the feature disorder of dense crowds.

[0089] The occlusion matching is specifically as follows: the outdoor scene can be divided into a simple scene and a complex scene, in the simple scene, targets are sparse and mutual occlusion is less, a single target feature is clear and stable; in the complex scene, multiple targets are covered, the occlusion condition is serious, and target features are mixed with each other and difficult to track.

[0090] Please refer to Figures 2 to 4 , the occlusion rate between the detection frames is calculated mainly based on the number of target detection in the scene and the position of the detection frame in the figure, when the number of detection frames is greater than a certain value, such as Figure 2 (a) shown, or the occlusion condition is more, such as Figure 2 (b) shown, the scene is determined as a complex scene, otherwise, it is a simple scene. For each person in the scene, feature extraction and tracking are performed, and a corresponding tracking chain is established, and the features of the corresponding target are saved in each tracking chain. For example, Figure 2 In (a), 9 tracking chains are generated for 9 targets respectively, and the features of the corresponding target are saved in each tracking chain, after the next picture enters, the features of the target are extracted first, and then the cosine distance calculation is performed on the features in each tracking chain, and the formula is: x1, x2 are feature values in the tracking chain and the feature values of the current target respectively.

[0091] For dense crowd situations, first calculate the dynamic threshold according to the current number of people and the occlusion situation, and set the dynamic threshold to high threshold 0.8 and low threshold 0.5. When the number of people is greater than N, N is set according to the size of the region set, that is, the minimum threshold 0.5 is adopted, otherwise 0.8 is adopted. The occlusion rate is calculated in a pairwise manner, that is, Figure 2 For example, (b), A will calculate 5 occlusion rates with B, C, D, E and F, and finally take the maximum value. The occlusion rate calculation uses the iou calculation, and the formula is cover= (Area_AB) / (Area_A + Area_B). Cover represents the occlusion rate, Area_AB represents the overlapping area of targets A and B, Area_A represents the area of target A, and Area_B represents the area of target B.

[0092] When the occlusion rate is greater than the threshold, compare the sizes of the two detection boxes, and filter out the detection box with smaller area; Area_ is smaller than Area_B, then filter the A target with smaller area, and the filtering means that the detection box will not participate in this tracking. As shown in Figure 3 , so as to avoid the phenomenon that the wandering person and other people are seriously overlapped, that is, the A target will not be tracked when it is occluded, and the features extracted cannot represent the A target, so as to improve the representativeness of the features, and the occlusion time is calculated according to the current system time and the time when the last feature of the tracking chain is recorded.

[0093] After the occlusion time passes, the occluded person shows the whole target, and the features of the target are extracted, and according to the duration of the tracking chain, a specific tracking threshold is calculated for the detection box that appears again (cos_cover=D_default+ (D_max-D_default) x min (T, Tmax) / Tmax, wherein the tracking threshold is cos_cover, the default tracking threshold D_default=0.25, the maximum tracking threshold D_max=0.35, the maximum occlusion time Tmax, ), when the cosine similarity between the target feature value and the last feature value of the tracking chain is less than the tracking threshold cos_cover, it represents that the tracking is completed, and the preliminary occlusion situation resistance is completed.

[0094] Specifically, please refer to Figure 5 and Figure 6 , the hierarchical matching includes:

[0095] According to the behavior characteristics of the wandering crowd, the wandering personnel stay in the scene for a long time, and have a long tracking chain. The ordinary personnel quickly pass through the scene, and a new tracking chain is generated, thereby generating a large number of partial occlusion events (occlusion events with a cover less than 0.5) in the scene. The hierarchical matching mechanism can well resist the case that the new tracking chain steals the original tracking chain.

[0096] As shown in Figure 5 (a), the wandering personnel often appear in the scene and are tracked all the time. When other personnel appear, the wandering personnel already have a longer tracking time and more tracking features. At this time, as shown in (b), the original wandering personnel is given a higher tracking level for subsequent matching.

[0097] When a small part of the occlusion occurs, a new tracking chain appears, and according to the hierarchical situation, the tracking chain with a high level is given a more preferential tracking order and a wider tracking threshold. At this time, the wandering personnel are matched to the detection frame in advance, as shown in Figure 6 (a), and become a tracked state. The remaining personnel match the remaining detection frame, as shown in Figure 6 (b), and become a tracked state. After matching, as shown in Figure 7 , the state is updated according to the personnel staying time, and the tracking level of the wandering personnel and the ordinary personnel is updated.

[0098] Specifically, please refer to Figures 8 to 10 , the wandering personnel will move and change their coordinate positions during the occlusion process, especially when the occluded personnel and the occluded personnel move synchronously, a long-time occlusion situation will be generated, and the position of the personnel itself will have a large displacement amount from the position before the occlusion. The present application proposes a time sequence matching mechanism to solve this problem.

[0099] The time sequence matching includes: as shown in Figure 8 (a), first, a threshold T_time is set artificially, whether the personnel staying time T exceeds the threshold T_time is determined to determine whether the time sequence matching level is reached, and is updated constantly, finally, part of the personnel will reach the time sequence matching level, as shown in Figure 8 (b), assuming that D and E have stayed in the area for a long time, A, C and F have just entered the area, at this time, D and E reach the time sequence matching level, the last time and coordinate position of the two tracking chains are recorded.

[0100] As shown in Figure 9As shown, the position where the tracking chain disappears is taken as the origin, and the matching range and matching threshold are determined according to the time when the tracking chain disappears, and the bounding box within the range participates in matching. The time matching range is according to the formula: r = min(50, min(T, 6) * 50), where r is the matching range, and T is the disappearing time. The matching threshold calculation formula is: if (distance < r) cos_time = cos_person / r * distance, where cos_person is the artificial threshold (0.05 when T is less than 3, and 0.1 when T is greater than 3), r is the matching range, distance is the distance from the target to the orange target origin, if (distance < r) indicates that the target must be within the matching range, and the matching threshold gradually decreases as the center point distance increases. As shown in Figure 9 As shown, the current D, E bounding box is still blocked by other people, and it is difficult to match.

[0101] As shown in Figure 10 After a period of time, the matching range is continued to be expanded according to the disappearing time, and the matching threshold is increased at the same time, so as to more easily search for a small part of the blocked bounding box, such as the E target, and so on, until the target bounding box that appears again after disappearing is matched or all tracking chains are traversed.

[0102] Specifically, in the identification of loitering personnel, the input picture is initialized and the logical judgment parameters are initialized; the picture is input into the detection model to obtain the target bounding box of the current personnel; the personnel features are extracted, and through a multi-layer matching mechanism, the loitering time of different personnel is recorded.

[0103] According to the matching result, the loitering personnel in the scene picture is identified, including: obtaining the loitering time of different targets in the scene according to the matching result; sorting the loitering time from large to small; judging whether the maximum value of the loitering time is greater than the potential loitering personnel threshold; if the maximum value of the loitering time is not greater than the potential loitering personnel threshold, there is no potential loitering personnel; if the maximum value of the loitering time is greater than the potential loitering personnel threshold, judging whether the loitering time of each target is greater than the time threshold of the temporary picture; if the loitering time of the target is greater than the time threshold of the temporary picture, the picture of the target is temporarily stored in the tracking chain; judging whether the loitering time of the target is greater than the loitering personnel threshold; if the loitering time of the target is greater than the loitering personnel threshold, the picture of the target is saved in the tracking chain for a long time; if the loitering time of the target is less than the loitering personnel threshold, judging whether the loitering time of the target is greater than the loitering personnel deletion threshold; then the picture of the target temporarily stored in the tracking chain is deleted. In this way, all tracking ids are traversed.

[0104] From the above embodiment, the loitering person detection method provided by the application can obtain a scene picture, extract the behavior features of each target in the scene picture, establish a tracking chain for each target in the scene picture, track the target, save the behavior features of the corresponding target in the tracking chain, and according to the behavior features, perform occlusion matching, hierarchical matching and time sequence matching on the tracked target, and according to the matching result, identify the loitering person in the scene picture; the application can track the nearby people in batches according to the personnel features and the personnel trend under the lens, the time, the position direction and the loitering time of the personnel disappearing under the lens and other factors, can guarantee the superior matching ability of the personnel after disappearing or being largely occluded, greatly weaken the tracking errors caused by similar clothing or personnel occlusion, and can efficiently and accurately prewarn the loitering person.

[0105] The application collects personnel images in a scene, extracts personnel features, records personnel time sequences, analyzes the loitering trend of the personnel in the current scene, updates the personnel staying status, extracts specific personnel images, and further prevents dangerous events in advance.

[0106] The application can grade the ordinary people, the potential loitering people and the loitering people by relying on the differential matching mechanism, avoid the problem of two tracking sequences of the same target, has better algorithm performance for improving the tracking effect and prolonging the tracking time, and can improve the matching ability of the occluded people and the people out of the shooting range when they appear again by relying on the time sequence state matching mechanism and matching the different tracking sequences by the personnel time sequence state to have different priorities, and improve the stability of the loitering detection.

[0107] Please refer to Figure 11 The application provides a loitering person detection device, which comprises:

[0108] The acquisition unit 11 is used for acquiring a scene picture.

[0109] The extraction unit 12 is used for extracting the behavior features of each target in the scene picture.

[0110] The tracking unit 13 is used for establishing a tracking chain for each target in the scene picture, tracking the target, and saving the behavior features of the corresponding target in the tracking chain.

[0111] The matching unit 14 is used for sequentially performing occlusion matching, hierarchical matching and time sequence matching on the tracking chain and the detection frame of the target, so as to continuously track the target dynamically.

[0112] The occlusion matching comprises: calculating an occlusion rate of the current tracked target and other targets in the scene picture respectively; filtering a detection frame of the occluded target when the occlusion rate is greater than an occlusion rate threshold; calculating a tracking threshold for the detection frame when the occluded target exposes the entire detection frame again; calculating a cosine similarity between a current behavior feature of the target and a last behavior feature in a tracking chain corresponding to the target; and matching the detection frame with the cosine similarity less than the tracking threshold to the tracking chain of the target to continue tracking the target.

[0113] The hierarchical matching comprises: hierarchically classifying the targets from high to low according to lengths of stay of the targets in the scene, and assigning a higher tracking order and a wider tracking threshold to a tracking chain of a target with a higher level according to the hierarchical classification.

[0114] The time sequence matching comprises: recording a last time and a coordinate position of a tracking chain of a target whose length of stay in the scene exceeds a time threshold; taking the position where the tracking chain disappears as a starting point, determining a matching range and a matching threshold according to a time when the tracking chain disappears, matching a detection frame in the range to the tracking chain; and after a time interval, continuing to expand the matching range according to the disappearance time, and increasing the matching threshold simultaneously until a detection frame of the target that appears again after disappearing is matched or all tracking chains are traversed.

[0115] The identification unit 15 is configured to identify the loitering person in the scene picture according to the matching result.

[0116] As can be seen from the above embodiments, the loitering person detection device provided by the application obtains a scene picture through the obtaining unit, extracts a behavior feature of each target in the scene picture through the extraction unit, establishes a tracking chain for each target in the scene picture through the tracking unit, tracks the target, saves the behavior feature of the corresponding target in the tracking chain, matches the tracked target through the matching unit according to the behavior feature, and identifies the loitering person in the scene picture according to the matching result. The loitering person detection device provided by the application can track the nearby person in batches according to the personnel features and the personnel trend under the lens, the time, the position direction and the loitering time when the personnel disappear under the lens and other factors, can guarantee a more superior matching ability of the personnel after disappearing or being largely occluded, greatly weakens the tracking error caused by similar clothing or personnel occlusion, and can efficiently and accurately prewarn the loitering person.

[0117] Please refer to Figure 12 The application further provides an electronic device, which comprises a memory 100 and a processor 200, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the above method.

[0118] The embodiments of the present application further provide a storage medium, and the storage medium stores a computer program. The computer program is executed by a processor to implement some or all steps in the embodiments of the method for detecting indoor and outdoor loitering person provided by the present application. The storage medium can be a disk, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

[0119] Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be realized by means of software and necessary general hardware platforms. Based on such understanding, the technical solutions in the embodiments of the present application can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disc, etc. The computer software product includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments of the present application or some parts of the embodiments.

[0120] The same or similar parts among the various embodiments in the present specification can be referred to each other. Especially, for the embodiments of the device for detecting indoor and outdoor loitering person, since they are basically similar to the method embodiments, the description is relatively simple, and the related parts can be referred to the description in the method embodiments.

[0121] The above-described embodiments of the present application do not constitute a limitation on the protection scope of the present application.

Claims

1. A method for detecting loitering individuals, characterized in that, include: Get scene images; Extract the behavioral features of each target in the scene image; A tracking chain is established for each target in the scene image, and the target is tracked. The tracking chain stores the behavioral features of the corresponding target. The tracking chain and detection box of the target are matched sequentially according to three matching mechanisms: occlusion matching, hierarchical matching, and temporal matching, so as to continuously and dynamically track the target. The occlusion matching includes: filtering the detection boxes of occluded targets based on the occlusion rate between the currently tracked target and other targets in the scene image; calculating a tracking threshold for the detection boxes when the occluded target re-exposes the entire detection box; and selecting the tracking chain of the detection box and the target for matching based on the tracking threshold. The hierarchical matching includes: classifying targets from high to low based on the length of time they stay in the scene, and assigning higher-level tracking chains a higher priority tracking order and a wider range of tracking thresholds according to the classification. The temporal matching includes: for targets whose dwell time in the scene exceeds a time threshold, recording the last time and coordinate position of the target's tracking chain; taking the position where the tracking chain disappears as the origin, determining the matching range and matching threshold based on the time when the tracking chain disappears, and matching the detection boxes within the range with the tracking chain; Based on the matching results, identify the loitering individuals in the scene image.

2. The method for detecting loitering individuals as described in claim 1, characterized in that, The behavioral characteristics include the target location, the time of target appearance, and target features.

3. The method for detecting loitering individuals as described in claim 1, characterized in that, Based on the occlusion rate of the currently tracked target and other targets in the scene image, the detection bounding boxes of occluded targets are filtered, including: Calculate the occlusion rate of the currently tracked target and other targets in the scene image respectively; Determine whether the maximum value of the occlusion rate is greater than the dynamic threshold; If the maximum value of the occlusion rate is determined to be greater than the dynamic threshold, the area of ​​the detection box of the current target is compared with the area of ​​the detection box of the target corresponding to the maximum occlusion rate, and the detection box with the smaller area is filtered out.

4. The method for detecting loitering individuals as described in claim 3, characterized in that, Matching a detection box with the target's tracking chain based on the tracking threshold includes: Calculate the occlusion time based on the current system time and the time of the last behavioral feature record of the tracking chain. After the occlusion time has passed, when the occluded target re-exposes the entire detection box, the target's behavioral features are extracted, and a tracking threshold is calculated for the detection box. Calculate the cosine similarity between the target's current behavioral feature and the last behavioral feature in the tracking chain corresponding to the target; The detection boxes with a cosine similarity less than the tracking threshold are matched with the tracking chain of the target, and the target is continued to be tracked.

5. The method for detecting loitering individuals as described in claim 1, characterized in that, The timing matching also includes: If no match is found, after a certain period of time, the matching range is expanded according to the disappearance time, and the matching threshold is increased accordingly. The search continues for occluded detection boxes until a target detection box that has disappeared and reappeared is matched or all tracking chains have been traversed.

6. The method for detecting loitering individuals as described in claim 1, characterized in that, Based on the matching results, identify loitering individuals in the scene image, including: The loitering time of different targets in the scene is obtained based on the matching results; Sort the hovering times from longest to shortest; Determine whether the maximum value of the loitering time is greater than the threshold for potential loitering individuals; If the maximum value of the loitering time is not greater than the potential loitering threshold, then there are no potential loitering individuals.

7. The method for detecting loitering individuals as described in claim 6, characterized in that, The method further includes: If the maximum value of the loitering time is greater than the threshold for potential loitering persons, determine whether the loitering time of each target is greater than the time threshold for temporarily stored images; If the target's hovering time is greater than the time threshold for temporarily storing images, the image of the target is temporarily stored in the tracking chain.

8. The method for detecting loitering individuals as described in claim 7, characterized in that, The method further includes: Determine whether the target's loitering time exceeds a loitering personnel threshold; If the target's loitering time exceeds the loitering personnel threshold, the target's image is stored in the tracking chain for an extended period. If the target's loitering time is less than the loitering personnel threshold, determine whether the target's loitering time is greater than the loitering personnel deletion threshold; If the target's loitering time exceeds the loitering personnel deletion threshold, the image of the target temporarily stored in the tracking chain will be deleted.

9. A device for detecting loitering individuals, characterized in that, include: The acquisition unit is used to acquire scene images; The extraction unit is used to extract the behavioral features of each target in the scene image; The tracking unit is used to establish a tracking chain for each target in the scene image and track the target. The tracking chain stores the behavioral features of the corresponding target. The matching unit is used to match the tracking chain and detection box of the target in sequence according to three matching mechanisms: occlusion matching, hierarchical matching and temporal matching, so as to continuously and dynamically track the target. The occlusion matching includes: filtering the detection boxes of occluded targets based on the occlusion rate between the currently tracked target and other targets in the scene image; calculating a tracking threshold for the detection boxes when the occluded target re-exposes the entire detection box; and selecting the tracking chain of the detection box and the target for matching based on the tracking threshold. The hierarchical matching includes: classifying targets from high to low based on the length of time they stay in the scene, and assigning higher-level tracking chains a higher priority tracking order and a wider range of tracking thresholds according to the classification. The temporal matching includes: for targets whose dwell time in the scene exceeds a time threshold, recording the last time and coordinate position of the target's tracking chain; taking the position where the tracking chain disappears as the origin, determining the matching range and matching threshold based on the time when the tracking chain disappears, and matching the detection boxes within the range with the tracking chain; The identification unit is used to identify loitering individuals in the scene image based on the matching results.

10. An electronic device, characterized in that, include: A memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1-8.

Images