A method, apparatus, equipment and medium for detecting loitering events

Abstract

This application discloses a method, apparatus, device, and medium for detecting loitering events. In the embodiments of this application, it is first determined whether the target object has moved based on the first detection box in the first image and the second detection box in the second image. After determining that the target object has moved, it is further determined whether the target object is moving inside or outside the detection area, thereby judging whether the target object is loitering. This effectively improves the accuracy of loitering event detection and reduces the false alarm rate. Furthermore, this application can determine whether the target object is loitering based on whether it moves back and forth inside or outside the detection area, thus compensating for the blind spots of existing loitering detection algorithms. Specifically, it compensates for the problem that loitering detection algorithms based on dwell time fail when the target object loiteres at the edge of the detection area. Moreover, the technical solution protected by this application has reliability, real-time performance, and generalizability, meeting the characteristics of dependability.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device and medium for detecting loitering events. Background Technology

[0002] With social development and the advancement of smart cities, intelligent video surveillance systems are receiving increasing attention. People's demands for the safety of their living environment are also growing, making intelligent video surveillance systems an important tool and means of security and defense. Among these systems, loitering detection algorithms are being applied to a wider range of technologies. For example, pedestrian loitering detection algorithms activate when a pedestrian or other object enters the frame captured by the image acquisition device. These algorithms are primarily used in scenarios such as unattended entrances and unattended parking lots.

[0003] In related technologies, the general approach is to determine whether the pedestrian's dwell time within the detection area meets the alarm criteria. If it does, an alarm is triggered. However, if the pedestrian is merely lingering at the edge of the detection area, the lingering detection algorithm based on dwell time will fail. Furthermore, since the pedestrian detection bounding box may be inaccurate due to object occlusion, algorithm stability, or other factors (e.g., the detection box suddenly shrinks or expands), if the pedestrian is only lingering at the edge of the detection area, it may be mistakenly identified as lingering even though no lingering has occurred. This leads to inaccurate lingering event detection results. Therefore, proposing a technical solution that can accurately determine whether a pedestrian is lingering is crucial. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for detecting loitering events, in order to solve the problem that existing technologies cannot accurately predict the detection results of loitering events.

[0005] This application provides a method for detecting loitering events, the method comprising:

[0006] Obtain a first detection bounding box of the target object contained in the first image, and determine a second detection bounding box of the target object in a second image acquired before the first image;

[0007] Based on the position information of the first detection box and the second detection box, determine whether the target object has moved;

[0008] If so, determine whether multiple preset points on the first detection frame of the target object are within the detection area, and determine the target number of preset points within the detection area;

[0009] Based on the number of targets and whether the target object in the second image is within the detection area, it is determined whether the target object is lingering.

[0010] Furthermore, the method also includes:

[0011] If it is determined that the target object has not moved, then it is determined that the target object has not lingered.

[0012] Further, determining whether the target object has moved based on the position information of the first detection box and the second detection box includes:

[0013] Based on the position information of the first detection box and the position information of the second detection box, determine the target area corresponding to the intersection of the first detection box and the second detection box;

[0014] Determine the first area corresponding to the first detection box and the second area of ​​the second detection box;

[0015] Determine a first ratio of the target area to the first area, and determine a second ratio of the target area to the second area;

[0016] If both the first ratio and the second ratio are greater than a preset ratio threshold, then the target object is determined to have moved; otherwise, the target object is determined not to have moved.

[0017] Further, determining whether the target object is lingering based on the number of targets and whether the target object in the second image is within the detection area includes:

[0018] If the target object in the second image is within the detection area, then if the number of targets is not greater than a preset first value, it is determined that the target object is loitering; if the number of targets is greater than the preset first value, it is determined that the target object is not loitering.

[0019] Further, determining whether the target object is lingering based on the number of targets and whether the target object in the second image is within the detection area includes:

[0020] If the target object in the second image is not within the detection area, then if the number of targets is not greater than a preset second value, it is determined that the target object is not lingering; if the number of targets is greater than the preset second value, it is determined that the target object is lingering.

[0021] Furthermore, after determining whether the target object is loitering, the method further includes:

[0022] If it is determined that the target object is lingering, then the sum of the saved number of lingering times of the target object and the preset target value is determined, and the sum is updated to the saved number of lingering times of the target object;

[0023] Determine whether the number of times the target object has been updated is greater than a preset threshold for the number of times it has been hovered. If so, output an alarm message.

[0024] Furthermore, the method also includes:

[0025] If the number of lingering attempts after the target object is updated is not greater than the preset lingering attempt threshold, then it is determined that no alarm information will be output.

[0026] Furthermore, the output alarm information includes:

[0027] Output the identification information of the target object.

[0028] Furthermore, after determining whether the target object is loitering, the method further includes:

[0029] If it is determined that the target object is lingering and the target object in the second image is within the detection area, then it is determined that the target object in the first image is outside the detection area.

[0030] If it is determined that the target object is lingering, and the target object in the second image is outside the detection area, then it is determined that the target object in the first image is within the detection area;

[0031] If it is determined that the target object is not lingering and the target object in the second image is within the detection area, then it is determined that the target object in the first image is within the detection area.

[0032] If it is determined that the target object is not lingering and the target object in the second image is outside the detection area, then it is determined that the target object in the first image is outside the detection area.

[0033] This application also provides a loitering event detection device, the device comprising:

[0034] The acquisition module is used to obtain a first detection box of a target object contained in a first image, and to determine a second detection box of the target object in a second image acquired before the first image.

[0035] The determination module is used to determine whether the target object has moved based on the position information of the first detection box and the second detection box; if so, it determines whether multiple preset points on the first detection box of the target object are within the detection area, and determines the number of targets at the preset points within the detection area; based on the number of targets and whether the target object in the second image is within the detection area, it determines whether the target object is wandering.

[0036] Furthermore, the determining module is also configured to determine that the target object is not wandering if it is determined that the target object has not moved.

[0037] Further, the determining module is specifically configured to: determine the target area corresponding to the intersection of the first detection frame and the second detection frame based on the position information of the first detection frame and the position information of the second detection frame; determine the first area corresponding to the first detection frame and the second area of ​​the second detection frame; determine the first ratio of the target area to the first area and the second ratio of the target area to the second area; determine whether both the first ratio and the second ratio are greater than a preset ratio threshold; if so, determine that the target object has moved; otherwise, determine that the target object has not moved.

[0038] Furthermore, the determining module is specifically configured to determine that if the target object in the second image is within the detection area, and if the number of targets is not greater than a preset first value, the target object is considered to be lingering; if the number of targets is greater than the preset first value, the target object is considered to be lingering.

[0039] Furthermore, the determining module is specifically configured to determine that the target object is not lingering if the target object in the second image is not within the detection area and the number of targets is not greater than a preset second value; and to determine that the target object is lingering if the number of targets is greater than the preset second value.

[0040] Furthermore, the device also includes:

[0041] The processing module is configured to, if it is determined that the target object is lingering, determine the sum of the saved number of lingering of the target object and the preset target value, and update the sum to the saved number of lingering of the target object; determine whether the updated number of lingering of the target object is greater than the preset number of lingering, and if so, output an alarm message.

[0042] Furthermore, the processing module is also configured to determine not to output alarm information if the updated number of hops of the target object is not greater than a preset hop count threshold.

[0043] Furthermore, the processing module is specifically used to output the identification information of the target object.

[0044] Furthermore, the determining module is further configured to: if it is determined that the target object is loitering and the target object in the second image is within the detection area, then determine that the target object in the first image is outside the detection area; if it is determined that the target object is loitering and the target object in the second image is outside the detection area, then determine that the target object in the first image is within the detection area; if it is determined that the target object is not loitering and the target object in the second image is within the detection area, then determine that the target object in the first image is within the detection area; if it is determined that the target object is not loitering and the target object in the second image is outside the detection area, then determine that the target object in the first image is outside the detection area.

[0045] This application also provides an electronic device including a processor, the processor being configured to execute a computer program stored in a memory to implement the steps of any of the above-described loitering event detection methods.

[0046] This application also provides a computer-readable storage medium storing a computer program executable by a terminal, which, when run on the terminal, causes the terminal to perform the steps of any of the above-described loitering event detection methods.

[0047] In this embodiment of the application, a first detection box of the target object contained in the first image is obtained, and a second detection box of the target object in the second image acquired before the first image is determined; based on the position information of the first detection box and the second detection box, it is determined whether the target object has moved; if so, it is determined whether multiple preset points on the first detection box of the target object are within the detection area, and the number of targets at the preset points within the detection area is determined; based on the number of targets and whether the target object in the second image is within the detection area, it is determined whether the target object is wandering. In this embodiment, the movement of the target object can be determined by first using the first detection box in the first image and the second detection box in the second image. After determining that the target object has moved, it can be further determined whether the target object is moving inside or outside the detection area, thereby determining whether the target object is lingering. This can effectively improve the accuracy of lingering event detection and reduce the false alarm rate. Moreover, this application can determine whether the target object is lingering based on whether the target object moves back and forth inside or outside the detection area, thus making up for the blind spot of the lingering detection algorithm in the prior art. That is, it makes up for the problem that the lingering detection algorithm based on dwell time will fail when the target object lingers at the edge of the detection area in the prior art. Furthermore, the technical solution protected by this application has reliability, real-time performance, and generalizability, and meets the characteristics of reliability. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of a loitering event detection process provided in an embodiment of this application;

[0050] Figure 2 Display diagrams showing the target object moving along the left and right edges of the detection area, as provided in some embodiments of this application;

[0051] Figure 3 A schematic diagram showing the intersection of a first detection frame and a second detection frame provided for some embodiments of this application;

[0052] Figure 4 A schematic diagram showing the target object located at the upper and lower edges of the detection area, provided for some embodiments of this application;

[0053] Figure 5 This is a schematic diagram illustrating the relationship between the number of targets and whether the target object lingers when a target object in a second image is in the detection area, as provided in some embodiments of this application.

[0054] Figure 6 This is a schematic diagram illustrating the relationship between the number of targets and whether the target object lingers when the target object in a second image is outside the detection area, as provided in some embodiments of this application.

[0055] Figure 7 This application provides a schematic diagram illustrating a process for determining whether to output an alarm message, based on some embodiments of the present application.

[0056] Figure 8 This application provides a schematic diagram of an alarm-generating process in some embodiments;

[0057] Figure 9 A schematic diagram of a loitering event detection device provided in some embodiments of this application;

[0058] Figure 10 This is a schematic diagram of the structure of an electronic device provided in some embodiments of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] In order to accurately determine whether pedestrians or other objects are loitering, embodiments of this application provide a loitering event detection method, apparatus, device, and medium.

[0061] In this embodiment of the application, a first detection box of the target object contained in the first image is obtained, and a second detection box of the target object in the second image acquired before the first image is determined; based on the position information of the first detection box and the second detection box, it is determined whether the target object has moved; if so, it is determined whether multiple preset points on the first detection box of the target object are within the detection area, and the number of targets at the preset points within the detection area is determined; based on the number of targets and whether the target object in the second image is within the detection area, it is determined whether the target object is wandering.

[0062] Figure 1 This application provides a schematic diagram of a loitering event detection process, which includes the following steps:

[0063] S101: Obtain a first detection box of the target object contained in the first image, and determine a second detection box of the target object in a second image acquired before the first image.

[0064] The loitering event detection method provided in this application is applied to an electronic device, which may be a smart terminal, a PC, or a server, etc.

[0065] In this embodiment, the electronic device first obtains the identifier (ID) and detection box of each object contained in a first image and a second image captured before the first image using an object detection algorithm and an object tracking algorithm. The object can be a pedestrian, vehicle, animal, or other object that may be loitering. To detect loitering events in real time, the first image can be an image captured at the current time. The second image can be an image captured before the first image and adjacent to it, or an image captured before the first image but separated from it by a preset number of frames, etc. It should be noted that to accurately determine whether a target object is loitering, the time interval between the capture time of the first image and the capture time of the second image should not be too large. The object detection algorithm can be a deep learning algorithm such as You Only Look Once (Yolo), an algorithm that applies deep learning to object detection (Region-CNN, RCNN), etc. Then, the first detection box of the target object contained in the first image and the second detection box of the target object in the second image are obtained. The target object can be any object.

[0066] Specifically, when detecting the bounding box of an object, the position of the object in the image can be identified first. Then, a candidate bounding box containing the object can be predicted based on the position. The human features contained in the candidate bounding box can be extracted. The human features include the position and size of the object's face, the position and size of the object's head, the object's human pose, etc. Finally, the complete bounding box of the object containing the occluded area can be predicted based on the human features.

[0067] S102: Determine whether the target object has moved based on the position information of the first detection box and the second detection box.

[0068] After determining the first detection frame and the second detection frame, the electronic device can determine whether the target object has moved based on the position information of the first detection frame and the position information of the second detection frame.

[0069] In one possible implementation, the distance between the center position of the first detection frame and the center position of the second detection frame can be determined first. Then, based on whether the distance is less than a preset distance threshold, it can be determined whether the target object has moved. Specifically, if the distance is less than the preset distance threshold, it is determined that the target object has not moved; otherwise, it is determined that the target object has moved.

[0070] S103: If yes, determine whether multiple preset points on the first detection frame of the target object are within the detection area, and determine the target number of preset points within the detection area.

[0071] After determining that the target object has moved, to determine whether the target object is loitering, multiple preset points on the first detection frame of the target object can be determined to be within the detection area. These multiple points can be two or more, and the number of targets at the preset points within the detection area can be determined. These preset points can be the midpoint of the bottom edge, the left vertex of the bottom edge, the right vertex of the bottom edge, etc., and can be set according to requirements. This application supports pre-defining one or more detection areas within a single monitoring screen; that is, each image can contain a single detection area or multiple detection areas, and the detection area supports polygon settings. This application uses the entire screen as the detection area by default. The edge of the detection area is a warning line, and the preset area including the edge is the warning area. The more targets are located at the preset points within the detection area, the greater the probability that the target object is within the detection area; the fewer targets are located at the preset points within the area, the greater the probability that the target object is outside the detection area.

[0072] S104: Determine whether the target object is lingering based on the number of targets and whether the target object in the second image is within the detection area.

[0073] After determining the number of targets, the electronic device determines whether the target object is loitering based on the number of targets and whether the target object is within the detection area in the second image. Specifically, if the target object is within the detection area in the second image, the fewer the number of targets, the greater the likelihood that the target object is loitering; conversely, if the target object is outside the detection area in the second image, the more the number of targets, the greater the likelihood that the target object is loitering.

[0074] In this embodiment, the movement of the target object can be determined by first using the first detection box in the first image and the second detection box in the second image. After determining that the target object has moved, it can be further determined whether the target object is moving inside or outside the detection area, thereby determining whether the target object is lingering. This can effectively improve the accuracy of lingering event detection and reduce the false alarm rate. Moreover, this application can determine whether the target object is lingering based on whether the target object moves back and forth inside or outside the detection area, thus making up for the blind spot of the lingering detection algorithm in the prior art. That is, it makes up for the problem that the lingering detection algorithm based on dwell time will fail when the target object lingers at the edge of the detection area in the prior art. Furthermore, the technical solution protected by this application has reliability, real-time performance, and generalizability, and meets the characteristics of reliability.

[0075] To accurately determine whether a target object is loitering, based on the above embodiments, the method in this application embodiment further includes:

[0076] If it is determined that the target object has not moved, then it is determined that the target object has not lingered.

[0077] In this embodiment of the application, after determining whether the target object has moved based on the position information of the first detection frame and the second detection frame, if it is determined that the target object has not moved, then it is determined that the target object has not lingered.

[0078] To accurately determine whether the target object has moved, based on the above embodiments, in this embodiment, determining whether the target object has moved based on the position information of the first detection frame and the second detection frame includes:

[0079] Based on the position information of the first detection box and the position information of the second detection box, determine the target area corresponding to the intersection of the first detection box and the second detection box;

[0080] Determine the first area corresponding to the first detection box and the second area of ​​the second detection box;

[0081] Determine a first ratio of the target area to the first area, and determine a second ratio of the target area to the second area;

[0082] If both the first ratio and the second ratio are greater than a preset ratio threshold, then the target object is determined to have moved; otherwise, the target object is determined not to have moved.

[0083] When the object moves a small distance along the edge of the detection area or remains stationary, such as Figure 2 As shown, where, Figure 2 This is a display diagram of the target object moving on the left and right edges of the detection area according to some embodiments of this application. Since the hovering event detection algorithm in the related technology determines the position of the object inside and outside the detection area based on three points: the midpoint of the bottom edge of the object's detection box, the vertex on the left side of the bottom edge, and the vertex on the right side of the bottom edge, if the detection box is prone to slight movement, becoming larger or smaller due to object occlusion, algorithm stability, etc., the position of the object inside and outside the detection area will constantly change, leading to an increase in the number of hovering events and ultimately resulting in false detections.

[0084] In this embodiment of the application, in order to accurately determine whether the target object is lingering and avoid false detection, the electronic device first accurately determines whether the target object is moving. Specifically, it can first determine the intersection of the first detection frame and the second detection frame based on the position information of the first detection frame and the position information of the second detection frame, and determine the target area corresponding to the intersection. The larger the target area, the greater the possibility that the target object has not moved.

[0085] Figure 3 This application provides a schematic diagram showing the intersection of a first detection frame and a second detection frame, based on some embodiments. Figure 3Please provide an explanation.

[0086] The first detection box and the second detection box have an intersection, and the target area corresponding to the intersection is A.

[0087] Then determine the first area corresponding to the first detection box and the second area of ​​the second detection box, and then determine the first ratio of the target area to the first area and the second ratio of the target area to the second area.

[0088] Specifically, the first ratio and the second ratio are determined according to the following formulas:

[0089]

[0090]

[0091] Intersection=(min(x2,x4)-max(x1,x3))*(min(y2,y4)-min(y1,y3));

[0092] Wherein, IO1 is the first ratio, IO2 is the second ratio, (x1, y1) is the coordinate of the upper left corner of the first detection box, (x2, y2) is the coordinate of the lower right corner of the first detection box, (x3, y3) is the coordinate of the upper left corner of the second detection box, and (x4, y4) is the coordinate of the lower right corner of the second detection box.

[0093] After determining the first ratio and the second ratio, it is determined whether both the first ratio and the second ratio are greater than a preset ratio threshold. If so, it is determined that the target object has moved; otherwise, it is determined that the target object has not moved. The preset ratio threshold can be 0.9, 0.85, etc., and can be set according to requirements.

[0094] To accurately determine whether a target object is loitering, based on the above embodiments, in this embodiment, determining whether the target object is loitering based on the number of targets and whether the target object in the second image is within the detection area includes:

[0095] If the target object in the second image is within the detection area, then if the number of targets is not greater than a preset first value, it is determined that the target object is loitering; if the number of targets is greater than the preset first value, it is determined that the target object is not loitering.

[0096] When the target object moves at the edge of the detection area, such as Figure 4 As shown, where, Figure 4This is a schematic diagram showing the target object located at the upper and lower edges of the detection area according to some embodiments of this application. Due to factors such as object occlusion and instability of the wandering detection algorithm, the object detection box may move slightly or suddenly increase or decrease in size, causing the bottom edge of the detection box corresponding to the target object to jump back and forth inside and outside the area, which in turn leads to an increase in the number of wanderings and ultimately results in false detections.

[0097] To accurately determine whether a target object is lingering and avoid false detections, when the target object in the second image is within the detection area, if the number of targets at preset points within the detection area of ​​the first detection box is not greater than a preset first value, it indicates that the target object is outside the detection area in the first image, and therefore, the target object is determined to be lingering. If the number of targets is greater than the preset first value, it indicates that the target object is within the detection area in the first image, and therefore, the target object is determined not to be lingering. The preset first value is not greater than the total number of preset points of the first detection box. In this embodiment, the preset points are preferably the midpoint of the bottom edge, the left vertex of the bottom edge, and the right vertex of the bottom edge. That is, the maximum number of targets at preset points within the detection area of ​​the first detection box is 3, so the first value is not greater than 3, and preferably, the first value is set to 0.

[0098] Figure 5 This application provides a schematic diagram illustrating the relationship between the number of targets and whether the target object lingers in a detection area when a target object in a second image is present in some embodiments. The diagram is now being used to illustrate this relationship. Figure 5 Please provide an explanation.

[0099] When the target object is within the detection area in the second image, if the number of targets is 0, it is determined that the target object is wandering; if the number of targets is 1, 2, or 3, it is determined that the target object is not wandering.

[0100] To accurately determine whether a target object is loitering, based on the above embodiments, in this embodiment, determining whether the target object is loitering based on the number of targets and whether the target object in the second image is within the detection area includes:

[0101] If the target object in the second image is not within the detection area, then if the number of targets is not greater than a preset second value, it is determined that the target object is not lingering; if the number of targets is greater than the preset second value, it is determined that the target object is lingering.

[0102] In this embodiment, to determine whether a target object is loitering, when the target object in the second image is not within the detection area (i.e., outside the detection area), if the number of targets at preset points within the detection area of ​​the first detection box is not greater than a preset second value, it indicates that the target object is outside the detection area in the first image, and therefore, it is determined that the target object is not loitering. If the number of targets is greater than the preset second value, it indicates that the target object is within the detection area in the first image, and therefore, it is determined that the target object is loitering. The preset second value is greater than a preset first value, and the preset second value is not greater than the total number of preset points in the first detection box. In this embodiment, the preset points are preferably the midpoint of the bottom edge, the left vertex of the bottom edge, and the right vertex of the bottom edge. That is, the maximum number of targets at preset points within the detection area of ​​the first detection box is 3, in which case the second value is not greater than 3. If the first value is 1, then the first value is greater than 1 but not greater than 3, and preferably, the first value is set to 2.

[0103] Figure 6 This application provides a schematic diagram illustrating the relationship between the number of targets and whether the target object lingers when it is outside the detection area in a second image, according to some embodiments of the application. The diagram is now being used to illustrate this relationship. Figure 6 Please provide an explanation.

[0104] If the target object in the second image is outside the detection area, then if the number of targets is 0 or 1, it is determined that the target object is not lingering; if the number of targets is 2 or 3, it is determined that the target object is lingering.

[0105] Based on the problems encountered in practical applications, this application proposes a logic optimization for the target object at the edge of the detection area. Specifically, it optimizes the combined logic for the positional changes of the target object at the left and right edges and the top and bottom edges of the detection area, thereby reducing false alarms and increasing accuracy. Furthermore, the loitering event detection algorithm in this application achieved an accuracy of over 90% in identifying loitering events during trial operation.

[0106] To ensure that an alarm message is output to prompt staff to perform safety maintenance when the number of loitering incidents is excessive, based on the above embodiments, in this embodiment of the application, after determining whether the target object is loitering, the method further includes:

[0107] If it is determined that the target object is lingering, then the sum of the saved number of lingering times of the target object and the preset target value is determined, and the sum is updated to the saved number of lingering times of the target object;

[0108] Determine whether the number of times the target object has been updated is greater than a preset threshold for the number of times it has been hovered. If so, output an alarm message.

[0109] In this embodiment of the application, after determining whether the target object is wandering, that is, after the position of the target object changes within or outside the detection area, if it is determined that the target object is wandering, the number of wanderings of the target object can be updated according to the number of wanderings of the target object and the preset target value. Specifically, the sum of the number of wanderings of the target object and the preset target value is determined, and the sum is updated to the number of wanderings of the target object. The preset target value is 1.

[0110] When a target object loiters excessively, it can be identified as a suspicious object. To ensure security, an alarm message can be output to remind staff to perform security maintenance. Specifically, to determine whether a target object is suspicious, the electronic device stores a loitering count threshold. When the updated loitering count of a target object exceeds the preset loitering count threshold, the target object is identified as a suspicious object.

[0111] To accurately determine whether to output alarm information, based on the above embodiments, the method in this application embodiment further includes:

[0112] If the number of lingering attempts after the target object is updated is not greater than the preset lingering attempt threshold, then it is determined that no alarm information will be output.

[0113] In this embodiment of the application, if the number of times the target object lingers after the update is not greater than the preset number of times it lingers, it means that the target object does not linger frequently. Therefore, the current target object can be identified as a non-suspicious object, and therefore no alarm information needs to be output.

[0114] The loitering event detection method provided in this application can effectively reduce the problem of false detection of loitering events. The following is a combination of... Figure 2 and Figure 4 Please provide an explanation.

[0115] like Figure 2As shown, when the target object moves a small distance along the edge of the detection area or remains stationary at the edge of the detection area, the detection box corresponding to the target object may undergo slight changes due to algorithm instability or object occlusion. If the position of the object within and outside the detection area is determined based on the three points of the object's detection box—the midpoint of the bottom edge, the vertex to the left of the bottom edge, and the vertex to the right of the bottom edge—and then whether the target object has moved, there may be cases where there is no wandering but it is falsely detected as wandering. However, in this embodiment, considering that when the target object moves a small distance along the edge of the detection area or remains stationary, its movement is much smaller than when the target object is moving, this application can accurately determine whether the target object is moving by determining whether the movement range of the target object is large. To determine whether a target object has moved, specifically, to ascertain whether its movement is significant, the method first determines the target area corresponding to the intersection of the first and second detection boxes based on their position information. Then, it determines the first area corresponding to the first detection box and the second area corresponding to the second detection box. Finally, it determines whether the target object has moved by checking if both the first ratio of the target area to the first area and the second ratio of the target area to the second area are greater than a preset ratio threshold. If both the first and second ratios are greater than the preset threshold, the target object has moved only slightly, thus determining that it has not moved. If neither the first nor the second ratio is greater than the preset threshold, the target object has moved significantly, thus determining that it has moved. This method for determining whether a target object has moved avoids misjudging its movement even when the first and second detection boxes undergo minor changes, i.e., when the target object's movement is small, thereby improving the accuracy of determining whether a target object has moved.

[0116] like Figure 4As shown, when a target object moves along the edge, due to factors such as object occlusion and the instability of the wandering detection algorithm, the object detection box may undergo slight movement or suddenly increase or decrease in size, causing the bottom edge of the detection box corresponding to the target object to jump back and forth inside and outside the detection area. This may result in a situation where wandering is not detected but is mistakenly identified as wandering. In this embodiment, considering that when the target object is within the detection area in the second image, even if not all the preset points on the first detection box of the target object are within the detection area, the target object may not be wandering, but rather some of its points may be outside the detection area due to a slight change in the detection box. Similarly, when the target object is outside the detection area in the second image, even if not all the preset points on the first detection box of the target object are outside the detection area, the target object may not be wandering, but rather some of its points may be within the detection area due to a slight change in the detection box. Therefore, in order to accurately determine whether the target object is wandering, this application can be based on... The method for determining whether a target object is loitering is determined by whether it is within or outside the detection area in the second image, and by the number of targets within the detection area at preset points on the first detection frame of the target object. Specifically, if the target object is within the detection area in the second image and the number of targets within the detection area at preset points on the first detection frame of the target object is relatively large, then the target object is determined not to be loitering. If the target object is within the detection area in the second image and the number of targets within the detection area at preset points on the first detection frame of the target object is relatively small, then the target object is determined to be loitering. If the target object is outside the detection area in the second image and the number of targets within the detection area at preset points on the first detection frame of the target object is relatively small, then the target object is determined not to be loitering. If the target object is outside the detection area in the second image and the number of targets within the detection area at preset points on the first detection frame of the target object is relatively large, then the target object is determined to be loitering. This method for determining whether a target object is loitering can accurately determine whether the target object is loitering even if the detection frame moves slightly.

[0117] Figure 7 This application provides a schematic diagram illustrating a process for determining whether to output alarm information, based on some embodiments of the present application. Figure 7 Please provide an explanation.

[0118] First, obtain the identifier (ID) of the target object contained in the first image and determine the first detection box corresponding to the target object in the first image. Then, obtain the second detection box corresponding to the target object contained in the second image. Based on the first and second detection boxes, determine whether the target object has moved. If it has not moved, the process ends. If it has moved, calculate the number C of targets within the detection area of ​​the midpoint of the bottom edge, the left vertex of the bottom edge, and the right vertex of the bottom edge of the first detection box, and obtain the state of the target object in the second image, i.e., determine whether the target object is within or outside the detection area of ​​the second image. If the target object is determined to be within the detection area of ​​the second image, determine whether C is less than 1. If yes, determine that the target object is lingering and increment the lingering count by 1; otherwise, the process ends. If it is outside the detection area, determine whether C is greater than 1. If yes, determine that the target object is lingering and increment the lingering count by 1; otherwise, the process ends.

[0119] After incrementing the target object's lingering count by 1 (i.e., updating the target object's lingering count), determine whether the updated lingering count of the target object is greater than a preset lingering count threshold. If so, output an alarm message.

[0120] To facilitate staff in identifying which object is suspicious, based on the above embodiments, in this embodiment, the output alarm information includes:

[0121] Output the identification information of the target object.

[0122] In this embodiment of the application, since the first image may contain multiple objects, in order to ensure that after the electronic device outputs alarm information, the staff can easily determine which object is suspicious, the identification information of the target object can be output when the alarm information is output after the target object is determined to be a suspicious object. For example, the voice output "the object identified as ID1 is a suspicious object" can be output.

[0123] To facilitate subsequent determination of whether the target object is loitering, based on the above embodiments, in this embodiment of the application, after determining whether the target object is loitering, the method further includes:

[0124] If it is determined that the target object is lingering and the target object in the second image is within the detection area, then it is determined that the target object in the first image is outside the detection area.

[0125] If it is determined that the target object is lingering, and the target object in the second image is outside the detection area, then it is determined that the target object in the first image is within the detection area;

[0126] If it is determined that the target object is not lingering and the target object in the second image is within the detection area, then it is determined that the target object in the first image is within the detection area.

[0127] If it is determined that the target object is not lingering and the target object in the second image is outside the detection area, then it is determined that the target object in the first image is outside the detection area.

[0128] In this embodiment, to facilitate subsequent determination of whether a target object is lingering, it is necessary to save the result of whether the target object is inside or outside the detection area in the first image. Specifically, to determine whether the target object is inside or outside the detection area in the first image, if it is determined that the target object is lingering, it means that the target object may have entered from outside the detection area or from outside the detection area. Therefore, if the target object in the second image is inside the detection area, it is determined that the target object in the first image is outside the detection area; if the target object in the second image is outside the detection area, it is determined that the target object in the first image is inside the detection area.

[0129] If it is determined that the target object is not lingering, it means that the target object has always been within the detection area or has always been outside the detection area. Therefore, if the target object in the second image is within the detection area, it is determined that the target object in the first image is also within the detection area; if the target object in the second image is outside the detection area, it is determined that the target object in the first image is also outside the detection area.

[0130] Figure 8 This application provides a schematic diagram of an alarm-generating process in some embodiments, which is now being discussed. Figure 8 Please provide an explanation.

[0131] In this embodiment of the application, in order to determine whether to issue an alarm, there are four main processes: acquiring video stream, detection and tracking, location determination, and algorithm alarm.

[0132] Specifically, the electronic device first obtains the video stream and decodes it to obtain N video frames. Each video frame is labeled with an analysis region, i.e., a detection region. Then, for each video frame, based on the video frame and the target detection algorithm and target tracking algorithm, the device identifies the corresponding identifier, detection box, and confidence level of the target object.

[0133] Based on the position information of the target object in the detection box in the first and second images, it is determined whether the target object has moved. If it has moved, a fusion decision is made based on whether the target object is in the detection area in the second image, and the number of targets in the detection area among the bottom midpoint, bottom left vertex, and bottom right vertex of the first detection box, to determine whether the target object is wandering.

[0134] After determining whether the target object is loitering, the loitering count of the target object is updated based on the result. When the updated loitering count is greater than the preset loitering count threshold, an alarm message is output.

[0135] The technical solution protected by this application is reliable, real-time, and generalizable, and meets the characteristics of reliability.

[0136] Figure 9 A schematic diagram of a loitering event detection device provided in some embodiments of this application is shown. The device includes:

[0137] The acquisition module 901 is used to acquire a first detection box of a target object contained in a first image, and to determine a second detection box of the target object in a second image acquired before the first image.

[0138] The determining module 902 is used to determine whether the target object has moved based on the position information of the first detection box and the second detection box; if so, it determines whether multiple preset points on the first detection box of the target object are within the detection area, and determines the number of targets at the preset points within the detection area; based on the number of targets and whether the target object in the second image is within the detection area, it determines whether the target object is wandering.

[0139] Furthermore, the determining module 902 is also used to determine that the target object is not wandering if it is determined that the target object has not moved.

[0140] Further, the determining module 902 is specifically configured to: determine the target area corresponding to the intersection of the first detection frame and the second detection frame based on the position information of the first detection frame and the position information of the second detection frame; determine the first area corresponding to the first detection frame and the second area of ​​the second detection frame; determine the first ratio of the target area to the first area and the second ratio of the target area to the second area; determine whether the first ratio and the second ratio are both greater than a preset ratio threshold; if so, determine that the target object has moved; otherwise, determine that the target object has not moved.

[0141] Furthermore, the determining module 902 is specifically used to determine that if the target object in the second image is within the detection area, and if the number of targets is not greater than a preset first value, the target object is lingering; if the number of targets is greater than the preset first value, the target object is determined not to linger.

[0142] Furthermore, the determining module 902 is specifically used to determine that the target object is not lingering if the target object in the second image is not within the detection area and the number of targets is not greater than a preset second value; and to determine that the target object is lingering if the number of targets is greater than the preset second value.

[0143] Furthermore, the device also includes:

[0144] The processing module 903 is used to determine, if it is determined that the target object is lingering, the sum of the saved number of lingering of the target object and the preset target value, and update the sum to the saved number of lingering of the target object; determine whether the updated number of lingering of the target object is greater than the preset number of lingering, and if so, output alarm information.

[0145] Furthermore, the processing module 903 is also configured to determine not to output alarm information if the updated number of wanderings of the target object is not greater than a preset number of wanderings threshold.

[0146] Furthermore, the processing module 903 is specifically used to output the identification information of the target object.

[0147] Furthermore, the determining module 902 is further configured to: if it is determined that the target object is loitering and the target object in the second image is within the detection area, then determine that the target object in the first image is outside the detection area; if it is determined that the target object is loitering and the target object in the second image is outside the detection area, then determine that the target object in the first image is within the detection area; if it is determined that the target object is not loitering and the target object in the second image is within the detection area, then determine that the target object in the first image is within the detection area; if it is determined that the target object is not loitering and the target object in the second image is outside the detection area, then determine that the target object in the first image is outside the detection area.

[0148] Based on the above embodiments, some embodiments of this application also provide an electronic device, such as... Figure 10 As shown, it includes: processor 1001, communication interface 1002, memory 1003 and communication bus 1004, wherein processor 1001, communication interface 1002 and memory 1003 communicate with each other through communication bus 1004.

[0149] The memory 1003 stores a computer program, which, when executed by the processor 1001, causes the processor 1001 to perform the following steps:

[0150] Obtain a first detection bounding box of the target object contained in the first image, and determine a second detection bounding box of the target object in a second image acquired before the first image;

[0151] Based on the position information of the first detection box and the second detection box, determine whether the target object has moved;

[0152] If so, determine whether multiple preset points on the first detection frame of the target object are within the detection area, and determine the target number of preset points within the detection area;

[0153] Based on the number of targets and whether the target object in the second image is within the detection area, it is determined whether the target object is lingering.

[0154] Furthermore, the processor 1001 is also configured to determine that the target object is not wandering if it is determined that the target object has not moved.

[0155] Further, the processor 1001 is specifically configured to: determine the target area corresponding to the intersection of the first detection frame and the second detection frame based on the position information of the first detection frame and the position information of the second detection frame; determine the first area corresponding to the first detection frame and the second area of ​​the second detection frame; determine the first ratio of the target area to the first area and the second ratio of the target area to the second area; determine whether the first ratio and the second ratio are both greater than a preset ratio threshold; if so, determine that the target object has moved; otherwise, determine that the target object has not moved.

[0156] Furthermore, the processor 1001 is specifically configured to determine that the target object is lingering if the target object in the second image is within the detection area and the number of targets is not greater than a preset first value; and to determine that the target object is not lingering if the number of targets is greater than the preset first value.

[0157] Furthermore, the processor 1001 is specifically configured to determine that the target object is not lingering if the target object in the second image is not within the detection area and the number of targets is not greater than a preset second value; and to determine that the target object is lingering if the number of targets is greater than the preset second value.

[0158] Furthermore, the processor 1001 is also configured to, if it is determined that the target object is lingering, determine the sum of the saved number of lingering of the target object and the preset target value, and update the sum to the saved number of lingering of the target object; determine whether the updated number of lingering of the target object is greater than the preset number of lingering threshold, and if so, output alarm information.

[0159] Furthermore, the processor 1001 is also configured to determine not to output alarm information if the updated number of hops of the target object is not greater than a preset hop count threshold.

[0160] Furthermore, the processor 1001 is specifically used to output the identification information of the target object.

[0161] Furthermore, the processor 1001 is further configured to: if it is determined that the target object is loitering and the target object in the second image is within the detection area, then determine that the target object in the first image is outside the detection area; if it is determined that the target object is loitering and the target object in the second image is outside the detection area, then determine that the target object in the first image is within the detection area; if it is determined that the target object is not loitering and the target object in the second image is within the detection area, then determine that the target object in the first image is within the detection area; if it is determined that the target object is not loitering and the target object in the second image is outside the detection area, then determine that the target object in the first image is outside the detection area.

[0162] The communication bus mentioned in the above server can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0163] The communication interface 1002 is used for communication between the above-mentioned electronic device and other devices.

[0164] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0165] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0166] Based on the above embodiments, some embodiments of this application also provide a computer-readable storage medium storing a computer program executable by an electronic device. When the program is run on the electronic device, the electronic device performs the following steps:

[0167] The memory stores a computer program, which, when executed by the processor, causes the processor to perform the following steps:

[0168] Obtain a first detection bounding box of the target object contained in the first image, and determine a second detection bounding box of the target object in a second image acquired before the first image;

[0169] Based on the position information of the first detection box and the second detection box, determine whether the target object has moved;

[0170] If so, determine whether multiple preset points on the first detection frame of the target object are within the detection area, and determine the target number of preset points within the detection area;

[0171] Based on the number of targets and whether the target object in the second image is within the detection area, it is determined whether the target object is lingering.

[0172] Furthermore, the method also includes:

[0173] If it is determined that the target object has not moved, then it is determined that the target object has not lingered.

[0174] Further, determining whether the target object has moved based on the position information of the first detection box and the second detection box includes:

[0175] Based on the position information of the first detection box and the position information of the second detection box, determine the target area corresponding to the intersection of the first detection box and the second detection box;

[0176] Determine the first area corresponding to the first detection box and the second area of ​​the second detection box;

[0177] Determine a first ratio of the target area to the first area, and determine a second ratio of the target area to the second area;

[0178] If both the first ratio and the second ratio are greater than a preset ratio threshold, then the target object is determined to have moved; otherwise, the target object is determined not to have moved.

[0179] Further, determining whether the target object is lingering based on the number of targets and whether the target object in the second image is within the detection area includes:

[0180] If the target object in the second image is within the detection area, then if the number of targets is not greater than a preset first value, it is determined that the target object is loitering; if the number of targets is greater than the preset first value, it is determined that the target object is not loitering.

[0181] Further, determining whether the target object is lingering based on the number of targets and whether the target object in the second image is within the detection area includes:

[0182] If the target object in the second image is not within the detection area, then if the number of targets is not greater than a preset second value, it is determined that the target object is not lingering; if the number of targets is greater than the preset second value, it is determined that the target object is lingering.

[0183] Furthermore, after determining whether the target object is loitering, the method further includes:

[0184] If it is determined that the target object is lingering, then the sum of the saved number of lingering times of the target object and the preset target value is determined, and the sum is updated to the saved number of lingering times of the target object;

[0185] Determine whether the number of times the target object has been updated is greater than a preset threshold for the number of times it has been hovered. If so, output an alarm message.

[0186] Furthermore, the method also includes:

[0187] If the number of lingering attempts after the target object is updated is not greater than the preset lingering attempt threshold, then it is determined that no alarm information will be output.

[0188] Furthermore, the output alarm information includes:

[0189] Output the identification information of the target object.

[0190] Furthermore, after determining whether the target object is loitering, the method further includes:

[0191] If it is determined that the target object is lingering and the target object in the second image is within the detection area, then it is determined that the target object in the first image is outside the detection area.

[0192] If it is determined that the target object is lingering, and the target object in the second image is outside the detection area, then it is determined that the target object in the first image is within the detection area;

[0193] If it is determined that the target object is not lingering and the target object in the second image is within the detection area, then it is determined that the target object in the first image is within the detection area.

[0194] If it is determined that the target object is not lingering and the target object in the second image is outside the detection area, then it is determined that the target object in the first image is outside the detection area.

[0195] In this embodiment, the movement of the target object can be determined by first using the first detection box in the first image and the second detection box in the second image. After determining that the target object has moved, it can be further determined whether the target object is moving inside or outside the detection area, thereby determining whether the target object is lingering. This can effectively improve the accuracy of lingering event detection and reduce the false alarm rate. Moreover, this application can determine whether the target object is lingering based on whether the target object moves back and forth inside or outside the detection area, thus making up for the blind spot of the lingering detection algorithm in the prior art. That is, it makes up for the problem that the lingering detection algorithm based on dwell time will fail when the target object lingers at the edge of the detection area in the prior art. Furthermore, the technical solution protected by this application has reliability, real-time performance, and generalizability, and meets the characteristics of reliability.

[0196] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0197] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0198] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0199] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0200] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A loitering event detection method characterized by, The method includes: Obtain a first detection bounding box of the target object contained in the first image, and determine a second detection bounding box of the target object in a second image acquired before the first image; Based on the position information of the first detection box and the second detection box, determine whether the target object has moved; If so, determine whether multiple preset points on the first detection frame of the target object are within the detection area, and determine the target number of preset points within the detection area, wherein the preset points include the midpoint of the bottom edge, the left vertex of the bottom edge, or the right vertex of the bottom edge. Based on the number of targets and whether the target object in the second image is within the detection area, it is determined whether the target object is lingering. The step of determining whether the target object is lingering based on the number of targets and whether the target object in the second image is within the detection area includes: If the target object in the second image is not within the detection area, then if the number of targets is not greater than a preset second value, it is determined that the target object is not lingering; if the number of targets is greater than the preset second value, it is determined that the target object is lingering.

2. The method of claim 1, wherein, The method further includes: If it is determined that the target object has not moved, then it is determined that the target object has not lingered.

3. The method of claim 1, wherein, Determining whether the target object has moved based on the position information of the first detection box and the second detection box includes: Based on the position information of the first detection box and the position information of the second detection box, determine the target area corresponding to the intersection of the first detection box and the second detection box; Determine the first area corresponding to the first detection box and the second area of ​​the second detection box; Determine a first ratio of the target area to the first area, and determine a second ratio of the target area to the second area; If both the first ratio and the second ratio are greater than a preset ratio threshold, then the target object is determined to have moved; otherwise, the target object is determined not to have moved.

4. The method of claim 1, wherein, The step of determining whether the target object is lingering based on the number of targets and whether the target object in the second image is within the detection area includes: If the target object in the second image is within the detection area, then if the number of targets is not greater than a preset first value, it is determined that the target object is loitering; if the number of targets is greater than the preset first value, it is determined that the target object is not loitering.

5. The method of claim 1, wherein, After determining whether the target object is loitering, the method further includes: If it is determined that the target object is lingering, then the sum of the saved number of lingering times of the target object and the preset target value is determined, and the sum is updated to the saved number of lingering times of the target object; Determine whether the number of times the target object has been updated is greater than a preset threshold for the number of times it has been hovered. If so, output an alarm message.

6. The method of claim 5, wherein, The method further includes: If the number of lingering attempts after the target object is updated is not greater than the preset lingering attempt threshold, then it is determined that no alarm information will be output.

7. The method of claim 5, wherein, The output alarm information includes: Output the identification information of the target object.

8. The method of claim 1, wherein, After determining whether the target object is loitering, the method further includes: If it is determined that the target object is lingering and the target object in the second image is within the detection area, then it is determined that the target object in the first image is outside the detection area. If it is determined that the target object is lingering, and the target object in the second image is outside the detection area, then it is determined that the target object in the first image is within the detection area; If it is determined that the target object is not lingering and the target object in the second image is within the detection area, then it is determined that the target object in the first image is within the detection area. If it is determined that the target object is not lingering and the target object in the second image is outside the detection area, then it is determined that the target object in the first image is outside the detection area.

9. An electronic device, comprising: The electronic device includes a processor and a memory, the memory being used to store program instructions, and the processor being used to execute the computer program stored in the memory to implement the steps of the loitering event detection method according to any one of claims 1-8.

Images