An event detection method and apparatus

Abstract

The application discloses an event detection method and device, which is used to avoid using a large number of preset behavior training materials, realizes event detection meeting an alarm condition based on video frames collected in a monitoring area, and improves the convenience and accuracy of detection. The event detection method provided by the application comprises the following steps: acquiring video frames collected in a preset monitoring area; performing target detection and tracking based on the video frames to obtain a target tracking result; wherein the target tracking result at least comprises one of the following target tracking results: a tracking result of a first category target, a tracking result of a second category target, and a tracking result of a third category target; and using the target tracking result to determine whether an event meeting a preset alarm condition exists, wherein the preset alarm condition comprises that a time and / or position relationship between different category targets meets a preset condition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an event detection method and apparatus. Background Technology

[0002] For a long time, due to the inadequate drainage and sewage facilities of some street-front shops, some shop owners have been indiscriminately dumping domestic sewage onto the road for convenience. This indiscriminate dumping of sewage not only causes road flooding and inconveniences pedestrians, but also emits a pungent odor in the hot summer; in winter, the sewage freezes on the road surface, seriously affecting the safety of passing vehicles and pedestrians. Sometimes, the discharged sewage even contains residue, which can cause blockages in storm drains if dumped there, hindering drainage during rainy days. Summary of the Invention

[0003] This application provides an event detection method and apparatus to avoid using a large amount of preset behavior training material, and to achieve event detection that meets alarm conditions based on video frames collected from the monitored area, thereby improving the convenience and accuracy of detection.

[0004] An event detection method provided in this application includes:

[0005] Acquire video frames collected for a preset monitoring area;

[0006] Target detection and tracking are performed based on the video frames to obtain target tracking results; wherein, the target tracking results include at least one of the following types of target tracking results: tracking results of a first type of target, tracking results of a second type of target, and tracking results of a third type of target;

[0007] Using the target tracking results, it is determined whether there are events that meet preset alarm conditions. The preset alarm conditions include the temporal and / or positional relationships between different types of targets meeting preset conditions.

[0008] This method performs target detection and tracking based on the video frames to obtain target tracking results. The target tracking results include at least one of the following categories: tracking results for a first category of targets, tracking results for a second category of targets, and tracking results for a third category of targets. Using these target tracking results, it determines whether an event meets preset alarm conditions. These preset alarm conditions include the temporal and / or positional relationships between different categories of targets meeting preset conditions. This avoids the need for extensive preset behavior training materials. Based on video frames collected from the monitored area, it achieves event detection that meets alarm conditions, thus improving the convenience of event detection. Furthermore, based on the target tracking results, by associating the time and / or position of different categories of targets with specific alarm event judgment logic, it can avoid false detection results in individual frames, thereby improving the accuracy of alarms.

[0009] In some implementations, determining whether an event meeting preset alarm conditions exists using the target tracking results includes:

[0010] When it is determined that there are candidate first-category targets, candidate second-category targets, and candidate third-category targets:

[0011] If, for any candidate first-category target, the candidate first-category target, the candidate second-category target associated with the candidate first-category target, and the candidate third-category target, the following preset conditions are met, then it is preliminarily determined that an event meeting the alarm conditions exists:

[0012] The candidate third category target first appears in the video frame earlier than the candidate first category target first appears in the video frame; and the candidate third category target first appears in the video frame earlier than the candidate second category target associated with the candidate first category target first appears in the video frame.

[0013] In some implementations, the candidate first category of targets satisfies one or a combination of the following conditions:

[0014] The center of the candidate first category target is located within a preset area;

[0015] The confidence level of the candidate first category target is higher than the preset first confidence threshold;

[0016] The detection boxes of the candidate first category target overlap with the detection boxes of at least one second category target.

[0017] In some implementations, the candidate second category of targets satisfies one or a combination of the following conditions:

[0018] The confidence level of the candidate second category target is higher than the preset second confidence threshold;

[0019] The detection boxes of the candidate second-category targets intersect with the detection boxes of at least one first-category target.

[0020] In some implementations, the candidate third category targets satisfy the following conditions:

[0021] The center of the candidate third category target is located within a preset area.

[0022] In some embodiments, the method further includes:

[0023] An event that meets the alarm criteria is finally determined to exist when the following preset conditions are met:

[0024] When the duration of the candidate third category target exceeds a preset threshold, an event that meets the alarm conditions is finally determined to exist.

[0025] In some embodiments, the method further includes:

[0026] When an event that meets the preset alarm conditions is determined, information about the first category of targets associated with the event is generated and reported based on the target tracking results.

[0027] Another embodiment of this application provides an event detection device, which includes a memory and a processor, wherein the memory is used to store program instructions, and the processor is used to call the program instructions stored in the memory and execute any of the methods described above according to the obtained program.

[0028] Furthermore, according to embodiments, for example, a computer program product for a computer is provided, which includes software code portions that, when the product is run on the computer, perform the steps of the methods defined above. The computer program product may include a computer-readable medium on which the software code portions are stored. Furthermore, the computer program product may be directly loaded into the computer's internal memory and / or sent via a network through at least one of an upload process, a download process, and a push process.

[0029] Another embodiment of this application provides a computer-readable storage medium storing computer-executable instructions for causing the computer to perform any of the methods described above. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of a wastewater dumping detection system provided in an embodiment of this application;

[0032] Figure 2 This is a schematic diagram of the monitoring area for the road in front of the shops along the street, provided in an embodiment of this application.

[0033] Figure 3 A schematic diagram of the processing flow of the deep network recognition module provided in the embodiments of this application;

[0034] Figure 4 A schematic diagram of the processing flow of the alarm logic judgment module provided in the embodiments of this application;

[0035] Figure 5 A flowchart illustrating an event detection method provided in an embodiment of this application;

[0036] Figure 6 This is a schematic diagram of the structure of an event detection device provided in an embodiment of this application. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0038] This application provides an event detection method and apparatus to avoid using a large amount of preset behavior training material, and to achieve event detection that meets alarm conditions based on video frames collected from the monitored area, thereby improving the convenience and accuracy of detection.

[0039] The method and apparatus are based on the same concept of the application. Since the methods and apparatus solve problems in similar ways, the implementation of the apparatus and methods can refer to each other, and the repeated parts will not be described again.

[0040] The terms "first," "second," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] The following examples and embodiments are to be understood as illustrative only. While this specification may refer to "a," "an," or "some" examples or embodiments in several places, this does not mean that every such reference relates to the same example or embodiment, nor does it mean that the feature applies only to a single example or embodiment. Individual features of different embodiments may also be combined to provide other embodiments. Furthermore, terms such as "comprising" and "including" should be understood not to limit the described embodiments to consisting only of those features mentioned; such examples and embodiments may also include features, structures, units, modules, etc., not specifically mentioned.

[0042] The various embodiments of this application will now be described in detail with reference to the accompanying drawings. It should be noted that the order in which the embodiments are presented in this application represents only a chronological order and does not represent the superiority or inferiority of the technical solutions provided by the embodiments.

[0043] In this embodiment, real-time monitoring is implemented on the road in front of street-facing shops to capture images of people pouring water along the street. The captured information and location information are promptly reported to the monitoring center, facilitating penalties for such behavior. This will greatly reduce the occurrence of uncivilized behavior, improve the city's appearance and sanitation, and create a better living environment for citizens.

[0044] It should be noted that the embodiments of this application use the detection of water spillage along the street as an example for illustration, but the technical solutions provided in the embodiments of this application are not limited to the detection of this type of alarm event, and can also be applied to the detection of other types of alarm events according to actual needs.

[0045] In the scenario of street-front shop monitoring, this application embodiment can accurately identify street-front sewage splashing behavior events (i.e., events that meet preset alarm conditions) by detecting human bodies (i.e., the first type of target), basins and buckets (i.e., the second type of target) and road water accumulation (i.e., the third type of target) and with strict alarm logic judgment, without the need for a large number of preset behavioral video images as training material.

[0046] This application uses deep learning as the basic method, and the overall system processing flow is as follows: Figure 1 As shown, the system first acquires surveillance image data of the road in front of shops along the street through cameras and sets the monitoring area. Then, a deep network recognition module detects targets such as people, buckets, and water accumulation on the road. A correlation is established between people and the buckets they are carrying. The integrated information is then sent to a multi-target tracking module, which determines the tracking results for each target. The tracking results for people and buckets include their movement trajectory and ID, while the tracking results for water accumulation targets include the area of ​​the water accumulation. These tracking results are then sent to an alarm logic judgment module to determine the nature of the illegal dumping of sewage along the street. Finally, the judgment result for the illegal dumping of sewage along the street is output.

[0047] Examples are given below. Figure 1 The processing flow of each module in the process:

[0048] Surveillance video data acquisition module:

[0049] Smart cameras are installed in front of shops along the street. These cameras need to clearly capture real-time footage of the area in front of the shops. The data is decoded using video encoding and decoding technology, and then transmitted to the subsequent algorithm module for analysis. A detection area (i.e., a preset area) is set within the pre-defined monitoring area of ​​the road in front of the shops. Figure 2 As shown in the image.

[0050] This is used in the deep network recognition module for image analysis and alarm logic judgment.

[0051] Deep network recognition module:

[0052] The deep network recognition module consists of a deep learning-based object detection network A and an object detection network B, such as... Figure 3 As shown, video sequences of the road in front of shops along the street were collected, and the video frames were used as images for training. First, a training dataset for object detection network A was created, with objects such as people and buckets labeled as positive samples, so that object detection network A can detect both people and buckets simultaneously. Then, a training dataset for object detection network B was created, with objects such as water accumulation on the road labeled as positive samples, so that object detection network B can identify water accumulation on the road.

[0053] See Figure 3 The deep network recognition module's internal processing of the current frame image includes:

[0054] Step 1: The current frame image is simultaneously input into both object detection network A and object detection network B; where:

[0055] The target detection network A detects two types of targets: human body and basin / bucket. It obtains information such as the target detection bounding box Rect, confidence score ConfOD (a decimal between 0 and 1), and target category TypeOD (category 1, representing human body; category 2, representing basin / bucket). The target detection bounding box includes human body detection bounding box and / or basin / bucket detection bounding box. For example, the detection bounding box of each target includes the coordinates of the upper left corner (x1, y1) and the lower right corner (x2, y2) of the target's bounding rectangle.

[0056] It should be noted that the detection frame described in the embodiments of this application can also be called a coordinate frame.

[0057] The target detection network B identifies the road surface water and obtains the target area R (i.e., the road surface water area) and the target category TypeOS (category 3, indicating road surface water).

[0058] Step 2: For each target type of basin or bucket, determine whether there is an intersection with the human body detection box (i.e., whether there is an IOU (Intersection Over Union) between the images within the detection boxes of the two). If there is an intersection, it means that the human body detection box and the basin or bucket detection box have overlapping areas. Then mark the flag of this human body carrying a basin or bucket as 1, indicating that the human body is carrying a basin or bucket.

[0059] In step two, the flag for the human carrying a bucket is set to 1, indicating that the human target is a candidate human target, meaning that event detection will be performed on such human targets in the future. Furthermore, an association is established between this candidate human target and the bucket target, for example, establishing a correspondence between the identifier of the human target and the identifier of the bucket target carried by the human target. See, for example... Figure 2 The human body in human body detection box A is not a candidate human body target, while the human body in human body detection box B is a candidate human body target.

[0060] Step 3: Output the detection results to the multi-target tracking module. Regarding the output detection results:

[0061] The first type, when the target category TypeOD is 1 (human body), the detection results input to the multi-target tracking module include: human body detection box Rect, confidence score ConfOD, current frame human body carrying bucket flag, bucket detection box Rect.

[0062] To distinguish them from other pedestrians on the road, people with the flag on the sign initially assumed they were the ones splashing water.

[0063] The second type, when the target category TypeOD is 2 (basin / bucket), the detection results input to the multi-target tracking module include: basin / bucket detection box Rect, confidence score ConfOD;

[0064] The third type, when the target category TypeOD is 3 (road surface water), the detection results input to the multi-target tracking module include: the segmented target area R (i.e., the road surface water area).

[0065] It should be noted that if all three types of targets exist in the same frame, the detection results output to the multi-target tracking module for that frame will include all three types of detection results.

[0066] Multi-target tracking module:

[0067] In this embodiment, the multi-target tracking algorithm in the multi-target tracking module employs, for example, the Tracking by Detection method. Based on the detection results output by the deep network recognition module for each frame of image, a matching algorithm (e.g., Hungarian matching algorithm) is used to match the current target detection object with existing tracking trajectories to form a new tracking trajectory for the target. In this embodiment, four states are set for the tracked target: Create, Update, Lost, and Delete.

[0068] The Tracking by Detection method involves multi-target tracking based on detection. Target detection is performed in each frame, and the results are then used for target tracking. The target detection algorithm outputs a single-frame detection result, while the target tracking algorithm is responsible for associating targets in consecutive frames and assigning them a unique track-id. Assuming M targets are detected in frame t and N targets are detected in frame t+1, the tracking algorithm is essentially an M->N matching and association process.

[0069] The matching process, which compares the current target detection object with existing tracking trajectories, includes calculating the Intersection over Union (IoU) between the target detection bounding box in the current frame and the tracking detection bounding box in the previous frame. A successful match is considered achieved when the IoU is greater than or equal to a preset threshold. If the target detection bounding box in the current frame matches multiple tracking detection bounding boxes in the previous frame, the tracking bounding box with the highest IoU is selected. Because the target is constantly moving, the target detection bounding box in the current frame may intersect with multiple target detection bounding boxes in the previous frame; therefore, the box with the largest intersection is considered to represent the same target. The following situations occur during the matching process:

[0070] Case 1: When the detected target can match an existing tracking trajectory, update the tracking trajectory and set the status bit of the tracked target to Update.

[0071] In this context, an IOU match between the target bounding box in the current frame and the tracking bounding box in the previous frame indicates a match between the detected object and the existing tracking trajectory. The tracking trajectory of the target refers to the sequence of positions of the target from the first frame it appears in to the current frame, specifically including the target category, target detection box coordinates, etc.

[0072] Scenario 2: If no matching tracking trajectory can be found for the currently detected object, it means that the target is a newly appeared tracking target. In this case, a tracking trajectory is created for it, including the creation of the starting target bounding box position, the tracking target's status bit being "Create", and the target being assigned a number (i.e., ID).

[0073] Case 3: If the tracking trajectory of a certain target does not have a matching target in the current frame, it means that the target has been lost in the video, and the status bit of the target is Lost.

[0074] Case 4: When the status bit of a tracked target is Lost for more than 12 frames, the status bit of the tracked target is updated to Delete, and the tracking trajectory of the tracked target is deleted.

[0075] When the target's status bit is Update, the target's tracking trajectory information and ID are transmitted to the alarm logic judgment module.

[0076] Alarm logic judgment module:

[0077] For the specific judgment process of the alarm logic judgment module, please refer to [link / reference]. Figure 4 ,include:

[0078] S401. Loop through and acquire the tracking results for each candidate target, i.e., acquire the tracking results for each target in the loop tracking module. The tracking results include: when the target category is a person or a bucket, obtain the target's category TypeOD, confidence level ConfOD, target's detection box Rect, the initial frame number f of the target's appearance, and the target's historical motion trajectory; when the target category is road surface water, obtain the target's target region R (see...). Figure 2 ), the outer rectangle S of the target region R, and the frame number f of the initial appearance of the target; in this embodiment, the target tracking module outputs the tracking trajectory of the current target, while the loop tracking module is used to traverse these tracked targets and obtain the historical trajectory information of each target.

[0079] S402. For the tracking result of each target in the loop tracking module, determine whether the target is a human body type, a bucket type, or a road surface water type. If it is a human body type, execute S403; if it is a bucket type, execute S406; if it is a road surface water type, execute S408.

[0080] S403. Determine the coordinates of the center point of this human body type target. ((x2+x1) / 2,(y2+y1) / 2) Is it within the designated area of ​​the road in front of the shop (e.g.) Figure 2 If the target is within the monitored area frame, execute S404; otherwise, return to the loop tracking module. Here, (x1, y1) is the coordinate of the upper left corner of the target's detection frame, and (x2, y2) is the coordinate of the lower right corner of the target's detection frame.

[0081] The return loop tracking module described in this application embodiment means that when the target does not meet the requirements, the target is not further judged, and the loop tracking module is returned to obtain the tracking results of other targets. The steps corresponding to the target category are then executed, that is, the process returns to step S401 to obtain the tracking results of another candidate target and execute subsequent steps.

[0082] S404. Determine whether the confidence level (ConfOD) of this human body type target is greater than or equal to the threshold (th1). If yes, proceed to S405; otherwise, return to the loop tracking module. The purpose of this step is to filter out false detection targets with low confidence levels, thereby further improving the accuracy of alarm judgment for illegal sewage dumping incidents along the street.

[0083] S405. Determine if the flag of the bucket carried by the current target human body is 1. If it is 1, execute S409; otherwise, return to the loop tracking module.

[0084] S406. Determine if the confidence level (ConfOD) of the target container is greater than or equal to the threshold (th2). If yes, proceed to S407; otherwise, return to the loop tracking module. This step is to filter out false detection targets with low confidence levels, thereby further improving the accuracy of alarm judgment for illegal sewage dumping along the street.

[0085] S407. Loop tracking results, that is, for each human target in the tracking results, calculate the relationship between the human target and the current basin / bucket position, and determine whether there is an intersection between the human target and the current basin / bucket based on the position relationship;

[0086] In other words, this step calculates whether there is an Interchange of Union (IOU) between each human detection box on the current frame image and the detection box of the current bucket target;

[0087] If it exists, execute S409; otherwise, return to the loop tracing module.

[0088] The purpose of this step is to filter out targets such as basins and buckets placed on the ground, thereby further improving the accuracy of alarm judgment on sewage dumping incidents along the street.

[0089] S408. Construct the minimum bounding rectangle for the waterlogged area R on the road surface, resulting in rectangle S, where the center point of rectangle S is (xs, ys). Determine whether this center point (xs, ys) is within the designated area of ​​the road in front of the shops (e.g., ...). Figure 2 If the detection area shown in the diagram is within the specified range, then execute S409; otherwise, return to the alarm logic judgment module.

[0090] S409. The frame number of the first appearance of the human body is f1, the frame number of the first appearance of the bucket / basin target bound to the human body is f2, and the frame number of the first appearance of the road water target is f3. Determine whether (f3-f1>0)&&(f3-f2>0) is true. If yes, execute S410; otherwise, return to the loop tracking module. This step is to determine the time relationship between the appearance of the human body, bucket / basin, and road water. The determination of (f3-f1>0)&&(f3-f2>0) is to ensure that the road water target appears after the human body and bucket / basin target, and to determine that the water target was splashed by the human body target.

[0091] S410. Summarize the information of human targets, bucket targets, and road surface water targets in the current frame, including, for example: the frame number of the first appearance of each target, the cumulative number of consecutive detection frames of road surface water targets, the coordinate frame, and the confidence level. Furthermore, increment the road surface water existence time t by 1; t is used to record the cumulative alarm time, and an alarm will only be reported if it exceeds the set threshold t1. Here, t1 can be configured by the user.

[0092] S411. Determine whether time t is greater than or equal to threshold t1. If so, execute S412. If less than, return to the alarm logic judgment module.

[0093] S412. The water-splashing human target obtained in S410 is uploaded to the monitoring center as an alarm result, and law enforcement officers can impose penalties on the behavior.

[0094] As can be seen, steps S403-S407 can provide human targets and bucket targets that meet the judgment conditions. Then, these targets and the water-covered targets on the road are judged according to the conditions in steps S409-S410. If the conditions are met, an alarm is triggered and the alarm result is uploaded to the monitoring center. The monitoring center locates the human target splashing water based on the alarm image.

[0095] In summary, the embodiments of this application have a complete logical judgment rule module, with progressively advancing steps, rigorous logic, and strong practicality. Based on the target trajectory output by the multi-target tracking module, the target category, the current frame position, and the target's first appearance time can be obtained. By calculating the positional relationship between the pedestrian and the bucket, it can be determined whether the pedestrian is carrying the bucket target. Then, by calculating the time relationship between the frame number of the first appearance of the human body, the frame number of the first appearance of the bucket target bound to the human body, and the frame number of the first appearance of the water accumulation target on the road, the behavior of dumping sewage in front of shops can be accurately identified, and a complete chain of evidence can be reported. Through judgment logic such as target position association and target appearance time sequence association, false detection results in individual frames are avoided, thereby improving the accuracy of alarms.

[0096] Furthermore, this embodiment utilizes a target detection network to detect human bodies, containers, and water-covered targets. Human bodies are associated with containers, and water-covered targets are associated with water-covered targets via time-series analysis. This method effectively identifies illegal dumping of sewage and offers strong real-time performance. The detection and identification results are sent to a multi-target tracking module to obtain the target's trajectory and ID. The trajectory allows for the calculation of the target's appearance time and location information, facilitating alarm logic judgment.

[0097] Through the above specific embodiments, see Figure 5 Therefore, the event detection method provided in this application includes:

[0098] S101. Acquire video frames for a preset monitoring area (e.g., target street);

[0099] S102. Target detection and tracking are performed based on the video frames to obtain target tracking results; wherein the target tracking results include at least one of the following types of target tracking results: tracking results of a first type of target (e.g., human body), tracking results of a second type of target (e.g., basin or bucket), and tracking results of a third type of target (e.g., water on the road).

[0100] Each type of target may have multiple target tracking results.

[0101] S103. Using the target tracking results, determine whether there is an event that meets the preset alarm conditions. The preset alarm conditions include the time and / or positional relationship between different types of targets meeting the preset conditions.

[0102] In some implementations, determining whether an event meeting preset alarm conditions exists using the target tracking results includes:

[0103] When it is determined that there are candidate first-category targets, candidate second-category targets, and candidate third-category targets:

[0104] If, for any candidate first-category target, the candidate first-category target, the candidate second-category target associated with the candidate first-category target, and the candidate third-category target (which may also be associated with the candidate first-category target) satisfy the following preset conditions, then it is preliminarily determined that an event meeting the alarm conditions exists:

[0105] The candidate third category target first appears in the video frame earlier than the candidate first category target first appears in the video frame; and the candidate third category target first appears in the video frame earlier than the candidate second category target associated with the candidate first category target first appears in the video frame.

[0106] In some embodiments, this can be used to determine whether an event that meets the alarm criteria exists. Of course, for a more accurate determination, further judgment steps can be performed to ultimately determine whether an event that meets the alarm criteria exists.

[0107] Among them, the first category of candidate targets, the second category of candidate targets, and the third category of candidate targets are the targets output to the alarm logic judgment module in the above embodiment.

[0108] In some implementations, the candidate first category of targets satisfies one or a combination of the following conditions:

[0109] The center of the candidate first category target is located in a preset area (e.g. Figure 2 Within the pedestrian walkway area shown;

[0110] The confidence level of the candidate first category target is higher than the preset first confidence threshold;

[0111] The detection boxes of the candidate first category target overlap with the detection boxes of at least one second category target.

[0112] In some implementations, the candidate second category of targets satisfies one or a combination of the following conditions:

[0113] The confidence level of the candidate second category target is higher than the preset second confidence threshold;

[0114] The detection boxes of the candidate second-category targets intersect with the detection boxes of at least one first-category target.

[0115] In some implementations, the candidate third category targets satisfy the following conditions:

[0116] The center of the candidate third category target is located within a preset area.

[0117] In some implementations, to further improve the accuracy of alarm event detection, the method further includes:

[0118] An event that meets the alarm criteria is finally determined to exist when the following preset conditions are met:

[0119] When the duration of the candidate third-category target exceeds a preset threshold, an event that meets the alarm conditions is finally determined to exist. For example, an alarm event is finally determined to exist only when the cumulative duration of water accumulation exceeds a threshold, that is, when water accumulation is detected in multiple consecutive frames (the cumulative number of consecutively detected water accumulation exceeds a preset number of frames).

[0120] In some embodiments, the method further includes:

[0121] When an event that meets the preset alarm conditions is determined, information about the first category of targets associated with the event is generated and reported based on the target tracking results, such as screenshots of the water splasher, time, location, etc.

[0122] The following describes the device or apparatus provided in the embodiments of this application, and the explanations or examples of the same or corresponding technical features as those described in the above methods will not be repeated hereafter.

[0123] See Figure 6 An event detection device provided in this application includes:

[0124] Processor 500 is used to read the program from memory 520 and execute the following procedures:

[0125] Acquire video frames collected for a preset monitoring area;

[0126] Target detection and tracking are performed based on the video frames to obtain target tracking results; wherein, the target tracking results include at least one of the following types of target tracking results: tracking results of a first type of target, tracking results of a second type of target, and tracking results of a third type of target;

[0127] Using the target tracking results, it is determined whether there are events that meet preset alarm conditions. The preset alarm conditions include whether the time and / or positional relationships between different types of targets meet preset conditions.

[0128] In some implementations, determining whether an event meeting preset alarm conditions exists using the target tracking results includes:

[0129] When it is determined that there are candidate first-category targets, candidate second-category targets, and candidate third-category targets:

[0130] If, for any candidate first-category target, the candidate first-category target, the candidate second-category target associated with the candidate first-category target, and the candidate third-category target, the following preset conditions are met, then it is preliminarily determined that an event meeting the alarm conditions exists:

[0131] The candidate third category target first appears in the video frame earlier than the candidate first category target first appears in the video frame; and the candidate third category target first appears in the video frame earlier than the candidate second category target associated with the candidate first category target first appears in the video frame.

[0132] In some implementations, the candidate first category of targets satisfies one or a combination of the following conditions:

[0133] The center of the candidate first category target is located within a preset area;

[0134] The confidence level of the candidate first category target is higher than the preset first confidence threshold;

[0135] The detection boxes of the candidate first category target overlap with the detection boxes of at least one second category target.

[0136] In some implementations, the candidate second category of targets satisfies one or a combination of the following conditions:

[0137] The confidence level of the candidate second category target is higher than the preset second confidence threshold;

[0138] The detection boxes of the candidate second-category targets intersect with the detection boxes of at least one first-category target.

[0139] In some implementations, the candidate third category targets satisfy the following conditions:

[0140] The center of the candidate third category target is located within a preset area.

[0141] In some implementations, the processor 500 is also configured to read a program from the memory 520 and execute the following processes:

[0142] An event that meets the alarm criteria is finally determined to exist when the following preset conditions are met:

[0143] When the duration of the candidate third category target exceeds a preset threshold, an event that meets the alarm conditions is finally determined.

[0144] In some implementations, the processor 500 is also configured to read a program from the memory 520 and execute the following processes:

[0145] When an event that meets the preset alarm conditions is determined, information about the first category of targets associated with the event is generated and reported based on the target tracking results.

[0146] Transceiver 510 is used to receive and send data under the control of processor 500.

[0147] Among them, Figure 6 In this context, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 500) and memory (memory 520). The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 510 may be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. The processor 500 is responsible for managing the bus architecture and general processing, and the memory 520 may store data used by the processor 500 during operation.

[0148] The processor 500 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD).

[0149] It should be noted that the division of units in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0150] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0151] This application provides a computing device, which may specifically be a desktop computer, portable computer, smartphone, tablet computer, personal digital assistant (PDA), etc. The computing device may include a central processing unit (CPU), memory, input / output devices, etc. Input devices may include a keyboard, mouse, touchscreen, etc., and output devices may include display devices, such as a liquid crystal display (LCD) or a cathode ray tube (CRT).

[0152] The memory may include read-only memory (ROM) and random access memory (RAM), and provides the processor with program instructions and data stored in the memory. In the embodiments of this application, the memory may be used to store the program of any of the methods provided in the embodiments of this application.

[0153] The processor executes any of the methods described in the embodiments of this application according to the program instructions stored in the memory.

[0154] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the methods described in the above embodiments. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0155] This application provides a computer-readable storage medium for storing computer program instructions used in the apparatus provided in the above-described embodiments, including a program for performing any of the methods provided in the above-described embodiments. The computer-readable storage medium may be a non-transitory computer-readable medium.

[0156] The computer-readable storage medium can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0157] It should be understood that:

[0158] The access technology used by entities in a communication network to transmit traffic can be any suitable current or future technology, such as WLAN (Wireless Local Access Network), WiMAX (Microwave Access Global Interoperability), LTE, LTE-A, 5G, Bluetooth, infrared, etc.; in addition, embodiments may also apply wired technologies, such as IP-based access technologies, such as wired networks or fixed lines.

[0159] An embodiment suitable for implementation as software code or as part thereof and for operation using a processor or processing function is independent of the software code and can be specified using any known or future-developed programming language, such as high-level programming languages ​​such as Objective-C, C, C++, C#, Java, Python, Javascript, other scripting languages, etc., or low-level programming languages ​​such as machine language or assembler.

[0160] The implementation of the embodiments is hardware-independent and can be implemented using any known or future-developed hardware technology or any combination thereof, such as microprocessors or CPUs (central processing units), MOS (metal-oxide-semiconductor), CMOS (complementary MOS), BiMOS (bipolar MOS), BiCMOS (bipolar CMOS), ECL (emitter-coupled logic), and / or TTL (transistor-transistor logic).

[0161] The embodiments may be implemented as individual devices, apparatuses, units, components or functions, or in a distributed manner. For example, one or more processors or processing functions may be used or shared in the process, or one or more processing segments or processing portions may be used and shared in the process, wherein one or more physical processors may be used to implement one or more processing portions dedicated to a particular process as described.

[0162] The device can be implemented by a semiconductor chip, a chipset, or a (hardware) module that includes such a chip or chipset.

[0163] The implementation can also be implemented as any combination of hardware and software, such as ASIC (Application-Specific IC (Integrated Circuit)) components, FPGA (Field Programmable Gate Array) or CPLD (Complex Programmable Logic Device) components or DSP (Digital Signal Processor) components.

[0164] The embodiments can also be implemented as computer program products, including a computer-usable medium in which computer-readable program code is embodied, the computer-usable program code being adapted to perform the processes described in the embodiments, wherein the computer-usable medium may be a non-transitory medium.

[0165] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0166] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0167] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0168] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, this application also intends to include such modifications and variations if they fall within the scope of the claims of this application and their equivalents.

Claims

1. An event detection method, characterized in that, The method is used for real-time monitoring of the road in front of street-facing shops, capturing images of people pouring water along the street, and promptly reporting the captured information and location information to the monitoring center. Acquire video frames collected for a preset monitoring area; Target detection and tracking are performed based on the video frames to obtain target tracking results; wherein, the target tracking results include: tracking results of a first category of targets, tracking results of a second category of targets, and tracking results of a third category of targets; wherein, the first category of targets is a human body, the second category of targets is a basin or bucket, and the third category of targets is water accumulation on the road surface; the tracking results of the first category of targets include the motion trajectory and ID of the human body, the tracking results of the second category of targets include the motion trajectory and ID of the basin or bucket, and the tracking results of the third category of targets include the area of ​​water accumulation on the road surface; Using the tracking results of the first category of targets, the tracking results of the second category of targets, and the tracking results of the third category of targets, it is determined whether there is a street-side water-pouring behavior event that meets the preset alarm conditions. The preset alarm conditions include that the time and location relationships between different categories of targets meet the preset conditions. The step of using the target tracking results to determine whether there is a street-side water-pouring behavior event that meets preset alarm conditions includes: When it is determined that there are candidate first-category targets, candidate second-category targets, and candidate third-category targets: If, for any candidate first-category target, the candidate first-category target, the candidate second-category target associated with the candidate first-category target, and the candidate third-category target, the following preset conditions are met, then it is preliminarily determined that there is a street-side water-pouring behavior event that meets the alarm conditions: The candidate third category target first appears in the video frame later than the candidate first category target first appears in the video frame; and the candidate third category target first appears in the video frame later than the candidate second category target associated with the candidate first category target first appears in the video frame. The method further includes: When the following preset conditions are met, it is finally determined that there is a street-side water-pouring behavior event that meets the alarm conditions: When the duration of the candidate third category target exceeds a preset threshold, it is finally determined that there is a street-side water-pouring behavior event that meets the alarm conditions.

2. The method according to claim 1, characterized in that, The candidate first category target satisfies one or a combination of the following conditions: The center of the candidate first category target is located within a preset area; The confidence level of the candidate first category target is higher than the preset first confidence threshold; The detection boxes of the candidate first category target overlap with the detection boxes of at least one second category target.

3. The method according to claim 1, characterized in that, The candidate second category target satisfies one or a combination of the following conditions: The confidence level of the candidate second category target is higher than the preset second confidence threshold; The detection boxes of the candidate second-category targets intersect with the detection boxes of at least one first-category target.

4. The method according to claim 1, characterized in that, The candidate third category target satisfies the following condition: The center of the candidate third category target is located within a preset area.

5. The method according to claim 1, characterized in that, The method further includes: When it is determined that there is a street-side water-pouring behavior event that meets the preset alarm conditions, information of the first category of targets associated with the street-side water-pouring behavior event is generated and reported based on the target tracking results.

6. An event detection device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method according to any one of claims 1 to 5.

7. A computer program product for use in a computer, characterized in that, Includes a software code portion that, when the product is run on the computer, is used to perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing the computer to perform the method according to any one of claims 1 to 5.

Images