Alarm event processing method and device
By automating the processing of surveillance videos from urban management cameras and using target detection and tracking algorithms to generate alarm events and law enforcement evidence, the problem of overlooked violations and low efficiency in urban governance has been solved, achieving efficient violation detection and evidence generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA TELECOM CORP LTD
- Filing Date
- 2024-03-05
- Publication Date
- 2026-06-26
AI Technical Summary
In the process of urban governance, existing technologies cannot fully capture images of violations, resulting in omissions of violations, and relying on manual methods to acquire images of violations is inefficient.
By acquiring surveillance videos from urban management cameras, the system uses target detection and multi-target tracking algorithms to extract illegal objects from the surveillance images, generate alarm events and law enforcement evidence, and merges and processes the data based on camera identifiers and feature vectors to achieve automated alarm event processing.
Automated urban governance has been achieved, improving the coverage and efficiency of violation detection, reducing manual intervention, and forming a traceable chain of alarm evidence.
Smart Images
Figure CN118097506B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and more specifically, to an alarm event handling method and apparatus. Background Technology
[0002] In the process of urban governance, there are still problems such as littering, sewage dumping, and illegal parking in back streets, alleys, and old residential areas. At present, urban governance teams mainly rely on manual guarding or manual review of video to obtain images of violations. However, this method cannot fully obtain images of violations, and it is easy to miss some violations. In addition, the work efficiency of obtaining images of violations manually is low.
[0003] There is currently no effective solution to the above problems. Summary of the Invention
[0004] This application provides an alarm event processing method and apparatus to at least solve the technical problem that the current urban governance scenario requires manual collection of evidence of violations of alarm events before processing, resulting in low overall efficiency.
[0005] According to one aspect of the embodiments of this application, an alarm event processing method is provided, comprising: acquiring multiple surveillance videos collected by multiple cameras used for urban management; for each surveillance video, extracting multiple frames of surveillance images from the surveillance video, and determining multiple violating objects in the surveillance video based on the multiple frames of surveillance images; for each violating object in the surveillance video, generating an alarm event corresponding to the violating object, determining the duration of the alarm event, generating law enforcement evidence of the alarm event based on the multiple frames of surveillance images corresponding to the violating object, and generating an evidence identifier for the law enforcement evidence based on the camera identifier corresponding to the surveillance video, the violation event type of the violating object, and the feature vector of the violating object in the corresponding surveillance image; merging the alarm events and law enforcement evidence corresponding to the same violating object of the same violation event type under different cameras based on the evidence identifier; and processing the alarm events according to the duration of the merged alarm events.
[0006] Optionally, before acquiring multiple surveillance videos collected by multiple cameras used for urban management, the method further includes: using a video access device to manage all cameras used for urban management within the target area and recording the camera identifier of each camera.
[0007] Optionally, for each surveillance video segment, multiple frames of surveillance images are extracted from the surveillance video, and multiple violations in the surveillance video are determined based on the multiple frames of surveillance images. This includes: for each surveillance video segment, performing frame extraction processing on the surveillance video according to a preset frame extraction interval to obtain multiple frames of surveillance images; using a target detection algorithm to determine the feature vector of each suspected violation object in each frame of surveillance images; using a multi-target tracking algorithm to analyze the feature vector of each suspected violation object in each frame of surveillance images, determining whether each suspected violation object is a violation object based on the analysis results, and recording the violation event type and feature vector corresponding to the violation object when determining the violation object.
[0008] Optionally, the duration of an alarm event includes at least one of the following: alarm start time, alarm end time, and alarm duration. For each violating object in the surveillance video, an alarm event corresponding to the violating object is generated, and the duration of the alarm event is determined, including: for each violating object, generating an alarm event corresponding to the violating object; determining the alarm start time of the alarm event based on the timestamp of the first frame of the surveillance image including the violating object; if the last frame of the surveillance image including the violating object is the last frame of the surveillance video, determining that the alarm event has not yet ended, and determining the alarm duration of the alarm event based on the timestamp of the last frame of the surveillance image including the violating object and the alarm start time; if the last frame of the surveillance image including the violating object is not the last frame of the surveillance video, determining that the alarm event has ended, determining the alarm end time of the alarm event based on the timestamp of the last frame of the surveillance image including the violating object, and determining the alarm duration of the alarm event based on the alarm start time and the alarm end time.
[0009] Optionally, law enforcement evidence for an alarm event is generated based on multiple frames of surveillance images corresponding to the violating object, including: forming an image evidence chain for the alarm event by arranging the multiple frames of surveillance images including the violating object in chronological order; extracting a video segment of a preset duration from the corresponding surveillance video as video auxiliary evidence for the alarm event, starting from the alarm start time of the alarm event; and using the image evidence chain and video auxiliary evidence together as law enforcement evidence for the alarm event.
[0010] Optionally, alarm events and law enforcement evidence corresponding to the same violation object of the same violation event type under different cameras are merged based on evidence identifiers, including: for each evidence identifier with different camera identifiers but the same violation event type, the feature vectors in each evidence identifier are compared pairwise for similarity, and the violation object corresponding to the evidence identifier with a similarity higher than a preset similarity threshold is determined to be the same violation object; all alarm events and law enforcement evidence corresponding to the same violation object of the same violation event type are merged.
[0011] Optionally, the alarm event can be processed according to its duration, including: if the alarm event has terminated, automatically closing the case; if the alarm event has not terminated, determining whether the alarm duration exceeds a preset duration threshold; if the alarm duration does not exceed the preset duration threshold, not processing the alarm event; if the alarm duration exceeds the preset duration threshold, dispatching law enforcement personnel to handle the alarm event on-site.
[0012] According to another aspect of the embodiments of this application, an alarm event processing device is also provided, comprising: an acquisition module for acquiring multiple segments of surveillance video collected by multiple cameras used for urban management; a determination module for extracting multiple frames of surveillance images from each surveillance video segment and determining multiple violating objects in the surveillance video based on the multiple frames of surveillance images; an alarm generation module for generating an alarm event corresponding to each violating object in the surveillance video, determining the duration of the alarm event, generating law enforcement evidence of the alarm event based on the multiple frames of surveillance images corresponding to the violating object, and generating an evidence identifier for the law enforcement evidence based on the camera identifier corresponding to the surveillance video, the violation event type of the violating object, and the feature vector of the violating object in the corresponding surveillance image; a merging module for merging alarm events and law enforcement evidence corresponding to the same violating object of the same violation event type under different cameras based on the evidence identifier; and an alarm processing module for processing the merged alarm events based on the duration of the alarm events.
[0013] According to another aspect of the embodiments of this application, a computer program product is also provided, the computer program product comprising: a computer program, wherein the computer program, when executed by a processor, implements the above-described alarm event handling method.
[0014] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described alarm event handling method through the computer program.
[0015] In this embodiment, multiple surveillance videos collected by multiple cameras used for urban management are acquired. For each surveillance video, multiple frames of surveillance images are extracted, and multiple violating objects in the surveillance video are identified based on the multiple frames of surveillance images. For each violating object in the surveillance video, an alarm event corresponding to the violating object is generated, and the duration of the alarm event is determined. Law enforcement evidence of the alarm event is generated based on the multiple frames of surveillance images corresponding to the violating object, and an evidence identifier of the law enforcement evidence is generated based on the camera identifier corresponding to the surveillance video, the violation event type of the violating object, and the feature vector of the violating object in the corresponding surveillance image. Based on the evidence identifier, alarm events and law enforcement evidence corresponding to the same violating object of the same violation event type under different cameras are merged. For each merged alarm event, the alarm event is processed according to the duration of the alarm event. Among them, by automatically extracting frames from surveillance videos of urban management cameras to identify violators, and by conducting multi-dimensional analysis based on camera identification, violation type, and characteristics of the violator, multiple alarm evidence chains are generated. By merging the evidence chains of the same violator under the same violation type from different cameras, traceable alarm events can be automatically formed, thereby achieving "contactless" urban governance, improving work efficiency, and effectively solving the technical problem of low overall efficiency in current urban governance scenarios that require manual collection of violation evidence for alarm events before processing. Attached Figure Description
[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0017] Figure 1 This is a schematic diagram of the structure of an optional computer terminal according to an embodiment of this application;
[0018] Figure 2 This is a flowchart illustrating an optional alarm event handling method according to an embodiment of this application;
[0019] Figure 3 This is a schematic diagram of an optional alarm event handling process in a vehicle illegal parking scenario according to an embodiment of this application;
[0020] Figure 4 This is an optional alarm event handling process in a trash can lid status detection scenario according to an embodiment of this application;
[0021] Figure 5 This is a schematic diagram of an optional alarm event processing device according to an embodiment of this application. Detailed Implementation
[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0023] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0024] Example 1
[0025] According to an embodiment of this application, an alarm event handling method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0026] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing an alarm event handling method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.
[0027] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).
[0028] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the alarm event handling method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned application vulnerability detection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0029] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.
[0030] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).
[0031] Under the above operating environment, this application embodiment provides an alarm event handling method, such as... Figure 2 As shown, the method includes the following steps:
[0032] Step S202: Acquire multiple surveillance videos collected by multiple cameras used for urban management;
[0033] Step S204: For each surveillance video segment, extract multiple frames of surveillance images from the surveillance video, and determine multiple violating objects in the surveillance video based on the multiple frames of surveillance images.
[0034] Step S206: For each violating object in the surveillance video, generate an alarm event corresponding to the violating object, determine the duration of the alarm event, generate law enforcement evidence of the alarm event based on the multi-frame surveillance images corresponding to the violating object, and generate evidence identifier of the law enforcement evidence based on the camera identifier corresponding to the surveillance video, the violation event type of the violating object, and the feature vector of the violating object in the corresponding surveillance image.
[0035] Step S208: Merge alarm events and law enforcement evidence corresponding to the same violation object of the same violation type under different cameras according to the evidence identifier;
[0036] Step S210: For each merged alarm event, process the alarm events according to their ongoing status.
[0037] The following section explains each step of the alarm event handling method in conjunction with the specific implementation process.
[0038] Before acquiring multiple surveillance videos from various cameras used for urban management, a video access device can be used to manage all cameras used for urban management within the target area and record the camera identifier of each camera.
[0039] Among them, the video access device can be a video server, video capture card and other related equipment, and the target area is the area that needs to be automated for management, which can be the entire city or a key area in the city.
[0040] As an optional implementation method, multiple frames of surveillance images can be extracted from the surveillance video in the following way, and multiple violations in the surveillance video can be determined based on the multiple frames of surveillance images: For each surveillance video segment, the surveillance video is processed by extracting frames according to a preset frame extraction interval to obtain multiple frames of surveillance images; the feature vector of each suspected violation object in each frame of surveillance images is determined using a target detection algorithm; the feature vector of each suspected violation object in each frame of surveillance images is analyzed using a multi-target tracking algorithm, and the analysis results are used to determine whether each suspected violation object is a violation object. When determining a violation object, the violation event type and feature vector corresponding to the violation object are recorded.
[0041] Specifically, the object detection algorithm can be the YOLOv8 algorithm, and the feature vectors of each violation object determined by the object detection algorithm can be stored in the Milvus vector database; the multi-object tracking algorithm can be the ByteTrack or DeepSort algorithm.
[0042] The above process can be understood as follows: Assuming the detection is for illegally parked vehicles, all vehicles are extracted from each frame of the surveillance image using an object detection algorithm. All vehicles are considered suspected violations, and a multi-object tracking algorithm is used to track and analyze each vehicle. Vehicles that remain stationary for more than a preset threshold across multiple consecutive frames are identified as violations. Similarly, assuming the detection is for elderly people falling, all elderly people are extracted from each frame of the surveillance image using an object detection algorithm. All elderly people are considered suspected violations, and a multi-object tracking algorithm is used to track and analyze each elderly person. Elderly people who remain in a fallen posture for more than multiple consecutive frames are identified as violations. Likewise, assuming the detection is for the state of trash can lids, all trash cans are extracted from each frame of the surveillance image using an object detection algorithm. All trash cans are considered suspected violations, and a multi-object tracking algorithm is used to track and analyze each trash can. Trash cans whose lids are open for more than a preset threshold across multiple consecutive frames are identified as violations.
[0043] Optionally, the persistent state of an alarm event includes at least one of the following: alarm start time, alarm end time, and alarm duration.
[0044] Specifically, for each violating object in the surveillance video, the duration of the alarm event corresponding to the violating object can be determined as follows: For each violating object, generate an alarm event corresponding to the violating object; determine the alarm start time of the alarm event based on the timestamp of the first frame of the surveillance image containing the violating object; if the last frame of the surveillance image containing the violating object is the last frame of the corresponding surveillance video, determine that the alarm event has not yet terminated, and determine the alarm duration of the alarm event based on the timestamp of the last frame of the surveillance image containing the violating object and the alarm start time; if the last frame of the surveillance image containing the violating object is not the last frame of the corresponding surveillance video, determine that the alarm event has terminated, determine the alarm termination time of the alarm event based on the timestamp of the last frame of the surveillance image containing the violating object, and determine the alarm duration of the alarm event based on the alarm start time and alarm termination time.
[0045] Understandably, after an alarm event for a violation is generated, regardless of whether the alarm event terminates, the duration of the alarm can be determined based on the alarm start time corresponding to the first frame of the monitoring image in which the violation occurs and the alarm termination time corresponding to the last frame of the monitoring image in which the violation occurs.
[0046] Subsequently, law enforcement evidence for alarm events can be generated based on multiple frames of surveillance images corresponding to the violating object in the following manner: The multiple frames of surveillance images including the violating object are arranged in chronological order to form an image evidence chain for the alarm event; starting from the alarm start time of the alarm event, a video segment of a preset duration is extracted from the corresponding surveillance video as video auxiliary evidence for the alarm event; the image evidence chain and the video auxiliary evidence are used together as law enforcement evidence for the alarm event.
[0047] It should be noted that the preset duration can be set according to actual needs. For example, it can be set to 5 minutes. Correspondingly, a video clip of the preset duration before the alarm event occurs needs to be extracted as video auxiliary evidence. This can prove that everything was normal at the location before the arrival of the violator. Taking the scenario of an uncovered trash can as an example, extracting the video of 5 minutes before the alarm start time is to prove that the trash can lid was in a normal closed state before the arrival of the violator, thereby improving the chain of evidence and providing strong evidence for law enforcement.
[0048] Optionally, alarm events and law enforcement evidence corresponding to the same violation object of the same violation event type under different cameras can be merged based on evidence identifiers in the following way: For each evidence identifier with different camera identifiers but the same violation event type, the feature vectors in each evidence identifier are compared pairwise for similarity, and the violation object corresponding to the evidence identifier with a similarity higher than a preset similarity threshold is determined to be the same violation object; all alarm events and law enforcement evidence corresponding to the same violation object of the same violation event type are merged.
[0049] Specifically, methods such as cosine similarity and Pearson correlation coefficient can be used for similarity comparison. The preset threshold for similarity can be set according to actual needs. Through the above process of similarity comparison, alarm events and law enforcement evidence merging, a large number of duplicate events can be filtered out, saving resources and improving processing efficiency.
[0050] After merging alarm events and law enforcement evidence corresponding to the same type of violation and the same violator from different cameras, the alarm events can be processed according to their duration status as follows: If the alarm event has terminated, the case is automatically closed; if the alarm event has not terminated, it is determined whether the alarm duration exceeds a preset time threshold; if the alarm duration does not exceed the preset time threshold, no action is taken on the alarm event; if the alarm duration exceeds the preset time threshold, law enforcement personnel are dispatched to handle the alarm event on-site.
[0051] Figure 3A schematic diagram of an optional alarm event handling process in a vehicle illegal parking scenario is shown. In this scenario, if the vehicle remains stationary for more than a preset threshold, an alarm event is automatically triggered. If the vehicle leaves or moves within the preset time threshold, no action is taken on the alarm event, and the case is automatically closed. If the vehicle remains stationary after the preset time threshold has been exceeded, law enforcement personnel are dispatched to the scene for handling. Figure 4 The diagram illustrates an optional alarm event handling process in a trash can lid status detection scenario. The alarm event handling process in the trash can lid status detection scenario is similar to that in the vehicle illegal parking scenario, and will not be described in detail here.
[0052] In this embodiment, multiple surveillance videos collected by multiple cameras used for urban management are acquired. For each surveillance video, multiple frames of surveillance images are extracted, and multiple violating objects in the surveillance video are identified based on the multiple frames of surveillance images. For each violating object in the surveillance video, an alarm event corresponding to the violating object is generated, and the duration of the alarm event is determined. Law enforcement evidence of the alarm event is generated based on the multiple frames of surveillance images corresponding to the violating object, and an evidence identifier of the law enforcement evidence is generated based on the camera identifier corresponding to the surveillance video, the violation event type of the violating object, and the feature vector of the violating object in the corresponding surveillance image. Based on the evidence identifier, alarm events and law enforcement evidence corresponding to the same violating object of the same violation event type under different cameras are merged. For each merged alarm event, the alarm event is processed according to the duration of the alarm event. Among them, by automatically extracting frames from surveillance videos of urban management cameras to identify violators, and by conducting multi-dimensional analysis based on camera identification, violation type, and characteristics of the violator, multiple alarm evidence chains are generated. By merging the evidence chains of the same violator under the same violation type from different cameras, traceable alarm events can be automatically formed, thereby achieving "contactless" urban governance, improving work efficiency, and effectively solving the technical problem of low overall efficiency in current urban governance scenarios that require manual collection of violation evidence for alarm events before processing.
[0053] Example 2
[0054] According to an embodiment of this application, an alarm event processing apparatus for implementing the alarm event processing method in Embodiment 1 is also provided, such as... Figure 5 As shown, the alarm event processing device includes at least: an acquisition module 51, a determination module 52, an alarm generation module 53, a merging module 54, and an alarm processing module 55, wherein:
[0055] The acquisition module 51 can acquire multiple surveillance videos collected by multiple cameras used for urban management;
[0056] The determination module 52 can extract multiple frames of monitoring images from each monitoring video and determine multiple violating objects in the monitoring video based on the multiple frames of monitoring images.
[0057] The alarm generation module 53 can generate an alarm event corresponding to each violating object in the monitoring video, determine the continuous status of the alarm event, generate law enforcement evidence of the alarm event based on the multi-frame monitoring images corresponding to the violating object, and generate evidence identifier of law enforcement evidence based on the camera identifier corresponding to the monitoring video, the violation event type of the violating object, and the feature vector of the violating object in the corresponding monitoring image.
[0058] The merging module 54 can merge alarm events and law enforcement evidence corresponding to the same violation object of the same violation type under different cameras based on evidence identifiers;
[0059] The alarm processing module 55 can process the merged alarm events according to their ongoing status.
[0060] The following section explains each step of the alarm event handling method in conjunction with the specific implementation process.
[0061] Before the acquisition module acquires multiple surveillance videos collected by multiple cameras used for urban management, the video access device can be used to manage all cameras used for urban management within the target area and record the camera identifier of each camera.
[0062] Among them, the video access device can be a video server, video capture card and other related equipment, and the target area is the area that needs to be automated for management, which can be the entire city or a key area in the city.
[0063] As an optional implementation, the determination module can extract multiple frames of monitoring images from the monitoring video in the following manner, and determine multiple violations in the monitoring video based on the multiple frames of monitoring images: For each monitoring video segment, the monitoring video is processed by extracting frames according to a preset frame extraction interval to obtain multiple frames of monitoring images; the feature vector of each suspected violation object in each frame of monitoring image is determined using a target detection algorithm; the feature vector of each suspected violation object in each frame of monitoring image is analyzed using a multi-target tracking algorithm, and the analysis results are used to determine whether each suspected violation object is a violation object. When determining a violation object, the violation event type and feature vector corresponding to the violation object are recorded.
[0064] Specifically, the object detection algorithm can be the YOLOv8 algorithm, and the feature vectors of each violation object determined by the object detection algorithm can be stored in the Milvus vector database; the multi-object tracking algorithm can be the ByteTrack or DeepSort algorithm.
[0065] The above process can be understood as follows: Assuming the detection is for illegally parked vehicles, all vehicles are extracted from each frame of the surveillance image using an object detection algorithm. All vehicles are considered suspected violations, and a multi-object tracking algorithm is used to track and analyze each vehicle. Vehicles that remain stationary for more than a preset threshold across multiple consecutive frames are identified as violations. Similarly, assuming the detection is for elderly people falling, all elderly people are extracted from each frame of the surveillance image using an object detection algorithm. All elderly people are considered suspected violations, and a multi-object tracking algorithm is used to track and analyze each elderly person. Elderly people who remain in a fallen posture for more than multiple consecutive frames are identified as violations. Likewise, assuming the detection is for the state of trash can lids, all trash cans are extracted from each frame of the surveillance image using an object detection algorithm. All trash cans are considered suspected violations, and a multi-object tracking algorithm is used to track and analyze each trash can. Trash cans whose lids are open for more than a preset threshold across multiple consecutive frames are identified as violations.
[0066] Optionally, the persistent state of an alarm event includes at least one of the following: alarm start time, alarm end time, and alarm duration.
[0067] Specifically, for each violation object in the surveillance video, the alarm generation module can determine the duration of the alarm event corresponding to the violation object in the following ways: For each violation object, generate an alarm event corresponding to the violation object; determine the alarm start time of the alarm event based on the timestamp of the first frame of the surveillance image including the violation object; if the last frame of the surveillance image including the violation object is the last frame of the corresponding surveillance video, determine that the alarm event has not yet terminated, and determine the alarm duration of the alarm event based on the timestamp of the last frame of the surveillance image including the violation object and the alarm start time; if the last frame of the surveillance image including the violation object is not the last frame of the corresponding surveillance video, determine that the alarm event has terminated, determine the alarm termination time of the alarm event based on the timestamp of the last frame of the surveillance image including the violation object, and determine the alarm duration of the alarm event based on the alarm start time and alarm termination time.
[0068] Understandably, after an alarm event for a violation is generated, regardless of whether the alarm event terminates, the duration of the alarm can be determined based on the alarm start time corresponding to the first frame of the monitoring image in which the violation occurs and the alarm termination time corresponding to the last frame of the monitoring image in which the violation occurs.
[0069] Subsequently, the alarm generation module can generate law enforcement evidence for alarm events based on multiple frames of monitoring images corresponding to the violating object in the following ways: The multiple frames of monitoring images including the violating object are arranged in chronological order to form an image evidence chain for the alarm event; Starting from the alarm start time of the alarm event, a video segment of a preset duration is extracted from the corresponding monitoring video as video auxiliary evidence for the alarm event; The image evidence chain and the video auxiliary evidence are used together as law enforcement evidence for the alarm event.
[0070] It should be noted that the preset duration can be set according to actual needs. For example, it can be set to 5 minutes. Correspondingly, a video clip of the preset duration before the alarm event occurs needs to be extracted as video auxiliary evidence. This can prove that everything was normal at the location before the arrival of the violator. Taking the scenario of an uncovered trash can as an example, extracting the video of 5 minutes before the alarm start time is to prove that the trash can lid was in a normal closed state before the arrival of the violator, thereby improving the chain of evidence and providing strong evidence for law enforcement.
[0071] Optionally, the merging module can merge alarm events and law enforcement evidence corresponding to the same violation object of the same violation event type under different cameras in the following way: For each evidence identifier with different camera identifiers but the same violation event type, the feature vectors in each evidence identifier are compared pairwise for similarity, and the violation object corresponding to the evidence identifier with a similarity higher than a preset similarity threshold is determined to be the same violation object; all alarm events and law enforcement evidence corresponding to the same violation object of the same violation event type are merged.
[0072] Specifically, methods such as cosine similarity and Pearson correlation coefficient can be used for similarity comparison. The preset threshold for similarity can be set according to actual needs. Through the above process of similarity comparison, alarm events and law enforcement evidence merging, a large number of duplicate events can be filtered out, saving resources and improving processing efficiency.
[0073] After merging alarm events and law enforcement evidence corresponding to the same type of violation from different cameras, the alarm processing module can process the alarm events based on their duration in the following ways: if the alarm event has ended, the case is automatically closed; if the alarm event has not ended, it is determined whether the alarm duration exceeds a preset time threshold; if the alarm duration does not exceed the preset time threshold, no processing is performed; if the alarm duration exceeds the preset time threshold, law enforcement personnel are dispatched to handle the alarm event on-site.
[0074] Figure 3A schematic diagram of an optional alarm event handling process in a vehicle illegal parking scenario is shown. In this scenario, if the vehicle remains stationary for more than a preset threshold, an alarm event is automatically triggered. If the vehicle leaves or moves within the preset time threshold, no action is taken on the alarm event, and the case is automatically closed. If the vehicle remains stationary after the preset time threshold has been exceeded, law enforcement personnel are dispatched to the scene for handling. Figure 4 The diagram illustrates an optional alarm event handling process in a trash can lid status detection scenario. The alarm event handling process in the trash can lid status detection scenario is similar to that in the vehicle illegal parking scenario, and will not be described in detail here.
[0075] It should be noted that each module in the alarm event processing device in this application embodiment corresponds one-to-one with each implementation step of the alarm event processing method in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.
[0076] Example 3
[0077] According to an embodiment of this application, a computer program product is also provided, which includes a computer program, wherein the computer program, when executed by a processor, implements the alarm event handling method described above.
[0078] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the alarm event handling method in Embodiment 1 by running the computer program.
[0079] According to an embodiment of this application, a processor is also provided for running a computer program, wherein the computer program executes the alarm event handling method in embodiment 1 during runtime.
[0080] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the alarm event handling method of Embodiment 1 through the computer program.
[0081] Specifically, the computer program executes the following steps during runtime: acquiring multiple surveillance video segments collected by multiple cameras used for urban management; for each surveillance video segment, extracting multiple frames of surveillance images and identifying multiple violating objects in the surveillance video based on the multiple frames of surveillance images; for each violating object in the surveillance video, generating an alarm event corresponding to the violating object and determining the duration of the alarm event; generating law enforcement evidence for the alarm event based on the multiple frames of surveillance images corresponding to the violating object; and generating an evidence identifier for the law enforcement evidence based on the camera identifier corresponding to the surveillance video, the violation event type of the violating object, and the feature vector of the violating object in the corresponding surveillance image; merging the alarm events and law enforcement evidence corresponding to the same violating object of the same violation event type under different cameras based on the evidence identifier; and processing the alarm events according to their duration for each merged alarm event.
[0082] The sequence numbers of the above embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0083] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0084] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.
[0085] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0086] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0087] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0088] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method for handling alarm events, characterized in that, include: Acquire multiple surveillance videos captured by various cameras used for urban management; For each surveillance video segment, multiple frames of surveillance images are extracted from the surveillance video, and multiple violating objects in the surveillance video are determined based on the multiple frames of surveillance images. For each violating object in the surveillance video, an alarm event corresponding to the violating object is generated, and the duration of the alarm event is determined. Law enforcement evidence of the alarm event is generated based on multiple frames of surveillance images corresponding to the violating object, and evidence identifier of the law enforcement evidence is generated based on the camera identifier corresponding to the surveillance video, the violation event type of the violating object, and the feature vector of the violating object in the corresponding surveillance image. Based on the aforementioned evidence identifier, alarm events and law enforcement evidence corresponding to the same violation object of the same violation type under different cameras are merged; For each merged alarm event, the alarm event is processed according to its ongoing status.
2. The method according to claim 1, characterized in that, Before acquiring multiple surveillance video clips captured by various cameras used for urban management, the method further includes: The system utilizes video access devices to manage all cameras used for urban management within the target area and records the camera identification of each camera.
3. The method according to claim 1, characterized in that, For each surveillance video segment, multiple frames of surveillance images are extracted from the video, and multiple violating entities in the video are identified based on these frames, including: For each segment of surveillance video, the surveillance video is processed by extracting frames according to a preset frame extraction interval to obtain multiple frames of surveillance images. The feature vectors of each suspected violation object in each frame of the surveillance image are determined using an object detection algorithm. A multi-target tracking algorithm is used to analyze the feature vectors of each suspected violation object in each frame of the monitoring image. Based on the analysis results, it is determined whether each suspected violation object is a violation object. When a violation object is determined, the violation event type and feature vector corresponding to the violation object are recorded.
4. The method according to claim 1, characterized in that, The duration of the alarm event includes at least one of the following: alarm start time, alarm end time, and alarm duration. For each violating object in the surveillance video, an alarm event corresponding to the violating object is generated, and the duration of the alarm event is determined, including: For each of the aforementioned violations, generate an alarm event corresponding to the violation object; The alarm start time of the alarm event is determined based on the timestamp of the first frame of the monitoring image including the object that violated the rules; If the last frame of the monitoring image including the violating object is the last frame of the corresponding monitoring video, it is determined that the alarm event has not yet ended, and the alarm duration of the alarm event is determined based on the timestamp of the last frame of the monitoring image including the violating object and the alarm start time. If the last frame of the monitoring image including the violating object is not the last frame of the corresponding monitoring video, it is determined that the alarm event has been terminated. The alarm termination time of the alarm event is determined based on the timestamp of the last frame of the monitoring image including the violating object, and the alarm duration of the alarm event is determined based on the alarm start time and the alarm termination time.
5. The method according to claim 4, characterized in that, The law enforcement evidence for the alarm event is generated based on multiple frames of monitoring images corresponding to the violating object, including: The multiple frames of surveillance images including the violating object are arranged in chronological order to form the image evidence chain of the alarm event; Starting from the alarm start time of the alarm event, a video segment of a preset duration is extracted from the corresponding monitoring video as video auxiliary evidence of the alarm event; The image evidence chain and the video auxiliary evidence are used together as law enforcement evidence for the alarm event.
6. The method according to claim 1, characterized in that, Based on the aforementioned evidence identifier, alarm events and law enforcement evidence corresponding to the same violation type and the same violating object under different cameras are merged, including: For each piece of evidence identifier that has different camera identification but the same type of violation, the feature vectors in each piece of evidence identifier are compared for similarity, and the violation object corresponding to the evidence identifier with a similarity higher than a preset similarity threshold is determined to be the same violation object. Merge all alarm events and enforcement evidence corresponding to the same violation object of the same violation type.
7. The method according to claim 4, characterized in that, The alarm event is processed according to its ongoing status, including: If the alarm event has terminated, the alarm event will be automatically closed. If the alarm event has not yet terminated, determine whether the alarm duration of the alarm event exceeds a preset duration threshold; If the alarm duration of the alarm event does not exceed the preset duration threshold, the alarm event will not be processed. If the alarm duration of the alarm event exceeds the preset duration threshold, law enforcement personnel will be dispatched to handle the alarm event on-site.
8. An alarm event handling device, characterized in that, include: The acquisition module acquires multiple surveillance videos collected by various cameras used for urban management. The determination module is used to extract multiple frames of monitoring images from each monitoring video segment and determine multiple violating objects in the monitoring video based on the multiple frames of monitoring images. The alarm generation module is used to generate an alarm event corresponding to each violating object in the monitoring video, determine the duration of the alarm event, generate law enforcement evidence of the alarm event based on multiple frames of monitoring images corresponding to the violating object, and generate an evidence identifier of the law enforcement evidence based on the camera identifier corresponding to the monitoring video, the violation event type of the violating object, and the feature vector of the violating object in the corresponding monitoring image. The merging module is used to merge alarm events and law enforcement evidence corresponding to the same violation object of the same violation type under different cameras based on the evidence identifier; The alarm processing module is used to process the merged alarm events according to their ongoing status.
9. A computer program product, characterized in that, include: A computer program, wherein when executed by a processor, the computer program implements the alarm event handling method according to any one of claims 1 to 7.
10. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the alarm event handling method of any one of claims 1 to 7 through the computer program.