Video monitoring method and system

Abstract

The invention provides a video monitoring method and system, and the method comprises the steps: obtaining monitoring parameters and configuration information of cameras in a monitoring region, and obtaining at least one recognition model correspondingly associated with a target camera in response to an event signal or at intervals; the target camera is associated with the event signal or is determined to perform monitoring based on the configuration information; sending an image acquisition request to a streaming media server to enable the streaming media server to receive a video stream of at least one target camera in the monitoring area through an input port; obtaining a target image generated by the streaming media server based on the video stream; calling an identification model to analyze the target image to generate an identification result; and judging the identification result based on the alarm strategy to generate alarm information. According to the method, analysis is started through event signals or time intervals, the recognition model is called based on the configuration information and the monitoring parameters, and efficient utilization of resources can be achieved while non-blind-area monitoring is guaranteed.

Description

Technical Field

[0001] This application relates to the field of video surveillance technology, and in particular to a video surveillance method and system. Background Technology

[0002] Video surveillance systems can be applied in various scenarios, such as data center management. In data center management, one or more cameras are installed within the data center. These cameras connect their video streams to the backend system of the video surveillance system to achieve real-time monitoring of the data center environment and alarms for abnormal events. To achieve alarms for abnormal events, the video streams can be identified and analyzed to monitor the data center environment and personnel behavior in real time, detect abnormal events, and issue alarms.

[0003] Multiple cameras can be installed in the same or different server rooms, and different cameras may use different transmission protocols. Therefore, the backend system needs to perform protocol conversion to receive video streams with different transmission protocols. Furthermore, video surveillance systems typically employ fixed analysis modes to achieve real-time monitoring and event detection. For example, continuously pulling and analyzing video streams; or pulling and analyzing video streams triggered by events.

[0004] However, in the above identification and analysis process, continuously pulling and analyzing video streams is inefficient and consumes a lot of resources. While pulling and analyzing video streams through event triggers saves resources, the monitoring behavior is determined by the event, lacking the ability to actively monitor areas or devices that need to be monitored but have no trigger signals. There are blind spots in the monitoring coverage. Therefore, existing video surveillance systems cannot achieve efficient resource utilization while ensuring real-time monitoring and coverage, resulting in low alarm efficiency. Summary of the Invention

[0005] This application provides a video surveillance method and system to solve the problem of low alarm efficiency in existing video surveillance systems.

[0006] In a first aspect, embodiments of this application provide a video surveillance method, comprising: acquiring monitoring parameters and configuration information of cameras within a monitoring area; wherein the configuration information characterizes the mapping relationship between the cameras and at least one detection capability, and each detection capability corresponds to at least one recognition model; in response to an event signal or at regular intervals, acquiring at least one recognition model associated with a target camera; the target camera is associated with an event signal or determined to be monitored based on the configuration information; the event signal is used to characterize changes in the state of the monitoring area; sending an image acquisition request to a streaming media server, so that the streaming media server receives video streams from at least one target camera within the monitoring area through an input port; wherein one target camera corresponds to one input port; acquiring a target image generated by the streaming media server based on the video stream; calling the recognition model to analyze the target image to generate a recognition result; and performing a judgment on the recognition result based on an alarm strategy to generate alarm information.

[0007] In one possible implementation, upon receiving an event signal, the configuration information corresponding to the target camera includes an event-triggered detection capability item; the recognition model is a first recognition model, the recognition result is a first recognition result, and the alarm information is a first alarm information; in response to the event signal or at regular intervals, at least one recognition model associated with the target camera is acquired, including: in response to the event signal, determining at least one target camera associated with the event signal, and acquiring at least one recognition model associated with the target camera; in the absence of an event signal, the monitoring parameters include a monitoring start time, a monitoring end time, and an interval time, and the configuration information corresponding to the target camera includes a non-event-triggered detection capability item; the recognition model is a second recognition model, the recognition result is a second recognition result, and the alarm information is a second alarm information; in response to the event signal or at regular intervals, at least one recognition model associated with the target camera is acquired, including: if the current time is between the monitoring start time and the monitoring end time, acquiring at least one second recognition model associated with the target camera based on the interval time.

[0008] In one possible implementation, the recognition result includes a first recognition result and a second recognition result; if the detection capability corresponding to the target camera includes video miniaturization, the method further includes: if the first recognition result includes image change information, the video stream is a first video stream corresponding to the first recognition result; and / or, if the second recognition result includes image change information, the video stream is a second video stream corresponding to the second recognition result; and according to the time sequence, multiple first video streams and second video streams within a first preset time period are stitched together to generate a miniaturized video.

[0009] In one possible implementation, the event signal includes an access control event signal; in response to the event signal, at least one target camera associated with the event signal is identified, and at least one recognition model associated with the target camera is obtained, and then the process includes: receiving the access control event signal; wherein the access control event signal includes an event type; parsing the access control event signal to obtain the event type; and if the event type is a door closing event, within a second preset time period, if the first recognition result does not include moving target features, stopping the sending of image acquisition requests to the streaming media server.

[0010] In one possible implementation, the event-triggered detection capability includes personnel entry / exit recognition; the first recognition model includes a tracking model, a morphology extraction model, and an identity extraction model; generating alarm information includes: if the event signal is a door opening event signal, extracting morphological vector information through the morphology extraction model, or extracting identity vector information through the identity extraction model, to match the target personnel; if the target personnel are matched, setting the entrance / exit area; when a moving target is detected entering the entrance / exit area, calling the tracking model to obtain the target identifier of one or more moving targets in the video stream; if the target identifier of the moving target matches the target personnel's target identifier... If the identifiers match, an exit event record for the target person is generated. If no target person is matched, within a third preset time period, the application calls the FFmpeg component to continuously receive multiple target images processed by the FFmpeg component. If no person information can be matched based on the target images, a first alarm message is generated, which represents an abnormal intrusion event. After generating the first alarm message, the process further includes: in response to generating the first alarm message, obtaining an alarm video sent by the streaming media server, where the alarm video is a video clip including the time when the first alarm message was generated; and associating and storing the video clip with the first alarm message to generate an alarm event record.

[0011] In one possible implementation, the detection capability includes multiple detection capability items; based on preset configuration information, at least one recognition model associated with the target camera is obtained, including: receiving at least one detection capability item from the target camera, wherein the detection capability item is a type of state change event to be recognized that is set by the user and / or pre-configured; and obtaining at least one recognition model associated with the target camera according to the correspondence between the detection capability item and the recognition model.

[0012] In one possible implementation, the configuration information also includes alarm event levels associated with detection capability items; the recognition model is associated with model parameter configuration, which includes input image size and confidence threshold; and the recognition result is judged based on the alarm strategy to generate alarm information, including: determining the priority of alarm information according to the alarm event level associated with the detection capability item corresponding to the recognition result; and generating alarm information based on the priority.

[0013] In one possible implementation, generating the recognition result includes: extracting feature vectors from the target image; performing similarity calculations between the feature vectors and multiple event feature vectors in an event database to obtain multiple calculation results; wherein the event database is used to store event feature vectors associated with different detection capability items; obtaining the target calculation result and the target detection capability item associated with the target calculation result, wherein the target calculation result is the calculation result with the highest similarity threshold among the multiple calculation results; and based on the target detection capability item, calling a recognition model to analyze the target image to generate the recognition result.

[0014] In one possible implementation, before performing a judgment on the recognition result based on the alarm strategy to generate alarm information, the method includes: extracting feature vectors of the target image, and performing similarity calculation between the feature vectors and multiple false alarm feature vectors in the false alarm feature library to generate a similarity result; wherein, the false alarm feature library is used to store feature vectors associated with historical false alarm events; if the similarity result is greater than or equal to a similarity threshold, no alarm information is generated; if the similarity result is less than the similarity threshold, alarm information is generated based on the recognition result; in response to a false alarm confirmation instruction from the user regarding the alarm information, the feature vectors of the target image are stored in the false alarm feature library.

[0015] In one possible implementation, the detection capability includes multiple detection capability items; an alarm strategy is associated with the detection capability items, and the alarm strategy includes strategy parameters, which are set based on the event type; the strategy parameters include a duration threshold and a suppression time threshold; the identification result is judged based on the alarm strategy to generate alarm information, including: if the identification result represents an abnormal state, starting a duration timer; the duration of the duration timer is set based on the alarm strategy associated with the current detection capability item; if the duration timer reaches the duration threshold, determining whether the abnormal state meets the alarm triggering condition; wherein the alarm triggering condition is: there is currently no valid suppression timer, or the time interval is greater than or equal to the suppression time threshold, and the time interval is the time interval elapsed since the last generation time of alarm information for the same detection capability item; if the alarm triggering condition is met, alarm information is generated.

[0016] Secondly, embodiments of this application provide a video surveillance system, including: an analysis unit configured to acquire monitoring parameters and configuration information of cameras within a monitoring area; wherein the configuration information characterizes the mapping relationship between the cameras and at least one detection capability, each detection capability corresponding to at least one recognition model; and, in response to an event signal or at regular intervals, acquire at least one recognition model associated with a target camera; the target camera is associated with an event signal or determined to be monitored based on the configuration information; the event signal is used to characterize changes in the state of the monitoring area; and, send an image acquisition request to a streaming media server; the streaming media server is configured to receive video streams from at least one camera within the monitoring area through an input port; wherein one target camera corresponds to one input port; the analysis unit is further configured to acquire a target image generated by the streaming media server based on the video stream; and, call the recognition model to analyze the target image to generate a recognition result; and, perform a judgment on the recognition result based on an alarm strategy to generate alarm information.

[0017] As described above, this application provides a video surveillance method and system. The method includes: acquiring monitoring parameters and configuration information of cameras within a monitoring area; wherein the configuration information represents the mapping relationship between the cameras and at least one detection capability, and each detection capability corresponds to at least one recognition model; in response to an event signal or at regular intervals, acquiring at least one recognition model associated with a target camera; the target camera is associated with an event signal or determined to be monitored based on the configuration information; the event signal is used to represent changes in the state of the monitoring area; sending an image acquisition request to a streaming media server, so that the streaming media server receives video streams from at least one target camera within the monitoring area through an input port; wherein one target camera corresponds to one input port; acquiring a target image generated by the streaming media server based on the video stream; calling a recognition model to analyze the target image to generate a recognition result; and performing a judgment on the recognition result based on an alarm strategy to generate alarm information. This method initiates the analysis process through event signals or time intervals, and dynamically calls recognition models matching the detection capabilities of the target cameras based on configuration information and monitoring parameters. It can be scheduled according to actual needs, thereby ensuring blind-spot-free monitoring while achieving efficient resource utilization and high alarm efficiency. Attached Figure Description

[0018] Figure 1 A flowchart illustrating the video surveillance method provided in this application embodiment; Figure 2 A schematic diagram illustrating the configuration information provided in the embodiments of this application; Figure 3 A flowchart illustrating the path for triggering monitoring and analysis provided in an embodiment of this application; Figure 4A schematic diagram illustrating the alarm setting strategy provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a video surveillance system provided in an embodiment of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of this application.

[0020] Before introducing the technical solutions of the embodiments of this application, the terminology involved in the embodiments of this application will be introduced by way of example.

[0021] 1. Nginx is an open-source web server and reverse proxy server. In video surveillance solutions, it is used for streaming media relay and load balancing. Specifically, Nginx receives video streams pushed by cameras using the RTSP (Real Time Streaming Protocol) protocol, and then calls external processing tools, such as the FFmpeg component, to perform transcoding and protocol conversion. For example, after transcoding the RTSP stream to an HTTP-FLV stream, it is then analyzed by other components.

[0022] In this architecture, Nginx is used to perform stream reception, transcoding scheduling and distribution. However, during the transcoding process, it needs to rely on external components, which introduces additional latency and has limited horizontal scalability.

[0023] 2. FFmpeg (Fast Forward MPEG) is a cross-platform audio and video processing component, including codec libraries. In existing technologies, FFmpeg is called by Nginx to perform tasks such as video stream transcoding and frame extraction. For example, it converts a video stream from one encoding format (such as H.265) to another format (such as H.264); another example is extracting images from the video stream at time intervals or according to instructions.

[0024] 3. ZLMediaKit is an open-source, lightweight streaming media server framework that supports multiple streaming media protocols, such as RTSP, RTMP, HLS, and HTTP-FLV. ZLMediaKit can receive video streams from cameras using different protocols and can convert the received video streams into various network protocol outputs.

[0025] FFmpeg, as a video encoding / decoding and processing tool, works in conjunction with ZLMediaKit to perform operations such as decapsulation, transcoding, and frame extraction of audio and video streams. Nginx is used to build the web service interface, undertaking functions such as distributing streaming media resources and forwarding management commands. In this embodiment, ZLMediaKit replaces Nginx as the streaming media server. By configuring streaming media protocol support capabilities, it can receive, process, and forward video streams of multiple protocols without relying on external tools for protocol conversion.

[0026] In the context of data center management, one or more cameras are installed in the data center. The cameras continuously collect video signals from inside the data center and transmit the video stream to a video monitoring system deployed in the background via the network to achieve real-time monitoring of the operating status of equipment, personnel activities, and environmental safety in the data center, as well as the identification of abnormal events.

[0027] To achieve automated alarm for abnormal events, the video surveillance system performs real-time or near real-time image recognition or analysis on the video stream. It can also trigger corresponding alarms by recognizing preset abnormal patterns, such as non-target personnel entering, people falling, smoke or open flames appearing, liquid leaks, etc.

[0028] Within the same data center or multiple data centers across different regions, cameras from different manufacturers and models may be used due to deployment time or other reasons. Consequently, the video streams output by these cameras may employ different transmission protocols, such as RTSP, RTMP, and HTTP-FLV. Therefore, video surveillance systems need to be capable of receiving and processing streaming media using multiple protocols. In some embodiments, protocol conversion is used to uniformly convert video streams of different formats into a processable intermediate format.

[0029] Existing video surveillance systems typically employ fixed analysis patterns. One such pattern involves continuously retrieving and analyzing video streams. This method requires the system to continuously retrieve video stream data from each camera and utilize artificial intelligence models to analyze the video content for continuous event detection. Furthermore, some systems, to enhance monitoring coverage, employ a feature-overlay approach, executing multiple event detection algorithms in parallel on the same video stream. However, this also relies on the continuous retrieval and decoding of the video stream.

[0030] Another mode is event-triggered analysis, where, for example, the relevant video stream is only retrieved and analyzed when a signal from the access control system is received. However, continuous retrieval and analysis leads to high computational and bandwidth consumption and low analysis efficiency. While event-triggered analysis saves resources, the monitoring behavior is determined by external events, lacking the ability to proactively monitor areas or devices that need monitoring but without trigger signals, resulting in blind spots in monitoring coverage. Therefore, existing video surveillance systems struggle to achieve efficient resource utilization while ensuring real-time monitoring and coverage, leading to low alarm efficiency.

[0031] To address the challenge of achieving efficient resource utilization while ensuring real-time monitoring and coverage, this application provides a video surveillance method and system. Unlike the fixed analysis modes described above, this method acquires the configuration information and monitoring parameters of cameras within the monitored area. It can flexibly respond to event signals or initiate analysis of the target video stream based on preset intervals. The configuration information represents the mapping relationship between the camera and at least one detection capability, with each detection capability corresponding to at least one recognition model. Therefore, based on actual monitoring needs (such as immediate response to personnel entry / exit events) and preset monitoring strategies (such as periodic inspection of equipment status), the analysis triggering method can be dynamically selected or switched. This ensures efficient resource utilization and improves alarm efficiency while maintaining real-time monitoring and coverage.

[0032] After receiving the video stream from the camera, ZLMediaKit uses the FFmpeg component to perform protocol parsing and image frame extraction. The backend application then calls the recognition model for analysis. In this embodiment, the video stream retrieval and analysis are not fixed but triggered by event signals or timing strategies.

[0033] Figure 1 This is a flowchart illustrating the video surveillance method provided in an embodiment of this application.

[0034] like Figure 1 As shown in the figure, this application provides a video surveillance method, which may include steps S100-S600.

[0035] S100: Obtain the monitoring parameters and configuration information of the cameras in the monitoring area; wherein, the configuration information represents the mapping relationship between the cameras and at least one detection capability, and each detection capability corresponds to at least one recognition model.

[0036] The monitoring area is the spatial range covered and monitored by the video surveillance system. One monitoring area corresponds to a location that needs to be managed, such as a computer room, a warehouse, an office floor, or an equipment room. One or more cameras may be deployed within a monitoring area, and multiple cameras can collectively cover the overall environment of the monitoring area from different angles and positions.

[0037] The boundaries and definitions of the monitoring area can be pre-configured. When an event signal is generated, one or more associated monitoring areas can be located based on the signal source, thereby determining which cameras need to be analyzed.

[0038] The monitoring parameters of a camera are time-series parameters used to control monitoring behavior. For example, for equipment areas that require periodic inspections, the monitoring can be set to run every 30 minutes from 9:00 to 18:00 on weekdays.

[0039] The camera configuration information is a pre-set data set used to define the mapping relationship between each camera and one or more detection capabilities. For example, the mapping relationship can be a lookup table, where the key is the camera's unique identifier and the value is a list of one or more detection capabilities that the camera is authorized or configured to be enabled.

[0040] like Figure 2 As shown, the configuration information may include basic camera information, camera capability configuration, and global policy parameters. Basic camera information includes the camera's unique identifier, the physical region or logical group to which the camera belongs, the address information of the streaming media server node used by the camera, and the camera's network access parameters. Camera capability configuration specifies one or more detection capabilities that each camera can be used for analysis. Detection capabilities may include various types such as operational anomalies, personnel behavior, capability analysis, and equipment anomalies. Global policy parameters include system-level configurations, such as the default image analysis interval, event recording retention time, and log level.

[0041] The detection capabilities represent the monitoring scenarios or abnormal events that can be identified. Each detection capability corresponds to a monitoring target that can be understood by business personnel, such as personnel entering and exiting, personnel falling, smoke generation, flames appearing, liquid leakage, or unauthorized construction activities. A camera can be configured with multiple detection capabilities simultaneously. For example, a camera at the entrance of a computer room may be configured with both personnel entry and exit recognition and area intrusion detection capabilities, while cameras inside the computer room may be configured with multiple detection capabilities such as personnel falling, smoke recognition, and water leakage detection. The configuration of detection capabilities allows for differentiated monitoring strategies to be implemented for cameras in different locations.

[0042] For each detection capability, there is at least one recognition model. The recognition model is an algorithm that has been trained and deployed on data and is capable of performing a specific pattern recognition task on image or video data.

[0043] S200: In response to an event signal or at regular intervals, acquire at least one recognition model associated with the target camera; the target camera is associated with the event signal or determined to be monitored based on configuration information; the event signal is used to characterize changes in the state of the monitored area.

[0044] Step S200 includes two paths for triggering monitoring and analysis: the first path is triggered in response to an event signal, and the second path is triggered at regular intervals, i.e., timed triggering. The two paths are described in detail below: First, for the first path, event signals are electronic signals used to indicate specific changes in the state of the monitored physical environment or the system itself. Specifically, event signals are used to characterize changes in the state of the monitored area, and can be door opening or closing signals generated by an access control system. In this case, the event signal is used to characterize changes in the access status of the controlled entrance or exit, for example, from "closed" to "open", or from "open" to "closed".

[0045] Event signals may include data such as source identifiers and timestamps. The signal source identifier is used to determine the address or number of the sensing device or system module that generated the event, and the timestamp is used to record the time when the event occurred.

[0046] Upon receiving an event signal, at least one target camera associated with the event signal is identified. The target camera is not all cameras in the monitored area, but rather one or more cameras logically and physically associated with the currently received event signal. The identification of the target camera can be achieved through predefined mapping rules, which associate different types of event signals with camera identifiers within the monitored area.

[0047] For example, when the access control system in the computer room detects someone swiping their card to open the door, it sends an event signal indicating that the door has been opened. Upon receiving this event signal, the system identifies the source of the signal and then, according to predefined mapping rules, queries the camera associated with that access point, i.e., the target camera. Subsequently, based on preset configuration information, it obtains at least one recognition model associated with the target camera.

[0048] Secondly, for the second path, if no event signal is received, a timed trigger is executed based on the monitoring parameters. If the current time is between the monitoring start and end times, the analysis task is triggered periodically based on the interval.

[0049] At this point, the target cameras are those that need to be monitored periodically based on the configuration information. For example, if the configuration information indicates that certain cameras have non-event-triggered detection capabilities such as smoke recognition and water seepage detection, and their monitoring parameters have been set, then these cameras will be identified as target cameras.

[0050] Whether in response to an event signal or at a time interval, after identifying the target camera, the system queries the target camera's configuration information and obtains the recognition model corresponding to the detection capability indicated by the configuration information. Each detection capability is mapped to one or more pre-trained recognition models.

[0051] For example, in timed trigger scenarios, these recognition models can be defined as second recognition models, whose corresponding detection capabilities are non-event-triggered types (such as fireworks, falls, water seepage, etc.).

[0052] Each detection capability requires one or more specific recognition models. These models are trained and deployed on data and are algorithms that can perform specific pattern recognition tasks on image or video data. For example, the ability to detect a person falling may require a target detection model trained on the YOLOv10 algorithm to identify the posture of the fall. The ability to recognize people entering and exiting requires the combined use of multiple models, such as pedestrian tracking models, face feature extraction models, and human body feature extraction models, to complete the recognition task.

[0053] When acquiring the recognition model, regardless of whether it is the first or second recognition model, it first receives at least one detection capability item from the target camera. Then, based on the correspondence between the detection capability item and the recognition model, it acquires at least one recognition model associated with the target camera. For example, the personnel entry and exit recognition detection capability item corresponds to the personnel entry and exit event type, and the smoke recognition detection capability item corresponds to the smoke occurrence event type.

[0054] Furthermore, the trained recognition models are deployed in a model service library, which is used to deploy and manage multiple first or second recognition models. The model service library provides a unified network interface for calling the recognition models via the network interface.

[0055] The steps of obtaining at least one recognition model associated with a target camera include: calling the recognition model associated with the target camera from the network interface of the model service library; wherein, the model service library deploys multiple recognition models; the recognition model is a trained model; the training method of the recognition model includes: obtaining a training set, which is an image or video including the target object; the target object corresponds to the same detection capability; labeling the position and range of the target object to obtain labeling information; the labeling information is used to indicate the position and category of the target object in the image; constructing an initial model based on the target detection algorithm; and using a deep learning framework to train the initial model according to the training set and the labeling information to obtain the recognition model.

[0056] During the training of the recognition model, the training set is first obtained. The training set is a collection of image or video data including the target object. All samples in the training set are collected for the same detection capability. For example, for the ability to detect people falling, the training set includes image samples of various falling scenarios.

[0057] The training set is labeled to indicate the location and extent of target objects, yielding labeled information. This labeled information indicates the specific location and category of target objects in the image. The labeling process can be completed using a labeling tool to ensure accuracy and consistency. An initial model is then built based on an object detection algorithm. For scenarios with high real-time requirements, the YOLO series of algorithms can be chosen as the initial model; for scenarios with high accuracy requirements, the Faster R-CNN series of algorithms may be selected.

[0058] Using a deep learning framework, an initial model is trained based on the training set and labeled information to obtain a recognition model. The training process may include steps such as forward propagation, loss calculation, backpropagation, and parameter update. After training, the recognition model is obtained.

[0059] The recognition model receives image data as input, performs internal calculations, and outputs a recognition result regarding whether the image contains a specific target or event. By invoking a recognition model matched to the target camera's detection capabilities based on configuration information, on-demand allocation and precise scheduling of analysis resources can be achieved.

[0060] S300: Sends an image acquisition request to the streaming media server so that the streaming media server can receive video streams from at least one target camera in the monitored area through an input port, wherein one target camera corresponds to one input port.

[0061] The streaming media server (ZLMediaKit) is used to receive, convert, distribute, and manage video streams generated by cameras. In this embodiment, ZLMediaKit integrates the FFmpeg component, which is used to process video streams with various transmission protocols received by ZLMediaKit. By calling the FFmpeg component, ZLMediaKit can perform decoding, format conversion, and frame extraction of the input video stream.

[0062] Understandably, whether triggered by an event or a timer, once the target camera and the recognition model to be invoked are determined, an image acquisition request will be sent to the streaming media server.

[0063] In traditional video surveillance systems, ZLMediaKit continuously and uninterruptedly pulls or receives video streams from cameras and forwards or records them, regardless of whether analysis is needed. This approach results in a significant waste of computational and bandwidth resources. In this embodiment, ZLMediaKit only identifies the designated target camera, establishes a session, and begins pulling the video stream from the target camera after receiving an image acquisition command.

[0064] The image acquisition request is used to request the capture of one or more images from a specified video stream. The image acquisition request includes a unique identifier for the target camera, which indicates to ZLMediaKit which connected video stream the screenshot should be taken from.

[0065] The image acquisition request also includes image format parameters, such as specifying whether the returned image is in JPEG or PNG format, and image resolution parameters, which specify the target width and height of the generated image. In some cases, the image acquisition request may also include timestamp or sequence number parameters to identify the image acquisition request or the desired capture of a specific moment in the video stream. After receiving the image acquisition request, ZLMediaKit's FFmpeg component performs a screenshot operation.

[0066] After responding to an image acquisition request, ZLMediaKit receives the video stream through the corresponding input port. This input port is a network communication endpoint used to receive video stream data sent by the camera using various transmission protocols. It's understandable that regardless of the camera's configuration settings, the camera needs to input its video stream to ZLMediaKit through the input port. The video stream received at the input port is then passed to the integrated FFmpeg component for processing.

[0067] ZLMediaKit can handle multiple streaming media protocols and enable access to heterogeneous video sources. This approach differs from the continuous streaming in traditional solutions, allowing for on-demand streaming to save resources.

[0068] S400: Obtain the target image generated by the streaming media server based on the video stream.

[0069] Upon receiving an image acquisition request, ZLMediaKit invokes the integrated FFmpeg component to perform a screenshot operation to generate the target image. In one implementation, the screenshot is taken based on the latest frame of the video stream. In this approach, the FFmpeg component continuously receives the video stream; when an image acquisition request is received, it decodes the latest video stream and outputs the decoded image frame as the target image. This minimizes latency and ensures that the acquired image is synchronized with the event.

[0070] In another implementation, screenshots can be taken based on a specific timestamp. In this method, the image acquisition request can carry a specific time point parameter. The FFmpeg component locates the data packet at the corresponding time point in the video stream based on the time point parameter, and generates the target image after decoding and image processing, which can reconstruct the scene at a specific moment.

[0071] Other implementation methods exist, such as taking screenshots based on keyframes. Keyframes in the video stream contain complete image information, and the FFmpeg component can prioritize keyframes for decoding and output to avoid the additional computational overhead required for decoding non-keyframes, thereby improving processing efficiency while ensuring image quality.

[0072] For example, batch screenshotting can continuously generate multiple target images in a single processing step. It can extract image frames from multiple time points in sequence from the video stream according to preset time interval parameters, forming an image sequence that can provide more comprehensive on-site information.

[0073] In some embodiments, the FFmpeg component includes a decoder and an image filter for processing the input video stream to generate a target image. Specifically, the FFmpeg component performs a screenshot according to one of the embodiments described above to generate the target image.

[0074] When generating target images, the FFmpeg component supports various post-processing options, such as image resizing, format conversion, and quality parameter adjustment. These post-processing options can be dynamically configured using parameters from the image acquisition request, ensuring that the generated target image meets the specific input format requirements of different recognition models. For example, some recognition models may require input images at a specific resolution, while others may have specific requirements for image compression quality; the FFmpeg component can meet these needs through appropriate parameter settings.

[0075] Furthermore, the generated target image includes one or more entities or specific states to be detected and identified within the image frame. For example, in a smart data center application scenario, the target image includes moving targets, such as people—individuals present in the data center environment. The number of moving targets can be one or more; for example, multiple moving targets may be in different postures such as walking, standing, squatting, or falling.

[0076] The number of target images depends on the specific parameter settings in the image acquisition request. A single request can acquire one or more target images. When requesting a single target image, ZLMediaKit can capture a frame at a preset time from the video stream. When requesting multiple target images, the application, such as a Java program, can directly call the FFmpeg component to continuously capture multiple frames at different times from the video stream, or capture several frames at specific time intervals. These images together constitute an image sequence.

[0077] The number of target images can also depend on the configuration of the image acquisition request and the complexity of the actual monitoring scenario. A single image acquisition request can request the generation of a single target image or a sequence of multiple consecutive target images. When there is only one target to be monitored, such as detecting whether someone is entering through a doorway, a single target image is sufficient for identification.

[0078] When monitoring scenarios are complex or require analyzing the dynamic behavior of a target, multiple consecutive images of the target may be necessary. For example, to determine whether a person has changed from walking to falling, a sequence of images consisting of several frames can be acquired, and an accurate judgment can be made by analyzing the continuous posture changes of the target in the image sequence.

[0079] Whether triggered by an event or a timer, the target image returned by the streaming media server will be obtained as input for subsequent recognition and analysis.

[0080] S500: Calls the recognition model to analyze the target image and generate recognition results.

[0081] Based on the current trigger path and the detection capability of the target camera, the corresponding recognition model (first recognition model or second recognition model) obtained in step S200 is invoked to analyze the target image.

[0082] In one implementation, the recognition model can be an object detection model, which is used to identify the presence and location information of specific target objects in an image. Taking a computer room monitoring scenario as an example, the object detection model is used to identify targets such as personnel, equipment, tools, or smoke and flames. It extracts features from the input image through a convolutional neural network, and then obtains the bounding box coordinates and category confidence of the target object through regression calculation.

[0083] In another implementation, the recognition model can also be a pose estimation model, which analyzes the joint positions and pose angles of the human body in an image. In a scenario where a person is being detected to have fallen, the pose estimation model first locates key points on the human body, and then calculates the current pose of the human body based on the geometric relationships between these key points.

[0084] In the recognition model, the target image is received as input, and the neural network inside the model performs feature extraction and transformation on the image. Different levels of neural networks capture a variety of information from low-level edge features to high-level semantic features. These features are processed by the model and transformed into recognition results.

[0085] The analysis process of the recognition model may include multiple steps, such as forward propagation computation, non-maximum suppression, and confidence threshold filtering, ultimately generating a recognition result. The recognition result reflects the target to be identified and its attributes in the image. The recognition result may include target category information, that is, which predefined category the identified target belongs to, such as people, smoke, flames, water stains, or other predefined anomaly types.

[0086] The recognition results may also include target location information, for example, represented in bounding box coordinates, indicating the target's specific location and coverage area in the image. The recognition results also include a confidence score, which represents the model's certainty about the recognition result; a higher score indicates a more reliable recognition result. For complex scenes, the recognition results may include identification information for multiple targets, each with independent category, location, and confidence data.

[0087] The quantity and level of detail of the recognition results correspond to the number of target images and the complexity of the recognition task. When a single target image is input, the recognition result may be the identification of a single target; for example, simply outputting whether a person is present in the image and their location. When a sequence of multiple consecutive target images is input, the recognition result may include the analysis results of dynamic behavior; for example, outputting the recognition results of the entire process of a person going from walking to falling, including posture determination and behavioral results at each time point. In some complex recognition tasks, the recognition result may include the recognition results of multiple related targets; for example, simultaneously outputting the person's location, the type of handheld tool, and the recognition results of the ongoing operational behavior.

[0088] For event-triggered personnel entry and exit identification, the identification results may include personnel information, location, etc.; for timed smoke detection, the identification results may include whether flames or smoke were detected and their confidence level.

[0089] S600: Based on the alarm policy, the identification results are judged to generate alarm information.

[0090] Alarm policies can be configured according to different monitoring scenarios and detection capabilities to judge the identification results and generate alarm information. Alarm policies can be based on duration, meaning that an alarm is triggered only when the identification result lasts for a certain period of time. For example, in a fall detection scenario, an alarm is only generated when the fall lasts for more than a preset time threshold, which can effectively distinguish between a real fall and a temporary squatting behavior.

[0091] Alarm strategies also include spatial relationship-based judgment methods, which analyze the target's position information in the image to determine whether to generate an alarm. For example, in a regional intrusion detection scenario, it checks whether the identified personnel target has entered a preset prohibited area. A regional intrusion alarm is only generated when the target crosses the virtual warning line and remains in the restricted area.

[0092] During the judgment process of the recognition results, by reading data such as target category, location information and confidence score in the recognition results, the corresponding alarm strategy is selected according to the current detection capability type, such as duration threshold, repeated detection number requirement, spatial area definition, etc.

[0093] The identification results are then compared with the alarm policies to determine whether the alarm triggering conditions are met. During this process, historical identification results can also be used for comprehensive analysis to ensure the accuracy of the alarm decision. An alarm message is generated when all policies are met.

[0094] Alarm information may include alarm type, alarm level, occurrence time, location coordinates, evidence image index, and processing suggestions. Depending on the triggering path and detection capabilities, the generated alarm information may correspond to different event types, such as personnel intrusion alarms, fire alarms, and water seepage alarms.

[0095] As described above, this application provides a video surveillance method and system. The method includes: acquiring monitoring parameters and configuration information of cameras within a monitoring area; wherein the configuration information represents the mapping relationship between the cameras and at least one detection capability, and each detection capability corresponds to at least one recognition model; in response to an event signal or at regular intervals, acquiring at least one recognition model associated with a target camera; the target camera is associated with an event signal or determined to be monitored based on the configuration information; the event signal is used to represent changes in the state of the monitoring area; sending an image acquisition request to a streaming media server, so that the streaming media server receives video streams from at least one target camera within the monitoring area through an input port; wherein one target camera corresponds to one input port; acquiring a target image generated by the streaming media server based on the video stream; calling the recognition model to analyze the target image to generate a recognition result; and performing a judgment on the recognition result based on an alarm strategy to generate alarm information. Based on the method provided by this application, the continuous consumption of computing resources and network bandwidth can be reduced, and the recognition efficiency is high.

[0096] Figure 3 This is a flowchart illustrating the path for triggering monitoring and analysis provided in an embodiment of this application.

[0097] The two trigger monitoring and analysis paths in S200 are described in detail here. First, for the first path, when an event signal is received, that is, when the access control system does not generate an open or close signal, the monitoring process is started through the built-in timer mechanism within the monitoring area to achieve periodic coverage of the monitoring area.

[0098] S301: In response to an event signal, identify at least one target camera associated with the event signal and obtain at least one recognition model associated with the target camera.

[0099] In scenarios triggered by event signals, the corresponding detection capability items include the detection capability items of event trigger type (such as personnel entry and exit recognition). At this time, the recognition model is defined as the first recognition model. After obtaining the first recognition model in S200, the steps of S300-S600 can be executed. They will not be elaborated here. The recognition result obtained is the first recognition result, and the alarm information is the first alarm information.

[0100] It is understandable that the first identification result may include information such as personnel location, feature vector, and tracking ID, and the first alarm information may be alarm information representing an intrusion event.

[0101] Specifically, the event signal includes access control event signal, wherein the access control event signal includes event type. After step S301, this embodiment of the application performs different identification steps based on the event type, specifically including steps S401-S402.

[0102] S401: Receive access control event signals; where the access control event signals include event types.

[0103] Access control event signals are electronic signals generated and sent by the access control system to convey information about changes in the access status at entrances and exits. Access control event signals include event types, which are used to distinguish between door opening events and door closing events.

[0104] S402: Parse the access control event signal to obtain the event type.

[0105] The received access control event signal is parsed to extract the event type. Based on predefined encoding rules, it is determined whether the signal is an open or closed event. If the event type is a closed event, steps S300-S500 are executed sequentially. During step S500, if the recognition result indicates that the first recognition target does not include moving target features within a second preset time period, the image acquisition request to the streaming media server is stopped.

[0106] Specifically, if the first identification result does not include moving target features within a second preset time period, for example, 5 minutes, indicating that the people in the monitored area have completely left and there are no other active targets, then the image acquisition request sent to ZLMediaKit will be stopped to ensure that the streaming stops after the people have completely left, thus avoiding the waste of resources caused by continuous monitoring of the empty area.

[0107] Furthermore, when the event type is a door opening event, i.e., the event signal is a door opening event signal, some cameras are equipped with the ability to detect people entering the door, and some cameras are equipped with the ability to detect people leaving the door, in order to detect personnel activity. In this case, the first recognition model specifically includes a tracking model, a morphological extraction model, and an identity extraction model.

[0108] The tracking model continuously tracks moving targets in a video sequence. By analyzing the correlation between consecutive frames, each detected moving target is assigned a unique identifier, and its trajectory in the sequence is recorded. The morphology extraction model extracts the overall morphological features of the target from the image. It extracts a global feature vector of the target using a deep neural network, which characterizes the target's appearance. The identity extraction model extracts facial biometric features to verify the target's identity.

[0109] When the event signal is a door opening event signal, the morphological vector information is first extracted through the morphological extraction model, or the identity vector information is extracted through the identity extraction model, in order to match the target person.

[0110] The morphology extraction model extracts human body vector information from the target image. For each tracked target, it crops the target region from the target image based on the coordinate position and then extracts the human body feature vector of the target region to match the target person.

[0111] Similarly, the identity recognition model extracts facial vector information of people from the target image. The model first detects the face region and then extracts facial feature vectors. The extracted feature vectors are compared with the pre-stored person feature vectors in the Milvus database. The similarity between the newly extracted feature vectors and the vectors stored in the database is calculated. The similarity score determines whether a match has been found with the target person. When the similarity exceeds a preset threshold, it indicates that a match with the target person has been successfully found.

[0112] Once the target person is matched, an entrance / exit area is then set. This entrance / exit area is a predefined region within the surveillance image used to identify the location of the passageway for personnel to enter or exit. The entrance / exit area can be defined using image coordinate ranges, for example, by using polygonal or rectangular regions to describe the spatial extent, thus determining the entrance / exit area.

[0113] When a moving target is detected entering the entrance / exit area, the tracking model is invoked to obtain identifiers for one or more moving targets in the video stream. The tracking model utilizes a pedestrian tracking algorithm, receives the target image, performs multi-object target tracking processing, detects all people targets in the image, and assigns a unique identifier to each target.

[0114] If the target identifier of the moving target matches the target identifier of the person, an exit event record for the person is generated. In other words, if the two identifiers match, it means that the same person is passing through the entrance / exit, and an exit event record for that person is generated, completing one person entry / exit identification process.

[0115] During the matching process, the target identifier, corresponding feature vector, and matched personnel information of the current target are recorded. The mapping relationship between the target identifier and personnel information is cached in the Redis database. The Redis database is used to temporarily store the recognition results and supports fast query and update.

[0116] If no target person is matched, within a third preset time period, the application calls the FFmpeg component to continuously receive multiple target images processed by the FFmpeg component. If no person information can be matched based on the target image, a first alarm message is generated, which represents an abnormal intrusion event.

[0117] The third preset duration can be 1-5 minutes. During this process, the FFmpeg component processes multiple target images, performs intensive analysis on them, and generates multiple target images.

[0118] Furthermore, the system records the abnormal status information of the target identifier in the Redis cache, and tracks and identifies the target for 1-5 minutes. If no matching personnel information can be found during this period, an intrusion alarm is sent, i.e., the first alarm. Through tracking models and multi-feature extraction, the accuracy of personnel identification is improved, and authorized and unauthorized personnel can be distinguished.

[0119] Furthermore, the method provided in this application embodiment can also generate an alarm event record. After the step of generating the first alarm information, in response to generating the first alarm information, an alarm video sent by a streaming media server is obtained, the alarm video being a video segment including the time when the first alarm information is generated; the video segment is associated with and stored with the first alarm information to generate an alarm event record, the alarm event record being used to completely record the relevant information of an alarm event.

[0120] ZLMediaKit can store video streams. After the first alarm message is generated, in order to save the video segment at the time the first alarm message was generated, a video acquisition request is sent to ZLMediaKit. After receiving the video acquisition request, ZLMediaKit extracts the specified video segment from the video stream. The specified video segment can be a video segment of a preset duration that is pushed forward and backward a certain period of time from the time the first alarm message was generated, for example, a video segment 30 seconds before and after the alarm time.

[0121] To ensure that the corresponding video segment can be found through the first alarm information, or the corresponding first alarm information can be found through the video segment, an alarm event record is generated by establishing a logical connection between the video segment and the first alarm information.

[0122] Secondly, for the second path, if no event signal is received, the monitoring process is run according to the monitoring parameters obtained in step S100. Within the monitoring area, the monitoring process is started through the built-in timer mechanism to achieve periodic coverage of the monitoring area.

[0123] Monitoring parameters can include monitoring start time, monitoring end time, and interval time, which are used to define the execution plan for timed triggering of analysis. The monitoring start time sets the start time of the automatic start of the inspection process each day, the monitoring end time sets the end time of the automatic stop of the inspection process each day, and the interval time specifies the static time between two consecutive image acquisition and analysis operations between the monitoring start time and the monitoring end time.

[0124] S302: If the current time is between the start and end time of monitoring, obtain at least one second recognition model associated with the target camera based on the interval time.

[0125] At this time, the configuration information corresponding to the target camera includes non-event-triggered detection capabilities, such as smoke recognition and water leakage detection.

[0126] When the current time reaches the monitoring start time, the system switches from idle state to scheduled inspection state. First, it acquires the monitoring parameters of all cameras to confirm which cameras participate in this inspection and their respective inspection time rules. Simultaneously, it acquires configuration information to clarify the detection tasks each camera needs to perform during the inspection, i.e., its configured non-personnel entry / exit detection capabilities. Based on the detection capabilities indicated by the configuration information, it acquires the second recognition model to perform these detection tasks.

[0127] For example, the environmental monitoring time for computer room B is set to 08:00 to 20:00 daily, with a check every 30 seconds. When the current time (e.g., 14:05) falls within the set monitoring time window, the camera configured with non-event-triggered detection capabilities is identified as the target camera based on the configuration information. According to this configuration, the corresponding recognition model, i.e., the second recognition model, is periodically acquired.

[0128] The second recognition model is an algorithmic model trained and deployed for detecting non-personnel entry and exit. For example, the second recognition model for smoke recognition is a convolutional neural network classification model trained on smoke images.

[0129] For example, the second recognition model can be an image classification model, which is used to determine whether an image belongs to a specific category. When determining whether a fire or water leak has occurred in the computer room environment, the image classification model can analyze the image features and output a probability score to indicate the likelihood that the image contains such anomalies.

[0130] Similar to the first recognition model, for a single capability detection item, there may be multiple second recognition models.

[0131] After obtaining the second recognition model in S200, steps S300-S600 are executed. At this point, the target image may include specific objects or equipment, such as server racks, power distribution boxes, fire extinguishers, or temporarily placed tools. Abnormal states of these devices, such as malfunctioning indicator lights, equipment displacement, or open cabinet doors, may also be targets to be identified. The target image may also be an abnormal phenomenon or event, such as water stains caused by liquid leaks, smoke or open flames caused by electrical short circuits, and smoke caused by people smoking.

[0132] The obtained recognition result is the second recognition result. The second recognition result is similar to the first recognition result, reflecting the target to be identified in the image and its attributes, such as whether an abnormal target is detected, the specific category of the target, its position coordinates in the image, and the confidence score of this recognition.

[0133] The alarm strategy is used to judge the second identification result and generate a second alarm message. The second alarm message can be based on repeated detection. After multiple detections within a specific time window, the second alarm message is generated. For example, in a smoke detection scenario, it may be required that smoke features are detected in multiple consecutive images and the confidence level reaches a certain level before it is determined to be a real fire and an alarm message is generated.

[0134] The second type of alarm information includes alarms for non-personnel entry / exit capability detection items such as smoking, falling, water seepage in walls, floor collapse, smoke, and fire. For example, an alarm message indicating that smoke was generated in computer room A.

[0135] For example, in a smart data center scenario, monitoring parameters are set to start inspections at 8:00 PM daily and end at 6:00 AM the following day, with intervals of 5 seconds or 30 seconds. The configuration information includes smoke detection and water leakage detection capabilities for the cameras inside the data center. At 10:00 PM, the system requests the current image from the cameras inside the data center from ZLMediaKit. After acquiring the image, it calls the smoke detection model for analysis. The model outputs a second recognition result, indicating suspected smoke features in the image with a confidence level of 75%. The alarm strategy requires the smoke confidence level to exceed 70% twice consecutively before triggering an alarm. Therefore, 30 minutes later, the image is requested and analyzed again. This time, the smoke confidence level is 80%, meeting the judgment condition, and a second alarm message for fire warning is generated.

[0136] By using intermittent streaming, image acquisition and recognition are only activated and performed at specific times and intervals defined by the monitored parameters, reducing average computing load, network bandwidth usage, and energy consumption.

[0137] In the process of target image recognition, whether using the first recognition model or the second recognition model, it is necessary to select a recognition model for recognition. To improve analysis efficiency, a pre-screening step is used, that is, the feature vector of the target image is extracted first and compared with the event feature library to preliminarily determine the most likely target detection capability item. The corresponding recognition model can be called in a targeted manner. Specifically, step S500 may also include the following steps S501-S504.

[0138] S501: Extract the feature vector of the target image.

[0139] The recognition model converts the target image into feature vectors. These feature vectors have fixed dimensions, and the value in each dimension represents the strength or presence of a certain feature attribute. This allows the similarity between different images to be quantified by the distance or similarity between the vectors. The recognition model can be a lightweight feature extraction network, such as a shallow layer of a pre-trained convolutional neural network.

[0140] S502: Perform similarity calculations between the feature vector and multiple event feature vectors in the event database to obtain multiple calculation results; wherein, the event database is used to store event feature vectors associated with different detection capability items.

[0141] The event database contains multiple event feature vectors, each associated with a specific detection capability. These event feature vectors are derived from feature extraction of various typical scene samples.

[0142] Similarity calculation can be performed using the cosine similarity algorithm. This algorithm calculates the cosine of the angle between the current feature vector and each event feature vector in the event database, resulting in multiple calculation results. Each result is a value between 0 and 1, with a larger value indicating a higher similarity.

[0143] Understandably, to improve comparison efficiency, there can be multiple event databases. For example, each camera can have its own event database, and each event database stores event feature vectors associated with different detection capability items.

[0144] S503: Obtain the target calculation result and the target detection capability item associated with the target calculation result. The target calculation result is the calculation result with the highest similarity threshold among multiple calculation results.

[0145] The target calculation result is the calculation result with the largest value. The target detection capability term is the detection capability term associated with the event feature vector corresponding to the largest value, indicating the event type most likely to be included in the target image.

[0146] S504: Based on the target detection capability, call the recognition model to analyze the target image to generate recognition results.

[0147] Based on the target detection capability, the corresponding recognition model is called to analyze the target image and generate the recognition result. This process ensures that only the recognition model most relevant to the current scene is called, avoiding unnecessary model calls and waste of computing resources, so as to achieve rapid recognition.

[0148] If the current path is event-triggered, the first recognition model is invoked to generate the first recognition result. If the current path is time-triggered, the second recognition model is invoked to generate the second recognition result. This method enables on-demand and precise scheduling of analysis resources, suitable for scenarios where a single camera is configured with multiple detection capabilities, and can significantly reduce unnecessary model inference calculations.

[0149] After generating the identification result, and before performing a judgment based on the alarm strategy to generate alarm information, in order to improve the accuracy of the alarm, a feature library constructed from historical false alarm samples can be used to pre-filter the current identification result, and the user-confirmed false alarms can be continuously learned. Specifically, step S600 may also include the following steps S601-S605.

[0150] S601: Extract the feature vector of the target image and perform similarity calculation between the feature vector and multiple false alarm feature vectors in the false alarm feature library to generate a similarity result; wherein, the false alarm feature library is used to store feature vectors associated with historical false alarm events.

[0151] After obtaining the recognition result, the alarm judgment is not immediately initiated. Instead, a false alarm risk screening is first performed. A general feature extractor is called to extract the feature vector of the target image. This feature vector is a compact representation of the image content. Then, the extracted feature vector is compared with multiple false alarm feature vectors stored in the false alarm feature library to calculate similarity (e.g., calculate cosine similarity) and generate similarity results.

[0152] S602: If the similarity result is greater than or equal to the similarity threshold, no alarm information will be generated.

[0153] The calculated similarity result is compared with the preset similarity threshold. If the similarity result is greater than or equal to the similarity threshold, it indicates that the current target image is highly similar to historical false alarm samples and there is a high risk of false alarm.

[0154] At this time, no alarm information will be generated, and the event may be recorded as a filtered suspected false alarm, thereby effectively reducing the interference of invalid alarms to users.

[0155] S603: If the similarity result is less than the similarity threshold, generate an alarm message based on the recognition result.

[0156] If the similarity result is less than the similarity threshold, it indicates that the current image is significantly different from known false alarms. Then, the subsequent alarm strategy judgment process is initiated, and alarm information is generated based on the recognition result, i.e., step S600 is executed.

[0157] S604: In response to the user's false alarm confirmation command for alarm information feedback, the feature vector of the target image is stored in the false alarm feature library.

[0158] For generated alarm information, a user feedback channel is provided to continuously optimize the system. When a user views alarm information through the application interface, such as a fire alarm in the alarm list, if it is determined to be a false alarm, for example, a light reflection, the user can click the false alarm button, which responds to the user's false alarm confirmation instruction.

[0159] Upon receiving a false alarm confirmation command, the feature vector of the target image that triggered the alarm is automatically stored in the false alarm feature library. Simultaneously, the context information of this false alarm (such as camera ID, time, and the type of error identified) can be stored in conjunction with it.

[0160] In this way, the false alarm feature library can be dynamically expanded and updated. When a similar scenario occurs again, it can be filtered in step S600, achieving self-learning and continuous optimization, and gradually reducing the false alarm rate.

[0161] Different alarm messages correspond to different priorities. The configuration information also includes the alarm event level associated with the detection capability item. The priority of the generated alarm message needs to be determined according to the alarm event level associated with the detection capability item corresponding to the identification result.

[0162] Alarm event levels are used to define the severity and urgency of response to different abnormal events. For example, in a smart data center scenario, emergency, danger, warning, and alert levels can be preset. The emergency level corresponds to events such as fires that may immediately cause significant losses; the danger level corresponds to events such as smoke that require immediate handling; the warning level corresponds to security violations such as intrusion into the area; and the alert level corresponds to general abnormal events such as equipment doors not being closed.

[0163] Alarm event levels can be preset by the administrator when configuring camera capabilities and will serve as the basis for determining their priority when generating alarm information.

[0164] In step S600, the identification result is judged based on the alarm policy. Here, the alarm policy needs to determine the priority of the generated alarm information according to the alarm event level associated with the detection capability item corresponding to the identification result.

[0165] Specifically, once the identification result meets the conditions for generating alarm information according to the alarm strategy, the configuration information is first queried to obtain the alarm event level associated with the detection capability item that triggered the alarm.

[0166] According to the preset priority mapping rules, the alarm event level is converted into the priority of the alarm information. For example, the mapping rule can be: emergency level is P0 (highest priority), high risk level is P1, warning level is P2, and alert level is P3.

[0167] Based on the determined priority, alarm information is generated. The generated alarm information should include a priority field. Higher priority alarm information can trigger stronger notification methods, faster response processes, or be highlighted on the monitoring screen.

[0168] For example, during a scheduled inspection, after the smoke recognition second recognition model of camera Cam_02 analyzes the target image, it outputs a second recognition result (confidence level 0.85, exceeding the threshold of 0.6). The alarm policy determines that the condition is met (e.g., smoke is detected twice consecutively). A query of the configuration reveals that the alarm event level associated with smoke recognition is high-risk. Based on this, the alarm information priority is determined to be P1, and a second alarm message with the content "Smoke detected in computer room B" and priority P1 is generated, triggering the audible and visual alarm and pushing it to the administrator's mobile device.

[0169] Alarm information is generated based on alarm policies. Furthermore, alarm policies are associated with detection capability items. If different detection capability items are set for the camera, then the corresponding alarm policies will also be different. Alarm policies include policy parameters, which are set based on event types. For example, the policy parameters set for personnel entry and exit events are different from those for other events.

[0170] like Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the alarm setting strategy provided in an embodiment of this application.

[0171] The strategy parameters include a duration threshold and a suppression time threshold. The duration threshold ensures the stability of events, the suppression time threshold prevents alarm flooding, and a suppression timer is used to manage alarm intervals. Specifically, step S600 includes the following steps S701-S703.

[0172] Understandably, the policy parameters can be applied to both the first and second alarm messages.

[0173] S701: When the identification result indicates an abnormal state, start the duration timer; the duration of the duration timer is based on the alarm policy setting associated with the current detection capability item.

[0174] When the recognition model analyzes the target image and outputs a recognition result that represents an abnormal state, such as recognizing a fall posture with a confidence level higher than a threshold, it does not immediately generate an alarm. Instead, it starts a duration timer for this abnormal state.

[0175] The duration of the timer is determined based on the duration threshold set in the alarm policy associated with the current detection capability. During monitoring, time thresholds are set, with different duration thresholds corresponding to different detection capabilities. For example, a 30-second duration threshold might be set for fall detection, while a 5-second duration threshold might be set for smoke detection, in order to filter out instantaneous anomalies or false alarms. An alarm message will only be generated if the abnormal state persists for a certain period of time.

[0176] S702: When the duration timer reaches the duration threshold, determine whether the abnormal state meets the alarm triggering conditions.

[0177] For abnormal states, monitor whether the abnormal state persists during the timer's operation. For example, if multiple consecutive frames of images identify a fall, and the duration timer reaches the duration threshold, further check whether the alarm triggering conditions are met.

[0178] First, calculate the time interval, which is the interval between the current time and the last time an alarm was generated for the same detection capability item. Maintain a record to track the time when the most recent alarm was generated for each detection capability item.

[0179] The alarm triggering conditions include conditions A or B. Condition A is that there is currently no valid suppression timer, and condition B is that the time interval is greater than or equal to the suppression time threshold. The time interval is the time interval that has elapsed since the last alarm information for the same detection capability item was generated.

[0180] If the alarm trigger condition is condition A, a suppression timer is maintained for each detection capability item. When it runs, it indicates that the alarm of that type is in a suppression period and should not be sent repeatedly. If the alarm trigger condition is condition B, that is, a sufficiently long time has passed since the last alarm of the same type was sent, then re-alerting is allowed.

[0181] The alarm is triggered when either condition A or condition B is met. If condition A is met, an alarm can be triggered regardless of the time interval; if condition A is not met, condition B must be met for an alarm to be triggered.

[0182] S703: Generate alarm information when the alarm triggering conditions are met.

[0183] When the alarm triggering conditions are met, an alarm message is generated, which can be either the first alarm message or the second alarm message. At the same time, a new suppression timer is reset and started for the detection capability item, and its duration is set to the suppression time threshold associated with the detection capability item (e.g., 5 minutes), thereby starting a new suppression period to prevent the same alarm from being generated repeatedly in a short period of time.

[0184] If the alarm triggering conditions are not met (i.e., conditions A and B are not met simultaneously), no alarm information will be generated, and the continuous abnormal state will only be recorded in the log for subsequent analysis.

[0185] For example, the water leakage detection capability of the camera inside the computer room has an alarm policy set with a duration threshold of 10 seconds and a suppression time threshold of 300 seconds.

[0186] When a suspected water stain is detected, a 10-second timer is started. The system continuously checks for 10 seconds. Once the timer reaches its threshold, it checks if the last leak alarm was triggered 2 minutes prior (120 seconds ago), and if the suppression timer for this capability is still running (180 seconds remaining). The alarm triggering conditions are then determined: Condition A is not met; Condition B is not met. Therefore, the triggering conditions are not met, and no alarm is generated this time.

[0187] If a leak is detected again for 10 seconds 6 minutes after the last alarm, the time interval (360 seconds) is greater than the suppression time threshold (300 seconds), condition B is met, a new leak alarm is generated, and the suppression timer is reset.

[0188] This embodiment also provides a video processing function for automatically generating video thumbnails during monitoring. After the recognition result is generated, the function is triggered based on the recognition result. It can be understood that video thumbnails can be generated regardless of the recognition result. In this embodiment, the first recognition result and the second recognition result are used for illustration.

[0189] Obtain the first recognition result and / or the image change information in the second recognition result. The image change information is a dynamic event that is determined to be meaningful after being recognized by the model, such as the detection of a new target, the movement of an existing target, a significant change in the target's posture, or an abnormal scene state.

[0190] If the first recognition result includes image change information, a video acquisition request is sent to ZLMediaKit, causing ZLMediaKit to capture multiple first video streams associated with the first recognition result based on the video acquisition request. And / or, if the second recognition result includes image change information, a video acquisition request is sent to ZLMediaKit, causing ZLMediaKit to capture multiple second video streams associated with the second recognition result based on the video acquisition request.

[0191] This embodiment can also directly generate recognition results based on image change information. For the video stream acquired by the camera, image change information in the video stream is acquired, and at least one target camera associated with the event signal is determined based on the image change information. Then, steps S200-S500 are executed in sequence.

[0192] For ZLMediaKit, the video streams acquired from the camera are stored, including a first video stream and / or a second video stream. The second video stream is technically identical to the first video stream, differing only in the type of recognition result associated with it and the triggering scenario. Both the first and second video streams can be preset in duration, such as a 30-second video.

[0193] Following chronological order, multiple first and second video streams within a first preset duration are stitched together to generate a summary video. A summary video is a composite video file generated by merging multiple video segments in chronological order. It is not a simple cut from the original video stream, but rather a summary video generated by stitching together multiple key event segments from different time points using video processing technology. The length of a summary video is less than the total duration of the original surveillance video, but it includes segments of changing images, and can be used to quickly review various events occurring within the monitored area.

[0194] The first preset duration is 24 hours, which means that by splicing video streams, it is possible to understand the segments of the scene changes that occurred in the monitored area on that day.

[0195] As can be seen from the above, the video surveillance method provided in this application, by acquiring the configuration information and monitoring parameters of the cameras, unifies event-triggered and timed-triggered modes within a single framework. This allows for flexible response to external signals such as access control events, instantly initiating analysis of associated cameras to achieve precise monitoring and resource conservation. Simultaneously, based on preset monitoring times and intervals, it automatically performs timed analysis on areas or devices requiring periodic inspection, ensuring no blind spots in monitoring. Analysis tasks are intelligently scheduled according to different triggering conditions, and a recognition model matching the camera's detection capabilities is invoked for analysis, combined with alarm strategies to generate alarm information. Therefore, this method effectively overcomes the contradiction between high resource consumption and low monitoring coverage in existing technologies, improving resource utilization efficiency and overall alarm effectiveness while ensuring real-time monitoring and full coverage.

[0196] Figure 5 This is a schematic diagram of the structure of an event-triggered video surveillance system provided in an embodiment of this application.

[0197] like Figure 5 As shown, the video surveillance system provided in this application embodiment may include an analysis unit 100. The analysis unit 100 acquires the monitoring parameters and configuration information of cameras within the monitoring area. The configuration information represents the mapping relationship between the camera and at least one detection capability, and each detection capability corresponds to at least one recognition model. In response to an event signal or at regular intervals, the analysis unit acquires at least one recognition model associated with the target camera. The target camera is associated with the event signal or determined to be monitored based on the configuration information. The event signal is used to represent changes in the state of the monitoring area. An image acquisition request is sent to the streaming media server.

[0198] Furthermore, the video surveillance system provided in this application embodiment may also include a streaming media server 200, which receives video streams from at least one target camera within the monitoring area through an input port, wherein one target camera corresponds to one input port.

[0199] The analysis unit 100 also acquires the target image generated by the streaming media server 200 based on the video stream; calls the recognition model to analyze the target image to generate recognition results; and performs a judgment on the recognition results based on the alarm strategy to generate alarm information.

[0200] In some implementations, upon receiving an event signal, the configuration information corresponding to the target camera includes a detection capability item of the event trigger type; the recognition model is a first recognition model, the recognition result is a first recognition result, and the alarm information is a first alarm information. The analysis unit 100 is also used to respond to the event signal, determine at least one target camera associated with the event signal, and obtain at least one recognition model associated with the target camera.

[0201] In some implementations, when no event signal is received, the monitoring parameters include the monitoring start time, monitoring end time, and interval time. The configuration information corresponding to the target camera includes non-event-triggered detection capability items. The recognition model is a second recognition model, the recognition result is a second recognition result, and the alarm information is a second alarm information. If the current time is within the monitoring start time and monitoring end time, the analysis unit 100 is also used to obtain at least one second recognition model associated with the target camera based on the interval time.

[0202] In some implementations, the recognition result includes a first recognition result and a second recognition result; if the detection capability corresponding to the target camera includes video miniaturization. The analysis unit 100 is further configured to, when the first recognition result includes image change information, use a first video stream corresponding to the first recognition result; and / or, when the second recognition result includes image change information, use a second video stream corresponding to the second recognition result; and to stitch together multiple first video streams and second video streams within a first preset duration in chronological order to generate a miniaturized video.

[0203] In some implementations, the event signal includes an access control event signal; in response to the event signal, at least one target camera associated with the event signal is identified, and at least one recognition model associated with the target camera is obtained. Then, the analysis unit 100 is also used to receive the access control event signal; wherein the access control event signal includes an event type; the access control event signal is parsed to obtain the event type; if the event type is a door closing event, within a second preset time period, if the first recognition result does not include moving target features, the sending of image acquisition requests to the streaming media server 200 is stopped.

[0204] In some implementations, the detection capability of the event-triggered type includes personnel entry and exit recognition; the first recognition model includes a tracking model, a morphological extraction model, and an identity extraction model; the analysis unit 100 is also used to extract morphological vector information through the morphological extraction model, or extract identity vector information through the identity extraction model, to match the target personnel when the event signal is a door opening event signal; when the target personnel are matched, an entrance / exit area is set; when a moving target is detected entering the entrance / exit area, the tracking model is invoked to obtain the target identifier of one or more moving targets in the video stream; if the target identifier of the moving target is consistent with the target identifier of the target personnel, an exit event record of the target personnel is generated; when no target personnel are matched, the FFmpeg component is invoked through the application within a third preset time period to continuously receive multiple target images processed by the FFmpeg component; when personnel information cannot be matched based on the target image, a first alarm message is generated, the first alarm message representing an abnormal intrusion event.

[0205] In some implementations, after generating the first alarm information, the analysis unit 100 is further configured to, in response to generating the first alarm information, acquire the alarm video sent by the streaming media server 200, wherein the alarm video is a video segment including the moment the first alarm information is generated; and associate the video segment with the first alarm information for storage to generate an alarm event record.

[0206] In some implementations, the configuration information also includes alarm event levels associated with detection capability items; the recognition model is associated with model parameter configuration, which includes input image size and confidence threshold; the analysis unit 100 is also used to determine the priority of alarm information based on the alarm event levels associated with the detection capability items corresponding to the recognition results; and generate alarm information based on the priority.

[0207] In some implementations, the analysis unit 100 is further used to extract feature vectors from the target image; perform similarity calculations between the feature vectors and multiple event feature vectors in the event database to obtain multiple calculation results; wherein, the event database is used to store event feature vectors associated with different detection capability items; obtain the target calculation result and the target detection capability item associated with the target calculation result, wherein the target calculation result is the calculation result with the highest similarity threshold among multiple calculation results; and based on the target detection capability item, call the recognition model to analyze the target image to generate recognition results.

[0208] In some implementations, the analysis unit 100 is further used to extract feature vectors of the target image and perform similarity calculations between the feature vectors and multiple false alarm feature vectors in the false alarm feature library to generate a similarity result; wherein, the false alarm feature library is used to store feature vectors associated with historical false alarm events; if the similarity result is greater than or equal to the similarity threshold, no alarm information is generated; if the similarity result is less than the similarity threshold, alarm information is generated based on the recognition result; in response to the user's false alarm confirmation instruction for the alarm information feedback, the feature vector of the target image is stored in the false alarm feature library.

[0209] In some implementations, the detection capability includes multiple detection capability items; the alarm strategy is associated with the detection capability items, and the alarm strategy includes strategy parameters, which are set based on the event type; the strategy parameters include a duration threshold and a suppression time threshold; the analysis unit 100 is also used to start a duration timer when the identification result represents an abnormal state; the duration of the duration timer is set based on the alarm strategy associated with the current detection capability item; when the duration timer reaches the duration threshold, it is determined whether the abnormal state meets the alarm triggering condition; wherein, the alarm triggering condition is: there is currently no valid suppression timer, or the time interval is greater than or equal to the suppression time threshold, and the time interval is the time interval elapsed since the last generation time of the alarm information for the same detection capability item; when the alarm triggering condition is met, an alarm information is generated.

[0210] In a specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps of the various embodiments of the video surveillance method provided by the present invention. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0211] It is readily understood that, based on the several embodiments provided in this application, those skilled in the art can combine, split, or reorganize the embodiments of this application to obtain other embodiments, none of which exceed the protection scope of this application.

[0212] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A video monitoring method characterized by, The method comprises: obtaining monitoring parameters and configuration information of a camera in a monitoring area; wherein the configuration information represents a mapping relationship between the camera and at least one detection capability, and each detection capability corresponds to at least one identification model; in response to an event signal or every interval, obtaining at least one identification model corresponding to a target camera; the target camera is associated with the event signal or determined to be monitored based on the configuration information; the event signal is used to represent a state change of the monitoring area; sending an image acquisition request to a streaming media server to enable the streaming media server to receive a video stream of at least one target camera in the monitoring area through an input port; wherein one target camera corresponds to one input port; obtaining a target image generated by the streaming media server based on the video stream; calling the identification model to analyze the target image to generate an identification result; judging the identification result based on an alarm strategy to generate alarm information.

2. The video monitoring method according to claim 1, characterized in that, In the case of receiving the event signal, the configuration information corresponding to the target camera includes a detection capability item of the event trigger type; the identification model is a first identification model, the identification result is a first identification result, and the alarm information is a first alarm information; In response to the event signal or every interval, obtaining at least one identification model corresponding to a target camera, comprising: in response to the event signal, determining at least one target camera associated with the event signal, and obtaining at least one identification model corresponding to the target camera; In the case of not receiving the event signal, the monitoring parameters include monitoring start time, monitoring end time and interval time, and the configuration information corresponding to the target camera includes a detection capability item of the non-event trigger type; the identification model is a second identification model, the identification result is a second identification result, and the alarm information is a second alarm information; In response to the event signal or every interval, obtaining at least one identification model corresponding to a target camera, comprising: if the current time is within the monitoring start time and the monitoring end time, based on the interval time, obtaining at least one second identification model corresponding to the target camera.

3. The video monitoring method of claim 1, wherein, The identification result includes a first identification result and a second identification result; If the detection capability corresponding to the target camera includes a video clip, the method further comprises: In the case of including picture change information in the first identification result, the video stream is a first video stream corresponding to the first identification result; and / or, In the case of including picture change information in the second identification result, the video stream is a second video stream corresponding to the second identification result; According to the time sequence, a plurality of first video streams and second video streams within a first preset time are spliced to generate a clip video.

4. The video monitoring method of claim 2, wherein, The event signal includes a gate control event signal; after determining at least one target camera associated with the event signal and obtaining at least one identification model corresponding to the target camera in response to the event signal, the method further comprises: receive the access control event signal; wherein the access control event signal comprises an event type; resolve the access control event signal to obtain the event type; if the first recognition result does not include a motion target feature within a second preset time length, stop sending an image acquisition request to the streaming media server in the case of the event type being a door closing event.

5. The video monitoring method according to claim 2, characterized by, The event trigger type detection capability item includes personnel access identification. The first recognition model includes a tracking model, a shape extraction model, and an identity extraction model. The generation of the alarm information includes: In the case of the event signal being a door opening event signal, extract shape vector information through the shape extraction model, or extract identity vector information through the identity extraction model to match a target person; In the case of matching the target person, set an entrance and exit area; When a motion target enters the entrance and exit area, call the tracking model to obtain a target identifier of one or more motion targets in the video stream; If the target identifier of the motion target is consistent with the target identifier of the target person, generate an exit event record of the target person; In the case of not matching the target person, within a third preset time length, call an FFmpeg component through an application program to continuously receive a plurality of target images processed by the FFmpeg component; In the case of being unable to match personnel information based on the target image, generate first alarm information representing an abnormal intrusion event; After generating the first alarm information, the method further includes: In response to generating the first alarm information, obtaining an alarm video sent by the streaming media server, the alarm video being a video segment including a time when the first alarm information is generated; Storing the video segment and the first alarm information in association to generate an alarm event record.

6. The video monitoring method of claim 1, wherein, The configuration information further includes an alarm event level associated with a detection capability item; the recognition model is associated with a model parameter configuration, and the model parameter configuration includes an input picture size and a confidence threshold; The judgment on the recognition result based on the alarm strategy to generate the alarm information includes: Determining a priority of the alarm information according to an alarm event level associated with a detection capability item corresponding to the recognition result; Generating the alarm information based on the priority.

7. The video monitoring method of claim 1, wherein, The generation of the recognition result includes: Extracting a feature vector of the target image; Performing similarity calculation on the feature vector and a plurality of event feature vectors in an event database to obtain a plurality of calculation results; wherein the event database is used to store event feature vectors associated with different detection capability items; Obtaining a target calculation result and a target detection capability item associated with the target calculation result, the target calculation result being a calculation result with the highest similarity threshold among the plurality of calculation results; Based on the target detection capability item, calling the recognition model to analyze the target image to generate a recognition result.

8. The video monitoring method of claim 1, wherein, Before the judgment on the recognition result based on the alarm strategy to generate the alarm information, the method further includes: extract a feature vector of the target image, and perform similarity calculation on the feature vector and a plurality of false alarm feature vectors in a false alarm feature library to generate a similarity result; wherein the false alarm feature library is used to store feature vectors associated with historical false alarm events; if the similarity result is greater than or equal to a similarity threshold, the alarm information is not generated; if the similarity result is less than the similarity threshold, alarm information is generated based on the identification result; in response to a false alarm confirmation instruction fed back by a user for the alarm information, the feature vector of the target image is stored to the false alarm feature library.

9. The video monitoring method of claim 1, wherein, The detection capability includes a plurality of detection capability items; the alarm strategy is associated with a detection capability item, and the alarm strategy includes a strategy parameter, which is set based on an event type; the strategy parameter includes a duration threshold and a suppression time threshold; The judgment on the identification result based on the alarm strategy to generate alarm information includes: in the case that the identification result represents an abnormal state, a duration timer is started; the duration of the duration timer is set based on the alarm strategy associated with the current detection capability item; in the case that the duration timer reaches the duration threshold, it is judged whether the abnormal state meets an alarm trigger condition; wherein the alarm trigger condition is that there is no valid suppression timer currently, or a time interval is greater than or equal to the suppression time threshold, the time interval being a time interval elapsed from the generation time of the last alarm information for the same detection capability item; in the case that the alarm trigger condition is met, alarm information is generated.

10. A video surveillance system characterized by, It includes: an analysis unit configured to obtain monitoring parameters of a camera in a monitoring area and configuration information; wherein the configuration information represents a mapping relationship between the camera and at least one detection capability, each detection capability corresponding to at least one identification model; and, in response to an event signal or every interval of a period of time, obtaining at least one identification model associated with a target camera; the target camera being associated with the event signal or being determined to be monitored based on the configuration information; the event signal being used to represent a change in the state of the monitoring area; and sending an image acquisition request to a streaming media server; a streaming media server configured to receive video streams of at least one camera in the monitoring area through an input port; wherein one target camera corresponds to one input port; the analysis unit is further configured to obtain a target image generated by the streaming media server based on the video stream; and call the identification model to analyze the target image to generate an identification result; and judge the identification result based on an alarm strategy to generate alarm information.

Images