Monitoring system facing network camera and data processing method thereof
The network camera monitoring system, which integrates data collection, labeling and model training functions, solves the problems of manual dependence of traditional monitoring systems and false alarms and missed alarms of deep learning systems, and realizes self-learning and efficient monitoring.
Patent Information
- Application Number
- CN202510840455.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-23
- Publication Date
- 2025-09-19
AI Technical Summary
Traditional monitoring systems rely on manual real-time inspections, which can easily lead to visual fatigue. Hardware resource limitations make it difficult to meet large-scale scene coverage. In addition, intelligent monitoring systems based on deep learning have a high risk of false alarms and missed alarms in complex scenarios and lack adaptive capabilities.
A surveillance system for network cameras is designed, which integrates data collection, annotation, model training, and update functions. It uses AI training servers, recognition servers, and file application servers to work together, and optimizes model performance through iterative updates of deep learning models and user feedback.
It achieves self-learning capability, reduces false alarm rate, improves the system's adaptability and monitoring accuracy in complex scenarios, and reduces human resource requirements.
Smart Images
Figure CN120673343A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a monitoring system for network cameras and a data processing method thereof. Background Art
[0002] Traditional surveillance systems rely on manual, real-time visual inspections, which present significant limitations. First, prolonged, high-intensity surveillance can easily lead to visual fatigue and decreased attention span, reducing the efficiency of anomaly identification. Second, due to limited human and hardware resources, there is a limit to the number of cameras that can be connected to a single terminal, making it difficult to meet the full coverage requirements of large-scale scenarios. While intelligent surveillance systems based on deep learning achieve automated target detection, they face multiple challenges in practical application. The algorithms lack the ability to adapt to complex scenarios, requiring significant human resources for data collection, annotation, model training, and iterative optimization. Furthermore, due to data bias or limited model generalization, the systems face a high risk of false positives and false negatives, impacting monitoring accuracy and reliability. Summary of the Invention
[0003] The purpose of the present invention is to disclose a monitoring system for network cameras and a data processing method thereof, so as to integrate operations such as data collection, labeling, model training and updating into the system, so that the system has self-learning capabilities and improves overall performance.
[0004] To achieve the above objectives, the present invention proposes a monitoring system for network cameras, including a client and a server cluster, wherein the server cluster includes: an AI training server, a file application server, and at least two AI recognition servers; Each of the AI recognition servers is configured to capture a set of images from network cameras using independent threads, and then sequentially detect the captured images based on the current version of the deep learning model to select Class C images with A ≤ target confidence ≤ B and Class D images with target confidence > B; The file application server is used to collect the detection results of each AI recognition server and store the Class C images and Class D images in different data storage blocks respectively; and output the classification results of the Class D images to the user for secondary confirmation, and filter out the Class E images whose alarm types are affected by other types and the Class F images that are misclassified as non-other types based on the user's secondary confirmation information; The AI training server is used to iteratively update the deep learning model based on images of categories C, E, and F, and to notify each of the AI recognition servers to perform update processing.
[0005] Preferably, the AI training server is used to determine whether the sum of the number of images in the temporary data set synthesized by the C, E and F class images and the current version data set is greater than a set threshold after the training task is started. If not, all the remaining images in the temporary data set are directly used as the training set to update the deep learning model again; if so, the merged temporary data set is randomly divided into a training set and a validation set, a deep learning model is obtained based on the training set, and the mAP value of each image in the validation set is calculated. After deleting some images in the validation set according to a preset deletion rule based on the mAP value, it is determined whether the sum of the number of images remaining in the temporary data set is greater than the set threshold. If so, the training set and validation set are re-divided to enter the next cycle until the sum of the number of images remaining in the temporary data set is less than or equal to the set threshold, and then all the remaining images in the temporary data set are used as the training set to update the deep learning model again; finally, each of the AI recognition servers is notified to perform the update process; Among them, during the training and iteration process of the deep learning model, the C, E and F category images are automatically labeled based on the integrated Grounding Dino multimodal large model.
[0006] Optionally, the deep learning model adopts the YOLOv11 algorithm.
[0007] Optionally, in the YOLOv11 algorithm, the constraint condition of "the target is in motion" is added to the intrusion behavior detection and the optical flow algorithm is used to implement the motion detection function within the field of view, and each pixel in the image is classified as a moving pixel or a stationary pixel. When the detected target is on a moving pixel, the detection result is retained; when the detected target is on a stationary pixel, the detection result is discarded.
[0008] Optionally, during the capture process, any AI recognition server maintains a conditional variable for each capture thread to ensure that at most only one latest image of the same network camera is retained in the buffer. At the same time, for each network camera, the position where the image is inserted into the buffer is determined by the recorded capture time. The calculation formula is: ; in, is the polling period, It takes time to capture pictures. It is the buffer position value based on first-in-first-out; Indicates the number of elements in the current buffer; and in the process of sorting the buffer images, the latest images are included one by one. When an element is inserted into the index position of the buffer, the elements at the original index position and the subsequent positions are moved back one position.
[0009] Optionally, the network camera is also used to set a binary mask image with the same resolution as the network camera when only a part of the field of view needs to be monitored, where the white area represents the monitored area of interest and the black area represents the non-monitored area. Before YOLOv11 inference, YOLOv11 training and Grounding Dino automatic labeling, the image is cropped according to the maximum bounding box of the binary mask image, and the non-monitored area in the image is blackened. The algorithm only processes the cropped sub-image.
[0010] Optionally, the AI training server is further configured to re-label the dataset in a traversal manner based on the Grounding Dino multimodal large model when identifying that the user has added or deleted new alarm types in the next version of the dataset.
[0011] Optionally, the file application server is also used to obtain the missed reporting information submitted by the client, and the missed reporting information includes the missed reporting time period and the device information of the missed reporting network camera, so that the AI training server can automatically capture the missed reporting images from the network camera according to the missed reporting information and add them to the temporary data set.
[0012] To achieve the above-mentioned object, the present invention further discloses a data processing method for a monitoring system for a network camera, which is applied to the above-mentioned system and at least comprises: Each of the AI recognition servers uses an independent thread to capture a group of network camera images, and then sequentially detects each captured image based on the current version of the deep learning model, screening out Class C images with A ≤ target confidence ≤ B and Class D images with target confidence > B; The file application server collects the detection results of each AI recognition server and stores the C-type images and the D-type images in different data storage blocks respectively; outputs the classification results of the D-type images to the user for secondary confirmation, and filters out the E-type images whose alarm types are affected by other types and the F-type images that are misclassified as non-other types based on the user's secondary confirmation information; The AI training server iteratively updates the deep learning model based on the C, E and F category images, and notifies each of the AI recognition servers to perform the update process.
[0013] The present invention has the following beneficial effects: 1. Integrate data collection, labeling, model training, and updating into the system. The servers in the cluster have clear division of labor, and the collaborative logic is simple and reliable, which enables the system as a whole to have self-learning capabilities and improves the overall performance of the system.
[0014] 2. The detection results of the current version of the deep learning model are classified according to confidence. Among the alarm events pushed to client agents, Class E images marked as other types of alarms are likely false positives from the current model. Adding them to the updated dataset will help improve the model's adaptability to new, complex, and changing scenarios, thereby reducing the false positive rate. Some Class C images detected by the current model are correct. Adding them to the updated dataset will increase the diversity of positive samples in the updated dataset, which will help improve the detection performance of the updated model.
[0015] 3. When formulating deletion rules, generally, the higher the mAP value, the closer the model's detection results are to the real results. If such images are added to subsequent new versions of the dataset, it will not increase the sample diversity of the subsequent new version of the dataset, nor will it improve the detection performance of the subsequent new version of the model. Instead, it will increase the training time of the subsequent new version of the model. Therefore, such images can be deleted. Similarly, excluding images whose classification results are consistent with manual confirmation from the temporary dataset is also based on the same purpose. For example, the specific deletion rules are to delete images ranked in the top 10% of mAP and delete images with mAP>0.98. If the ratio of the dataset to the validation set is 50% each, it can be guaranteed that at least 5% of the images in the temporary dataset are deleted in one iteration, thereby ensuring that the number of images in the temporary dataset will be less than or equal to the set threshold within a limited number of iterations.
[0016] 4. After determining that the temporary dataset meets the expected threshold conditions, it is used entirely for model training, ensuring that the information of all images can be effectively focused on, further improving the reliability and accuracy of the model.
[0017] The present invention will be described in further detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings: Figure 1 This is a distribution diagram of the constituent nodes of the network camera-oriented monitoring system disclosed in Example 1 of the present invention.
[0019] Figure 2 This is a flow chart of a data processing method for a monitoring system for network cameras disclosed in Example 2 of the present invention. DETAILED DESCRIPTION
[0020] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered by the claims.
[0021] Example 1 This embodiment discloses a monitoring system for network cameras. Its essence is a self-learning intelligent monitoring system based on human feedback. It fully utilizes the system characteristics and the powerful zero-shot generalization capability of the Grounding Dino multimodal large model to integrate operations such as data collection, labeling, model training, and updating into the system. This enables the system to have self-learning capabilities, greatly reducing the workload of R&D and operation and maintenance personnel. At the same time, it enables the system to quickly adapt to complex scenarios and the possibility of widespread and large-scale deployment, reducing the risk of false alarms and missed alarms in the system and improving the overall performance of the system.
[0022] 1. System topology.
[0023] The system presents a star topology, such as Figure 1 As shown, the server cluster can be located in the same local area network and use the local area network for data transmission and communication.
[0024] The system consists of network cameras, an AI recognition server, an AI training server, a file application server, an NVR (Network Video Recorder), and a client. The network cameras support HTTP protocol image capture, allowing programs to automatically capture real-time images from the network cameras. The AI recognition server is equipped with multiple AI inference cards (such as the Huawei Ascend 300I PRO) and runs the AI recognition program. The AI training server is equipped with one or more AI training cards (such as the Huawei Ascend 300I DUO) and runs the AI training program. The NVR automatically stores video stream data from all network cameras in real time. The file application server runs a database management program and a business processing program. The client runs a web visualization program, providing a human-computer interaction interface.
[0025] 2. System workflow.
[0026] Suppose the system has M network cameras and N AI recognition servers. Typically, M is much larger than N, for example, M = 1000 and N = 5. Assuming all AI recognition servers have identical configurations, each server is responsible for monitoring approximately M / N network camera fields of view. The AI recognition server runs an AI recognition program to identify moving targets, such as people and vehicles, within its field of view (assuming the target recognition confidence threshold is A, which can be 0.05). The server then sends the image, network camera ID, and target information to the file application server. A service processing program merges the results of multiple AI recognition programs and records them in a database. When an image contains an object with a confidence level greater than B (B > A, which can be 0.3), the image and related information are pushed to the client interface, generating an alarm event. Client agents are required to perform a secondary confirmation of each alarm event to determine its true alarm type, labeling the alarm as an intrusion, human impact, vehicle impact, animal impact, or other impact type. The secondary confirmation results are then recorded in the database. If it's an intrusion alarm, the system can dispatch personnel to the scene to handle it. In this application, "other types" refer to those that are not currently receiving focus but are being monitored to provide dataset samples for subsequent expansion of new types. "Non-other types" refer to those that have already been explicitly prioritized by the system, including the aforementioned intrusion alarms, human impact, vehicle impact, and animal impact.
[0027] 3. AI recognition program design.
[0028] The AI recognition program runs on the AI recognition server, enabling the recognition of moving targets such as people and vehicles. Specifically, the YOLOv11 algorithm, a deep learning technology, is used for target detection. An optical flow algorithm is used to detect motion within the field of view, classifying each pixel in the image as either a moving pixel or a stationary pixel. When the detected target is located on a moving pixel, the detection result is retained. When the detected target is located on a stationary pixel, the detection result is discarded. The retained recognition result is the output of the AI recognition program. Because intrusions inevitably involve two motion processes: entering and leaving the monitored area, it is reasonable to add the "target in motion" constraint to intrusion behavior recognition. This does not affect the final alarm rate, while also reducing the false alarm rate for fixed objects such as fire hydrants, light poles, and shadows.
[0029] Alarm delay is an important performance indicator of the intelligent monitoring system. For example, it is generally required to be less than 2 seconds. For a certain network camera, the alarm delay time is the sum of the capture time, target recognition waiting time, target recognition time, and network transmission time to the file application server and client. Among them, the capture time is the most unstable, and the capture may fail, and the target recognition waiting time is programmable. When an AI recognition server is responsible for P network cameras and contains Q AI inference cards, the AI recognition program run by the server needs to solve the multi-producer-multi-consumer problem consisting of P capture threads and Q recognition threads, so as to reduce the alarm delay mean and variance of these P network cameras as much as possible, thereby ensuring that the alarm delay of all network cameras is less than 2 seconds with a greater probability. To this end, this embodiment discloses an improved semaphore algorithm that dynamically adjusts the target recognition waiting time according to the actual capture time to solve the multi-producer-multi-consumer problem here.
[0030] Specifically, assume that the AI recognition server is configured with P network cameras (for example, 200) and has Q AI inference cards (for example, 4). The AI recognition program uses P independent threads to capture images from each of the P network cameras, and uses Q independent threads to execute the target recognition algorithm. The algorithm pseudocode is shown in Table 1 below. Two improvements are made to the original semaphore algorithm for the multi-producer-multi-consumer problem. First, a conditional variable is maintained for each capture thread to ensure that at most one of the latest images from a given network camera is retained in the buffer, thereby reducing the target recognition wait time and, in turn, the mean alarm delay. Second, for a given network camera, the recorded capture time is used to infer the location where the image data is inserted into the buffer, ensuring that the sum of the capture time and the wait time is as stable as possible, thereby reducing the variance of the alarm delay.
[0031] The inference method is shown in the following formula (1), where period is the polling period, which is determined by the target recognition time and the number of network cameras P. It is generally stable, for example, 2 seconds. captrue_time is the time taken for this capture, index is the buffer position value, and len(buffer) represents the number of elements in the buffer at the current moment.
[0032] Formula (1) The buffer is a linear linked list containing at most P elements. Inserting an element at position index in the buffer means that the elements at and after the previous index position must be shifted back one position. Popping the zeroth element from the buffer means that the elements at and after the previous index position must be shifted forward one position. Buffer elements may contain fields such as image data and network camera numbers.
[0033] 4. AI training program design.
[0034] The AI training program generates new datasets, trains and updates the YOLOv11 model, and aims to improve the performance of the YOLOv11 object detection algorithm. This program can be run periodically, weekly or monthly, or manually. It doesn't require real-time performance and does not affect the normal operation of business systems.
[0035] The AI training program's operational flow is as follows: The program first downloads images from alarm events manually labeled as "other impacts" from the file application server, and also downloads images containing objects with a confidence level between [A, B], to obtain a new image set. The Grounding Dino algorithm, part of the multimodal large model technology, is then used to automatically annotate the new image set, generating a new dataset. This dataset is then merged with the existing dataset to form a temporary dataset. In this embodiment, the temporary dataset also includes images with "other impacts" and misclassified images of non-other impacts, filtered out based on user secondary confirmation. If the number of images in the temporary dataset is no greater than a set threshold (e.g., 100,000), the temporary dataset is considered the new dataset. All images in the new dataset are then used as the training set to update the deep learning model again. The YOLOv11 training program is then run to train a new object detection model. Finally, the new model is updated to all AI recognition servers. If the number of images in the temporary dataset is greater than the set threshold, dataset reduction is performed to ensure that the number of images in the new dataset remains limited, thereby ensuring that the training time of the YOLOv11 model is always limited.
[0036] Preferably, the dataset reduction technique of this embodiment first randomly divides the temporary dataset into 50% as a training set and the remaining 50% as a validation set. The YOLOv11 training program is then run to obtain a temporary detection model. The temporary model is then used to evaluate its detection performance on the validation set, removing images with the top 10% mAP (Mean Average Precision) values and images with an mAP greater than 0.98. Finally, the training set and the validation set (after some images have been removed) are merged to form a new temporary dataset. A higher mAP value indicates that the detection results of the temporary model are closer to the true results. Adding such images to a subsequent new dataset will not increase the sample diversity of the dataset or improve the detection performance of the model, but will increase the training time of the model. Therefore, such images can be removed. Removing images with the top 10% mAP and images with an mAP greater than 0.98 ensures that at least 5% of the images in the temporary dataset are removed per iteration, thereby ensuring that the number of images in the temporary dataset remains less than or equal to a set threshold within a limited number of iterations. When the loop iterates until the sum of the number of images remaining in the temporary dataset is less than or equal to the set threshold, all the remaining images in the temporary dataset are used as training sets to update the deep learning model again; finally, each AI recognition server is notified to perform the update process.
[0037] Among the alarm events pushed to client agents, images marked as other types of alarms are likely false positives for the current model. Adding these to the updated dataset will help improve the model's adaptability to new, complex, and changing scenarios, thereby reducing false positives. In the image set where the current model detects objects with a confidence level between [A, B], some of these images are correct detections. Adding these to the updated dataset will increase the diversity of positive examples in the dataset, improving the model's detection performance.
[0038] Grounding Dino is a high-performance, multimodal, open-vocabulary object detection model based on a transformer architecture. While its recognition speed is relatively slow and cannot be used for real-time object detection tasks, it achieves very high accuracy and precision in detecting common types such as people, vehicles, and animals. Therefore, in this example, it is used as an automatic annotation tool, using its detection results as the standard answer for the new model, thereby achieving knowledge distillation.
[0039] In addition, the first version of the YOLOv11 model and the first version of the dataset were manually given during the system deployment phase.
[0040] 5. Other supplementary situations that can be optimized.
[0041] A. Defense zone map function.
[0042] When only a portion of the IP camera's field of view needs to be monitored, the zone map feature can be introduced. The zone map is a binary mask with the same resolution as the IP camera. White areas represent the monitored area of interest, and black areas represent the unmonitored area. Before YOLOv11 recognition, YOLOv11 training, and Grounding Dino automatic annotation, the image is cropped according to the zone map's maximum bounding box, and the unmonitored areas in the image are blacked out. The algorithm only processes the cropped image, which allows for more efficient use of computing resources and further improves the system's overall alarm performance.
[0043] B. One-key add and delete function for alarm target types When adding or removing target types for an alert, the system leverages the Grounding Dino multimodal large model's powerful generalization capabilities in open-set object detection. After adding or removing target types, the system simply executes an AI training program, automatically re-annotates the accumulated dataset, and updates the newly trained model to the AI recognition server. For example, if a user initially focuses on human intrusions, the target type will be person. Later, if the user wants to focus on animal intrusions, they can simply add the animal type on the client interface, effectively tracking both human and animal intrusions.
[0044] C. Add the function of missing samples.
[0045] Smart surveillance systems generally undergo a gradual improvement process. During actual use or testing, if any underreporting is detected, the client can manually submit information such as the underreported time period and network camera number. The system will automatically download images from the NVR for that time period, import them into the AI training server, and execute the AI training program, thereby improving the detection performance of the new model and reducing the risk of subsequent underreporting.
[0046] Example 2 Corresponding to the above embodiment 1, this embodiment discloses a data processing method for a monitoring system for a network camera, and the method is applied to the above system, such as Figure 2 As shown, including at least: Step S1: Each AI recognition server uses an independent thread to capture a group of images from network cameras, and then detects the captured images in sequence based on the current version of the deep learning model, screening out Class C images with A ≤ target confidence ≤ B and Class D images with target confidence > B.
[0047] In step S2, the file application server collects statistics on the detection results of each AI recognition server and stores the Class C images and Class D images in different data storage blocks respectively; and outputs the classification results of the Class D images to the user for secondary confirmation, and filters out the Class E images whose alarm types are affected by other types and the Class F images that are misclassified as non-other types based on the user's secondary confirmation information.
[0048] Step S3: The AI training server iteratively updates the deep learning model based on images of categories C, E, and F, and notifies each AI recognition server to perform the update process.
[0049] Preferably, the specific implementation of this step can be: after the training task is started, the AI training server is used to determine whether the sum of the number of images in the temporary data set synthesized by the C, E and F class images and the current version data set is greater than the set threshold. If not, all the remaining images in the temporary data set are directly used as the training set to update the deep learning model again; if so, the merged temporary data set is randomly divided into a training set and a validation set, the deep learning model is updated based on the data set and the mAP value of each image in the validation set is calculated. After deleting some images in the validation set according to the preset deletion rule based on the mAP value, it is determined whether the sum of the remaining images in the temporary data set is greater than the set threshold. If so, the training set and validation set are re-divided to enter the next cycle until the sum of the remaining images in the temporary data set is less than or equal to the set threshold, and then all the remaining images in the temporary data set are used as training sets to update the deep learning model again; finally, each AI recognition server is notified to perform the update process.
[0050] Furthermore, during the training and iteration process of the deep learning model, images of categories C, E, and F are automatically annotated based on the integrated Grounding Dino multimodal large model.
[0051] The rest of the logic of this embodiment is the same as that of embodiment 1. For example, during the image capture process, any AI recognition server maintains a conditional variable for each capture thread to ensure that at most only one latest image from the same network camera is retained in the buffer. At the same time, for each network camera, the position where the image is inserted into the buffer is estimated based on the recorded capture time. The calculation formula is: ; in, is the polling period, It takes time to capture pictures. It is the buffer position value based on first-in-first-out; Indicates the number of elements in the current buffer; and in the process of sorting the buffer images, the latest images are included one by one. When an element is inserted into the index position of the buffer, the elements at the original index position and the subsequent positions are moved back one position.
[0052] Furthermore, this embodiment also includes: when the network camera only needs to monitor a part of the field of view, a binary mask image with the same resolution as the network camera is set, where the white area represents the monitored area of interest and the black area represents the non-monitored area. Before YOLOv11 inference, YOLOv11 training and Grounding Dino automatic labeling, the image is cropped according to the maximum bounding box of the binary mask image, and the non-monitored area in the image is blackened. The algorithm only processes the cropped sub-image.
[0053] Alternatively, the method of this embodiment also includes: the file application server obtains the missed reporting information submitted by the client, the missed reporting information includes the missed reporting time period and the device information of the missed reporting network camera, and in conjunction with this, the AI training server automatically captures the missed reporting images from the network camera according to the missed reporting information and adds them to the temporary data set.
[0054] In summary, the network camera-oriented monitoring system and data processing method thereof disclosed in this embodiment have at least the following beneficial effects: 1. Integrate data collection, labeling, model training, and updating into the system. The servers in the cluster have clear division of labor, and the collaborative logic is simple and reliable, which enables the system as a whole to have self-learning capabilities and improves the overall performance of the system.
[0055] 2. The current version of the deep learning model classifies alarm events pushed to client agents based on confidence. Class E images marked as other alarm types are likely false positives for the current model. Adding these images to the updated dataset will help improve the model's adaptability to new, complex, and changing scenarios, thereby reducing false positives. Some of the Class C images detected by the current model are correct. Adding these images to the updated dataset will increase the diversity of positive samples in the dataset, improving the model's detection performance.
[0056] 3. When formulating deletion rules, generally, the higher the mAP value, the closer the model's detection results are to the real results. If such images are added to subsequent new versions of the dataset, it will not increase the sample diversity of the subsequent new version of the dataset, nor will it improve the detection performance of the subsequent new version of the model. Instead, it will increase the training time of the subsequent new version of the model. Therefore, such images can be deleted. Similarly, excluding images whose classification results are consistent with manual confirmation from the temporary dataset is also based on the same purpose. For example, the specific deletion rules are to delete images ranked in the top 10% of mAP and delete images with mAP>0.98. If the ratio of the dataset to the validation set is 50% each, it can be guaranteed that at least 5% of the images in the temporary dataset are deleted in one iteration, thereby ensuring that the number of images in the temporary dataset will be less than or equal to the set threshold within a limited number of iterations.
[0057] 4. After determining that the temporary dataset meets the expected threshold conditions, it is used entirely for model training, ensuring that the information of all images can be effectively focused on, further improving the reliability and accuracy of the model.
[0058] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.
Claims
1. A monitoring system for network cameras, including a client and a server cluster, characterized in that: The server cluster includes: an AI training server, a file application server and at least two AI recognition servers; Each of the AI recognition servers is configured to capture a set of images from network cameras using independent threads, and then sequentially detect the captured images based on the current version of the deep learning model to select Class C images with A ≤ target confidence ≤ B and Class D images with target confidence > B; The file application server is used to collect the detection results of each AI recognition server and store the Class C images and Class D images in different data storage blocks respectively; and output the classification results of the Class D images to the user for secondary confirmation, and filter out the Class E images whose alarm types are affected by other types and the Class F images that are misclassified as non-other types based on the user's secondary confirmation information; The AI training server is used to iteratively update the deep learning model based on images of categories C, E, and F, and to notify each of the AI recognition servers to perform update processing.
2. The network camera-oriented monitoring system according to claim 1, characterized in that: The AI training server is used to determine whether the sum of the number of images in the temporary dataset synthesized by the C, E and F category images and the current version dataset is greater than a set threshold after the training task is started. If not, all the remaining images in the temporary dataset are directly used as the training set to update the deep learning model again; If so, randomly divide the merged temporary data set into a training set and a validation set, obtain a deep learning model based on the training set and calculate the mAP value of each image in the validation set, delete some images in the validation set according to the preset deletion rule based on the mAP value, and then determine whether the sum of the number of images remaining in the temporary data set is greater than the set threshold. If so, re-divide the training set and validation set to enter the next cycle until the sum of the number of images remaining in the temporary data set is less than or equal to the set threshold, and then use all the remaining images in the temporary data set as the training set to update the deep learning model again; Among them, during the training and iteration process of the deep learning model, the C, E and F category images are automatically labeled based on the integrated Grounding Dino multimodal large model.
3. The monitoring system for network cameras according to claim 2, characterized in that: The deep learning model adopts the YOLOv11 algorithm.
4. The monitoring system for network cameras according to claim 3, characterized in that: The YOLOv11 algorithm adds the "target in motion" constraint to intrusion detection and uses an optical flow algorithm to detect motion within the field of view. Each pixel in the image is classified as either moving or stationary. If the detected target is located on a moving pixel, the detection result is retained; if the detected target is located on a stationary pixel, the detection result is discarded.
5. The network camera monitoring system according to any one of claims 1 to 4, characterized in that: During the capture process, any AI recognition server maintains a conditional variable for each capture thread to ensure that the buffer retains at most one latest image from the same network camera. At the same time, for each network camera, the position where the image is inserted into the buffer is estimated based on the recorded capture time. The calculation formula is: ； in, is the polling period, It takes time to capture pictures. It is the buffer position value based on first-in-first-out; Indicates the number of elements in the current buffer; and in the process of sorting the buffer images, the latest images are included one by one. When an element is inserted into the index position of the buffer, the elements at the original index position and the subsequent positions are moved back one position.
6. The network camera monitoring system according to claim 5, characterized in that: The network camera is also used to set a binary mask image with the same resolution as the network camera when only a part of the field of view needs to be monitored. The white area represents the monitored area of interest, and the black area represents the non-monitored area. Before YOLOv11 inference, YOLOv11 training and Grounding Dino automatic labeling, the image is cropped according to the maximum bounding box of the binary mask image, and the non-monitored area in the image is blackened. The algorithm only processes the cropped sub-image.
7. The network camera monitoring system according to claim 6, characterized in that: The AI training server is also used to re-label the dataset in a traversal manner based on the Grounding Dino multimodal large model when identifying that the user has added or deleted new alarm types in the next version of the dataset.
8. The network camera monitoring system according to claim 5, characterized in that: The file application server is also used to obtain the missed reporting information submitted by the client, which includes the missed reporting period and the device information of the missed network camera, so that the AI training server can automatically capture the missed images from the network camera according to the missed reporting information and add them to the temporary data set.
9. A data processing method for a monitoring system for a network camera, characterized in that: The method is applied to the system according to any one of claims 1 to 8, and at least comprises: Each of the AI recognition servers uses an independent thread to capture a group of network camera images, and then sequentially detects each captured image based on the current version of the deep learning model, screening out Class C images with A ≤ target confidence ≤ B and Class D images with target confidence > B; The file application server collects the detection results of each AI recognition server and stores the C-type images and the D-type images in different data storage blocks respectively; outputs the classification results of the D-type images to the user for secondary confirmation, and filters out the E-type images whose alarm types are affected by other types and the F-type images that are misclassified as non-other types based on the user's secondary confirmation information; The AI training server iteratively updates the deep learning model based on the C, E and F category images, and notifies each of the AI recognition servers to perform the update process.