Video monitoring task processing method and device

Abstract

The invention discloses a video monitoring task processing method and device. The method comprises the steps that before a video monitoring task is started, a first target detection model with corresponding precision is selected according to an initial task level and a preset first mapping relation table, the table is associated with models with different task levels and data processing precision, each model comprises a feature extraction and analysis sub-model, and the precision of the first target detection model is not lower than that of the former; in the task execution period, a multi-view camera is used for obtaining multiple frames of images at each collection moment, and a target detection result is obtained through feature extraction and analysis of a first target detection model; and meanwhile, monitoring and calculating the resource usage amount, adjusting the task level according to the target detection result, and selecting a matched new target detection model through the first mapping relation table to replace the original model for subsequent detection. According to the method and the device, the technical problem that the calculation resource waste is easily caused by fixed model calculation precision in a related video monitoring scheme is solved.

Description

Technical Field

[0001] The present application relates to the field of video surveillance technology, and in particular to a method and device for processing video surveillance tasks. Background Art

[0002] Multi-camera security cameras can capture scene information from multiple angles, providing a more comprehensive and three-dimensional monitoring perspective. However, with the increase in the number of cameras, the improvement in image resolution, and the increasing complexity of monitoring scenes, processing the data collected by these multi-cameras faces huge challenges. From a technical perspective, the image data collected by multi-cameras is characterized by high dimensionality, large scale, and high complexity. In the data processing process, many complex algorithms and computing operations are involved, such as feature extraction, target detection, and behavior analysis. Traditional computing methods usually use a single precision (such as full precision) for calculation. Although this can ensure the accuracy of the calculation results, it brings huge computing resource consumption. On the one hand, full-precision calculations require a large amount of memory to store data and intermediate results, which is a heavy burden for devices with limited memory resources. On the other hand, the amount of calculation of full-precision calculations is huge, resulting in extended calculation time, which is difficult to meet the real-time requirements of security monitoring. Especially in scenarios that require rapid response and handling of emergencies, calculation delays may lead to serious consequences. In addition, multi-camera security cameras often need to run continuously for a long time. The high-energy consumption computing method not only increases the operating cost, but also may cause problems such as equipment heating and decreased stability, affecting the reliability and service life of the system.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a method and device for processing a video surveillance task, so as to at least solve the technical problem that the model calculation accuracy in the related video surveillance scheme is fixed, which easily causes waste of computing resources.

[0005] According to one aspect of an embodiment of the present application, a video surveillance task processing method is provided, comprising: before the video surveillance task starts, obtaining an initial task level of the video surveillance task, and determining a first target detection model corresponding to the initial task level according to a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing precisions, each target detection model includes a feature extraction submodel and a feature analysis submodel, and the data processing precision of the feature analysis submodel is not lower than the data processing precision of the feature extraction submodel; during the execution of the video surveillance task, cyclically executing the following process until the video surveillance task is terminated: at each acquisition moment, obtaining multiple frames of images acquired by multiple cameras, and using the first feature extraction submodel in the first target detection model to extract image features in the multiple frames of images, and using the first feature analysis submodel in the first target detection model to analyze the image features to obtain a target detection result corresponding to the acquisition moment; obtaining the computing resource usage of the monitoring system during the target detection process, and adjusting the task level of the video surveillance task according to the computing resource usage and the target detection result, determining a second target detection model corresponding to the new task level according to the first mapping relationship table, and using the second target detection model to replace the first target detection model to perform target detection at the next acquisition moment.

[0006] Optionally, obtaining the initial task level of the video surveillance task includes: obtaining the initial task level set for the target object; or determining the target task type of the video surveillance task, and determining the initial task level corresponding to the target task type based on a preset second mapping relationship table, wherein the second mapping relationship table stores a mapping relationship between different task types and different task levels, and the task type includes at least one of the following: target recognition, target position detection, and target tracking.

[0007] Optionally, the training process of multiple target detection models in the first mapping relationship table includes: constructing multiple feature extraction sub-models and feature analysis sub-models with different data processing precisions; obtaining multiple initial detection models based on the multiple feature extraction sub-models and feature analysis sub-models, wherein each initial detection model includes a feature extraction sub-model and a feature analysis sub-model, and the data processing precision of the feature analysis sub-model is not lower than the data processing precision of the feature extraction sub-model; obtaining multiple images as training samples, and determining the sample label corresponding to each training sample; using multiple training samples to iteratively train multiple initial detection models respectively; and using the multiple models after training as target detection models.

[0008] Optionally, after iteratively training multiple initial detection models using multiple training samples, the method also includes: for each model after the training is completed, pruning the model according to a preset pruning method, and fine-tuning the model parameters of the pruned model to obtain a target detection model; or, for each model after the training is completed, using the model as a teacher model, training according to the knowledge distillation method to obtain a student model corresponding to the teacher model, and using the student model as the target detection model.

[0009] Optionally, the first feature extraction submodel in the first target detection model is used to extract image features from multiple frames of images, and the first feature analysis submodel in the first target detection model is used to analyze the image features, including: performing image preprocessing on the acquired multiple frames of images, wherein the image preprocessing includes at least one of the following: image scaling, image grayscale; performing feature extraction on the preprocessed multiple frames of images using the first feature extraction submodel to obtain first image features; when the data processing accuracy of the first feature extraction submodel and the first feature analysis submodel are the same, the first image features are input into the first feature analysis submodel to obtain target detection results output by the first feature analysis submodel; when the data processing accuracy of the first feature extraction submodel and the first feature analysis submodel are different, the first image features are converted into second image features that match the data processing accuracy of the first feature analysis submodel, and the second image features are input into the first feature analysis submodel to obtain target detection results output by the first feature analysis submodel.

[0010] Optionally, a first feature extraction submodel is used to perform feature extraction on multiple frames of images after image preprocessing to obtain a first image feature, including: for each frame of image after image preprocessing, using the first feature extraction submodel to extract a third image feature in the image, and determining a preset feature weight corresponding to the camera corresponding to the image; and performing feature fusion on multiple third image features according to the preset feature weights corresponding to each third image feature to obtain a first image feature.

[0011] Optionally, the task level of the video surveillance task is adjusted based on the computing resource usage and the target detection result, including: when the target detection result indicates that the target has not been detected and has continued for a preset time, lowering the task level of the video surveillance task; and / or, when the target detection result indicates that the target has been detected and the computing resource usage does not exceed a first preset threshold, increasing the task level of the video surveillance task; and / or, when the computing resource usage exceeds a first preset threshold, lowering the task level of the video surveillance task.

[0012] According to another aspect of the embodiment of the present application, a video surveillance task processing device is also provided, including: a model selection module, used to obtain the initial task level of the video surveillance task before the start of the video surveillance task, and determine the first target detection model corresponding to the initial task level according to a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing accuracies, each target detection model includes a feature extraction sub-model and a feature analysis sub-model, and the data processing accuracy of the feature analysis sub-model is not lower than the data processing accuracy of the feature extraction sub-model; a data processing module, used to cyclically execute the following during the execution of the video surveillance task The process ends until the video surveillance task is terminated: at each acquisition moment, multiple frames of images acquired by multiple cameras are obtained, and the image features in the multiple frames of images are extracted using the first feature extraction submodel in the first target detection model, and the image features are analyzed using the first feature analysis submodel in the first target detection model to obtain the target detection result corresponding to the acquisition moment; the computing resource usage of the monitoring system in the target detection process is obtained, and the task level of the video surveillance task is adjusted according to the computing resource usage and the target detection result, and the second target detection model corresponding to the new task level is determined according to the first mapping relationship table, and the second target detection model is used to replace the first target detection model to perform target detection at the next acquisition moment.

[0013] According to another aspect of an embodiment of the present application, a computer program product is further provided, the computer program product comprising: a computer program, wherein when the computer program is executed by a processor, the above-mentioned video surveillance task processing method is implemented.

[0014] According to another aspect of an embodiment of the present application, an electronic device is further provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the above-mentioned video surveillance task processing method through the computer program.

[0015] In an embodiment of the present application, before the start of a video surveillance task, an initial task level of the video surveillance task is obtained, and a first target detection model corresponding to the initial task level is determined according to a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing precisions, and each target detection model includes a feature extraction submodel and a feature analysis submodel, and the data processing precision of the feature analysis submodel is not lower than the data processing precision of the feature extraction submodel; during the execution of the video surveillance task, the following process is executed in a loop until the video surveillance task is terminated: at each acquisition moment, multiple frames of images acquired by a multi-eye camera are acquired, and image features in the multiple frames of images are extracted using a first feature extraction submodel in the first target detection model, and the image features are analyzed using a first feature analysis submodel in the first target detection model to obtain a target detection result corresponding to the acquisition moment; the computing resource usage of the monitoring system during the target detection process is obtained, and the task level of the video surveillance task is adjusted according to the computing resource usage and the target detection result, and a second target detection model corresponding to the new task level is determined according to the first mapping relationship table, and the second target detection model is used to replace the first target detection model to perform target detection at the next acquisition moment. In this way, the task level and target detection model can be dynamically adjusted according to the current computing resource usage and target detection results, and target detection models of different accuracy can be selected according to different task levels to ensure the best detection accuracy and improve resource utilization efficiency in different scenarios, thereby solving the technical problem of fixed model calculation accuracy in related video surveillance solutions, which easily leads to waste of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a flowchart of an optional video surveillance task processing method according to an embodiment of the present application;

[0018] Figure 2 is a structural schematic diagram of an optional video surveillance task processing device according to an embodiment of the present application;

[0019] Figure 3 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] Example 1

[0023] According to an embodiment of the present application, a video surveillance task processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] Figure 1 is a flow chart of a video surveillance task processing method provided according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0025] Step S102, before the video surveillance task starts, the initial task level of the video surveillance task is obtained, and a first target detection model corresponding to the initial task level is determined according to a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing precisions, each target detection model includes a feature extraction sub-model and a feature analysis sub-model, and the data processing precision of the feature analysis sub-model is not lower than the data processing precision of the feature extraction sub-model;

[0026] Step S104, during the execution of the video surveillance task, the following process is executed in a loop until the video surveillance task is terminated: at each acquisition moment, multiple frames of images acquired by multiple cameras are acquired, and image features in the multiple frames of images are extracted using the first feature extraction submodel in the first target detection model, and the image features are analyzed using the first feature analysis submodel in the first target detection model to obtain the target detection result corresponding to the acquisition moment; the computing resource usage of the monitoring system during the target detection process is acquired, and the task level of the video surveillance task is adjusted according to the computing resource usage and the target detection result, and the second target detection model corresponding to the new task level is determined according to the first mapping relationship table, and the second target detection model is used to replace the first target detection model to perform target detection at the next acquisition moment.

[0027] The following describes each step of the video surveillance task processing method in conjunction with a specific implementation process.

[0028] Before a video surveillance task begins, the system will first obtain the initial task level of the task.

[0029] It should be noted that this initial level can be set manually by the user, or automatically determined by the system according to the task type (such as target recognition, target location detection, target tracking, etc.). For example, if the monitoring task is used for security protection of important areas, a higher task level may be set by default to ensure detection accuracy.

[0030] As an optional implementation, when obtaining the initial task level of a video surveillance task, in addition to obtaining the initial task level set for the target object, the target task type of the video surveillance task can also be determined, and the initial task level corresponding to the target task type can be determined based on a preset second mapping relationship table, wherein the second mapping relationship table stores a mapping relationship between different task types and different task levels, and the task type includes at least one of the following: target identification, target position detection, and target tracking.

[0031] According to the initial task level, a first target detection model corresponding to the initial task level is determined according to a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing accuracies, each target detection model includes a feature extraction sub-model and a feature analysis sub-model, and the data processing accuracy of the feature analysis sub-model is not lower than the data processing accuracy of the feature extraction sub-model.

[0032] The training process of multiple target detection models in the first mapping relationship table may include the following steps:

[0033] S1, construct multiple feature extraction sub-models and feature analysis sub-models with different data processing precisions;

[0034] For example, for the low-precision feature extraction sub-model, a simpler convolutional neural network (CNN), such as MobileNet, is used, which has fewer layers and parameters and is suitable for resource-constrained environments; for the medium-precision feature extraction sub-model, a medium-complexity CNN, such as ResNet-18, is used, which has a moderate number of layers and parameters to balance accuracy and computational cost; for the high-precision feature extraction sub-model, a complex CNN, such as ResNet-50, is used, which has a larger number of layers and parameters and is suitable for scenarios requiring high-precision detection.

[0035] For example, for the low-precision feature analysis sub-model, a simple fully connected layer or a lightweight attention mechanism is used, which has a fast processing speed but low accuracy; for the medium-precision feature analysis sub-model, a medium-complexity fully connected layer and an attention mechanism are used to balance speed and accuracy; for the high-precision feature analysis sub-model, a complex fully connected layer and a multi-head attention mechanism are used, which has a slower processing speed but high accuracy.

[0036] S2, obtaining multiple initial detection models based on multiple feature extraction sub-models and feature analysis sub-models, wherein each initial detection model includes a feature extraction sub-model and a feature analysis sub-model, and the data processing accuracy of the feature analysis sub-model is not lower than the data processing accuracy of the feature extraction sub-model;

[0037] S3, obtaining multiple images as training samples, and determining a sample label corresponding to each training sample;

[0038] To obtain training samples, multiple images from different scenes can be collected, including images in various environments such as indoors, outdoors, during the day, and at night. The image content covers different types of monitoring objects, such as pedestrians, vehicles, animals, etc. Then each image is manually annotated to determine the location and category of the target in the image. The generated sample labels include the bounding box coordinates and category labels of the target.

[0039] S4, iteratively training multiple initial detection models using multiple training samples;

[0040] Among them, the training samples can be preprocessed, including image scaling, normalization, data augmentation (such as rotation, flipping) and other operations to increase the generalization ability of the model. During the training process, the validation set is regularly used to evaluate the performance of the model to avoid overfitting. After the training is completed, the model is finally evaluated using an independent test set to ensure that the model also performs well on unseen data.

[0041] S5, the multiple models after training are used as target detection models.

[0042] In addition, after training, multiple models can be comprehensively evaluated, including indicators such as accuracy, recall rate, and F1 score, to ensure that the performance of each model in different scenarios meets expectations. According to the evaluation results, each model is matched with the corresponding task level to form a first mapping relationship table, and the trained model is deployed to the monitoring system, and the appropriate model is automatically selected for target detection according to the task level.

[0043] After iteratively training multiple initial detection models using multiple training samples, the models can be further optimized, including:

[0044] For each model after training, the model is pruned according to the preset pruning method, and the model parameters of the pruned model are fine-tuned to obtain the target detection model. Pruning can remove unimportant weights and connections in the model, which can significantly reduce the size of the model and make it easier to deploy the model on devices with limited resources.

[0045] For each model after training, the model is used as the teacher model, and the student model corresponding to the teacher model is trained according to the knowledge distillation method, and the student model is used as the target detection model. By transferring the knowledge of the large model (teacher model) to the small model (student model), the size and amount of calculation of the model can be significantly reduced while maintaining high accuracy.

[0046] During the execution of the video surveillance task, the system collects multiple frames of images taken by multiple cameras at each acquisition moment, and then processes these images using the selected first target detection model. The specific steps include: image preprocessing, feature extraction, and feature analysis, and finally obtains the target detection results.

[0047] Specifically, during the execution of the video surveillance task, the following process is executed in a loop until the video surveillance task is terminated: at each acquisition moment, multiple frames of images acquired by multiple cameras are acquired, and image features in the multiple frames of images are extracted using the first feature extraction submodel in the first target detection model, and the image features are analyzed using the first feature analysis submodel in the first target detection model to obtain the target detection result corresponding to the acquisition moment; the computing resource usage of the monitoring system during the target detection process is acquired, and the task level of the video surveillance task is adjusted according to the computing resource usage and the target detection result, and the second target detection model corresponding to the new task level is determined according to the first mapping relationship table, and the second target detection model is used to replace the first target detection model to perform target detection at the next acquisition moment.

[0048] As an optional implementation, the first feature extraction submodel in the first target detection model is used to extract image features in multiple frames of images, and the first feature analysis submodel in the first target detection model is used to analyze the image features. The following steps may be taken:

[0049] S1, performing image preprocessing on the acquired multiple frames of images, wherein the image preprocessing includes at least one of the following: image scaling and image grayscale conversion;

[0050] S2, using the first feature extraction sub-model to extract features from the multiple frames of images after image preprocessing to obtain first image features;

[0051] S3, when the data processing accuracy of the first feature extraction sub-model and the first feature analysis sub-model is the same, input the first image feature into the first feature analysis sub-model to obtain the target detection result output by the first feature analysis sub-model;

[0052] S4, when the data processing accuracy of the first feature extraction sub-model is different from that of the first feature analysis sub-model, the first image feature is converted into a second image feature that matches the data processing accuracy of the first feature analysis sub-model, and the second image feature is input into the first feature analysis sub-model to obtain the target detection result output by the first feature analysis sub-model.

[0053] As an optional implementation, since multiple cameras can capture the same scene from different angles, the images taken by each camera contain different perspective information. Through feature fusion, these multi-perspective information can be combined to generate a more comprehensive feature representation, thereby improving the accuracy of target detection. Specifically, the first feature extraction submodel is used to extract features from multiple frames of images after image preprocessing to obtain first image features. The following method can also be used: for each frame of image after image preprocessing, the first feature extraction submodel is used to extract the third image features in the image, and the preset feature weights corresponding to the camera corresponding to the image are determined; multiple third image features are feature fused according to the preset feature weights corresponding to each third image feature to obtain the first image feature.

[0054] During the execution of the video surveillance task, the system will monitor the usage of computing resources during the target detection process in real time, and decide whether the task level needs to be adjusted based on the target detection results.

[0055] Specifically, adjusting the task level of the video surveillance task based on the computing resource usage and the target detection result may include the following adjustment methods: when the target detection result indicates that the target has not been detected and has lasted for a preset time, lowering the task level of the video surveillance task; when the target detection result indicates that the target has been detected and the computing resource usage does not exceed a first preset threshold, increasing the task level of the video surveillance task; when the computing resource usage exceeds a first preset threshold, lowering the task level of the video surveillance task.

[0056] For example, if the system fails to detect any target for a long time (for example, 10 minutes in a row), it will downgrade the task level. This can save computing resources for other more important tasks.

[0057] For example, if the system detects a target and the current computing resource usage does not exceed a preset threshold (such as 80% CPU usage), the task level will be increased. This can improve detection accuracy and better monitor the target.

[0058] If the system's computing resource usage exceeds a preset threshold (such as 90% CPU usage), the task level will be downgraded even if the target is detected. This is done to avoid system overload and ensure overall stable operation.

[0059] For example, if no target is detected for a period of time, the task level can be appropriately lowered to save resources; on the contrary, if an important target is detected and there is still enough computing resources, the task level can be increased to obtain more accurate results. Once the task level changes, the system will reselect the most suitable target detection model based on the latest level and replace the old model with the new model to continue to perform subsequent monitoring tasks.

[0060] In an embodiment of the present application, before the start of a video surveillance task, an initial task level of the video surveillance task is obtained, and a first target detection model corresponding to the initial task level is determined according to a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing precisions, and each target detection model includes a feature extraction submodel and a feature analysis submodel, and the data processing precision of the feature analysis submodel is not lower than the data processing precision of the feature extraction submodel; during the execution of the video surveillance task, the following process is executed in a loop until the video surveillance task is terminated: at each acquisition moment, multiple frames of images acquired by a multi-eye camera are acquired, and image features in the multiple frames of images are extracted using a first feature extraction submodel in the first target detection model, and the image features are analyzed using a first feature analysis submodel in the first target detection model to obtain a target detection result corresponding to the acquisition moment; the computing resource usage of the monitoring system during the target detection process is obtained, and the task level of the video surveillance task is adjusted according to the computing resource usage and the target detection result, and a second target detection model corresponding to the new task level is determined according to the first mapping relationship table, and the second target detection model is used to replace the first target detection model to perform target detection at the next acquisition moment. In this way, the task level and target detection model can be dynamically adjusted according to the current computing resource usage and target detection results, and target detection models of different accuracy can be selected according to different task levels to ensure the best detection accuracy and improve resource utilization efficiency in different scenarios, thereby solving the technical problem of fixed model calculation accuracy in related video surveillance solutions, which easily leads to waste of computing resources.

[0061] Example 2

[0062] According to an embodiment of the present application, a video surveillance task processing device for implementing the video surveillance task processing method in embodiment 1 is also provided. Figure 2 As shown, the video monitoring task processing device at least includes: a model selection module 21, a data processing module 22, wherein:

[0063] The model selection module 21 is used to obtain the initial task level of the video surveillance task before the video surveillance task starts, and determine the first target detection model corresponding to the initial task level according to a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing precisions, each target detection model includes a feature extraction sub-model and a feature analysis sub-model, and the data processing precision of the feature analysis sub-model is not lower than the data processing precision of the feature extraction sub-model;

[0064] The data processing module 22 is used to loop the following process during the execution of the video surveillance task until the video surveillance task is terminated: at each acquisition moment, obtain multiple frames of images acquired by multiple cameras, and use the first feature extraction submodel in the first target detection model to extract image features in the multiple frames of images, and use the first feature analysis submodel in the first target detection model to analyze the image features to obtain the target detection result corresponding to the acquisition moment; obtain the computing resource usage of the monitoring system during the target detection process, and adjust the task level of the video surveillance task according to the computing resource usage and the target detection result, determine the second target detection model corresponding to the new task level according to the first mapping relationship table, and use the second target detection model to replace the first target detection model to perform target detection at the next acquisition moment.

[0065] The functions of each module of the video surveillance task processing device are described below in conjunction with a specific implementation process.

[0066] Before the video surveillance task begins, the model selection module first obtains the initial task level of the task.

[0067] It should be noted that this initial level can be set manually by the user, or automatically determined by the system according to the task type (such as target recognition, target location detection, target tracking, etc.). For example, if the monitoring task is used for security protection of important areas, a higher task level may be set by default to ensure detection accuracy.

[0068] As an optional implementation, when obtaining the initial task level of a video surveillance task, in addition to obtaining the initial task level set for the target object, the target task type of the video surveillance task can also be determined, and the initial task level corresponding to the target task type can be determined based on a preset second mapping relationship table, wherein the second mapping relationship table stores a mapping relationship between different task types and different task levels, and the task type includes at least one of the following: target identification, target position detection, and target tracking.

[0069] The model selection module determines the first target detection model corresponding to the initial task level according to the initial task level and a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing accuracies, each target detection model includes a feature extraction sub-model and a feature analysis sub-model, and the data processing accuracy of the feature analysis sub-model is not lower than the data processing accuracy of the feature extraction sub-model.

[0070] The training process of multiple target detection models in the first mapping relationship table includes:

[0071] S1, construct multiple feature extraction sub-models and feature analysis sub-models with different data processing precisions;

[0072] S2, obtaining multiple initial detection models based on multiple feature extraction sub-models and feature analysis sub-models, wherein each initial detection model includes a feature extraction sub-model and a feature analysis sub-model, and the data processing accuracy of the feature analysis sub-model is not lower than the data processing accuracy of the feature extraction sub-model;

[0073] S3, obtaining multiple images as training samples, and determining a sample label corresponding to each training sample;

[0074] S4, iteratively training multiple initial detection models using multiple training samples;

[0075] S5, multiple models after training are used as target detection models.

[0076] In addition, after training, multiple models can be comprehensively evaluated, including indicators such as accuracy, recall rate, and F1 score, to ensure that the performance of each model in different scenarios meets expectations. According to the evaluation results, each model is matched with the corresponding task level to form a first mapping relationship table, and the trained model is deployed to the monitoring system, and the appropriate model is automatically selected for target detection according to the task level.

[0077] After iteratively training multiple initial detection models using multiple training samples, the models can be further optimized, including:

[0078] For each model after training, the model is pruned according to the preset pruning method, and the model parameters of the pruned model are fine-tuned to obtain the target detection model. Pruning can remove unimportant weights and connections in the model, which can significantly reduce the size of the model and make it easier to deploy the model on devices with limited resources.

[0079] For each model after training, the model is used as the teacher model, and the student model corresponding to the teacher model is trained according to the knowledge distillation method, and the student model is used as the target detection model. By transferring the knowledge of the large model (teacher model) to the small model (student model), the size and amount of calculation of the model can be significantly reduced while maintaining high accuracy.

[0080] During the execution of the video surveillance task, the system collects multiple frames of images taken by multiple cameras at each acquisition moment, and then processes these images using the selected first target detection model. The specific steps include image preprocessing, feature extraction, and feature analysis, and finally obtains the target detection results.

[0081] Specifically, during the execution of the video surveillance task, the data processing module cyclically executes the following process until the video surveillance task is terminated: at each acquisition moment, multiple frames of images acquired by multiple cameras are acquired, and image features in the multiple frames of images are extracted using the first feature extraction submodel in the first target detection model, and the image features are analyzed using the first feature analysis submodel in the first target detection model to obtain the target detection result corresponding to the acquisition moment; the computing resource usage of the monitoring system during the target detection process is acquired, and the task level of the video surveillance task is adjusted according to the computing resource usage and the target detection result, and the second target detection model corresponding to the new task level is determined according to the first mapping relationship table, and the second target detection model is used to replace the first target detection model to perform target detection at the next acquisition moment.

[0082] As an optional implementation, the first feature extraction submodel in the first target detection model is used to extract image features in multiple frames of images, and the first feature analysis submodel in the first target detection model is used to analyze the image features. The following steps may be adopted:

[0083] S1, performing image preprocessing on the acquired multiple frames of images, wherein the image preprocessing includes at least one of the following: image scaling and image grayscale conversion;

[0084] S2, using the first feature extraction sub-model to extract features from the multiple frames of images after image preprocessing to obtain first image features;

[0085] S3, when the data processing accuracy of the first feature extraction sub-model and the first feature analysis sub-model is the same, input the first image feature into the first feature analysis sub-model to obtain the target detection result output by the first feature analysis sub-model;

[0086] S4, when the data processing accuracy of the first feature extraction sub-model is different from that of the first feature analysis sub-model, the first image feature is converted into a second image feature that matches the data processing accuracy of the first feature analysis sub-model, and the second image feature is input into the first feature analysis sub-model to obtain the target detection result output by the first feature analysis sub-model.

[0087] Since multiple cameras can capture the same scene from different angles, the images taken by each camera contain different perspective information. Through feature fusion, these multi-perspective information can be combined to generate a more comprehensive feature representation, thereby improving the accuracy of target detection. Specifically, the first feature extraction submodel is used to extract features from multiple frames of images after image preprocessing to obtain first image features. The following method can also be used: for each frame of image after image preprocessing, the first feature extraction submodel is used to extract the third image feature in the image, and the preset feature weight corresponding to the camera corresponding to the image is determined; the multiple third image features are feature fused according to the preset feature weights corresponding to each third image feature to obtain the first image feature.

[0088] During the execution of the video surveillance task, the system will monitor the usage of computing resources during the target detection process in real time, and decide whether the task level needs to be adjusted based on the target detection results.

[0089] Specifically, adjusting the task level of the video surveillance task based on the computing resource usage and the target detection result may include the following adjustment methods: when the target detection result indicates that the target has not been detected and has lasted for a preset time, lowering the task level of the video surveillance task; when the target detection result indicates that the target has been detected and the computing resource usage does not exceed a first preset threshold, increasing the task level of the video surveillance task; when the computing resource usage exceeds a first preset threshold, lowering the task level of the video surveillance task.

[0090] It should be noted that each module in the video surveillance task processing device in the embodiment of the present application corresponds one-to-one to each implementation step of the video surveillance task processing method in Example 1. Since a detailed description has been given in Example 1, some details not reflected in this embodiment can be referred to Example 1 and will not be repeated here.

[0091] Example 3

[0092] According to an embodiment of the present application, a computer program product is also provided. The computer program product includes a computer program, wherein when the computer program is executed by a processor, the video surveillance task processing method in Example 1 is implemented.

[0093] According to an embodiment of the present application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the video surveillance task processing method in Example 1 by running the computer program.

[0094] According to an embodiment of the present application, a processor is also provided, which is used to run a computer program, wherein the video surveillance task processing method in Example 1 is executed when the computer program is running.

[0095] According to an embodiment of the present application, an electronic device is also provided, which includes: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the video surveillance task processing method in Example 1 through the computer program.

[0096] Specifically, the computer program executes the following steps when it is running: before the video surveillance task starts, the initial task level of the video surveillance task is obtained, and a first target detection model corresponding to the initial task level is determined according to a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing precisions, each target detection model includes a feature extraction submodel and a feature analysis submodel, and the data processing precision of the feature analysis submodel is not lower than the data processing precision of the feature extraction submodel; during the execution of the video surveillance task, the following process is executed cyclically until the video surveillance task is terminated: at each acquisition moment, multiple frames of images acquired by multiple cameras are obtained, and the image features in the multiple frames of images are extracted using the first feature extraction submodel in the first target detection model, and the image features are analyzed using the first feature analysis submodel in the first target detection model to obtain the target detection result corresponding to the acquisition moment; the computing resource usage of the monitoring system during the target detection process is obtained, and the task level of the video surveillance task is adjusted according to the computing resource usage and the target detection result, and the second target detection model is determined to correspond to the new task level according to the first mapping relationship table, and the second target detection model is used to replace the first target detection model to perform target detection at the next acquisition moment.

[0097] As an optional implementation, the electronic device may be in the form of a mobile terminal, a computer terminal or a similar computing device. Figure 3 FIG. 1 shows a hardware structure block diagram of an electronic device for implementing a video monitoring task processing method. Figure 3 As shown, the electronic device 30 may include one or more (302a, 302b, ..., 302n are used to illustrate) processors 302 (the processor 302 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 304 for storing data, and a transmission device 306 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 3 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 3More or fewer components as shown, or with Figure 3 Different configurations are shown.

[0098] It should be noted that the one or more processors 302 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the electronic device 30. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0099] The memory 304 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the video surveillance task processing method in the embodiment of the present application. The processor 302 executes various functional applications and data processing by running the software programs and modules stored in the memory 304, that is, realizing the vulnerability detection method of the above-mentioned application program. The memory 304 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 304 may further include a memory remotely arranged relative to the processor 302, and these remote memories may be connected to the electronic device 30 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0100] The transmission device 306 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the electronic device 30. In one example, the transmission device 306 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 306 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0101] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the electronic device 30 .

[0102] The serial numbers of the above embodiments are only for description and do not represent the advantages or disadvantages of the embodiments.

[0103] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0105] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0106] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0107] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0108] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A video monitoring task processing method, applied to a monitoring system, characterized in that: include: Before the video surveillance task starts, the initial task level of the video surveillance task is obtained, and a first target detection model corresponding to the initial task level is determined according to a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing precisions, each of the target detection models includes a feature extraction submodel and a feature analysis submodel, and the data processing precision of the feature analysis submodel is not lower than the data processing precision of the feature extraction submodel; During the execution of the video monitoring task, the following process is executed cyclically until the video monitoring task is terminated: At each acquisition moment, multiple frames of images acquired by multiple cameras are acquired, and image features in the multiple frames of images are extracted using the first feature extraction submodel in the first target detection model, and the image features are analyzed using the first feature analysis submodel in the first target detection model to obtain the target detection result corresponding to the acquisition moment; the computing resource usage of the monitoring system during the target detection process is acquired, and the task level of the video monitoring task is adjusted according to the computing resource usage and the target detection result, a second target detection model corresponding to the new task level is determined according to the first mapping relationship table, and the second target detection model is used to replace the first target detection model to perform target detection at the next acquisition moment.

2. The method according to claim 1, characterized in that Obtaining the initial task level of the video surveillance task includes: Obtaining the initial task level set for the target object; or, Determine the target task type of the video surveillance task, and determine the initial task level corresponding to the target task type based on a preset second mapping relationship table, wherein the second mapping relationship table stores a mapping relationship between different task types and different task levels, and the task type includes at least one of the following: target recognition, target position detection, and target tracking.

3. The method according to claim 1, characterized in that: The training process of the plurality of target detection models in the first mapping relationship table includes: Construct multiple feature extraction sub-models and feature analysis sub-models with different data processing precisions; A plurality of initial detection models are obtained by splicing the plurality of feature extraction sub-models and the feature analysis sub-models, wherein each of the initial detection models includes a feature extraction sub-model and a feature analysis sub-model, and the data processing accuracy of the feature analysis sub-model is not lower than the data processing accuracy of the feature extraction sub-model; Obtain multiple images as training samples, and determine the sample label corresponding to each training sample; Iteratively training the multiple initial detection models respectively using the multiple training samples; The multiple models after training are all used as the target detection models.

4. The method according to claim 3, characterized in that After iteratively training the multiple initial detection models respectively using the multiple training samples, the method further includes: For each model after training, pruning the model is performed according to a preset pruning method, and model parameters of the pruned model are fine-tuned to obtain the target detection model; or, For each model after training, the model is used as a teacher model, and a student model corresponding to the teacher model is obtained by training according to a knowledge distillation method, and the student model is used as the target detection model.

5. The method according to claim 1, characterized in that Extracting image features from the multiple frames of images using a first feature extraction submodel in the first target detection model, and analyzing the image features using a first feature analysis submodel in the first target detection model, including: Performing image preprocessing on the acquired multiple frames of images, wherein the image preprocessing includes at least one of the following: image scaling and image grayscale conversion; Using the first feature extraction sub-model to perform feature extraction on the multiple frames of images after image preprocessing to obtain first image features; In a case where the data processing accuracy of the first feature extraction sub-model is the same as that of the first feature analysis sub-model, inputting the first image feature into the first feature analysis sub-model to obtain the target detection result output by the first feature analysis sub-model; When the data processing accuracy of the first feature extraction sub-model is different from that of the first feature analysis sub-model, the first image feature is converted into a second image feature that matches the data processing accuracy of the first feature analysis sub-model, and the second image feature is input into the first feature analysis sub-model to obtain the target detection result output by the first feature analysis sub-model.

6. The method according to claim 5, characterized in that The first feature extraction sub-model is used to extract features from the multiple frames of images after image preprocessing to obtain first image features, including: For each frame of the image after image preprocessing, extract a third image feature in the image using the first feature extraction sub-model, and determine a preset feature weight corresponding to the camera corresponding to the image; The first image feature is obtained by performing feature fusion on a plurality of the third image features according to the preset feature weights corresponding to the respective third image features.

7. The method according to claim 1, characterized in that Adjusting the task level of the video surveillance task according to the computing resource usage and the target detection result includes: When the target detection result indicates that no target has been detected and the detection has lasted for a preset period of time, lowering the task level of the video monitoring task; and / or, When the target detection result indicates that a target is detected and the computing resource usage does not exceed a first preset threshold, increasing the task level of the video monitoring task; and / or, When the computing resource usage exceeds the first preset threshold, the task level of the video monitoring task is reduced.

8. A video surveillance task processing device, characterized in that: include: A model selection module is used to obtain the initial task level of the video surveillance task before the video surveillance task begins, and determine a first target detection model corresponding to the initial task level according to a preset first mapping relationship table, wherein the first mapping relationship table stores a mapping relationship between target detection models with different task levels and different data processing precisions, each of the target detection models includes a feature extraction submodel and a feature analysis submodel, and the data processing precision of the feature analysis submodel is not lower than the data processing precision of the feature extraction submodel; A data processing module is used to loop the following process during the execution of the video surveillance task until the video surveillance task is terminated: at each acquisition moment, obtain multiple frames of images acquired by multiple cameras, and use the first feature extraction submodel in the first target detection model to extract image features in the multiple frames of images, and use the first feature analysis submodel in the first target detection model to analyze the image features to obtain the target detection result corresponding to the acquisition moment; obtain the computing resource usage of the monitoring system during the target detection process, and adjust the task level of the video surveillance task according to the computing resource usage and the target detection result, determine the second target detection model corresponding to the new task level according to the first mapping relationship table, and use the second target detection model to replace the first target detection model to perform target detection at the next acquisition moment.

9. A computer program product, characterized in that include: A computer program, wherein when the computer program is executed by a processor, the video surveillance task processing method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the video surveillance task processing method according to any one of claims 1 to 7 through the computer program.

Images