Video target detection method and system based on visual AI model

Abstract

This application proposes a video target detection method and system based on a visual AI model, belonging to the field of computer vision technology. The method includes: establishing background video frames of the monitored area; when a moving target is determined to exist in the monitored area, acquiring the current area video frame containing the moving target, and determining the target detection complexity of the moving target based on the current area video frame and the background video frame; determining the number of local model calls and the number of online model calls based on the target detection complexity, and determining multiple local and multiple online model call recognition models based on the local and online model call numbers; performing target recognition, obtaining the local and online type probability distributions, and generating the moving target recognition result. The method adaptively determines the number of local and online model calls based on the target detection complexity of the moving target, achieving a match between recognition model resources and actual detection requirements.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a video target detection method and system based on a visual AI model. Background Technology

[0002] With the continuous development of intelligent security technology, video target detection methods are widely used in intrusion detection and alarm scenarios in monitored areas. Existing video target detection methods typically employ a fixed model invocation strategy to identify moving targets in surveillance videos, that is, pre-setting the number of calls to local and online recognition models, and uniformly executing the same model invocation scheme for all detection scenarios.

[0003] However, the detection complexity of moving targets varies significantly across different monitoring scenarios. When the detection complexity is low, a fixed model invocation scheme may consume excessive recognition model resources, leading to unnecessary resource consumption. Conversely, when the detection complexity is high, a fixed model invocation scheme may not utilize sufficient recognition model resources, resulting in decreased recognition accuracy. Existing methods cannot dynamically allocate the number of calls to local and online recognition models based on the detection complexity of moving targets, leading to a mismatch between recognition model resource allocation and actual detection needs. This makes it difficult to achieve accurate and reliable target recognition across detection scenarios with varying levels of complexity. Summary of the Invention

[0004] This invention addresses the technical problem in existing video target detection methods that cannot dynamically allocate the number of local and online recognition models based on the detection complexity of moving targets, resulting in a mismatch between recognition model resource allocation and actual detection needs. It provides a video target detection method and system based on a visual AI model to solve this problem.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0006] In a first aspect, the present invention provides a video target detection method based on a visual AI model, comprising: in the absence of a moving target, acquiring video frames of a monitored area using a camera to establish a background video frame of the monitored area; when it is determined that a moving target exists in the monitored area, acquiring a current area video frame containing the moving target, and determining the target detection complexity of the moving target based on the current area video frame and the background video frame; determining the number of local model calls and the number of online model calls based on the target detection complexity, and determining multiple local call recognition models and multiple online call recognition models based on the number of local model calls and the number of online model calls; performing target recognition on the current area video frame based on the multiple local call recognition models and the multiple online call recognition models, obtaining the local type probability distribution and the online type probability distribution, and generating a moving target recognition result.

[0007] Secondly, this invention provides a video target detection system based on a visual AI model, comprising: a background establishment module, used to acquire video frames of a monitored area through a camera in the absence of a moving target, and establish background video frames of the monitored area; a complexity determination module, used to acquire a current area video frame containing the moving target when it is determined that a moving target exists in the monitored area, and determine the target detection complexity of the moving target based on the current area video frame and the background video frame; a dynamic scheduling module, used to determine the number of local model calls and the number of online model calls based on the target detection complexity, and determine multiple local call recognition models and multiple online call recognition models based on the number of local model calls and the number of online model calls; and a target recognition module, used to perform target recognition on the current area video frame based on the multiple local call recognition models and the multiple online call recognition models, acquire local type probability distribution and online type probability distribution, and generate a moving target recognition result.

[0008] The beneficial effects of this invention are:

[0009] In the absence of moving targets, video frames are captured from the monitored area using cameras to establish background video frames for the monitored area, providing a benchmark reference for subsequent moving target detection. When a moving target is detected in the monitored area, video frames of the current area containing the moving target are acquired. The target detection complexity of the moving target is determined based on the current area video frames and background video frames, thus quantifying the difficulty of the current detection scenario into a unified index, providing a basis for the dynamic scheduling of subsequent recognition models. The number of local model calls and online model calls is determined based on the target detection complexity, and multiple local and online recognition models are selected based on these numbers. This allows for the dynamic allocation of local and online recognition model resources according to actual detection needs, ensuring that the number of model calls matches the target detection complexity. Target recognition is performed on the current area video frames based on multiple local and online recognition models, obtaining the local and online type probability distributions to generate moving target recognition results. Finally, the judgment results of the local and online recognition models are combined to output the final target recognition conclusion.

[0010] The above technical solution adaptively determines the number of local and online model calls based on the target detection complexity of moving targets, and determines the corresponding local and online recognition models to identify moving targets based on the number of calls. This achieves dynamic matching between recognition model resources and actual detection needs, solving the problem in existing video target detection methods that cannot dynamically allocate the number of local and online recognition models based on the detection complexity of moving targets, resulting in a mismatch between recognition model resource allocation and actual detection needs. It achieves the technical effect of adaptively determining the number of local and online model calls based on the target detection complexity of moving targets and matching recognition model resources with actual detection needs. Attached Figure Description

[0011] Figure 1 A flowchart illustrating the video target detection method based on a visual AI model provided by this invention;

[0012] Figure 2 This is a schematic diagram of the structure of the video target detection system based on a visual AI model provided by the present invention.

[0013] In the attached diagram, the components represented by each number are as follows:

[0014] Background establishment module 11, complexity determination module 12, dynamic scheduling module 13, target recognition module 14. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0017] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0018] Example 1, as Figure 1 As shown, this embodiment of the invention provides a video target detection method based on a visual AI model, including:

[0019] S1. In the absence of a moving target, video frames are captured from the monitored area using a camera to establish background video frames for the monitored area.

[0020] Specifically, during the initialization phase, the camera continuously captures video frames of the monitored area. In this phase, the monitored area is free of moving targets, and the video frames captured by the camera only contain static scene information of the monitored area, such as the pixel distribution of background elements like the ground, walls, and fixed structures. The multiple video frames captured in the free-target state are processed to extract stable color and brightness values ​​for each pixel in the monitored area, and the processing results are stored as background video frames. These background video frames completely record the pixel distribution characteristics of the monitored area in the free-target state, forming a baseline reference template for subsequent moving target detection. When a new video frame of the current area is captured during monitoring, pixel-level differences between the current area video frame and the background video frame can be compared to identify areas of pixel change introduced by moving targets, thus providing a data foundation for subsequent target detection complexity calculations and model invocation.

[0021] S2. When it is determined that there is a moving target in the monitored area, the current area video frame containing the moving target is obtained, and the target detection complexity of the moving target is determined based on the current area video frame and the background video frame.

[0022] Specifically, during monitoring operations, video frames are continuously acquired from the monitored area. The differences between the real-time acquired video frames and background video frames are calculated pixel-by-pixel. For each pixel, the color value difference between the current video frame and the background video frame at that pixel is calculated. When the difference exceeds a preset pixel difference threshold, the pixel is marked as a difference pixel. The number of difference pixels in the current video frame is counted. When the number of difference pixels exceeds a preset threshold, a significant pixel difference is determined between the current video frame and the background video frame, indicating the presence of a moving target in the monitored area. A current area video frame containing the moving target is then acquired. This current area video frame completely records the pixel distribution state of the monitored area when the moving target appears, including all pixel information of both the moving target and the background area.

[0023] After acquiring video frames for the current region, the target detection complexity of the moving target is determined based on the current region video frames and background video frames. Target detection complexity quantifies the difficulty of detecting the moving target, comprehensively considering key factors affecting detection difficulty, such as the pixel size and color distribution characteristics of the moving target. Higher target detection complexity indicates greater difficulty in identifying the moving target, requiring more online recognition models to participate in detection to ensure accuracy; lower target detection complexity indicates obvious moving target features and lower recognition difficulty, allowing local recognition models to meet detection requirements, thus effectively reducing system resource consumption. By introducing target detection complexity, adaptive dynamic allocation of the ratio of local to online model calls is achieved, providing a quantitative basis for determining the number of subsequent model calls.

[0024] Furthermore, determining the target detection complexity of the moving target based on the current region video frame and the background video frame includes:

[0025] S21. Compare the differences between the current region video frame and the background video frame, and extract the target region of the moving target from the current region video frame;

[0026] S22. Calculate the pixel size of the target region to obtain the target pixel size, and normalize the target pixel size based on a preset size range to obtain the normalized target size;

[0027] S23. Perform histogram statistics on the pixel colors of the target area to obtain a color histogram, calculate the information entropy of the color histogram to obtain the color distribution entropy, and normalize the color distribution entropy based on a preset entropy value range to obtain the normalized color distribution entropy.

[0028] S24. Based on preset size weights and preset color weights, the normalized target size and the normalized color distribution entropy are weighted and summed to obtain the target detection complexity of the moving target.

[0029] In a preferred embodiment, when determining the target detection complexity of a moving target based on the current region video frame and the background video frame, the following steps are taken: First, the differences between the current region video frame and the background video frame are compared, and the target region of the moving target is extracted from the current region video frame. Specifically, the color value difference between the current region video frame and the background video frame at corresponding pixels is calculated pixel by pixel. Pixels with a difference value exceeding a preset pixel difference threshold are marked as difference pixels. Connectivity analysis is performed on the regions formed by all difference pixels, and connected components with an area exceeding a preset area threshold are determined as the target region where the moving target is located. This filters out interference from isolated noise pixels caused by factors such as lighting changes and camera noise, ensuring that the extracted target region accurately corresponds to the actual contour range of the moving target.

[0030] Then, the pixel size of the target region is calculated to obtain the target pixel size. Specifically, the total number of pixels contained in the target region is counted, and this total number of pixels is taken as the target pixel size. The target pixel size directly reflects the pixel scale occupied by the moving target in the video frame. The larger the pixel size, the higher the proportion of the moving target in the frame, the richer the visual information contained in the target, and the higher the computational requirements of the recognition model. The target pixel size is normalized based on a preset size range. Specifically, the difference between the target pixel size and the minimum value of the preset size range is divided by the difference between the maximum and minimum values ​​of the preset size range, linearly mapping the target pixel size to a numerical range of 0 to 1, obtaining the normalized target size. This eliminates the influence of differences in the dimensions of different pixel sizes on the subsequent calculation results of the target detection complexity. The closer the normalized target size is to 1, the larger the pixel size of the moving target. At the same time, histogram statistics are performed on the pixel colors of the target region to obtain a color histogram, and the information entropy of the color histogram is calculated to obtain the color distribution entropy. Specifically, the color values ​​of all pixels within the target area are binned and statistically analyzed according to preset color intervals. The pixel frequency within each color interval is counted, and a color histogram is constructed with the color interval as the horizontal axis and the pixel frequency as the vertical axis. The color histogram fully describes the distribution pattern of pixel colors within the target area. The richer the color variety and the more uniform the frequency distribution across color intervals, the more complex the visual texture features of the moving target. Based on this, the information entropy of the color histogram is calculated using the information entropy formula to obtain the color distribution entropy. Specifically, let the color histogram be divided into N color intervals, the pixel frequency in the i-th color interval be f(i), and the total number of pixels in the target area be F. Then, the probability value p(i) corresponding to the i-th color interval is the ratio of the pixel frequency f(i) in that color interval to the total number of pixels F, i.e., p(i) = f(i) / F. Based on this, the product of each probability value p(i) and its base-2 logarithmic value log2(p(i)) is calculated for all color intervals. The summation of the corresponding product values ​​for all color intervals is then taken as the negative value to obtain the color distribution entropy H, i.e., H = -Σp(i) × log2(p(i)). The summation range includes all color intervals with probability values ​​p(i) greater than 0. Color intervals with probability values ​​of 0 are not included in the calculation to avoid meaningless logarithmic operations. The higher the color distribution entropy, the more uniform and dispersed the color distribution of the target area, the greater the difficulty in visually identifying the moving target, and the higher the requirement for the feature extraction capability of the recognition model. The lower the color distribution entropy, the more concentrated the color of the target area is in a few color intervals, the simpler the color features of the moving target, and the relatively easier the recognition. The color distribution entropy is normalized based on a preset entropy value range, linearly mapping the color distribution entropy to a numerical range of 0 to 1 to obtain the normalized color distribution entropy. The closer the normalized color distribution entropy is to 1, the more complex the color distribution of the moving target.

[0031] Next, based on preset size weights and preset color weights, the normalized target size and normalized color distribution entropy are weighted and summed to obtain the target detection complexity of the moving target. Specifically, the product of the normalized target size and the preset size weight and the product of the normalized color distribution entropy and the preset color weight are summed to obtain the target detection complexity. Both the preset size weight and the preset color weight are values ​​greater than 0 and less than 1, and their sum is 1. This can be flexibly configured according to the differences in the influence of size and color factors on the detection difficulty in actual application scenarios. The target detection complexity ranges from 0 to 1. A higher target detection complexity indicates a greater overall difficulty in detecting moving targets, requiring more online recognition models with strong feature extraction capabilities to participate in the detection; a lower target detection complexity indicates a lower overall difficulty in detecting moving targets, where local recognition models can meet the recognition accuracy requirements, thus effectively reducing system resource consumption while ensuring recognition performance.

[0032] S3. Determine the number of local model calls and the number of online model calls based on the target detection complexity, and determine multiple local call recognition models and multiple online call recognition models based on the number of local model calls and the number of online model calls.

[0033] Specifically, the total number of local model calls and online model calls is fixed at the total number of standard models. The ratio between local and online model calls is dynamically adjusted based on the object detection complexity. When the object detection complexity is low, local recognition models are prioritized for object detection to reduce the consumption of cloud server resources. When the object detection complexity is high, the number of online recognition model calls is increased accordingly. This allows online recognition models with stronger feature extraction capabilities to compensate for the insufficient recognition accuracy of local recognition models in complex scenarios, thereby achieving a dynamic balance between system resource consumption and recognition accuracy.

[0034] After determining the number of local model calls and the number of online model calls, a corresponding number of recognition models are extracted from the pre-built local model pool and online model pool, respectively, resulting in multiple locally called recognition models and multiple online called recognition models, which are then used for target recognition on the current region's video frames. By dynamically extracting recognition models from the model pools for detection, rather than fixedly calling the same model, model diversity is effectively introduced, reducing the risk of systematic errors in recognition results due to training bias of a single model, and improving the robustness of the overall recognition results.

[0035] Furthermore, the number of local model calls and the number of online model calls are determined based on the target detection complexity, including:

[0036] S31. Obtain the preset basic detection complexity and the total number of standard models;

[0037] S32. When the target detection complexity is less than or equal to the preset basic detection complexity, the total number of standard models is determined as the number of local model calls, and the number of online model calls is determined as 0.

[0038] S33. When the target detection complexity is greater than the preset basic detection complexity, calculate the ratio of the preset basic detection complexity to the target detection complexity to obtain the local call coefficient. Take the integer part of the product of the total number of standard models and the local call coefficient to obtain the local model call count. Determine the difference between the total number of standard models and the local model call count as the online model call count.

[0039] In a preferred embodiment, firstly, a preset basic detection complexity and the total number of standard models are obtained. The preset basic detection complexity is a complexity threshold pre-configured based on the actual detection capability of the local recognition model. It represents the maximum target detection complexity that the local recognition model can independently meet the detection accuracy requirements. The value of the preset basic detection complexity can be flexibly configured according to the hardware performance of the local device and the actual recognition capability of the local recognition model. The total number of standard models is the pre-configured total number of recognition models participating in this target detection. The value of the total number of standard models can be configured according to the comprehensive requirements of detection accuracy and system resource consumption in the actual application scenario. The sum of the number of local model calls and the number of online model calls is always equal to the total number of standard models, ensuring that the total number of models participating in recognition remains stable in each target detection process.

[0040] When the target detection complexity is less than or equal to the preset basic detection complexity, it indicates that the detection difficulty reflected by the pixel size and color distribution features of the current moving target is within the capability range of the local recognition model. The local recognition model can meet the recognition accuracy requirements of the current detection scenario with its own feature extraction and classification capabilities, without the need to call additional cloud-based online recognition models to participate in the detection. At this time, the total number of standard models is set to the local model call count, meaning that all models participating in the recognition process are drawn from the local model pool, and the online model call count is set to 0, without initiating any model call requests to the cloud server. This effectively avoids unnecessary cloud server resource consumption, reduces system network communication overhead, and improves the overall system response efficiency while meeting the recognition accuracy requirements.

[0041] When the target detection complexity exceeds the preset basic detection complexity, it indicates that the detection difficulty of the current moving target exceeds the independent processing capability of the local recognition model. Relying solely on the local recognition model is insufficient to guarantee recognition accuracy, necessitating the introduction of an online recognition model with stronger feature extraction capabilities to collaborate in the detection. In this case, the ratio of the preset basic detection complexity to the target detection complexity is calculated to obtain the local call coefficient. The local call coefficient ranges from 0 to 1, and its physical meaning represents the proportion of detection work that the local recognition model should undertake under the current detection difficulty. The higher the target detection complexity, the smaller the local call coefficient, indicating a lower proportion of the local recognition model's work in this detection, and correspondingly a higher proportion of the online recognition model's work. Conversely, the closer the target detection complexity is to the preset basic detection complexity, the closer the local call coefficient is to 1, indicating that the local recognition model undertakes the vast majority of the recognition task in this detection, requiring only a small number of online recognition models to supplement it. The product of the total number of standard models and the local call coefficient is rounded down to obtain the number of local model calls. The rounding operation ensures that the number of local model calls is an integer so that the corresponding number of local recognition models can be extracted from the local model pool. The difference between the total number of standard models and the number of local model calls is determined as the number of online model calls. This enables adaptive dynamic allocation of the ratio between the number of local model calls and the number of online model calls based on the complexity of target detection, ensuring recognition accuracy while taking into account the rational use of system resources.

[0042] Furthermore, based on the number of local model calls and the number of online model calls, multiple local call identification models and multiple online call identification models are determined, including:

[0043] S34. Connect the pre-built local model pool and the online model pool, wherein the local model pool contains multiple local recognition models and the online model pool contains multiple online recognition models;

[0044] S35. Randomly select a number of local recognition models corresponding to the number of local model calls from the local model pool to obtain multiple local call recognition models;

[0045] S36. When the number of online model calls is greater than 0, randomly select the number of online recognition models corresponding to the number of online model calls from the online model pool to obtain multiple online call recognition models.

[0046] In a preferred embodiment, firstly, a pre-built local model pool and an online model pool are connected. The local model pool contains multiple local recognition models, and the online model pool contains multiple online recognition models. The local model pool is deployed on a local device, while the online model pool is deployed on a cloud server. The local and online model pools operate independently, and are connected via a network communication interface. The multiple local recognition models in the local model pool differ in model architecture and training strategies, and the multiple online recognition models in the online model pool also possess model diversity. This provides a sufficient foundation of model diversity for subsequent random sampling, ensuring that the combinations of recognition models obtained from multiple samplings are differentiated.

[0047] A number of local recognition models corresponding to the number of local model calls are randomly selected from the local model pool to obtain multiple local call recognition models. Specifically, a random sampling strategy is used to select a number of local recognition models corresponding to the number of local model calls from the local model pool, ensuring that each local recognition model has the same probability of being selected. This avoids introducing systematic recognition bias due to fixed calls to specific local recognition models. The multiple local recognition models obtained constitute the set of local call recognition models participating in the recognition process in this target detection.

[0048] When the number of online model calls is greater than 0, a number of online recognition models corresponding to the number of online model calls are randomly selected from the online model pool, resulting in multiple online call recognition models. Specifically, when the target detection complexity exceeds the preset basic detection complexity and the number of online model calls is determined to be greater than 0, a number of online recognition models corresponding to the number of online model calls are selected from the online model pool based on a random sampling strategy. The multiple selected online recognition models constitute the set of online call recognition models participating in the target detection process. When the number of online model calls is equal to 0, no selection request is initiated to the online model pool, and the target detection process is entirely undertaken by the local call recognition model, without the need to establish a model call communication connection with the cloud server.

[0049] S4. Based on the multiple local call recognition models and the multiple online call recognition models, target recognition is performed on the current area video frame to obtain the local type probability distribution and the online type probability distribution, and a moving target recognition result is generated.

[0050] Specifically, the current region's video frames are input into multiple local and online target recognition models for parallel target recognition. Each model, based on its feature extraction and classification capabilities, outputs the corresponding target type recognition result for moving targets in the current region's video frames. The recognition results from multiple local models are statistically summarized to obtain the local type probability distribution; similarly, the recognition results from multiple online models are statistically summarized to obtain the online type probability distribution. The local and online type probability distributions describe the probability of a moving target belonging to each target type from the perspectives of the local and online recognition models, respectively. Together, they constitute the input data for subsequent fusion processing.

[0051] After obtaining the local and online type probability distributions, the two are fused to generate the moving target recognition result. By introducing a fusion mechanism of local and online type probability distributions, instead of directly using the recognition result of a single model, the recognition advantages of the local and online recognition models are effectively combined. This reduces the risk of misjudgment in the final recognition result due to the recognition bias of a single model, and improves the accuracy and reliability of the moving target recognition result.

[0052] By adaptively determining the number of local model calls and online model calls based on the target detection complexity of moving targets, and generating moving target recognition results based on the fusion processing of local type probability distribution and online type probability distribution, dynamic matching of recognition model resources with actual detection needs is achieved, thereby improving the accuracy of moving target recognition and the efficiency of system resource utilization.

[0053] Furthermore, the construction steps of the online model pool include:

[0054] S371. Collect historical target detection records of multiple sample monitoring areas, and construct a sample video frame set and a sample target type label set based on the historical target detection records;

[0055] S372. Construct multiple online recognition model architectures, using the sample video frame set as input and the sample target type label set as target, train the multiple online recognition model architectures until convergence, and obtain multiple online recognition models.

[0056] S373. Integrate the multiple online recognition models into an online model pool, wherein the online model pool is deployed on a cloud server.

[0057] In a preferred embodiment, firstly, historical target detection records from multiple sample monitoring areas are collected, and a sample video frame set and a sample target type label set are constructed based on these historical target detection records. Specifically, historical target detection records are collected from sample monitoring areas under different scenarios and environmental conditions. These historical target detection records contain video frames collected during historical monitoring and their corresponding target type annotation information. Historical video frames are extracted from the historical target detection records and aggregated to construct a sample video frame set; target type annotation information corresponding to each historical video frame is extracted and aggregated to construct a sample target type label set. The samples in the sample video frame set and the sample target type label set correspond one-to-one, together forming the supervised learning dataset for subsequent online recognition model training. By collecting historical target detection records from multiple sample monitoring areas, it is ensured that the training dataset covers target type samples under various scenario conditions, thereby improving the generalization ability of the online recognition model.

[0058] Then, multiple online recognition model architectures are constructed. Using a set of sample video frames as input and a set of sample target type labels as targets, these architectures are trained until convergence, resulting in multiple online recognition models. Specifically, multiple online recognition model architectures with differences in network structure, network depth, or feature extraction strategies are constructed. Sample video frames from the sample video frame set are used as model input, and the corresponding target type labels from the sample target type label set are used as supervision targets. Each online recognition model architecture is iteratively trained, continuously optimizing its network parameters until the loss function of each architecture converges to a stable state, resulting in multiple online recognition models with target recognition capabilities. The differences in model architecture among these online recognition models ensure that each model exhibits a certain degree of result diversity when recognizing the same target, providing a valid basis for subsequent probabilistic statistics based on the multi-model recognition results. For example, when constructing an online recognition model architecture, multiple online recognition model architectures can be built using various types of deep learning network architectures. These architectures differ in the number of network layers, feature extraction methods, and classification strategies, ensuring that each online recognition model has a different focus when recognizing the same moving target. For instance, a ResNet-50 residual network architecture can be used to construct the first online recognition model. During training, the input sample video frames are uniformly scaled to a resolution of 224×224 pixels. Data augmentation strategies such as random cropping, horizontal flipping, and color dithering are used to expand the sample video frame set. A stochastic gradient descent optimizer with a batch size of 32, an initial learning rate of 0.01, and a momentum coefficient of 0.9 is used to iteratively update the network parameters. A cosine annealing strategy is used to dynamically adjust the learning rate. The training epochs are set to 100. The model is considered converged when the classification loss function value on the validation set does not decrease for 10 consecutive epochs, resulting in the first online recognition model. Simultaneously, a visual Transformer architecture can be used to construct the second online recognition model. During training, the input sample video frames were uniformly scaled to a resolution of 224×224 pixels, and the video frames were divided into a sequence of image blocks of 16×16 pixels. The AdamW optimizer was used to iteratively update the network parameters. The initial learning rate was set to 0.001, the weight decay coefficient was set to 0.05, the batch size was set to 16, the training epochs were set to 200, and the Dropout mechanism was introduced to prevent the model from overfitting. When the classification loss function value on the validation set tended to stabilize, the model was considered to have converged, and the second online recognition model was obtained.In practical applications, depending on the complexity of the detection scenario and the requirements for recognition accuracy, more different types of deep learning network architectures can be used to build a corresponding number of online recognition models according to the above training method, and these models can be uniformly integrated into the online model pool to ensure that the online model pool has a sufficient number and diversity of models to meet the extraction requirements of the dynamically changing number of online model calls under different target detection complexity scenarios.

[0059] Subsequently, multiple online recognition models are integrated into an online model pool, which is deployed on a cloud server. Specifically, the trained online recognition models are uniformly integrated into the online model pool for unified management and scheduling. The online model pool is deployed on a cloud server, relying on the ample storage and computing resources of the cloud server to support concurrent calls and efficient inference of multiple online recognition models, providing stable service support for local devices to dynamically call online recognition models based on the complexity of target detection.

[0060] Furthermore, the construction steps of the local model pool include:

[0061] S374. Lightweighting is performed on the multiple online recognition models in the online model pool to obtain multiple local recognition models;

[0062] S375. Integrate the multiple local recognition models into a local model pool, wherein the local model pool is deployed on a local device and the local model pool operates independently of the online model pool.

[0063] In a preferred embodiment, multiple online recognition models in the online model pool are lightweighted to obtain multiple local recognition models. Specifically, since local devices have significant limitations in storage resources and computing power compared to cloud servers, directly deploying online recognition models on local devices can lead to slow inference speeds or even failure to run properly. Therefore, it is necessary to lightweight the online recognition models to reduce the model's parameter size and computational complexity. Lightweighting methods include, but are not limited to, model pruning, model quantization, and knowledge distillation. Model pruning reduces the number of model parameters by identifying and removing redundant network connections or neurons in the online recognition model that contribute little to the recognition results, while retaining the main feature extraction capabilities. Model quantization converts the floating-point network parameters in the online recognition model into low-order integer representations, reducing model storage usage while improving the inference speed on local devices. Knowledge distillation uses the online recognition model as a teacher model to train student models with smaller parameter sizes to mimic the output distribution of the teacher model, thereby retaining as much recognition capability as possible within the lightweight network architecture. The local recognition models obtained through lightweight processing are significantly smaller and less computationally complex than their corresponding online recognition models, enabling efficient inference under limited hardware resources on local devices.

[0064] Multiple local recognition models are integrated into a local model pool, which is deployed on a local device and operates independently of the online model pool. Specifically, multiple lightweight local recognition models are uniformly integrated into the local model pool for unified management and scheduling. The local model pool is persistently stored on the local device and can respond to local model call requests without relying on a network connection, effectively ensuring the normal operation of the object detection task in scenarios with unstable network connections or zero online model calls. The local model pool and the online model pool operate independently, with no direct data interaction between them. The local model pool is only responsible for responding to local model call requests, and the online model pool is only responsible for responding to online model call requests. Both provide model call services for the object detection task through their respective independent scheduling mechanisms.

[0065] Furthermore, based on the multiple local call recognition models and the multiple online call recognition models, target recognition is performed on the current region video frames to obtain local type probability distributions and online type probability distributions, generating moving target recognition results, including:

[0066] S41. Input the current region video frame into the multiple local call recognition models respectively to obtain multiple local recognition results. Count the number of occurrences of each target type in the multiple local recognition results. Calculate the ratio of the number of occurrences of each target type to the number of local model calls to obtain the local probability value of each target type. Summarize to obtain the local type probability distribution.

[0067] S42. Input the current region video frame into the multiple online call recognition models respectively to obtain multiple online recognition results. Count the number of occurrences of each target type in the multiple online recognition results. Calculate the ratio of the number of occurrences of each target type to the number of online model calls to obtain the online probability value of each target type. Summarize to obtain the online type probability distribution.

[0068] S43. The local type probability distribution and the online type probability distribution are fused to obtain the moving target recognition result.

[0069] In a preferred embodiment, firstly, the current region video frame is input into multiple local invocation recognition models to obtain multiple local recognition results. The occurrence frequency of each target type in the multiple local recognition results is counted, and the ratio of the occurrence frequency of each target type to the number of local model calls is calculated to obtain the local probability value of each target type. The results are then summarized to obtain the local type probability distribution. Specifically, the same current region video frame is input into all local invocation recognition models in parallel. Each local invocation recognition model outputs a target type recognition result for the moving target in the current region video frame based on its own feature extraction and classification capabilities. The recognition results of all local invocation recognition models are summarized to obtain multiple local recognition results. The occurrence frequency of each target type in the multiple local recognition results is counted, and the occurrence frequency of each target type in the multiple local recognition results is counted. The occurrence frequency of each target type is divided by the number of local model calls to obtain the local probability value corresponding to each target type. This local probability value reflects the proportion of models in the local invocation recognition model group that consider the moving target to belong to that target type. All target types and their corresponding local probability values ​​are summarized to form a local type probability distribution, and the sum of the local probability values ​​of all target types in the local type probability distribution is 1.

[0070] The current region's video frame is input into multiple online recognition models, resulting in multiple online recognition results. The frequency of each target type in these online recognition results is counted, and the ratio of the frequency of each target type to the number of online model calls is calculated to obtain the online probability value for each target type. These results are then aggregated to obtain the online type probability distribution. Specifically, the same current region's video frame is input into all online recognition models in parallel. Each online recognition model outputs a target type recognition result for a moving target in the current region's video frame. The recognition results from all online recognition models are aggregated to obtain multiple online recognition results. The frequency of each target type in the multiple online recognition results is counted, and the frequency of each target type is divided by the number of online model calls to obtain the online probability value for each target type. This online probability value reflects the proportion of models in the online recognition model group that consider the moving target to belong to that target type. All target types and their corresponding online probability values ​​are aggregated to form an online type probability distribution, where the sum of the online probability values ​​for all target types in the online type probability distribution is 1.

[0071] The local and online type probability distributions are fused to obtain the moving target recognition result. By fusing the local and online type probability distributions, and combining the recognition judgments of both the local and online recognition model groups, the risk of misjudgment due to the recognition bias of a single model group is effectively reduced, thus improving the accuracy and reliability of the moving target recognition result.

[0072] For example, in a park entrance / exit monitoring scenario, target types include three categories: normal pedestrians, people carrying large items, and people loitering abnormally. Assuming the local model is called 5 times and the online model is called 3 times during this target detection process, the current area video frames are input into the 5 local recognition models, resulting in 5 local recognition results: normal pedestrians, people carrying large items, and so on. The frequency of each target type is counted: normal pedestrians appear 3 times, people carrying large items appear 2 times, and people loitering abnormally appear 0 times. Dividing the frequency of each target type by the local model call count of 5 yields the local probability value for each target type: the local probability value for normal pedestrians is 3 / 5 = 0.6, the local probability value for people carrying large items is 2 / 5 = 0.4, and the local probability value for people loitering abnormally is 0. The summation yields the local type probability distribution. The current area video frames are input into three online recognition models, resulting in three online recognition results: normal pedestrians, people carrying large items, and people with large items. The frequency of each target type is counted: normal pedestrians appear once, people carrying large items appear twice, and people loitering abnormally appear zero times. Dividing the frequency of each target type by the number of online model calls (3) yields the online probability value for each target type: the online probability value for normal pedestrians is approximately 1 / 3 ≈ 0.33, for people carrying large items it is approximately 2 / 3 ≈ 0.67, and for people loitering abnormally it is zero. The online type probability distribution is then obtained by summing these values. Subsequent processing combines the local and online type probability distributions to obtain the final moving target recognition result.

[0073] For example, in a road traffic monitoring scenario, target types include pedestrians, non-motorized vehicles, and motorized vehicles. Assuming the number of local model calls and online model calls is 4 in this target detection process, the current area video frames are input into the four local model calls, resulting in four local recognition results: non-motorized vehicle, non-motorized vehicle, pedestrian, and non-motorized vehicle. The occurrence frequency of each target type is counted: non-motorized vehicles appear 3 times, pedestrians appear 1 time, and motorized vehicles appear 0 times. Dividing the occurrence frequency of each target type by the number of local model calls (4) yields the local probability value for each target type: the local probability value for non-motorized vehicles is 3 / 4 = 0.75, the local probability value for pedestrians is 1 / 4 = 0.25, and the local probability value for motorized vehicles is 0. The summation yields the local type probability distribution. The current area video frames are input into four online recognition models, resulting in four online recognition results: non-motorized vehicles, pedestrians, non-motorized vehicles, and pedestrians. The occurrence frequency of each target type is counted: non-motorized vehicles appear twice, pedestrians appear twice, and motorized vehicles appear zero times. The occurrence frequency of each target type is divided by the number of online model calls (4) to obtain the online probability value for each target type: the online probability value for non-motorized vehicles is 2 / 4 = 0.5, the online probability value for pedestrians is 2 / 4 = 0.5, and the online probability value for motorized vehicles is 0. The online type probability distribution is then obtained by summing the results. Subsequent processing combines the local type probability distribution and the online type probability distribution to obtain the final moving target recognition result.

[0074] Furthermore, the local type probability distribution and the online type probability distribution are fused to obtain the moving target recognition result, including:

[0075] S431. Sort the probability values ​​of each target type in the local type probability distribution in descending order to obtain the local probability sorting result;

[0076] S432. Determine the first local probability value and the second local probability value based on the local probability ranking result, calculate the difference between the first local probability value and the second local probability value, and obtain the local distribution concentration.

[0077] S433. Arrange the probability values ​​of each target type in the online type probability distribution in descending order to obtain the online probability ranking result;

[0078] S434. Determine the first online probability value and the second online probability value based on the online probability ranking result, calculate the difference between the first online probability value and the second online probability value, and obtain the online distribution concentration.

[0079] S435. Sum the local distribution concentration degree and the online distribution concentration degree to obtain the total concentration degree. Calculate the ratio of the local distribution concentration degree to the total concentration degree to obtain the local fusion weight. Calculate the ratio of the online distribution concentration degree to the total concentration degree to obtain the online fusion weight.

[0080] S436. Based on the local fusion weight and the online fusion weight, the local type probability distribution and the online type probability distribution are fused to obtain the fused type probability distribution;

[0081] S437. Obtain the target type with the highest probability in the fusion type probability distribution, and use it as the moving target identification result of the moving target.

[0082] In a preferred embodiment, firstly, the probability values ​​of each target type in the local type probability distribution are sorted in descending order to obtain a local probability ranking result. Specifically, all target types in the local type probability distribution are sorted from high to low according to their corresponding local probability values ​​to obtain a local probability ranking result. The local probability ranking result intuitively reflects the order of the strength of the local calling recognition model group's tendency to recognize each target type. Then, a first local probability value and a second local probability value are determined based on the local probability ranking result, and the difference between the first local probability value and the second local probability value is calculated to obtain the local distribution concentration. Specifically, the local probability value corresponding to the target type ranked first in the local probability ranking result is determined as the first local probability value, and the local probability value corresponding to the target type ranked second is determined as the second local probability value. The difference between the first local probability value and the second local probability value is calculated to obtain the local distribution concentration. The local distribution concentration reflects the degree of concentration of the local calling recognition model group in recognizing the highest-ranked target type. The larger the local distribution concentration, the more consistent the local calling recognition model group is in recognizing the highest-ranked target type and the higher the confidence level. The smaller the local distribution concentration, the closer the recognition results of the local calling recognition model group are to the top two target types, and there is a certain degree of recognition uncertainty.

[0083] Simultaneously, the probability values ​​of each target type in the online probability distribution are sorted in descending order to obtain the online probability ranking result. Specifically, all target types in the online probability distribution are sorted from high to low according to their corresponding online probability values, resulting in the online probability ranking result. The online probability ranking result intuitively reflects the strength order of the online recognition model's tendency to recognize each target type. Based on the online probability ranking result, the first and second online probability values ​​are determined, and the difference between the first and second online probability values ​​is calculated to obtain the online distribution concentration. Specifically, the online probability value corresponding to the target type ranked first in the online probability ranking result is determined as the first online probability value, and the online probability value corresponding to the target type ranked second is determined as the second online probability value. The difference between the first and second online probability values ​​is calculated to obtain the online distribution concentration. The online distribution concentration reflects the degree of concentration of the online identification model group's identification tendency towards the highest-ranked target type. The greater the online distribution concentration, the more consistent the online identification model group's identification tendency towards the highest-ranked target type and the higher the confidence level. The smaller the online distribution concentration, the closer the online identification model group's identification results for the top two target types are, and there is a certain degree of identification uncertainty.

[0084] Subsequently, the local distribution concentration and the online distribution concentration are summed to obtain the total concentration. The ratio of the local distribution concentration to the total concentration is calculated to obtain the local fusion weight, and the ratio of the online distribution concentration to the total concentration is calculated to obtain the online fusion weight. Specifically, the local distribution concentration and the online distribution concentration are added to obtain the total concentration. The local distribution concentration is divided by the total concentration to obtain the local fusion weight, and the online distribution concentration is divided by the total concentration to obtain the online fusion weight. The sum of the local fusion weight and the online fusion weight is 1. The magnitudes of the local fusion weight and the online fusion weight reflect the proportions of the local type probability distribution and the online type probability distribution in the subsequent fusion processing, respectively. The side with higher distribution concentration receives a larger fusion weight. That is, the model group with more concentrated identification tendency and higher confidence plays a greater dominant role in the fusion result, thereby achieving adaptive weighting of the identification confidence levels of the local and online called identification model groups.

[0085] Next, the local type probability distribution and the online type probability distribution are fused based on the local fusion weight and the online fusion weight to obtain the fused type probability distribution. Specifically, all target types involved in the local type probability distribution and the online type probability distribution are traversed. For each target type, the product of its corresponding local probability value and local fusion weight and the product of its online probability value and online fusion weight are summed to obtain the fusion probability value corresponding to that target type. The fusion probability values ​​of all target types are then summarized to form the fused type probability distribution.

[0086] Next, the target type with the highest probability in the fusion type probability distribution is obtained as the mobile target identification result. Specifically, the fusion probability values ​​of all target types in the fusion type probability distribution are compared, and the target type with the highest fusion probability value is determined as the mobile target identification result. This target type comprehensively reflects the final judgment of the type of the mobile target after adaptive weighted fusion of the local calling identification model group and the online calling identification model group.

[0087] Furthermore, the local type probability distribution and the online type probability distribution are fused based on the local fusion weight and the online fusion weight to obtain a fused type probability distribution, including:

[0088] S4361. Determine multiple target types from the local type probability distribution and the online type probability distribution;

[0089] S4362. Traverse the multiple target types to obtain the first target type;

[0090] S4363. Obtain the local probability value corresponding to the first target type from the local type probability distribution to obtain the local probability value of the first target type;

[0091] S4364. Obtain the online probability value corresponding to the first target type from the online type probability distribution to obtain the online probability value of the first target type.

[0092] S4365. Summing the product of the local probability value of the first target type and the local fusion weight and the product of the online probability value of the first target type and the online fusion weight, we obtain the fusion probability value of the first target type.

[0093] S4366. Continue to traverse the multiple target types until the traversal is complete, obtain the fusion probability values ​​of multiple target types, and summarize them to form the fusion type probability distribution.

[0094] In a preferred embodiment, firstly, multiple target types are determined from the local type probability distribution and the online type probability distribution. Specifically, all target types appearing in the local type probability distribution and the online type probability distribution are summarized, and their union is taken to determine the multiple target types involved in this fusion process, ensuring that no target type that has appeared in either the local or online type probability distribution is omitted from the fusion type probability distribution. Then, the multiple target types are traversed to obtain the first target type. Specifically, the multiple target types are traversed sequentially according to a preset traversal order, and the target type currently traversed is determined as the first target type, which is used as the processing object for the fusion probability value calculation in this round.

[0095] Next, the local probability value corresponding to the first target type is obtained from the local type probability distribution, thus obtaining the local probability value of the first target type. Specifically, the local probability value corresponding to the first target type is searched in the local type probability distribution, and this probability value is determined as the local probability value of the first target type. If there is no corresponding record for the first target type in the local type probability distribution, the local probability value of the first target type is assigned to 0, indicating that no model in the local call identification model group identifies the moving target as the first target type. Simultaneously, the online probability value corresponding to the first target type is obtained from the online type probability distribution, thus obtaining the online probability value of the first target type. Specifically, the online probability value corresponding to the first target type is searched in the online type probability distribution, and this probability value is determined as the online probability value of the first target type. If there is no corresponding record for the first target type in the online type probability distribution, the online probability value of the first target type is assigned to 0, indicating that no model in the online call identification model group identifies the moving target as the first target type.

[0096] Subsequently, the product of the local probability value of the first target type and the local fusion weight, and the product of the online probability value of the first target type and the online fusion weight are summed to obtain the fusion probability value of the first target type. Specifically, the local probability value of the first target type is multiplied by the local fusion weight to obtain the local weighted probability value of the first target type, and the online probability value of the first target type is multiplied by the online fusion weight to obtain the online weighted probability value of the first target type. The sum of the two yields the fusion probability value of the first target type. This fusion probability value comprehensively reflects the combined judgment of the local invocation identification model group and the online invocation identification model group, after adaptive weighting, that the moving target belongs to the first target type.

[0097] Continue iterating through multiple target types until the iteration is complete, obtaining the fusion probability values ​​for multiple target types, and summing them to form a fusion type probability distribution. Specifically, calculate the fusion probability value for each target type in the multiple target types in turn according to the above steps until all target types have been traversed. Summarize all target types and their corresponding fusion probability values ​​to form a fusion type probability distribution, in which the sum of the fusion probability values ​​of all target types in the fusion type probability distribution is 1.

[0098] Example 2, as Figure 2 As shown, based on the same inventive concept as the video target detection method based on a visual AI model provided in Embodiment 1, this embodiment of the invention also provides a video target detection system based on a visual AI model, including:

[0099] Background creation module 11 is used to capture video frames of the monitored area through a camera when there is no moving target, and to create background video frames of the monitored area.

[0100] The complexity determination module 12 is used to obtain a current area video frame containing the moving target when it is determined that there is a moving target in the monitoring area, and determine the target detection complexity of the moving target based on the current area video frame and the background video frame.

[0101] The dynamic scheduling module 13 is used to determine the number of local model calls and the number of online model calls based on the target detection complexity, and to determine multiple local call identification models and multiple online call identification models based on the number of local model calls and the number of online model calls.

[0102] The target recognition module 14 is used to perform target recognition on the current area video frame based on the multiple local call recognition models and the multiple online call recognition models, obtain the local type probability distribution and the online type probability distribution, and generate moving target recognition results.

[0103] Furthermore, the complexity determination module 12 is also used for:

[0104] The difference between the current region video frame and the background video frame is compared, and the target region of the moving target is extracted from the current region video frame;

[0105] Calculate the pixel size of the target region to obtain the target pixel size, and normalize the target pixel size based on a preset size range to obtain the normalized target size;

[0106] Histogram statistics are performed on the pixel colors of the target region to obtain a color histogram, and the information entropy of the color histogram is calculated to obtain the color distribution entropy. The color distribution entropy is normalized based on a preset entropy value range to obtain the normalized color distribution entropy.

[0107] Based on preset size weights and preset color weights, the normalized target size and the normalized color distribution entropy are weighted and summed to obtain the target detection complexity of the moving target.

[0108] Furthermore, the dynamic scheduling module 13 is also used for:

[0109] Obtain the preset basic detection complexity and the total number of standard models;

[0110] When the target detection complexity is less than or equal to the preset basic detection complexity, the total number of standard models is determined as the number of local model calls, and the number of online model calls is determined as 0.

[0111] When the target detection complexity is greater than the preset basic detection complexity, the ratio of the preset basic detection complexity to the target detection complexity is calculated to obtain the local call coefficient. The product of the total number of standard models and the local call coefficient is rounded down to obtain the local model call count. The difference between the total number of standard models and the local model call count is determined as the online model call count.

[0112] Furthermore, the dynamic scheduling module 13 is also used for:

[0113] Connect a pre-built local model pool and an online model pool, wherein the local model pool contains multiple local recognition models and the online model pool contains multiple online recognition models;

[0114] Randomly select a number of local recognition models corresponding to the number of local model calls from the local model pool to obtain multiple local call recognition models;

[0115] When the number of online model calls is greater than 0, a number of online recognition models corresponding to the number of online model calls are randomly selected from the online model pool to obtain multiple online call recognition models.

[0116] Furthermore, the construction steps of the online model pool include:

[0117] Collect historical target detection records from multiple sample monitoring areas, and construct a sample video frame set and a sample target type label set based on the historical target detection records;

[0118] Multiple online recognition model architectures are constructed. The sample video frame set is used as input and the sample target type label set is used as the target. The multiple online recognition model architectures are trained until convergence, resulting in multiple online recognition models.

[0119] The multiple online recognition models are integrated into an online model pool, which is deployed on a cloud server.

[0120] Furthermore, the construction steps of the local model pool include:

[0121] The multiple online recognition models in the online model pool are each subjected to lightweight processing to obtain multiple local recognition models;

[0122] The multiple local recognition models are integrated into a local model pool, wherein the local model pool is deployed on a local device and operates independently of the online model pool.

[0123] Furthermore, the target recognition module 14 is also used for:

[0124] The current region video frames are input into the multiple local call recognition models respectively to obtain multiple local recognition results. The occurrence frequency of each target type in the multiple local recognition results is counted, the ratio of the occurrence frequency of each target type to the number of local model calls is calculated, the local probability value of each target type is obtained, and the local type probability distribution is obtained by summarizing.

[0125] The current region video frames are input into the multiple online recognition models respectively to obtain multiple online recognition results. The occurrence frequency of each target type in the multiple online recognition results is counted, the ratio of the occurrence frequency of each target type to the number of online model calls is calculated, the online probability value of each target type is obtained, and the online type probability distribution is obtained by summarizing.

[0126] The local type probability distribution and the online type probability distribution are fused to obtain the moving target recognition result.

[0127] Furthermore, the target recognition module 14 is also used for:

[0128] The probability values ​​of each target type in the local type probability distribution are sorted in descending order to obtain the local probability ranking result;

[0129] Based on the local probability ranking results, a first local probability value and a second local probability value are determined, and the difference between the first local probability value and the second local probability value is calculated to obtain the local distribution concentration.

[0130] The probability values ​​of each target type in the online type probability distribution are sorted in descending order to obtain the online probability ranking result;

[0131] Based on the online probability ranking results, determine the first online probability value and the second online probability value, calculate the difference between the first online probability value and the second online probability value, and obtain the online distribution concentration.

[0132] The local distribution concentration is summed with the online distribution concentration to obtain the total concentration. The ratio of the local distribution concentration to the total concentration is calculated to obtain the local fusion weight. The ratio of the online distribution concentration to the total concentration is calculated to obtain the online fusion weight.

[0133] Based on the local fusion weight and the online fusion weight, the local type probability distribution and the online type probability distribution are fused to obtain the fused type probability distribution;

[0134] The target type with the highest probability in the fusion type probability distribution is obtained as the moving target identification result of the moving target.

[0135] Furthermore, the target recognition module 14 is also used for:

[0136] Multiple target types are determined from the local type probability distribution and the online type probability distribution;

[0137] Iterate through the multiple target types to obtain the first target type;

[0138] The local probability value corresponding to the first target type is obtained from the local type probability distribution to obtain the local probability value of the first target type;

[0139] The online probability value corresponding to the first target type is obtained from the online type probability distribution to obtain the online probability value of the first target type;

[0140] The product of the local probability value of the first target type and the local fusion weight and the product of the online probability value of the first target type and the online fusion weight are summed to obtain the fusion probability value of the first target type.

[0141] Continue traversing the multiple target types until the traversal is complete, obtain the fusion probability values ​​of the multiple target types, and summarize them to form the fusion type probability distribution.

[0142] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0143] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0145] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0147] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0148] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for video object detection based on visual AI model, characterized in that, The method includes: In the absence of moving targets, video frames are captured from the monitored area using a camera to establish background video frames of the monitored area. When it is determined that there is a moving target in the monitored area, the current area video frame containing the moving target is obtained, and the target detection complexity of the moving target is determined based on the current area video frame and the background video frame. Determining the number of local model calls and the number of online model calls based on the target detection complexity includes: obtaining a preset basic detection complexity and the total number of standard models; when the target detection complexity is less than or equal to the preset basic detection complexity, determining the total number of standard models as the number of local model calls and setting the number of online model calls to 0; when the target detection complexity is greater than the preset basic detection complexity, calculating the ratio of the preset basic detection complexity to the target detection complexity to obtain the local call coefficient, rounding down the product of the total number of standard models and the local call coefficient to obtain the number of local model calls, and determining the difference between the total number of standard models and the number of local model calls as the number of online model calls; Based on the number of local model calls and the number of online model calls, multiple local call identification models and multiple online call identification models are determined. Based on the multiple local call recognition models and the multiple online call recognition models, target recognition is performed on the current area video frame to obtain local type probability distribution and online type probability distribution, generating moving target recognition results. This includes: inputting the current area video frame into the multiple local call recognition models to obtain multiple local recognition results; counting the occurrence frequency of each target type in the multiple local recognition results; calculating the ratio of the occurrence frequency of each target type to the number of local model calls to obtain the local probability value of each target type; and summing these to obtain the local type probability distribution; inputting the current area video frame into the multiple online call recognition models to obtain multiple online recognition results; counting the occurrence frequency of each target type in the multiple online recognition results; calculating the ratio of the occurrence frequency of each target type to the number of online model calls to obtain the online probability value of each target type; and summing these to obtain the online type probability distribution; and fusing the local type probability distribution and the online type probability distribution to obtain the moving target recognition result.

2. The method of claim 1, wherein, Determining the target detection complexity of the moving target based on the current region video frame and the background video frame includes: The difference between the current region video frame and the background video frame is compared, and the target region of the moving target is extracted from the current region video frame; Calculate the pixel size of the target region to obtain the target pixel size, and normalize the target pixel size based on a preset size range to obtain the normalized target size; Histogram statistics are performed on the pixel colors of the target region to obtain a color histogram, and the information entropy of the color histogram is calculated to obtain the color distribution entropy. The color distribution entropy is normalized based on a preset entropy value range to obtain the normalized color distribution entropy. Based on preset size weights and preset color weights, the normalized target size and the normalized color distribution entropy are weighted and summed to obtain the target detection complexity of the moving target.

3. The method according to claim 1, characterized in that, Based on the number of local model calls and the number of online model calls, multiple local call identification models and multiple online call identification models are determined, including: Connect a pre-built local model pool and an online model pool, wherein the local model pool contains multiple local recognition models and the online model pool contains multiple online recognition models; Randomly select a number of local recognition models corresponding to the number of local model calls from the local model pool to obtain multiple local call recognition models; When the number of online model calls is greater than 0, a number of online recognition models corresponding to the number of online model calls are randomly selected from the online model pool to obtain multiple online call recognition models.

4. The method according to claim 3, characterized in that, The steps for constructing the online model pool include: Collect historical target detection records from multiple sample monitoring areas, and construct a sample video frame set and a sample target type label set based on the historical target detection records; Multiple online recognition model architectures are constructed. The sample video frame set is used as input and the sample target type label set is used as the target. The multiple online recognition model architectures are trained until convergence, resulting in multiple online recognition models. The multiple online recognition models are integrated into an online model pool, which is deployed on a cloud server.

5. The method according to claim 4, characterized in that, The steps for constructing the local model pool include: The multiple online recognition models in the online model pool are each subjected to lightweight processing to obtain multiple local recognition models; The multiple local recognition models are integrated into a local model pool, wherein the local model pool is deployed on a local device and operates independently of the online model pool.

6. The method according to claim 1, characterized in that, The local type probability distribution and the online type probability distribution are fused to obtain the moving target recognition result, including: The probability values ​​of each target type in the local type probability distribution are sorted in descending order to obtain the local probability ranking result; Based on the local probability ranking results, a first local probability value and a second local probability value are determined, and the difference between the first local probability value and the second local probability value is calculated to obtain the local distribution concentration. The probability values ​​of each target type in the online type probability distribution are sorted in descending order to obtain the online probability ranking result; Based on the online probability ranking results, determine the first online probability value and the second online probability value, calculate the difference between the first online probability value and the second online probability value, and obtain the online distribution concentration. The local distribution concentration is summed with the online distribution concentration to obtain the total concentration. The ratio of the local distribution concentration to the total concentration is calculated to obtain the local fusion weight. The ratio of the online distribution concentration to the total concentration is calculated to obtain the online fusion weight. Based on the local fusion weight and the online fusion weight, the local type probability distribution and the online type probability distribution are fused to obtain the fused type probability distribution; The target type with the highest probability in the fusion type probability distribution is obtained as the moving target identification result of the moving target.

7. The method according to claim 6, characterized in that, The local type probability distribution and the online type probability distribution are fused based on the local fusion weight and the online fusion weight to obtain a fused type probability distribution, including: Multiple target types are determined from the local type probability distribution and the online type probability distribution; Iterate through the multiple target types to obtain the first target type; The local probability value corresponding to the first target type is obtained from the local type probability distribution to obtain the local probability value of the first target type; The online probability value corresponding to the first target type is obtained from the online type probability distribution to obtain the online probability value of the first target type; The product of the local probability value of the first target type and the local fusion weight and the product of the online probability value of the first target type and the online fusion weight are summed to obtain the fusion probability value of the first target type. Continue traversing the multiple target types until the traversal is complete, obtain the fusion probability values ​​of the multiple target types, and summarize them to form the fusion type probability distribution.

8. A video target detection system based on a visual AI model, characterized in that, The system for implementing the method as described in any one of claims 1 to 7, the system comprising: The background creation module is used to capture video frames of the monitored area through a camera when there is no moving target, and to create background video frames of the monitored area. The complexity determination module is used to obtain a current area video frame containing the moving target when it is determined that there is a moving target in the monitoring area, and determine the target detection complexity of the moving target based on the current area video frame and the background video frame. A dynamic scheduling module is used to determine the number of local model calls and the number of online model calls based on the target detection complexity, including: obtaining a preset basic detection complexity and the total number of standard models; when the target detection complexity is less than or equal to the preset basic detection complexity, determining the total number of standard models as the number of local model calls and setting the number of online model calls to 0; when the target detection complexity is greater than the preset basic detection complexity, calculating the ratio of the preset basic detection complexity to the target detection complexity to obtain a local call coefficient, rounding down the product of the total number of standard models and the local call coefficient to obtain the number of local model calls, and determining the difference between the total number of standard models and the number of local model calls as the number of online model calls; Based on the number of local model calls and the number of online model calls, multiple local call identification models and multiple online call identification models are determined. The target recognition module is used to perform target recognition on the current area video frame based on the multiple local call recognition models and the multiple online call recognition models, obtain local type probability distributions and online type probability distributions, and generate moving target recognition results. The module includes: inputting the current area video frame into the multiple local call recognition models to obtain multiple local recognition results; counting the occurrence frequency of each target type in the multiple local recognition results; calculating the ratio of the occurrence frequency of each target type to the number of local model calls to obtain the local probability value of each target type; and summing these to obtain the local type probability distribution; inputting the current area video frame into the multiple online call recognition models to obtain multiple online recognition results; counting the occurrence frequency of each target type in the multiple online recognition results; calculating the ratio of the occurrence frequency of each target type to the number of online model calls to obtain the online probability value of each target type; and summing these to obtain the online type probability distribution; and fusing the local type probability distribution and the online type probability distribution to obtain the moving target recognition result.

Images