A multi-target recognition-based video monitoring dynamic control method

Abstract

The application provides a video monitoring dynamic arrangement and control method based on multi-target identification, and relates to the technical field of computer vision, which detects the personnel density condition of the region through the data stream of the camera, and dynamically adjusts the shooting angle and focal length of the camera according to the detection result. The method in the application can solve the technical problems that the personnel gathering state of the key region in the outdoor large venue is difficult to be perceived in real time, the security personnel response is not timely, and the human resource deployment efficiency is low to a certain extent.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and more specifically, to a video monitoring dynamic deployment method based on multi-target recognition. BACKGROUND

[0002] With the rapid development of video monitoring technology, intelligent analysis-based video monitoring systems have become an important part of public safety, traffic management, and security in important venues. Traditional video monitoring systems mainly rely on real-time monitoring by humans, which cannot fully utilize the large amount of data collected by cameras, and is prone to problems such as monitoring blind spots or information omission. In recent years, with the breakthroughs in computer vision and deep learning technologies, target detection, target tracking, and behavior analysis technologies have become increasingly widely used in the monitoring field. These technologies can achieve identification and tracking of specific targets through intelligent analysis of video data, thereby significantly improving monitoring efficiency. However, most current intelligent monitoring technologies still have certain limitations, such as over-reliance on single target recognition algorithms, difficulty in simultaneously handling the dynamic characteristics of multiple targets, or decreased recognition accuracy and tracking effectiveness in complex scenes with dense targets. In addition, the global scheduling strategy of cameras is still relatively rough, and cannot finely adjust the shooting angle, focal length, and priority of cameras according to real-time changes in the scene, resulting in low resource utilization efficiency.

[0003] In the prior art, multi-target detection algorithms can already detect and mark targets in video monitoring scenes in real time, but there are still many deficiencies in the actual application of dynamic deployment. First, the feature extraction of existing algorithms for targets mostly stays at the level of simple color or shape, lacking depth analysis of target regions, making it difficult to accurately extract key features of targets in dense or overlapping scenes. Second, dynamic region analysis based on target features has not been fully realized, and existing technologies often cannot effectively combine the distribution characteristics of key areas of venues to accurately calculate and dynamically map the density of targets, which leads to insufficient attention to high-risk areas. In addition, the dynamic adjustment mechanism of cameras is not perfect, and existing automatic zoom and angle adjustment algorithms usually rely on fixed logic and cannot dynamically optimize camera scheduling strategies according to the distribution characteristics of targets and real-time needs of the venue. Therefore, in complex scenes, existing technologies cannot achieve efficient monitoring of multiple targets and optimal allocation of resources. SUMMARY

[0004] To solve the above technical problems, the present application provides a video monitoring dynamic deployment method based on multi-target recognition, which can to some extent solve the technical problem that the gathering state of personnel in key areas in outdoor large venues is difficult to perceive in real time, leading to delayed response of security personnel and low efficiency of human resource deployment.

[0005] According to one aspect of the present application, a multi-target recognition-based video monitoring dynamic control method is provided, which comprises:

[0006] collecting video data streams of multiple cameras in a venue and obtaining monitoring scene images in the video data streams;

[0007] performing target detection on the monitoring scene images to obtain a target detection frame, performing matting processing on the target based on the target detection frame to obtain a target image, and simultaneously performing feature decomposition on the target image according to an HSV color space to obtain a target feature map;

[0008] based on the target feature map, performing connected domain labeling on the target feature map using a region growing algorithm to obtain a target attention region, calculating a personnel density distribution map in the target attention region, and simultaneously performing spatial mapping of the personnel density distribution map and a preset key area of the venue to obtain a region attention coefficient;

[0009] constructing a camera priority queue according to the region attention coefficient, dynamically adjusting the shooting angle and focal length of the camera according to the priority queue, and turning a camera with a higher priority to the target attention region for automatic zooming.

[0010] Further, the target detection uses a target detection network to identify and locate human targets in the monitoring scene images.

[0011] The target detection network optimizes the resolution transition of the feature map through smooth up-sampling and down-sampling, and uses a GIoU loss to improve the positioning accuracy of low overlap targets.

[0012] Further, the up-sampling and down-sampling determines a weighting coefficient by calculating the relative distance between target pixels and source pixels, and performs step-by-step interpolation in the horizontal and vertical directions, combined with boundary padding to ensure sampling continuity, thereby generating a smooth and high-quality target size feature map.

[0013] Further, the GIoU loss comprises:

[0014] calculating the intersection area and the union area of the predicted frame and the real frame to obtain an IoU value;

[0015] determining the smallest closed region that can enclose both the predicted frame and the real frame, and calculating the area of the smallest closed region;

[0016] subtracting the union area of the predicted frame and the real frame from the area of the smallest closed region, and dividing by the area of the smallest closed region to obtain a penalty term;

[0017] adding the penalty term to 1 minus the IoU value to obtain a GIoU loss value.

[0018] Further, the matting processing comprises the following steps:

[0019] Obtaining the target frame coordinates output by the target detection network, analyzing the target posture and calculating the main direction angle;

[0020] Rotating and aligning the target frame according to the main direction angle to obtain a minimum circumscribed rectangle parallel to the image coordinate axis;

[0021] Expanding the boundary of the minimum circumscribed rectangle, adjusting the part exceeding the image boundary and filling with the boundary pixels, and retaining the context information on the target periphery;

[0022] Constructing a projection transformation matrix based on the vertex coordinates of the minimum circumscribed rectangle, and calculating the color value of each pixel of the target image through bilinear interpolation to complete the matting processing.

[0023] Further, the region growing algorithm comprises:

[0024] Selecting the point with the strongest HSV feature response as the seed point through adaptive grid division on the target feature map;

[0025] Calculating the HSV feature vector of the seed point, and using eight-neighborhood search in the region growing process to judge whether the adjacent pixels join the current region according to the weighted combination of the HSV spatial distance and the position distance;

[0026] When multiple regions are adjacent, judging whether to merge through a feature similarity matrix, merging the regions and updating the statistical features when the threshold condition is met;

[0027] Completing the connected domain labeling using a two-pass scanning algorithm to ensure that the pixels in the same region have the same label;

[0028] Post-processing the labeled regions to remove the regions with small area or irregular shape through area filtering and shape optimization to obtain the final labeling result.

[0029] Further, a density estimation network is constructed to calculate the personnel density distribution map, and the density estimation network adopts a multi-branch structure, including a global density estimation branch and a local detail enhancement branch.

[0030] Further, the personnel density distribution map is spatially mapped with the preset key area of the venue to obtain a spatial mapping relationship;

[0031] Based on the spatial mapping relationship, the cumulative density value in each key area is calculated;

[0032] Based on the cumulative density value, the area attention coefficient is obtained in combination with the area importance weight.

[0033] Further, the calculation of the cumulative density value is based on spatial position weight and timing smoothing factor, and the cumulative density value of the key area is obtained by hierarchical weighted accumulation.

[0034] Further, the dynamic adjustment adjusts the camera according to the priority queue in sequence, calculates the horizontal rotation angle, the pitch angle and the focal length through the spherical coordinate system, ensures that the angle and the focal length are within the mechanical limit and the optical range, and preferentially selects the camera with the best view angle to adjust when the target areas overlap.

[0035] Compared with the prior art, the video monitoring dynamic control method based on multi-target recognition provided by the application detects the personnel density of the area through the data stream of the camera, and dynamically adjusts the shooting angle and focal length of the camera according to the detection result. In this way, to a certain extent, the technical problem that the personnel gathering state of the key area in the outdoor large venue is difficult to perceive in real time, resulting in that the security personnel response is not timely and the efficiency of human resource allocation is low can be solved. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, below will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor. In the drawings:

[0037] Figure 1 The flow chart of the video monitoring dynamic control method based on multi-target recognition according to the embodiment of the present application.

[0038] Figure 2 The flow chart of the bilinear interpolation in the video monitoring dynamic control method based on multi-target recognition according to the embodiment of the present application. DETAILED DESCRIPTION

[0039] In the following, the example embodiments according to the present application will be described in detail with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.

[0040] Figure 1 The flow chart of the video monitoring dynamic control method based on multi-target recognition according to the embodiment of the present application. As shown in Figure 1 In the video monitoring dynamic control method based on multi-target recognition, it includes:

[0041] S1: collecting video data streams of multiple cameras in the venue, and acquiring monitoring scene images in the video data streams;

[0042] Video data streams of multiple cameras in the venue are collected, the cameras including fixed cameras of entrances and exits of the venue, stand areas, square areas and passage areas, and rotatable pan-tilt cameras, and real-time sampling of the video data streams at a frame rate of 25 fps is performed to obtain monitoring scene images in the video data streams; the resolution of the monitoring scene images is 1920x1080 pixels to ensure the definition of image details, the monitoring scene images are stored and transmitted in YUV420 format, wherein the Y component represents luminance information, and the UV component represents chrominance information, and image enhancement processing is performed on the monitoring scene images, including luminance equalization and color correction, to eliminate image quality differences caused by illumination changes at different times.

[0043] S2: target detection is performed on the monitoring scene images to obtain a target detection frame, and based on the target detection frame, a matting processing is performed on the target to obtain a target image, and the target image is feature-decomposed according to an HSV color space to obtain a target feature map;

[0044] Target detection is performed on the monitoring scene images, an improved YOLOv5 target detection network is used to recognize and locate human targets in the monitoring scene images, a CSPDarknet53 structure is used for a backbone feature extraction network of the target detection network, a receptive field range is improved through an SPP spatial pyramid pooling module, and target detection frame coordinate information with a confidence score is output; based on the coordinate information of the target detection frame, a minimum circumscribed rectangle method is used to perform matting processing on a target region to obtain a target image, and size normalization processing is performed on the target image to uniformly scale the image to 224x224 pixels; the target image is converted from an RGB color space to an HSV color space, wherein the H component represents hue information, the S component represents saturation information, and the V component represents brightness information, independent feature decomposition is performed on the three components of the HSV to extract color, texture and brightness features of the target respectively, and a target feature map is obtained; the target feature map uses a multi-scale feature fusion manner to combine HSV features at different scales in a weighted manner to form a feature representation with rich details.

[0045] The improved YOLOv5 object detection network is specifically optimized based on the standard YOLOv5 network. When inputting a surveillance scene image, it first extracts features through the improved CSPDarknet53 backbone network. The CSPDarknet53 backbone network replaces the standard convolution operation with depthwise separable convolution and introduces a channel attention mechanism in each CSP module to highlight the weights of important feature channels. The feature maps extracted by the backbone network are then fused at multiple scales through the improved Feature Pyramid Network (FPN). When the feature map scale changes, bilinear interpolation is used for upsampling and downsampling operations to ensure the feature map is properly segmented. The resolution transition is smooth; an improved spatial attention module is added to the neck layer, which adaptively adjusts the weight coefficients of different regions of the feature map by calculating the response intensity of spatial location when the target exhibits scale changes or dense distribution; a dense prediction mechanism is introduced in the detection head layer, which improves the position accuracy of the bounding box by refining the prediction offset when there is a large deviation between the predicted box and the ground truth box; at the same time, the original cross-entropy loss function is improved to a GIoU loss function based on IOU, which can provide more effective gradient information to guide network optimization when the overlap between the predicted box and the ground truth box is low; if target occlusion occurs, a cascaded detection mechanism is set up to perform secondary detection and optimization of the occluded area.

[0046] The improved Feature Pyramid Network (FPN) employs an adaptive feature fusion strategy. When the backbone network outputs multiple feature maps of different scales, it first constructs a feature pyramid structure from deep to shallow layers, containing five scale levels from P2 to P6. For the deepest feature map, P6, the number of channels is adjusted using a 1×1 convolutional layer, and it is directly used as the top-level feature of the pyramid. When it is necessary to construct the P5 feature map, the P6 feature map is upsampled by a factor of 2 and adaptively fused with the feature map of the corresponding level in the backbone network. The adaptive fusion includes calculating the channel attention weights and spatial attention weights of the two feature maps. When the importance of different regions and channels in the feature map is different, the contribution level is dynamically adjusted through an attention mechanism. The same process is repeated to construct the P4, P3, and P2 feature maps, with each level of feature fusion employing an adaptive weighting method.

[0047] After top-down feature fusion is completed, the feature maps of each level are optimized using 3×3 convolutions. The optimization includes eliminating the aliasing effect caused by upsampling and enhancing the discriminative power of the features. If small-scale targets are detected, the high-resolution feature maps of the P2 and P3 levels are mainly used. When detecting large-scale targets, the large receptive field feature maps of the P5 and P6 levels are mainly relied upon. A dense connection mechanism is introduced between feature maps of different scales. When the features of a certain level are insufficient, they can be supplemented by the features of adjacent levels. At the same time, a spatial attention module is embedded in the feature map of each level. When the target is occluded or deformed, the feature representation of important regions is dynamically highlighted by calculating the spatial response map.

[0048] It should be noted that when upsampling the feature map is required, the feature map of the target size is first determined. For each pixel position in the target feature map, its corresponding floating-point coordinates are found in the source feature map. The floating-point coordinates are usually located between four adjacent pixels in the source feature map. At this time, the relative distance between the target position and these four source pixels is calculated, and the contribution weight of each source pixel to the target position is determined according to the relative distance. When calculating the pixel value of the target position, the values ​​of the four source pixels are weighted and averaged according to their weights.

[0049] When performing downsampling, pixels in adjacent regions of the source feature map are also fused using a distance-weighted method. If the size of the source feature map is an integer multiple of the target feature map, the pixel values ​​in the corresponding region are weighted according to their relative positions. When the ratio of the source feature map to the target feature map is not an integer, the source image region corresponding to each target pixel is determined through reverse mapping, and the pixels in that region are weighted and averaged. If sampling occurs in the boundary region of the source feature map, boundary pixel filling is used to ensure the continuity of the sampling operation.

[0050] In the bilinear interpolation process, the interpolation operations in the horizontal and vertical directions can be performed step by step. When performing horizontal interpolation, linear interpolation calculations are first performed on each row; after completing the horizontal interpolation, vertical linear interpolation is then performed on the intermediate results to finally obtain the feature map of the target size; if the feature map contains multiple channels, the same interpolation operation is performed independently on each channel, specifically as follows: Figure 2 As shown.

[0051] The IoU-based GIoU loss function is an improved loss function designed for the bounding box regression problem in object detection. When the predicted box and the ground truth box completely overlap, the GIoU value is 1; when the predicted box and the ground truth box are completely separated, the GIoU value is less than or equal to 0. The GIoU loss function first calculates the intersection area and the union area of ​​the predicted box and the ground truth box to obtain the IoU value. Simultaneously, it calculates the minimum closure region C that can simultaneously enclose both the predicted box and the ground truth box. The area of ​​the minimum closure region C minus the area of ​​the union region of the predicted box and the ground truth box, and then divided by the area of ​​the minimum closure region C, yields a penalty term. When the IoU values ​​of the predicted box and the ground truth box are the same, this penalty term makes the predicted box more inclined to move closer to the ground truth box. If the distance between the predicted box and the ground truth box is large, the penalty term is larger, providing stronger gradient information to guide network optimization. The final GIoU loss value is equal to 1 minus the IoU value plus the penalty term. As the predicted box gradually approaches the ground truth box, the loss value monotonically decreases. More specifically, the loss function L... GIoU As shown in the following formula:

[0052]

[0053] Among them, B p To predict the region of the bounding box, B g Let C be the region of the ground truth bounding box, and let C be the minimum closure region that simultaneously encloses the predicted box and the ground truth bounding box. |·| represents the area of ​​the calculated region.

[0054] Compared to the traditional IoU loss function, the proposed GIoU loss function effectively addresses the gradient vanishing problem when the predicted bounding box and the ground truth bounding box do not overlap by introducing the concept of a closure region. It also considers the overlap and relative positional relationship between the predicted and ground truth bounding boxes, thus more accurately guiding the network to learn the positional information of the bounding boxes. It exhibits better convergence and stability, especially when dealing with targets with large scale variations.

[0055] For example, when a large number of people are densely distributed in a monitoring scene, the target detection network will generate a predicted bounding box for each human target. If a human target is detected in the stands area, the predicted bounding box and the ground truth bounding box will partially overlap. In this case, the intersection area and the union area of ​​the two boxes will be calculated to obtain the IoU value. At the same time, the area of ​​the minimum closure region C that can completely surround the predicted bounding box and the ground truth bounding box will be calculated. When the predicted bounding box and the ground truth bounding box are close, the area of ​​the minimum closure region C is relatively small. In this case, the calculated penalty term value is smaller, and the final GIoU loss value is mainly determined by the IoU value.

[0056] If the predicted bounding box is significantly offset from the ground truth bounding box, resulting in a decrease in the overlap between the two boxes, the IoU value decreases, and the area of ​​the minimum closure region C increases, causing the calculated penalty term to increase accordingly. Ultimately, the GIoU loss value increases significantly. Through the change in the loss value, the network obtains more explicit gradient information during backpropagation, driving the predicted bounding box to adjust to the position of the ground truth bounding box. When the predicted bounding box and the ground truth bounding box do not overlap at all, the traditional IoU loss will exhibit gradient vanishing, while the GIoU loss function can still calculate an effective loss value through the closure region C, continuing to provide directional guidance for network optimization.

[0057] On the other hand, the specific steps of the minimum bounding rectangle method for matting the target region are as follows: After obtaining the target bounding box coordinates output by the target detection network, the target bounding box is first subjected to pose analysis. If the target is tilted, the principal direction angle of the target is calculated. The principal direction angle is based on the pixel distribution characteristics of the target region and is obtained by calculating the eigenvalues ​​and eigenvectors of the second-order matrix of the region pixels. After determining the principal direction, the target bounding box is rotated and aligned to be parallel to the image coordinate axis, thereby obtaining the minimum bounding rectangle of the target. After obtaining the minimum bounding rectangle, the rectangle boundary is expanded. When the target is close to the image edge, the expansion range is adjusted according to the actual situation. If it exceeds the image boundary, the boundary pixels are used for filling. The expansion operation aims to preserve the context information around the target. When the target is partially occluded, the context information is used to assist in subsequent feature extraction. If the aspect ratio of the target region is abnormal, it is cropped or filled by setting a threshold to ensure the rationality of the matting result.

[0058] When performing the image matting operation, the projection transformation matrix from the original image to the target image is first constructed based on the coordinates of the four vertices of the minimum bounding rectangle. When performing pixel mapping, the color value of each pixel position in the target image is calculated using the bilinear interpolation method.

[0059] The process of constructing the projection transformation matrix from the original image to the target image is as follows: First, obtain the coordinates of the four vertices of the minimum bounding rectangle and arrange them in the order of top left, top right, bottom right, and bottom left to form the source point coordinate set; at the same time, according to the preset size of the target image (224×224 pixels), construct the corresponding target point coordinate set so that the four corner points of the target image correspond to the four vertices of the minimum bounding rectangle respectively; after the source point coordinate set and the target point coordinate set are determined, the homography matrix solution method based on the least squares method is used to establish a 3×3 projection transformation matrix H from the source space to the target space; the projection transformation matrix H contains transformation information such as rotation, scaling, and translation. When there are four pairs of source points and target points, eight linear equations can be listed, and the nine elements of matrix H are solved by the singular value decomposition (SVD) method.

[0060] The constructed projection transformation matrix H must satisfy the following properties: when any point in the source image is transformed, the product of its homogeneous coordinates and matrix H is the position of the corresponding point in the target image; if there is an error in the point position before and after the transformation, the element values ​​of matrix H are optimized by minimizing the reprojection error; once matrix H is determined, a projection transformation is performed on each pixel in the original image to obtain the mapped position in the target image coordinate system, thereby achieving geometric correction and size normalization of the image. More specifically, the projection transformation matrix is ​​shown in the following equation:

[0061]

[0062] Among them, H * p is a 3×3 projection transformation matrix. i Let p' be the homogeneous coordinates (x_i, y_i, 1)^T of the i-th vertex in the source image. i Let α be the homogeneous coordinates (x'_i, y'_i, 1)^T of the i-th vertex in the target image. i Let d be the weight coefficient of the i-th vertex pair (calculated based on the reliability of the vertex position). i Let be the Euclidean distance between the i-th pair of vertices, β be the distance decay factor, and μ be the regularization coefficient. 2 Denotes the L2 norm, ∥*∥ F Describing the Frobenius norm

[0063] S3: Based on the target feature map, the connected component labeling of the target feature map is performed using the region growing algorithm to obtain the target attention region, and the personnel density distribution map in the target attention region is calculated. At the same time, the personnel density distribution map is spatially mapped with the preset key areas of the venue to obtain the regional attention coefficient.

[0064] Based on the target feature map, an improved region growing algorithm is used to label connected components in the target feature map. When a seed point is detected in the target feature map, the region is expanded outward from the seed point. The selection of the seed point is based on the similarity measure of the HSV feature space. When the difference between the HSV feature value of an adjacent pixel and the seed point is less than the dynamically calculated growth threshold, the pixel is merged into the current region. If multiple growth regions appear, a region merging criterion is used to determine whether to merge adjacent regions. The region merging criterion considers the feature similarity and spatial distance relationship between regions. After the region growing is completed, all connected components are labeled to obtain the target region of interest.

[0065] The improved region growing algorithm is an optimization of the standard region growing algorithm. First, a set of candidate seed points is established on the target feature map. When a local maximum point exists in the image, it is used as the initial seed point. The seed point selection process adopts an adaptive grid partitioning method, which divides the feature map into multiple sub-regions. In each sub-region, the point with the strongest HSV feature response is selected as the candidate seed point. After the seed point is determined, the HSV feature vector of the point is calculated, including a weighted combination of hue component, saturation component and brightness component.

[0066] During region growing, an eight-neighbor search strategy is employed. When an adjacent pixel is encountered, the feature distance between that pixel and the current seed pixel is calculated. This feature distance includes a weighted combination of HSV spatial distance and spatial location distance. If the feature distance is less than the adaptive growth threshold, the pixel is added to the current region, and the region's statistical features are updated. When multiple growing regions are adjacent, a region merging mechanism is initiated. The feature similarity matrix between regions is calculated to determine whether merging is necessary. This feature similarity matrix considers the region's average HSV feature, region shape feature, and region boundary gradient information. If the similarity between two adjacent regions exceeds the merging threshold, and the boundary gradient between the regions is weak, these two regions are merged into a new region. After a region merge occurs, the statistical features of the merged region are recalculated, and the region growing parameters are updated. In the connected component labeling stage, a two-pass scanning algorithm is used to label all growing regions. The first pass scans the image from top to bottom and from left to right, assigning a temporary label to each pixel. The second pass solves the equivalent label problem, ensuring that pixels in the same region have the same label. After labeling is completed, each connected component is post-processed, including area filtering and shape optimization, to remove regions that are too small or have irregular shapes, resulting in the final target region of interest labeling result.

[0067] Based on the target region of interest, a density estimation network is constructed to calculate the personnel density distribution map. The density estimation network adopts a multi-branch structure, including a global density estimation branch and a local detail enhancement branch. When performing density estimation, the global density estimation branch extracts large-scale features through downsampling convolution to obtain the overall density distribution of the scene. The local detail enhancement branch maintains the original resolution and extracts local detail features through dilated convolution to accurately characterize the personnel distribution in high-density areas. The feature maps of the two branches are adaptively fused to obtain the final personnel density distribution map.

[0068] The global density estimation branch first downsamples the input feature map. When the feature map size is large, it is downsampled sequentially through max pooling with a stride of 2 until the feature map size is reduced to 1 / 8 of the original size. The downsampled feature map is then processed through multiple residual blocks, each containing two 3x3 convolutional layers and a shortcut connection. As the feature map passes through the residual blocks, global semantic information of the scene is extracted step by step. A spatial pyramid pooling module is set after the residual blocks, using adaptive pooling operations at three scales: 1x1, 2x2, and 4x4, to capture global contextual information under different receptive fields. If there is uneven density distribution in the scene, an attention mechanism is used to enhance the feature response of high-density regions.

[0069] The local detail enhancement branch maintains the original resolution of the input feature map and uses multi-scale dilated convolution for feature extraction. When performing dilated convolution, 3x3 convolution kernels with dilation rates of 1, 2, and 4 are used, and an instance normalization layer and LeakyReLU activation function are added after each convolutional layer. The multi-scale features are adaptively fused through a feature selection module. When a feature at a certain scale has a strong expressive power for the current region, the weight of the feature at that scale is enhanced through a soft attention mechanism. After the feature selection module, a detail enhancement module is set up to enhance the density details of the local region through residual learning.

[0070] The feature fusion process of the two branches adopts an adaptive weighting strategy. First, the feature map of the global branch is upsampled to the original resolution through bilinear interpolation. When performing feature fusion, the channel attention weights and spatial attention weights of the feature maps of the two branches are calculated. The channel attention weights are calculated by the squeeze-and-excitation module and are used to select important feature channels. The spatial attention weights are calculated based on the spatial response intensity of the feature map. When a certain position has a strong response in both branches, that position receives a higher fusion weight.

[0071] The feature map after feature fusion is subjected to final density estimation by a density regression head, which contains multiple 1x1 convolutional layers to map the feature map into a single-channel density map. When generating the density map, the local consistency of density values ​​is ensured by the Poisson fusion algorithm. If there are regions with abnormal density values, they are corrected by density value smoothing.

[0072] This dual-branch structure achieves accurate estimation of scene density distribution through the complementary effects of global and local branches. Particularly when dealing with scenes exhibiting significant differences in crowd density, it maintains both the accuracy of the global density distribution and the clarity of local details. Furthermore, the adaptive feature fusion mechanism effectively integrates density information at different scales, improving the robustness and accuracy of density estimation.

[0073] The personnel density distribution map is spatially mapped to the preset key areas of the venue. When the key areas are irregularly shaped, perspective transformation is used to align the density distribution map to the coordinate system of the venue floor plan. Based on the spatial mapping relationship, the cumulative density value of each key area is calculated, and combined with the regional importance weight, the regional attention coefficient is obtained.

[0074] The cumulative density value is calculated using a layered weighted accumulation method. When the spatially mapped density distribution map is obtained, the key area is first divided into multiple grid cells of equal area. For each grid cell, the density value of the corresponding pixel in the density distribution map is calculated. When a grid cell covers multiple pixels, the average density value of the grid cell is calculated using bilinear interpolation. The density value of the grid cell is related to its spatial position in the key area. When the grid cell is located at the center of the area, a higher position weight coefficient is assigned, and when the grid cell is located at the edge of the area, a lower position weight coefficient is assigned. If the density value of the grid cell changes continuously over time, a temporal smoothing factor is introduced to avoid drastic fluctuations in the density value. The weighted density values ​​of all grid cells are accumulated to obtain the cumulative density value of the key area. More specifically, the cumulative density value is shown in the following formula:

[0075]

[0076] in,

[0077]

[0078] Among them, D total The cumulative density value for the key area, where N is the total number of grid cells, and D is the cumulative density value for the key area. i W represents the average density value of the i-th grid cell. i d is the spatial location weighting coefficient. i T is the distance from the grid cell to the center of the region, σ is the spatial weight decay parameter, and T is the distance from the grid cell to the center of the region. i D is the time series smoothing factor. i (t) represents the density value at the current time, D i (t-1) represents the density value at the previous time step, and α is the time-series smoothing adjustment parameter.

[0079] S4: Construct a camera priority queue based on the area attention coefficient, dynamically adjust the shooting angle and focal length of the cameras according to the priority queue, and automatically zoom the higher priority cameras towards the target attention area, while pushing the location information and personnel density distribution map of the target attention area to the security personnel terminal.

[0080] First, each camera is processed sequentially according to the priority queue. When a camera to be adjusted is determined, the required horizontal rotation angle and pitch angle are calculated based on its installation position and the spatial coordinates of the target area of ​​interest. The angle calculation adopts a spherical coordinate system transformation method. When the calculated angle exceeds the mechanical limit of the camera, the closest feasible angle is selected. If the target area is within the common coverage area of ​​multiple cameras, the camera with the best viewing angle is selected for adjustment.

[0081] For focal length adjustment, calculations are made based on the area of ​​the target region and the observation distance. When the target region is large, the field of view is expanded by reducing the focal length value; when the target region is small and far away, the image is magnified by increasing the focal length value. The focal length adjustment must be ensured to be within the optical zoom range of the camera. When the calculated focal length value is outside the range, the closest available focal length value is used.

[0082] For example, suppose a stadium has five spherical cameras (labeled A, B, C, D, and E). According to the current priority queue, the northwest corner of the stands has the highest attention coefficient (covered by cameras A and B), followed by the southeast entrance area (covered by camera C), and then the central area (covered by cameras D and E). For the highest priority northwest corner area, the system first analyzes the positional conditions of cameras A and B: camera A is 12 meters from the target area with a 15-degree viewing angle; camera B is 18 meters from the target area with an 8-degree viewing angle. Therefore, camera A, with its superior viewing angle, is selected for adjustment. The specific adjustment process is as follows:

[0083] Horizontal angle adjustment: The current horizontal angle of camera A is 225 degrees, and the target area needs to be 245 degrees. Therefore, the horizontal rotation is completed in 3 steps (each rotation is about 7 degrees).

[0084] Pitch angle adjustment: The current pitch angle is 35 degrees, and the target area requires 42 degrees. The pitch adjustment will be completed in two steps (each adjustment is about 3.5 degrees).

[0085] Focus adjustment: The target area is about 80 square meters and the distance is 12 meters. The calculated required focal length is 18mm. The current focal length is 25mm. Adjust to the target focal length in 4 steps.

[0086] Once camera A is in position, the system switches to the southeast gate area, which has the second highest priority, and controls camera C to make corresponding adjustments. This step-by-step adjustment method ensures that the monitoring system always focuses on key areas in optimal condition. When the area priority changes, the system will recalculate and execute new adjustment instructions.

[0087] In summary, the video surveillance dynamic deployment method based on multi-target recognition, as described in this invention, is explained. It uses camera data streams to detect the population density in a given area and dynamically adjusts the camera's shooting angle and focal length based on the detection results. This can, to some extent, solve the technical problem of difficulty in real-time perception of crowd gathering in key areas of large outdoor venues, leading to delayed security personnel response and low efficiency in human resource allocation.

[0088] Here, those skilled in the art will understand that the specific operations of each step in the above-described dynamic deployment method for video surveillance based on multi-target recognition have been referenced above. Figure 1 and Figure 2 The method for dynamic deployment of video surveillance based on multi-target recognition has been described in detail, and therefore, its repeated description will be omitted.

Claims

1. A video surveillance dynamic deployment method based on multi-target recognition, characterized in that, include: Collect video data streams from multiple cameras in the venue and acquire images of the monitored scene from the video data streams; Target detection is performed on the monitored scene image to obtain a target detection box, and the target is matted based on the target detection box to obtain a target image. At the same time, the target image is decomposed according to the HSV color space to obtain a target feature map. Based on the target feature map, a region growing algorithm is used to label the connected components of the target feature map to obtain the target attention region, and the personnel density distribution map within the target attention region is calculated. At the same time, the personnel density distribution map is spatially mapped with the preset key areas of the venue to obtain the regional attention coefficient. A camera priority queue is constructed based on the area attention coefficient. The shooting angle and focal length of the cameras are dynamically adjusted according to the priority queue, and the cameras with higher priority are turned to the target area of ​​attention for automatic zoom. The target detection uses a target detection network to identify and locate human targets in the monitored scene images; The target detection network optimizes the feature map resolution transition through smooth upsampling and downsampling, and improves the localization accuracy of low-overlap targets by utilizing GIoU loss. The upsampling and downsampling determine the weighting coefficients by calculating the relative distance between the target pixel and the source pixel, and interpolate in the horizontal and vertical directions step by step, combined with boundary filling to ensure sampling continuity, thereby generating a smooth and high-quality target size feature map. The GIoU loss includes: Calculate the intersection area and union area of ​​the predicted bounding box and the ground truth bounding box to obtain the IoU value; Determine the minimum closure region that can simultaneously enclose the predicted bounding box and the ground truth bounding box, and calculate the area of ​​the minimum closure region; Subtract the area of ​​the union of the predicted box and the ground truth box from the area of ​​the minimum closure region, and divide by the area of ​​the minimum closure region to obtain the penalty term; Add the value obtained by subtracting the IoU value from 1 to the penalty term to obtain the GIoU loss value.

2. The video surveillance dynamic deployment method based on multi-target recognition according to claim 1, characterized in that, The image cutout process includes the following steps: Obtain the bounding box coordinates output by the object detection network, analyze the target pose, and calculate the principal direction angle; The target box is rotated and aligned according to the main direction angle to obtain the smallest bounding rectangle parallel to the image coordinate axis; Expand the minimum bounding rectangle boundary, adjust the portion that exceeds the image boundary and fill it with boundary pixels, and preserve the surrounding context information of the target; The projection transformation matrix is ​​constructed based on the vertex coordinates of the minimum bounding rectangle, and the color value of each pixel in the target image is calculated by bilinear interpolation to complete the image matting process.

3. The video surveillance dynamic deployment method based on multi-target recognition according to claim 1, characterized in that, The region growing algorithm includes: On the target feature map, the point with the strongest HSV feature response is selected as the seed point by adaptive grid partitioning; Calculate the HSV feature vector of the seed point, and use eight-neighbor search during the region growing process to determine whether neighboring pixels should be added to the current region based on the weighted combination of HSV spatial distance and positional distance. When multiple regions are adjacent, the feature similarity matrix is ​​used to determine whether to merge them. If the threshold condition is met, the regions are merged and the statistical features are updated. A two-pass scanning algorithm is used to complete the connected component labeling, ensuring that pixels within the same region have the same label; The marked regions are post-processed by area filtering and shape optimization to remove regions that are too small or have irregular shapes, resulting in the final marked result.

4. The video surveillance dynamic deployment method based on multi-target recognition according to claim 1, characterized in that, A density estimation network is constructed to calculate the density distribution map of personnel. The density estimation network adopts a multi-branch structure, including a global density estimation branch and a local detail enhancement branch.

5. The video surveillance dynamic deployment method based on multi-target recognition according to claim 1, characterized in that, The personnel density distribution map is spatially mapped to the preset key areas of the venue to obtain the spatial mapping relationship; The cumulative density value within each key region is calculated based on the spatial mapping relationship; The regional attention coefficient is obtained by combining the cumulative density value with the regional importance weight.

6. The video surveillance dynamic deployment method based on multi-target recognition according to claim 5, characterized in that, The cumulative density value is calculated based on spatial location weights and temporal smoothing factors, and the cumulative density value of key areas is obtained by hierarchical weighted accumulation.

7. The video surveillance dynamic deployment method based on multi-target recognition according to claim 1, characterized in that, The dynamic adjustment adjusts the cameras sequentially according to a priority queue, calculates the horizontal rotation angle, pitch angle, and focal length using a spherical coordinate system, ensures that the angle and focal length are within mechanical and optical limits, and prioritizes the adjustment of the camera with the best viewing angle when the target areas overlap.

Images