Smart city camera multi-target tracking method and system

By combining deep learning and convolutional neural network technology, the problems of multi-scale recognition, occlusion analysis and multi-camera data integration in the multi-objective tracking method of smart city cameras are solved, and more efficient target tracking and abnormal behavior recognition are achieved, improving monitoring System response capability and event processing efficiency.

CN118297984BActive Publication Date: 2024-11-08HUNAN POLICE ACAD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410114313.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-26
Publication Date
2024-11-08
Estimated Expiration
2044-01-26

AI Technical Summary

Technical Problem

The existing multi-objective tracking methods for smart city cameras have shortcomings in multi-scale target recognition, target occlusion analysis and multi-camera data integration, resulting in limited recognition capabilities, low tracking accuracy, slow response speed, and difficult to accurately obtain occlusion relationships and space-time relationships.

Method used

Using technical means such as deep separation convolutional neural network, scale invariant feature conversion, feature pyramid network, attention mechanism, mask area convolutional neural network, graph convolutional neural network, time convolutional neural network and long and short-term memory neural network , perform image processing, multi-scale feature fusion, adaptive scale adjustment, occlusion relationship analysis, cross-camera data integration and abnormal behavior recognition.

Benefits of technology

Improve the accuracy and flexibility of target tracking, enhance the ability to identify multi-scale targets, ensure accurate acquisition of scene depth information, improve the construction efficiency of space-time relationships and the accuracy of behavioral analysis, and significantly enhance smart city monitoring System response capability and event processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118297984B_ABST
    Figure CN118297984B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of computer vision technology, specifically to a multi-target tracking method and system for smart city cameras, comprising the following steps: based on monitoring images, using a deep separation convolutional neural network and scale-invariant feature conversion to perform image processing and preliminary target recognition, and perform preliminary detection of targets. In the present invention, a feature pyramid network and an attention mechanism accurately identify targets of different scales, strengthen multi-scale feature extraction and fusion, adaptive scale adjustment optimizes target tracking flexibility and accuracy, mask area convolutional neural network and monocular depth estimation fusion solve target occlusion, graph convolutional neural network enhances multi-camera data integration, improves the efficiency of spatiotemporal association construction, combines time convolutional neural network and long short-term memory neural network, and uses a single-class support vector machine for abnormal behavior recognition and warning, improves the accuracy of behavior analysis and the timeliness of warning, and enhances the responsiveness and event processing efficiency of smart city monitoring systems.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a multi-target tracking method and system for smart city cameras. Background Art

[0002] Computer vision is a field of science and engineering that studies how to enable computers to understand and interpret images or videos. It involves the use of algorithms and models to process, analyze, and understand digital image or video data to identify, track, and understand objects, actions, and events in a scene.

[0003] Among them, the smart city camera multi-target tracking method aims to track multiple targets using the smart city camera system. The purpose of this method is to automatically detect, track and identify multiple targets by analyzing the moving objects in the video surveillance screen, thereby providing information about the target location, motion trajectory and behavior pattern. In order to achieve this effect, this method is generally achieved through means such as target detection, target tracking, multi-target tracking and behavior analysis. These technical means are usually based on algorithms and methods in related fields such as deep learning, image processing and pattern recognition, and improve the accuracy and robustness of tracking by training a large amount of labeled data.

[0004] In the existing smart city camera multi-target tracking methods, the existing methods often lack effective multi-scale feature fusion and adaptive scale adjustment mechanisms, resulting in limited recognition capabilities when processing multi-scale targets, and low tracking accuracy for long-distance or small-scale targets. In addition, traditional methods often lack effective deep information analysis tools when dealing with target occlusion problems, making it difficult to accurately obtain occlusion relationships, resulting in insufficient understanding of the relationship between targets. In terms of multi-camera data integration, existing methods often cannot fully utilize information from different perspectives due to the lack of efficient network structures, affecting the establishment of spatiotemporal associations. At the same time, in terms of behavior learning and early warning, traditional methods are often limited to surface features and ignore deep-level pattern recognition, making the early warning of abnormal behavior not timely or accurate enough, limiting the response speed of the overall monitoring system and the ability to handle complex events. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose a smart city camera multi-target tracking method and system.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: a multi-target tracking method of a smart city camera, comprising the following steps:

[0007] S1: Based on the surveillance image, a deep separation convolutional neural network and scale-invariant feature transformation are used for image processing and preliminary target recognition, and preliminary target detection is performed to generate preliminary target recognition information;

[0008] S2: Based on the preliminary target recognition information, a feature pyramid network and an attention mechanism are used to perform multi-scale feature fusion, and the recognition capability of multi-scale targets is enhanced to generate multi-scale target feature information;

[0009] S3: Based on the multi-scale target feature information, adaptive scale adjustment is performed using a scale estimation module and an anchor frame mechanism to optimize the size of the target tracking frame and generate tracking information after adaptive scale adjustment;

[0010] S4: Based on the tracking information after the adaptive scale adjustment, a mask region convolutional neural network and a monocular depth estimation algorithm are used to process the target occlusion problem, obtain the occlusion relationship between targets and the scene depth information, and generate the target occlusion relationship and depth information;

[0011] S5: Based on the target occlusion relationship and depth information, the data of multiple camera perspectives are integrated through a graph convolutional neural network, and the spatiotemporal correlation of the target in the field of view of multiple cameras is established to generate cross-camera target tracking information;

[0012] S6: Based on the cross-camera target tracking information, the target pattern is learned by combining the temporal convolutional neural network and the long short-term memory neural network, and the abnormal behavior is identified and warned by using a single-class support vector machine, and an abnormal behavior identification and warning report is generated;

[0013] The preliminary target recognition information includes the position, size and shape information of the target. The multi-scale target feature information is specifically a multi-level, multi-scale target feature description, including the details and structural information of the target at multiple scales. The tracking information after adaptive scale adjustment is specifically the tracking frame and feature information of the matching target at multiple scales. The target occlusion relationship and depth information specifically refers to the recognition information of the occluded target and the depth position of the target in the scene. The cross-camera target tracking information includes the position, time association and behavior information of key targets between multiple cameras. The abnormal behavior recognition and warning report is specifically the discovered illegal behavior or abnormal behavior, and its time, place and nature are described and warned.

[0014] As a further solution of the present invention, based on the monitoring image, a deep separation convolutional neural network and a scale-invariant feature transformation are used to perform image processing and preliminary target recognition, and preliminary detection of the target is performed. The steps of generating preliminary target recognition information are specifically as follows:

[0015] S101: Based on the original monitoring image, histogram equalization and Gaussian blur algorithm are used to perform image enhancement and noise reduction to generate an enhanced image;

[0016] S102: Based on the enhanced image, a depthwise separable convolution algorithm is used to extract basic features, reduce the amount of calculation and maintain performance, and generate a basic feature map;

[0017] S103: Based on the basic feature map, multi-scale spatial pyramid pooling is used to enhance scale-invariant feature extraction to generate a vector feature map;

[0018] S104: Based on the vector feature map, a region proposal network is used in combination with bounding box regression to perform preliminary detection of the target, accurately locate the target, and generate preliminary target recognition information.

[0019] As a further solution of the present invention, based on the preliminary target recognition information, a feature pyramid network and an attention mechanism are used to perform multi-scale feature fusion, and the recognition capability of multi-scale targets is enhanced. The steps of generating multi-scale target feature information are specifically as follows:

[0020] S201: Based on the preliminary target recognition information, a feature pyramid network is used to achieve fusion between feature layers, enhance the detection capability of multi-scale targets, and generate a fusion feature map;

[0021] S202: Based on the fused feature map, an attention mechanism is used to enhance the expressiveness of features and generate an attention-weighted feature map;

[0022] S203: Based on the attention weighted feature map, an anchor frame optimization algorithm is used to optimize and adjust the anchor frame, so as to improve the fitting accuracy of the model for multi-scale targets and generate optimized anchor frame feature information;

[0023] S204: Based on the optimized anchor frame feature information, features are refined through a cascaded convolutional network to enhance the representation of multi-scale features and generate multi-scale target feature information.

[0024] As a further solution of the present invention, based on the multi-scale target feature information, the scale estimation module and the anchor frame mechanism are used to perform adaptive scale adjustment, optimize the size of the target tracking frame, and generate the tracking information after adaptive scale adjustment in the following steps:

[0025] S301: Based on the multi-scale target feature information, a real-time scale estimation algorithm is used to dynamically predict the target size, adapt to changes in the target scale, and generate scale adjustment information;

[0026] S302: Based on the scale adjustment information, an anchor frame generation algorithm is used to adjust the scale of the tracking frame so that the tracking frame fits the actual scale of the target, and a tracking frame with an adaptive scale is generated;

[0027] S303: Based on the tracking frame of the adaptive scale, a non-maximum suppression algorithm is used to eliminate overlapping tracking frames, reduce redundancy, and generate a non-maximum suppressed tracking frame;

[0028] S304: Based on the non-maximum suppressed tracking frame, the final target tracking frame is located in combination with a tracking learning algorithm to generate tracking information after adaptive scale adjustment.

[0029] As a further solution of the present invention, based on the tracking information after the adaptive scale adjustment, the mask area convolutional neural network and the monocular depth estimation algorithm are used to process the target occlusion problem, obtain the occlusion relationship between targets and the scene depth information, and generate the target occlusion relationship and depth information in the following steps:

[0030] S401: Based on the tracking information after the adaptive scale adjustment, a mask region convolutional neural network enhanced by a feature pyramid network is used to perform target instance segmentation, and occlusion boundaries are accurately determined to generate target segmentation and occlusion boundary information;

[0031] S402: Based on the target segmentation and occlusion boundary information, a deep learning monocular depth estimation method is used to perform three-dimensional reconstruction of the scene to generate a scene depth map;

[0032] S403: Based on the scene depth map, pixel-level fusion technology is used to analyze the occlusion relationship between objects and generate an object occlusion relationship map;

[0033] S404: Based on the target occlusion relationship graph, combined with the depth sorting algorithm and the occlusion processing technology, the occlusion level and depth between the targets are determined, and the target occlusion relationship and depth information are generated.

[0034] As a further solution of the present invention, based on the target occlusion relationship and depth information, the data of multiple camera perspectives are integrated through a graph convolutional neural network, and the spatiotemporal association of the target in the multi-camera field of view is established, and the steps of generating cross-camera target tracking information are specifically as follows:

[0035] S501: Based on the target occlusion relationship and depth information, a graph convolutional neural network is applied, and feature extraction and optimization strategies are combined to integrate multi-view data to generate preliminary cross-camera target association information;

[0036] S502: Based on the preliminary cross-camera target association information, a multi-camera geometric calibration method is used to refine the target position relationship through a calibration technology based on epipolar constraints to generate multi-view target position information;

[0037] S503: Based on the multi-view target position information, using time series analysis technology, performing time series data analysis through dynamic time warping, optimizing target spatiotemporal association, and generating time-optimized target association information;

[0038] S504: Based on the time-optimized target association information, the multi-camera data is integrated, and the spatiotemporal association is completed through a multi-view tracking fusion algorithm to generate cross-camera target tracking information.

[0039] As a further solution of the present invention, based on the cross-camera target tracking information, the target pattern is learned by combining the temporal convolutional neural network and the long short-term memory neural network, and the abnormal behavior is identified and warned by using a single-class support vector machine. The steps of generating abnormal behavior identification and warning reports are specifically as follows:

[0040] S601: Based on the cross-camera target tracking information, deep feature learning is performed in combination with a temporal convolutional neural network and a long short-term memory network to generate a target behavior feature learning result;

[0041] S602: Based on the target behavior feature learning result, a feature clustering algorithm is used to perform spatiotemporal feature analysis, extract the target behavior pattern, and generate spatiotemporal clustered behavior features;

[0042] S603: Based on the behavior characteristics after the spatiotemporal clustering, use a single-class support vector machine to identify and model abnormal behaviors to generate an abnormal behavior identification model;

[0043] S604: Based on the abnormal behavior recognition model and in combination with real-time monitoring data, abnormal behavior warning is performed to generate an abnormal behavior recognition and warning report.

[0044] A smart city camera multi-target tracking system, the smart city camera multi-target tracking system is used to execute the above-mentioned smart city camera multi-target tracking method, the system includes an image preprocessing module, a basic feature extraction module, a preliminary target recognition module, a feature enhancement and optimization module, a target tracking frame preprocessing module, a target association module, and a behavior learning and early warning module.

[0045] As a further solution of the present invention, the image preprocessing module uses histogram equalization and Gaussian blur algorithm based on the original monitoring image to perform image enhancement and noise reduction to generate an enhanced image;

[0046] The basic feature extraction module extracts basic features based on the enhanced image, adopts a depth-separable convolution algorithm to reduce the amount of calculation and maintain performance, and generates a basic feature map;

[0047] The preliminary target recognition module performs preliminary target recognition based on the basic feature map, adopts multi-scale spatial pyramid pooling to enhance scale-invariant feature extraction, and adopts a region proposal network combined with bounding box regression to locate the target and generate preliminary target recognition information;

[0048] The feature enhancement and optimization module performs feature enhancement and optimization based on preliminary target recognition information, uses a feature pyramid network to achieve fusion between feature layers and adopts an attention mechanism to enhance the expressiveness of features, optimizes and adjusts the frame by using an anchor frame optimization algorithm, and extracts features through a cascaded convolutional network to generate multi-scale target feature information;

[0049] The target tracking frame preprocessing module processes the tracking frame based on multi-scale target feature information, uses a real-time scale estimation algorithm to dynamically predict the target size, adjusts the tracking frame scale through an anchor frame generation algorithm, and combines a non-maximum suppression algorithm to eliminate overlapping tracking frames, and generates tracking information after adaptive scale adjustment;

[0050] The target association module runs a target association algorithm based on the tracking information after adaptive scale adjustment, uses a mask area convolutional neural network enhanced by a feature pyramid network to perform target instance segmentation, reconstructs the scene in three dimensions through a deep learning monocular depth estimation method, uses pixel-level fusion technology for processing, and determines the occlusion level and depth between targets through a depth sorting algorithm and occlusion processing technology, and generates target occlusion relationship and depth information;

[0051] The behavior learning and warning module performs behavior learning and warning based on target occlusion relationship and depth information, uses temporal convolutional neural network and long short-term memory network for deep feature learning, performs spatiotemporal feature analysis through feature clustering algorithm, uses single-class support vector machine to identify and model abnormal behavior, combines real-time monitoring data for abnormal behavior warning, and generates abnormal behavior identification and warning report.

[0052] As a further solution of the present invention, the image preprocessing module includes a noise reduction submodule, an image enhancement submodule, an abnormality filtering submodule, and an image normalization submodule;

[0053] The basic feature extraction module includes a deep convolution submodule, a feature extraction submodule, a feature compression submodule, and a feature noise screening submodule;

[0054] The preliminary target recognition module includes a target positioning submodule, a target detection submodule, a target screening submodule, and a target confirmation submodule;

[0055] The feature enhancement and optimization module includes a feature fusion submodule, a feature attention weighting submodule, an anchor frame optimization submodule, and a feature extraction submodule;

[0056] The target tracking frame preprocessing module includes a real-time scale estimation submodule, an anchor frame generation submodule, a non-maximum suppression submodule, and a target tracking frame positioning submodule;

[0057] The target association module includes a target instance segmentation submodule, a scene 3D reconstruction submodule, a target occlusion relationship analysis submodule, and an occlusion level and depth determination submodule;

[0058] The behavior learning and warning module includes a deep feature learning submodule, a spatiotemporal feature analysis submodule, an abnormal behavior recognition submodule, and a real-time abnormal warning submodule.

[0059] Compared with the prior art, the advantages and positive effects of the present invention are:

[0060] In the present invention, by integrating the deep separation convolutional neural network and the scale-invariant feature conversion, the accuracy of image processing is effectively improved, and the ability of preliminary target recognition is enhanced. The application of feature pyramid network and attention mechanism makes the recognition of targets of different scales more accurate, and enhances the extraction and fusion of multi-scale target features. Adaptive scale adjustment optimizes the size of the tracking frame through the scale estimation module and the anchor frame mechanism, improving the flexibility and accuracy of target tracking. The fusion of mask area convolutional neural network and monocular depth estimation algorithm provides an effective means to solve the problem of target occlusion and ensures the accurate acquisition of scene depth information. The introduction of graph convolutional neural network strengthens the integration ability of multi-camera data and improves the efficiency of building spatiotemporal association. Finally, the combination of time convolutional neural network and long short-term memory neural network, as well as the use of single-class support vector machine in abnormal behavior recognition and early warning, greatly improves the accuracy of behavior analysis and the timeliness of early warning, and significantly enhances the overall response ability and event processing efficiency of smart city monitoring system to complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0062] Figure 2 This is a detailed flow chart of S1 of the present invention;

[0063] Figure 3 This is a detailed flow chart of S2 of the present invention;

[0064] Figure 4 This is a detailed flow chart of S3 of the present invention;

[0065] Figure 5 This is a detailed flow chart of S4 of the present invention;

[0066] Figure 6 This is a detailed flow chart of S5 of the present invention;

[0067] Figure 7 This is a detailed flow chart of S6 of the present invention;

[0068] Figure 8 is a system flow chart of the present invention;

[0069] Fig. 9 Schematic diagram of the system framework of the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0071] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined. Embodiment 1

[0072] See also Figure 1 The present invention provides a technical solution: a smart city camera multi-target tracking method, comprising the following steps:

[0073] S1: Based on the surveillance image, a deep separation convolutional neural network and scale-invariant feature transformation are used for image processing and preliminary target recognition, and preliminary target detection is performed to generate preliminary target recognition information;

[0074] S2: Based on the preliminary target recognition information, the feature pyramid network and attention mechanism are used to fuse multi-scale features, enhance the recognition ability of multi-scale targets, and generate multi-scale target feature information;

[0075] S3: Based on the multi-scale target feature information, the scale estimation module and anchor frame mechanism are used to perform adaptive scale adjustment, optimize the size of the target tracking frame, and generate tracking information after adaptive scale adjustment;

[0076] S4: Based on the tracking information after adaptive scale adjustment, the mask region convolutional neural network and monocular depth estimation algorithm are used to deal with the target occlusion problem, obtain the occlusion relationship between targets and the scene depth information, and generate the target occlusion relationship and depth information;

[0077] S5: Based on the target occlusion relationship and depth information, the graph convolutional neural network is used to integrate the data from multiple camera perspectives, and the spatiotemporal correlation of the target in the multi-camera field of view is established to generate cross-camera target tracking information;

[0078] S6: Based on cross-camera target tracking information, the target pattern is learned by combining the temporal convolutional neural network and the long short-term memory neural network, and a single-class support vector machine is used to identify and warn abnormal behaviors, and generate abnormal behavior identification and warning reports;

[0079] Preliminary target recognition information includes the target's position, size and shape information. Multi-scale target feature information refers to a multi-level, multi-scale target feature description, including the details and structural information of the target at multiple scales. The tracking information after adaptive scale adjustment refers to the tracking frame and feature information of the matching target at multiple scales. The target occlusion relationship and depth information refers to the recognition information of the occluded target and the depth position of the target in the scene. The cross-camera target tracking information includes the position, time correlation and behavior information of key targets between multiple cameras. The abnormal behavior recognition and warning report refers to the discovered illegal or abnormal behavior, and describes and warns of its time, place and nature.

[0080] By using the smart city camera system for multi-target tracking, multiple targets can be monitored and identified in real time, providing rich information on target locations, motion trajectories, and behavior patterns. This method uses deep separation convolutional neural networks and scale-invariant feature transformation to improve tracking accuracy. At the same time, multi-scale feature fusion is performed through feature pyramid networks and attention mechanisms to enhance the recognition of multi-scale targets. In order to solve the problem of target occlusion, this method uses mask region convolutional neural networks and monocular depth estimation algorithms to process target occlusion relationships and scene depth information. In addition, the graph convolutional neural network integrates data from multiple camera perspectives to achieve cross-camera target tracking and obtain more comprehensive target information. Finally, the temporal convolutional neural network and long short-term memory neural network are combined to learn the target pattern, and a single-class support vector machine is used to identify and warn abnormal behaviors.

[0081] See also Figure 2 ,Based on the surveillance image, a deep separation convolutional neural network and scale-invariant feature transformation are used for image processing and preliminary target recognition, and preliminary target detection is performed. The specific steps for generating preliminary target recognition information are as follows:

[0082] S101: Based on the original monitoring image, histogram equalization and Gaussian blur algorithm are used to perform image enhancement and noise reduction to generate an enhanced image;

[0083] S102: Based on the enhanced image, a depth-separable convolution algorithm is used to extract basic features, reduce the amount of calculation and maintain performance, and generate a basic feature map;

[0084] S103: Based on the basic feature map, multi-scale spatial pyramid pooling is used to enhance scale-invariant feature extraction to generate a vector feature map;

[0085] S104: Based on the vector feature map, a region proposal network combined with bounding box regression is used to perform preliminary detection of the target, accurately locate the target, and generate preliminary target recognition information.

[0086] First, the original surveillance image is processed by histogram equalization and Gaussian blur algorithm to enhance the contrast and clarity of the image and reduce the influence of noise. This can generate an enhanced image and provide better input for subsequent target recognition.

[0087] Next, the enhanced image is subjected to feature extraction using the depthwise separable convolution algorithm. This algorithm decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution, which reduces the amount of computation while maintaining good performance. Through this step, a basic feature map containing basic feature information can be generated.

[0088] In order to enhance the recognition ability of objects of different scales, a multi-scale spatial pyramid pooling method is used to process the basic feature map. This method extracts scale-invariant features from feature maps at different levels and fuses them together to generate a vector feature map. This can obtain a richer and more discriminative feature representation.

[0089] Based on the generated vector feature map, the region proposal network is used in combination with bounding box regression to perform preliminary detection of the target. The region proposal network can generate a series of candidate boxes, and these candidate boxes are adjusted and optimized through bounding box regression to accurately locate the position and size of the target. Through this step, preliminary target recognition information can be generated, including key information such as the position, size and shape of the target.

[0090] In the present invention, after the preliminary target identification information is generated, in order to better lock the preliminary target identification information and improve the accuracy of the selected target, it is necessary to calculate the accuracy of the preliminary target selected in the candidate frame based on the preliminary target identification information, that is, calculate the possibility that the preliminary target selected is the final target, and judge it through a target matching index.

[0091] Specifically, the present invention also includes the following steps:

[0092] S104a: Based on the preliminary target recognition information, obtain the position, size and shape of the target;

[0093] S104b: performing comparisons in a preset target database according to the position of the target, the size of the target, and the shape of the target, respectively, to obtain the similarity of the position item of the target, the similarity of the size item of the target, and the similarity of the shape item of the target;

[0094] S104c: Calculate the target matching index of the preliminary target according to the target position item similarity, the target size item similarity and the target shape item similarity.

[0095] In this step, the calculation formula of the target matching index of the preliminary target is expressed as:

[0096] ;

[0097] in, represents the target fit index of the preliminary target, The benchmark value of the target fit index representing the preliminary target, Represents the location item similarity of the target, represents the weight factor of the target's position term, Indicates the similarity of the size of the target, represents the weight factor of the size term of the target, Represents the shape item similarity of the target, Represents the weight factor of the shape term of the target.

[0098] It can be understood that by comparing the calculated target match index of the preliminary target with a preset target match index threshold, preliminary targets with very low probabilities can be screened out, thereby increasing the possibility that the preliminary target is the final target.

[0099] See also Figure 3 Based on the preliminary target recognition information, the feature pyramid network and attention mechanism are used to fuse multi-scale features and enhance the recognition ability of multi-scale targets. The specific steps for generating multi-scale target feature information are as follows:

[0100] S201: Based on the preliminary target recognition information, a feature pyramid network is used to achieve fusion between feature layers, enhance the detection capability of multi-scale targets, and generate a fusion feature map;

[0101] S202: Based on the fused feature map, an attention mechanism is used to enhance the expressiveness of the features and generate an attention-weighted feature map;

[0102] S203: Based on the attention weighted feature map, an anchor frame optimization algorithm is used to optimize and adjust the anchor frame, improve the fitting accuracy of the model for multi-scale targets, and generate optimized anchor frame feature information;

[0103] S204: Based on the optimized anchor frame feature information, features are refined through a cascaded convolutional network to enhance the representation of multi-scale features and generate multi-scale target feature information.

[0104] First, a feature pyramid network is constructed based on the preliminary target recognition information. Feature pyramid is a multi-scale feature representation method, which usually includes feature maps of multiple resolutions. A convolutional neural network (CNN) is used to extract features of each resolution and ensure that these features have the same semantic information. In this way, feature representations of different scales can be obtained, thereby enhancing the detection ability of multi-scale targets. The generated fused feature map will contain information from different scales.

[0105] Based on the fused feature map, an attention mechanism is introduced to enhance the expressiveness of features. Through the attention mechanism, the model can automatically focus on the most important parts for target recognition. The attention weights for each position are calculated, and then these weights are applied to the fused feature map to generate an attention-weighted feature map. This helps the model better focus on key areas and improves the recognition ability of multi-scale targets.

[0106] The generated attention-weighted feature map is optimized using the anchor box optimization algorithm. Anchor boxes are predefined bounding boxes used for object detection, and their size and position are critical for objects of different scales. By optimizing the anchor boxes, the model's fitting accuracy for multi-scale objects can be improved. This step involves adjusting the scale and position of the anchor boxes to better adapt them to the scales and shapes of different objects.

[0107] Based on optimizing the anchor box feature information, a cascaded convolutional network is introduced to further refine the features and enhance the representation ability of multi-scale features. The cascaded convolutional network is a stack of a series of convolutional layers to capture higher-level semantic information. This helps to further improve the recognition performance of multi-scale objects. The generated multi-scale object feature information will include information from each step, providing a more comprehensive object representation.

[0108] See also Figure 4 Based on the multi-scale target feature information, the scale estimation module and anchor frame mechanism are used to perform adaptive scale adjustment, optimize the size of the target tracking frame, and generate the tracking information after adaptive scale adjustment as follows:

[0109] S301: Based on multi-scale target feature information, a real-time scale estimation algorithm is used to dynamically predict the target size, adapt to changes in the target scale, and generate scale adjustment information;

[0110] S302: Based on the scale adjustment information, an anchor frame generation algorithm is used to adjust the scale of the tracking frame so that the tracking frame fits the actual scale of the target, and a tracking frame with an adaptive scale is generated;

[0111] S303: Based on the adaptive scale tracking frame, a non-maximum suppression algorithm is used to eliminate overlapping tracking frames, reduce redundancy, and generate a non-maximum suppressed tracking frame;

[0112] S304: Based on the tracking frame after non-maximum suppression, the final target tracking frame is located in combination with the tracking learning algorithm to generate tracking information after adaptive scale adjustment.

[0113] Based on the multi-scale target feature information, a real-time scale estimation algorithm is used to dynamically predict the target size. The algorithm can predict the size change of the target in the next frame based on the target feature information of the current frame and the target feature information of the historical frame. Through this step, scale adjustment information can be generated for subsequent adaptive scale adjustment.

[0114] Based on the generated scale adjustment information, an anchor box generation algorithm is used to adjust the scale of the tracking box. The algorithm adjusts the scale of the tracking box to a size that matches the actual scale of the target based on the scale adjustment information. Through this step, an adaptive scale tracking box can be generated to better adapt to changes in the target scale.

[0115] Based on the generated adaptive scale tracking frame, the non-maximum suppression algorithm is used to eliminate overlapping tracking frames. The algorithm compares the confidence of different tracking frames, retains the tracking frame with the highest confidence, and suppresses other overlapping tracking frames. Through this step, redundant tracking frames can be reduced and the accuracy and efficiency of tracking can be improved.

[0116] Based on the tracking frame after non-maximum suppression, the final target tracking frame is located in combination with the tracking learning algorithm. The algorithm uses the target position information and motion information between the previous and next frames to further optimize and locate the tracking frame. Through this step, adaptively scaled tracking information can be generated, including key information such as the target's position, size, and shape.

[0117] See also Figure 5 Based on the tracking information after adaptive scale adjustment, the mask area convolutional neural network and monocular depth estimation algorithm are used to deal with the target occlusion problem, obtain the occlusion relationship between targets and the scene depth information, and the steps of generating the target occlusion relationship and depth information are as follows:

[0118] S401: Based on the tracking information after adaptive scale adjustment, a mask region convolutional neural network enhanced by a feature pyramid network is used to perform target instance segmentation, and occlusion boundaries are accurately determined to generate target segmentation and occlusion boundary information;

[0119] S402: Based on the target segmentation and occlusion boundary information, a monocular depth estimation method based on deep learning is used to perform three-dimensional reconstruction of the scene and generate a scene depth map;

[0120] S403: Based on the scene depth map, pixel-level fusion technology is used to analyze the occlusion relationship between targets and generate a target occlusion relationship map;

[0121] S404: Based on the target occlusion relationship graph, combined with the depth sorting algorithm and the occlusion processing technology, the occlusion level and depth between the targets are determined, and the target occlusion relationship and depth information are generated.

[0122] Based on the tracking information after adaptive scale adjustment, the mask region convolutional neural network enhanced by the feature pyramid network is used to segment the target instance. This algorithm can accurately determine the boundary between the target and the background and generate target segmentation and occlusion boundary information. Through this step, the target can be accurately segmented and the location of the occlusion boundary can be determined.

[0123] Based on the generated target segmentation and occlusion boundary information, the scene is reconstructed in 3D using a deep learning monocular depth estimation method. This method can infer the depth information of objects in the scene by analyzing the pixel information in the image and generate a scene depth map. Through this step, the relative position and depth information of each object in the scene can be obtained.

[0124] Based on the generated scene depth map, the pixel-level fusion technology is used to analyze the occlusion relationship between targets. This technology can determine the occlusion relationship between targets based on the depth information in the scene depth map and generate a target occlusion relationship map. Through this step, the occlusion situation between targets can be accurately described.

[0125] Based on the generated target occlusion relationship graph, the depth sorting algorithm and occlusion processing technology are combined to determine the occlusion level and depth between targets. The algorithm can sort the targets according to the information in the target occlusion relationship graph and determine the occlusion level and depth of each target. Through this step, a detailed description of the occlusion relationship between targets and the scene depth information can be obtained.

[0126] See also Figure 6 Based on the target occlusion relationship and depth information, the graph convolutional neural network is used to integrate the data from multiple camera perspectives, and the spatiotemporal correlation of the target in the multi-camera field of view is established. The specific steps for generating cross-camera target tracking information are as follows:

[0127] S501: Based on the target occlusion relationship and depth information, a graph convolutional neural network is applied, combined with feature extraction and optimization strategies to integrate multi-view data and generate preliminary cross-camera target association information;

[0128] S502: Based on the preliminary cross-camera target association information, a multi-camera geometric calibration method is used to refine the target position relationship through a calibration technology based on epipolar constraints to generate multi-view target position information;

[0129] S503: Based on the multi-view target position information, using time series analysis technology, performing time series data analysis through dynamic time warping, optimizing the target spatiotemporal association, and generating time-optimized target association information;

[0130] S504: Based on the time-optimized target association information, the multi-camera data is integrated, and the spatiotemporal association is completed through a multi-view tracking fusion algorithm to generate cross-camera target tracking information.

[0131] Based on the target occlusion relationship and depth information, a graph convolutional neural network is used in combination with feature extraction and optimization strategies to integrate multi-view data. This algorithm can fuse the view data from different cameras and generate preliminary cross-camera target association information. Through this step, the view information of multiple cameras can be fully utilized to improve the accuracy and robustness of target tracking.

[0132] Based on the generated preliminary cross-camera target association information, a multi-camera geometric calibration method is used to refine the target position relationship through the calibration technology based on epipolar constraints to generate multi-view target position information. This method can further improve the accuracy and consistency of target position by calibrating the geometric relationship between multiple cameras.

[0133] Based on the generated multi-view target position information, time series analysis technology is used to analyze time series data through dynamic time warping, optimize the target time-space association, and generate time-optimized target association information. The algorithm can model and predict the target's motion trajectory based on the information in the time dimension, further improving the accuracy and stability of target tracking.

[0134] Based on the generated time-optimized target association information, the multi-camera data is integrated, and the spatiotemporal association is completed through the multi-view tracking fusion algorithm to generate cross-camera target tracking information. This algorithm can fuse the target association information from different cameras to obtain the final cross-camera target tracking result. Through this step, global tracking and association analysis of targets under multiple cameras can be achieved.

[0135] See also Figure 7 Based on the cross-camera target tracking information, the target pattern is learned by combining the temporal convolutional neural network and the long short-term memory neural network, and the abnormal behavior is identified and warned using a single-class support vector machine. The specific steps for generating abnormal behavior identification and warning reports are as follows:

[0136] S601: Based on the cross-camera target tracking information, deep feature learning is performed by combining the temporal convolutional neural network and the long short-term memory network to generate target behavior feature learning results;

[0137] S602: Based on the target behavior feature learning result, a feature clustering algorithm is used to perform spatiotemporal feature analysis, extract the target behavior pattern, and generate spatiotemporal clustered behavior features;

[0138] S603: Based on the behavior characteristics after spatiotemporal clustering, a single-class support vector machine is used to identify and model abnormal behaviors to generate an abnormal behavior identification model;

[0139] S604: Based on the abnormal behavior recognition model and in combination with real-time monitoring data, abnormal behavior warning is performed to generate an abnormal behavior recognition and warning report.

[0140] In this embodiment, based on the above abnormal behavior recognition model, the monitoring data will be analyzed and classified, and the corresponding abnormal behavior comprehensive value will be calculated by calculating the abnormal index corresponding to each type of abnormal behavior. Specifically, the calculation formula of the abnormal behavior comprehensive value is expressed as:

[0141] ;

[0142] in, Represents the comprehensive value of abnormal behavior, The base value representing the comprehensive value of abnormal behavior, Indicates The weight factor of abnormal behavior, Indicates Calibration factor for abnormal behavior, Indicates The abnormality index of abnormal behavior, Indicates The baseline anomaly index of abnormal behavior, The maximum number of categories representing anomalous behavior.

[0143] It can be understood that when the calculated comprehensive value of abnormal behavior is greater than the preset abnormal behavior threshold, an early warning can be triggered to assist in discovering dangerous situations as early as possible and making emergency plans in a timely manner.

[0144] Based on cross-camera target tracking information, deep feature learning is performed by combining temporal convolutional neural networks and long short-term memory networks. This algorithm can learn and extract the target's behavior pattern and generate target behavior feature learning results. Through this step, the target's behavior characteristics in different time and space can be obtained, providing a basis for subsequent abnormal behavior identification.

[0145] Based on the generated target behavior feature learning results, a feature clustering algorithm is used to perform spatiotemporal feature analysis, extract target behavior patterns, and generate spatiotemporal clustered behavior features. This algorithm can cluster targets with similar behavior patterns to obtain target behavior features of different categories. Through this step, the target behavior features can be further refined to improve the accuracy of abnormal behavior recognition.

[0146] Based on the generated spatiotemporal clustering behavior features, a single-class support vector machine is used to identify and model abnormal behaviors and generate an abnormal behavior recognition model. This algorithm can distinguish between normal and abnormal behaviors by training a binary classifier and generate a corresponding abnormal behavior recognition model. Through this step, automatic identification and early warning of abnormal behaviors can be achieved.

[0147] Based on the generated abnormal behavior recognition model, combined with real-time monitoring data, abnormal behavior warning is carried out, and abnormal behavior recognition and warning reports are generated. The algorithm can analyze and predict the target's behavior in real time based on real-time monitoring data. Once abnormal behavior is found, a warning signal can be issued and a corresponding abnormal behavior recognition and warning report can be generated.

[0148] See also Figure 8 ,Smart city camera multi-target tracking system,The smart city camera multi-target tracking system is used to execute the above-mentioned smart city camera multi-target tracking method. The system includes an image preprocessing module, a basic feature extraction module, a preliminary target recognition module, a feature enhancement and optimization module, a target tracking frame preprocessing module, a target association module, and a behavior learning and early warning module.

[0149] The image preprocessing module uses histogram equalization and Gaussian blur algorithm to perform image enhancement and noise reduction based on the original surveillance image to generate an enhanced image;

[0150] The basic feature extraction module extracts basic features based on the enhanced image, adopts the depth-separable convolution algorithm to reduce the amount of calculation and maintain performance, and generates a basic feature map;

[0151] The preliminary target recognition module performs preliminary target recognition based on the basic feature map, uses multi-scale spatial pyramid pooling to enhance scale-invariant feature extraction, and uses a region proposal network combined with bounding box regression to locate the target and generate preliminary target recognition information;

[0152] The feature enhancement and optimization module performs feature enhancement and optimization based on the preliminary target recognition information. It uses the feature pyramid network to achieve the fusion between feature layers and adopts the attention mechanism to enhance the expressiveness of features. It optimizes the frame by using the anchor frame optimization algorithm and extracts features through the cascaded convolutional network to generate multi-scale target feature information.

[0153] The target tracking frame preprocessing module processes the tracking frame based on multi-scale target feature information, uses a real-time scale estimation algorithm to dynamically predict the target size, adjusts the tracking frame scale through the anchor frame generation algorithm, and combines the non-maximum suppression algorithm to eliminate overlapping tracking frames, generating tracking information after adaptive scale adjustment;

[0154] The target association module runs the target association algorithm based on the tracking information after adaptive scale adjustment, uses the mask area convolutional neural network enhanced by the feature pyramid network to perform target instance segmentation, reconstructs the scene in three dimensions through the monocular depth estimation method based on deep learning, uses pixel-level fusion technology to process, and determines the occlusion level and depth between targets through the depth sorting algorithm and occlusion processing technology, and generates target occlusion relationship and depth information;

[0155] The behavior learning and warning module performs behavior learning and warning based on target occlusion relationships and depth information, uses temporal convolutional neural networks and long short-term memory networks for deep feature learning, performs spatiotemporal feature analysis through feature clustering algorithms, and uses single-class support vector machines to identify and model abnormal behaviors. It combines real-time monitoring data to provide abnormal behavior warnings and generate abnormal behavior identification and warning reports.

[0156] The image preprocessing module uses histogram equalization and Gaussian blur algorithms for image enhancement and noise reduction, which can effectively improve the accuracy and stability of subsequent feature extraction. The basic feature extraction module uses a deep separable convolution algorithm to reduce the amount of calculation while maintaining performance and improving the efficiency of feature extraction. The preliminary target recognition module uses a multi-scale spatial pyramid pooling and region proposal network combined with a bounding box regression method to accurately locate and identify targets, providing a reliable foundation for subsequent target tracking.

[0157] In addition, the feature enhancement and optimization module uses a feature pyramid network to achieve fusion between feature layers and introduces an attention mechanism to enhance the expressiveness of features, further improving the accuracy of target recognition.

[0158] The target tracking frame preprocessing module uses a real-time scale estimation algorithm to dynamically predict the target size, and adjusts the tracking frame scale through the anchor frame generation algorithm. It combines the non-maximum suppression algorithm to eliminate overlapping tracking frames, and realizes the generation of tracking information after adaptive scale adjustment.

[0159] The target association module uses a mask area convolutional neural network enhanced by a feature pyramid network to perform target instance segmentation, reconstructs the scene in three dimensions through a deep learning monocular depth estimation method, processes it using pixel-level fusion technology, and determines the occlusion level and depth between targets through a depth sorting algorithm and occlusion processing technology, generating target occlusion relationships and depth information.

[0160] The behavior learning and warning module uses temporal convolutional neural networks and long short-term memory networks for deep feature learning, performs spatiotemporal feature analysis through feature clustering algorithms, uses single-class support vector machines to identify and model abnormal behaviors, combines real-time monitoring data for abnormal behavior warnings, and generates abnormal behavior identification and warning reports.

[0161] See also Fig. 9 ,The image preprocessing module includes a noise reduction submodule, an image enhancement submodule, an anomaly filtering submodule, and an image normalization submodule;

[0162] The basic feature extraction module includes a deep convolution submodule, a feature extraction submodule, a feature compression submodule, and a feature noise screening submodule;

[0163] The preliminary target recognition module includes a target positioning submodule, a target detection submodule, a target screening submodule, and a target confirmation submodule;

[0164] The feature enhancement and optimization module includes feature fusion submodule, feature attention weighting submodule, anchor box optimization submodule, and feature extraction submodule;

[0165] The target tracking frame preprocessing module includes a real-time scale estimation submodule, an anchor frame generation submodule, a non-maximum suppression submodule, and a target tracking frame positioning submodule;

[0166] The target association module includes a target instance segmentation submodule, a scene 3D reconstruction submodule, a target occlusion relationship analysis submodule, and an occlusion level and depth determination submodule;

[0167] The behavior learning and warning module includes a deep feature learning submodule, a spatiotemporal feature analysis submodule, an abnormal behavior recognition submodule, and a real-time abnormal warning submodule.

[0168] In the image preprocessing module, the noise reduction submodule uses the Gaussian blur algorithm to reduce the noise of the original monitoring image; the image enhancement submodule uses the histogram equalization algorithm to enhance the image; the abnormal filtering submodule is used to remove abnormal values ​​in the image; and the image normalization submodule standardizes the image data.

[0169] In the basic feature extraction module, the deep convolution submodule uses the deep separable convolution algorithm to extract basic features; the feature extraction submodule is used to extract key features from the enhanced image; the feature compression submodule reduces the amount of calculation by reducing the feature dimension; and the feature noise screening submodule is used to screen out effective features.

[0170] In the preliminary target recognition module, the target localization submodule uses a region proposal network combined with bounding box regression to locate the target; the target detection submodule is used to detect multiple targets; the target screening submodule screens out reliable targets based on preset thresholds and confidence levels; and the target confirmation submodule is used to confirm the identity information of the target.

[0171] In the feature enhancement and optimization module, the feature fusion submodule uses the feature pyramid network to achieve fusion between feature layers; the feature attention weighting submodule enhances the expressiveness of features by introducing the attention mechanism; the anchor box optimization submodule optimizes the target position estimation by adjusting the tracking box scale; the feature extraction submodule further refines features through a cascaded convolutional network.

[0172] In the target tracking frame preprocessing module, the real-time scale estimation submodule uses a real-time scale estimation algorithm to dynamically predict the target size; the anchor frame generation submodule adjusts the tracking frame scale through the anchor frame generation algorithm; the non-maximum suppression submodule is used to eliminate overlapping tracking frames; and the target tracking frame positioning submodule is used to determine the position of the tracking frame.

[0173] In the target association module, the target instance segmentation submodule uses the mask area convolutional neural network enhanced by the feature pyramid network to perform target instance segmentation; the scene 3D reconstruction submodule reconstructs the scene in 3D through the monocular depth estimation method based on deep learning; the target occlusion relationship analysis submodule uses pixel-level fusion technology to process, and determines the occlusion level and depth between targets through the depth sorting algorithm and occlusion processing technology; the occlusion level and depth determination submodule is used to determine the occlusion relationship and depth information between targets.

[0174] In the behavior learning and warning module, the deep feature learning submodule uses temporal convolutional neural networks and long short-term memory networks for deep feature learning; the spatiotemporal feature analysis submodule performs spatiotemporal feature analysis through feature clustering algorithm; the abnormal behavior recognition submodule uses a single-class support vector machine to identify and model abnormal behaviors; the real-time abnormal warning submodule combines real-time monitoring data to warn of abnormal behaviors and generates abnormal behavior recognition and warning reports.

[0175] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A multi-target tracking method for smart city cameras, characterized in that: The following steps are involved: Based on the surveillance images, a deep separation convolutional neural network and scale-invariant feature transformation are used for image processing and preliminary target recognition, and preliminary target detection is performed to generate preliminary target recognition information; Based on the preliminary target recognition information, a feature pyramid network and an attention mechanism are used to fuse multi-scale features, and the recognition capability of multi-scale targets is enhanced to generate multi-scale target feature information; Based on the multi-scale target feature information, a scale estimation module and an anchor frame mechanism are used to perform adaptive scale adjustment, optimize the size of the target tracking frame, and generate tracking information after adaptive scale adjustment; Based on the tracking information after the adaptive scale adjustment, a mask area convolutional neural network and a monocular depth estimation algorithm are used to process the target occlusion problem, obtain the occlusion relationship between targets and the scene depth information, and generate the target occlusion relationship and depth information; Based on the target occlusion relationship and depth information, the data of multiple camera perspectives are integrated through a graph convolutional neural network, and the spatiotemporal correlation of the target in the field of view of multiple cameras is established to generate cross-camera target tracking information; Based on the cross-camera target tracking information, the target pattern is learned by combining the temporal convolutional neural network and the long short-term memory neural network, and the abnormal behavior is identified and warned by using a single-class support vector machine to generate an abnormal behavior identification and warning report; The preliminary target recognition information includes the position, size and shape information of the target. The multi-scale target feature information is specifically a multi-level, multi-scale target feature description, including the details and structural information of the target at multiple scales. The tracking information after adaptive scale adjustment is specifically the tracking frame and feature information of the matching target at multiple scales. The target occlusion relationship and depth information specifically refers to the recognition information of the occluded target and the depth position of the target in the scene. The cross-camera target tracking information includes the position, time association and behavior information of key targets between multiple cameras. The abnormal behavior recognition and warning report is specifically the abnormal behavior discovered, and its time, location and nature are described and warned.

2. The smart city camera multi-target tracking method according to claim 1 is characterized in that: Based on the surveillance image, a deep separation convolutional neural network and scale-invariant feature transformation are used for image processing and preliminary target recognition, and preliminary detection of the target is performed. The specific steps for generating preliminary target recognition information are as follows: Based on the original surveillance image, histogram equalization and Gaussian blur algorithm are used to perform image enhancement and noise reduction to generate an enhanced image; Based on the enhanced image, a depthwise separable convolution algorithm is used to extract basic features, reduce the amount of calculation and maintain performance, and generate a basic feature map; Based on the basic feature map, multi-scale spatial pyramid pooling is used to enhance scale-invariant feature extraction to generate a vector feature map; Based on the vector feature map, a region proposal network combined with bounding box regression is used to perform preliminary detection of the target, accurately locate the target, and generate preliminary target recognition information.

3. The smart city camera multi-target tracking method according to claim 1, characterized in that: Based on the preliminary target recognition information, a feature pyramid network and an attention mechanism are used to fuse multi-scale features and enhance the recognition capability of multi-scale targets. The steps for generating multi-scale target feature information are as follows: Based on the preliminary target recognition information, a feature pyramid network is used to achieve fusion between feature layers, enhance the detection capability of multi-scale targets, and generate a fusion feature map; Based on the fused feature map, an attention mechanism is adopted to enhance the expressiveness of features and generate an attention-weighted feature map; Based on the attention weighted feature map, an anchor frame optimization algorithm is used to optimize and adjust the anchor frame, improve the fitting accuracy of the model for multi-scale targets, and generate optimized anchor frame feature information; Based on the optimized anchor frame feature information, features are refined through a cascaded convolutional network, the representation power of multi-scale features is enhanced, and multi-scale target feature information is generated.

4. The smart city camera multi-target tracking method according to claim 1, characterized in that: Based on the multi-scale target feature information, the scale estimation module and the anchor frame mechanism are used to perform adaptive scale adjustment, optimize the size of the target tracking frame, and generate the tracking information after adaptive scale adjustment in the following steps: Based on the multi-scale target feature information, a real-time scale estimation algorithm is used to dynamically predict the target size, adapt to changes in the target scale, and generate scale adjustment information; Based on the scale adjustment information, an anchor frame generation algorithm is used to adjust the scale of the tracking frame so that the tracking frame fits the actual scale of the target, thereby generating a tracking frame with an adaptive scale; Based on the tracking frame of the adaptive scale, a non-maximum suppression algorithm is used to eliminate overlapping tracking frames, reduce redundancy, and generate a non-maximum suppressed tracking frame; Based on the non-maximum suppressed tracking frame, the final target tracking frame is positioned in combination with a tracking learning algorithm to generate tracking information after adaptive scale adjustment.

5. The smart city camera multi-target tracking method according to claim 1, characterized in that: Based on the tracking information after the adaptive scale adjustment, the mask area convolutional neural network and the monocular depth estimation algorithm are used to process the target occlusion problem, obtain the occlusion relationship between targets and the scene depth information, and generate the target occlusion relationship and depth information in the following steps: Based on the tracking information after the adaptive scale adjustment, a mask region convolutional neural network enhanced by a feature pyramid network is used to perform target instance segmentation, and occlusion boundaries are accurately determined to generate target segmentation and occlusion boundary information; Based on the target segmentation and occlusion boundary information, a monocular depth estimation method based on deep learning is used to perform three-dimensional reconstruction of the scene to generate a scene depth map; Based on the scene depth map, pixel-level fusion technology is used to analyze the occlusion relationship between targets and generate a target occlusion relationship map; Based on the target occlusion relationship graph, combined with the depth sorting algorithm and occlusion processing technology, the occlusion level and depth between targets are determined, and the target occlusion relationship and depth information are generated.

6. The smart city camera multi-target tracking method according to claim 1, characterized in that: Based on the target occlusion relationship and depth information, the data of multiple camera perspectives are integrated through a graph convolutional neural network, and the spatiotemporal association of the target in the multi-camera field of view is established. The steps of generating cross-camera target tracking information are as follows: Based on the target occlusion relationship and depth information, a graph convolutional neural network is applied to integrate multi-view data in combination with feature extraction and optimization strategies to generate preliminary cross-camera target association information; Based on the preliminary cross-camera target association information, a multi-camera geometric calibration method is adopted to refine the target position relationship through a calibration technology based on epipolar constraints to generate multi-view target position information; Based on the multi-view target position information, using time series analysis technology, performing time series data analysis through dynamic time warping, optimizing target spatiotemporal association, and generating time-optimized target association information; Based on the time-optimized target association information, multi-camera data is integrated, and spatiotemporal association is completed through a multi-view tracking fusion algorithm to generate cross-camera target tracking information.

7. The smart city camera multi-target tracking method according to claim 1, characterized in that: Based on the cross-camera target tracking information, the target pattern is learned by combining the temporal convolutional neural network and the long short-term memory neural network, and the abnormal behavior is identified and warned by using a single-class support vector machine. The steps of generating the abnormal behavior identification and warning report are as follows: Based on the cross-camera target tracking information, deep feature learning is performed by combining a temporal convolutional neural network and a long short-term memory neural network to generate a target behavior feature learning result; Based on the target behavior feature learning result, a feature clustering algorithm is used to perform spatiotemporal feature analysis, extract the target behavior pattern, and generate spatiotemporal clustered behavior features; Based on the behavior characteristics after the spatiotemporal clustering, a single-class support vector machine is used to identify and model abnormal behaviors to generate an abnormal behavior identification model; Based on the abnormal behavior recognition model and combined with real-time monitoring data, abnormal behavior warning is performed to generate abnormal behavior recognition and warning reports.

8. Smart city camera multi-target tracking system, characterized by: The system is used to execute the smart city camera multi-target tracking method described in any one of claims 1-7, and the system includes an image preprocessing module, a basic feature extraction module, a preliminary target recognition module, a feature enhancement and optimization module, a target tracking frame preprocessing module, a target association module, and a behavior learning and early warning module.

9. The smart city camera multi-target tracking system according to claim 8, characterized in that: The image preprocessing module uses histogram equalization and Gaussian blur algorithm to perform image enhancement and noise reduction based on the original monitoring image to generate an enhanced image; The basic feature extraction module extracts basic features based on the enhanced image, adopts a depth-separable convolution algorithm to reduce the amount of calculation and maintain performance, and generates a basic feature map; The preliminary target recognition module performs preliminary target recognition based on the basic feature map, adopts multi-scale spatial pyramid pooling to enhance scale-invariant feature extraction, and adopts a region proposal network combined with bounding box regression to locate the target and generate preliminary target recognition information; The feature enhancement and optimization module performs feature enhancement and optimization based on preliminary target recognition information, uses a feature pyramid network to achieve fusion between feature layers and adopts an attention mechanism to enhance the expressiveness of features, optimizes and adjusts the frame by using an anchor frame optimization algorithm, and extracts features through a cascaded convolutional network to generate multi-scale target feature information; The target tracking frame preprocessing module processes the tracking frame based on multi-scale target feature information, uses a real-time scale estimation algorithm to dynamically predict the target size, adjusts the tracking frame scale through an anchor frame generation algorithm, and combines a non-maximum suppression algorithm to eliminate overlapping tracking frames, and generates tracking information after adaptive scale adjustment; The target association module runs a target association algorithm based on the tracking information after adaptive scale adjustment, uses a mask area convolutional neural network enhanced by a feature pyramid network to perform target instance segmentation, reconstructs the scene in three dimensions through a deep learning monocular depth estimation method, uses pixel-level fusion technology for processing, and determines the occlusion level and depth between targets through a depth sorting algorithm and occlusion processing technology, and generates target occlusion relationship and depth information; The behavior learning and warning module performs behavior learning and warning based on target occlusion relationship and depth information, uses temporal convolutional neural network and long short-term memory neural network for deep feature learning, performs spatiotemporal feature analysis through feature clustering algorithm, uses single-class support vector machine to identify and model abnormal behavior, combines real-time monitoring data for abnormal behavior warning, and generates abnormal behavior identification and warning report.

10. The smart city camera multi-target tracking system according to claim 8, characterized in that: The image preprocessing module includes a noise reduction submodule, an image enhancement submodule, an abnormality filtering submodule, and an image normalization submodule; The basic feature extraction module includes a deep convolution submodule, a feature extraction submodule, a feature compression submodule, and a feature noise screening submodule; The preliminary target recognition module includes a target positioning submodule, a target detection submodule, a target screening submodule, and a target confirmation submodule; The feature enhancement and optimization module It includes feature fusion submodule, feature attention weighting submodule, anchor box optimization submodule, and feature extraction submodule; The target tracking frame preprocessing module includes a real-time scale estimation submodule, an anchor frame generation submodule, a non-maximum suppression submodule, and a target tracking frame positioning submodule; The target association module includes a target instance segmentation submodule, a scene 3D reconstruction submodule, a target occlusion relationship analysis submodule, and an occlusion level and depth determination submodule; The behavior learning and warning module includes a deep feature learning submodule, a spatiotemporal feature analysis submodule, an abnormal behavior recognition submodule, and a real-time abnormal warning submodule.