Monitoring video dynamic target tracking method and system coupled with geographic space information

Abstract

The invention relates to the technical field of target tracking and positioning, in particular to a monitoring video dynamic target tracking method and system coupled with geographic space information, and the method comprises the steps: carrying out the perspective transformation of a monitoring video frame sequence, so as to build a mapping relation model for the conversion from an image space to a geographic space; the perspective transformation processing is used for establishing a mapping relation model by performing orthographic perspective transformation on a monitoring video image; and detecting and tracking the target in the monitoring video, extracting a target movement track, mapping the target movement track to the geographic space according to the mapping relation model so as to determine the coordinate of the target in the geographic space, and visually displaying the target movement track in the geographic space. According to the method, geographic space information is fully combined, the motion trail of the target object can be accurately tracked and mapped in the geographic space of the real world, monitoring personnel can conveniently carry out geographic space-time analysis on the target, and the integration effect of the monitoring video and the geographic information is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking and positioning, and in particular to a monitoring video dynamic target tracking method and system coupled with geographic space information. Background Art

[0002] With the advancement of digitalization and intelligence in cities, monitoring equipment has been widely deployed in various places in cities, playing a vital role in public security protection, crime prevention and detection, accident warning and post-event handling. Video surveillance systems have become an indispensable tool in social security management, and various monitoring systems generate massive amounts of surveillance video data every day. The traditional way of managing surveillance videos is for monitoring personnel to observe and analyze on the display screen, which is time-consuming and labor-intensive. This method not only requires high requirements for monitoring personnel, but also requires them to be familiar with the geographical scenes of each monitoring area. When the scale of video data generated by the monitoring system is greater than the processing capacity of the monitoring personnel, it will make it difficult for monitoring personnel to detect abnormal situations in time, and important events or information may be ignored or missed, resulting in serious consequences such as waste of resources, false or missed information, and information overload. Therefore, combining surveillance videos with actual geographic spatial information and exploring the expression of geographic hypermedia has become a new direction of current research.

[0003] In recent years, with the rapid development of computer vision algorithms such as deep learning and machine learning, video intelligent processing has become a research hotspot in the field of computer vision. However, current research on video mainly focuses on improving the accuracy of target detection or tracking, targeting only the video image itself, ignoring the importance of geospatial information in video surveillance, and the degree of coupling analysis with geospatial information is low. The spatial information of the video surveillance area is not fully utilized, and it is difficult to meet the monitoring personnel's needs for the real geographical location, movement direction and movement trajectory information when analyzing the target of interest. The coupling analysis of geospatial information and video information can fully grasp the security situation of the monitoring area based on the information such as the geographical environment around the target, and help the monitoring personnel make decisions in time, thereby overcoming the limitations of simple monitoring video images. However, the existing fusion of monitoring video and geographic information only uses a unified geographic reference system to integrate the location information of the monitoring camera into the geographic information framework, and associates the monitoring cameras in a specific area with each other. Its role is more inclined to provide schematic geographic information, and the integration effect is poor; or the internal and external parameters of the camera are calculated, and the camera video image is simply superimposed on the geographic base map through transformation, but this will cause the video content to overlap with the geographic base map information, resulting in visual confusion and information redundancy, and unable to highlight key targets and information. In general, current research focuses on combining surveillance video with geographic information using geographic information as supplementary information, with the emphasis on video display, while access to geospatial data is limited to the current camera view, making it impossible to conduct comprehensive analysis and decision support for the target of interest and the surrounding larger area facilities. Summary of the invention

[0004] To this end, the present invention provides a monitoring video dynamic target tracking method and system coupled with geographic space information, which solves the problems of low coupling between existing monitoring video and geographic space information and inability to fully combine geographic space information.

[0005] According to the design scheme provided by the present invention, on the one hand, a monitoring video dynamic target tracking method coupled with geographic space information is provided, comprising:

[0006] Performing perspective transformation processing on the surveillance video frame sequence to establish a mapping relationship model for converting from image space to geographic space, wherein the perspective transformation processing is used to establish the mapping relationship model by performing orthographic perspective transformation on the surveillance video image;

[0007] Detect and track targets in surveillance videos and extract target motion trajectories. Map target motion trajectories to geographic space based on a mapping relationship model to determine the coordinates of the target in geographic space, and visualize the target motion trajectory in geographic space.

[0008] As a monitoring video dynamic target tracking method coupled with geographic spatial information of the present invention, further, perspective transformation processing is performed on the monitoring video frame sequence, comprising:

[0009] The perspective transformation projection relationship is represented by using the two-dimensional coordinates of the pixels of the surveillance video image and the corresponding homogeneous coordinates and mapping parameters, wherein the mapping parameters include: image linear transformation parameters, image translation parameters, image perspective transformation parameters and constant parameters;

[0010] The perspective transformation projection relationship is used to obtain the homogeneous coordinates of the pixel points of the surveillance video image after the perspective transformation, and the two-dimensional plane coordinates of the pixel points of the surveillance video image after the perspective transformation are obtained through normalization processing;

[0011] Assign values ​​to constant parameters, and obtain the perspective transformation expression of image pixels according to the two-dimensional plane coordinates after perspective transformation; solve the perspective transformation expression using the four pairs of corresponding points specified before and after perspective transformation in the surveillance video image, and obtain the mapping matrix that transforms the image pixels to the new viewing plane according to the solution result, and use the mapping matrix as a mapping relationship model.

[0012] As a monitoring video dynamic target tracking method coupled with geographic spatial information of the present invention, further, the two-dimensional plane coordinates of the monitoring video image pixel points after perspective transformation are obtained by normalization processing, including:

[0013] Normalize the Z in the homogeneous coordinates (X, Y, Z) of the surveillance video image pixel point to obtain the normalized homogeneous coordinates (X', Y', 1) of the surveillance video image pixel point, where X' = X / Z and Y' = Y / Z;

[0014] According to the normalized homogeneous coordinates (X', Y', 1) of the pixel point of the surveillance video image, the two-dimensional plane coordinates (X', Y') of the corresponding pixel point after perspective transformation are obtained.

[0015] As a monitoring video dynamic target tracking method coupled with geographic spatial information of the present invention, further, detecting and tracking the target in the monitoring video and extracting the target motion trajectory include:

[0016] Detect and identify the target in the monitoring identification using the pre-trained target recognition detection model, and output the target detection and recognition result, which includes the target detection frame, target category and target confidence;

[0017] The target detection and recognition results are input into the pre-trained multi-target tracker to track the target and extract the target motion trajectory using the multi-target tracker.

[0018] As a surveillance video dynamic target tracking method coupled with geographic spatial information of the present invention, further, the target recognition detection model is constructed using a yolov5 network, and the multi-target tracker is constructed using a DeepSort algorithm.

[0019] As a monitoring video dynamic target tracking method coupled with geographic spatial information of the present invention, further, a multi-target tracker is used to track the target and extract the target motion trajectory, comprising:

[0020] The Kalman filter is used to predict the target motion state and obtain the target motion characteristics, wherein the target motion characteristics include target position description information and target state description information;

[0021] Based on the target motion features, target appearance features and target motion Mahalanobis distance, the Hungarian algorithm is used to perform cascade matching and IOU matching on the tracking target and the detection frame in the current video frame, and then updated through the Kalman filter to obtain the position and identity information of the tracking target in the video frame;

[0022] The target trajectory node point set is obtained according to the pixel coordinates of the target in the video frame, and the target motion trajectory is drawn according to the trajectory node point set.

[0023] As a monitoring video dynamic target tracking method coupled with geographic space information of the present invention, further, the target motion trajectory in the geographic space is visualized and displayed, including:

[0024] The target tracking visualization monitoring platform is used to visualize the movement trajectory of the tracked target. The target tracking visualization monitoring platform adopts a B / S architecture and provides front-end display and back-end management of integrated monitoring video and two-dimensional map based on the star map online map and monitoring video.

[0025] In another aspect, the present invention further provides a monitoring video dynamic target tracking system coupled with geographic space information, comprising: a video conversion module and a target tracking module, wherein:

[0026] A video transformation module, used to perform perspective transformation processing on a surveillance video frame sequence to establish a mapping relationship model for conversion from image space to geographic space, wherein the perspective transformation processing is used to establish a mapping relationship model by performing orthographic perspective transformation on the surveillance video image;

[0027] The target tracking module is used to detect and track targets in surveillance videos and extract target motion trajectories, map target motion trajectories to geographic space based on a mapping relationship model to determine the coordinates of the target in geographic space, and visualize the target motion trajectory in geographic space.

[0028] Beneficial effects of the present invention:

[0029] The present invention uses the perspective transformation principle to establish a mapping relationship model, performs orthographic perspective transformation on the surveillance video shooting scene, and maps and matches it with the satellite image of the location, obtains the real geographic coordinates of the target in the image, and displays its motion trajectory on the map, thereby realizing a highly coupled expression of the surveillance video and geographic information; at the same time, an interactive surveillance video target tracking visualization system platform is used to map and visualize the motion trajectory of the target of interest in the system platform, which is convenient for monitoring personnel to perform geographic spatiotemporal analysis on the target, improves the integration effect of surveillance video and geographic information, and provides monitoring personnel with a more intuitive and convenient analysis tool, which has good application prospects in the field of dynamic target monitoring and identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the monitoring video dynamic target tracking process coupled with geographic space information in the embodiment;

[0031] Figure 2 This is a schematic diagram of the monitoring video dynamic target tracking algorithm principle in the embodiment;

[0032] Figure 3 This is a schematic diagram of the target recognition and tracking process based on YOLOv5+Deepsort in the embodiment;

[0033] Figure 4 Schematic diagram of the comparison of the monitoring video before and after the perspective transformation in the embodiment;

[0034] Figure 5 This is a schematic diagram of the monitoring video tracking result in the embodiment;

[0035] Figure 6 Schematic diagram of the target tracking trajectory visualization result in the embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and technical solutions.

[0037] Cross-view image matching and positioning refers to the technology of determining the geographic location of a ground-view query image by matching it with an aerial-view reference image with a geographic tag. Current cross-view image matching and positioning technologies mainly use CNNs with fixed receptive fields or Transformers with global modeling capabilities as feature extraction backbone networks. However, these methods fail to fully consider the scale differences between different features in the image, and due to the high number of network parameters and computational complexity, lightweight deployment faces significant challenges. To this end, the embodiments of the present invention, see Figure 1 As shown, a method for dynamic target tracking of surveillance video coupled with geographic spatial information is provided, comprising:

[0038] S101. Perform perspective transformation processing on a surveillance video frame sequence to establish a mapping relationship model for converting from image space to geographic space. The perspective transformation processing is used to establish a mapping relationship model by performing orthographic perspective transformation on surveillance video images.

[0039] Specifically, the surveillance video frame sequence is processed by perspective transformation, which can be designed to include:

[0040] The perspective transformation projection relationship is represented by using the two-dimensional coordinates of the pixels of the surveillance video image and the corresponding homogeneous coordinates and mapping parameters, wherein the mapping parameters include: image linear transformation parameters, image translation parameters, image perspective transformation parameters and constant parameters;

[0041] The perspective transformation projection relationship is used to obtain the homogeneous coordinates of the pixel points of the surveillance video image after the perspective transformation, and the two-dimensional plane coordinates of the pixel points of the surveillance video image after the perspective transformation are obtained through normalization processing;

[0042] Assign values ​​to constant parameters, and obtain the perspective transformation expression of image pixels according to the two-dimensional plane coordinates after perspective transformation; solve the perspective transformation expression using the four pairs of corresponding points specified before and after perspective transformation in the surveillance video image, and obtain the mapping matrix that transforms the image pixels to the new viewing plane according to the solution result, and use the mapping matrix as a mapping relationship model.

[0043] The images captured by surveillance videos map real geographic spatial entity information onto a two-dimensional image plane. Therefore, by establishing a mutual mapping model between two-dimensional images and geographic spatial coordinates, the conversion of image pixel coordinates to actual geographic coordinates can be achieved, and the targets of interest can be matched with their geographic locations. This helps to better understand and analyze events in surveillance videos and conduct geographic and spatiotemporal analysis of the targets of interest.

[0044] Due to the particularity of the camera, the angle of surveillance video shooting is fixed. We can use the fixed background information in the surveillance video to map with the real geographic space and establish a mapping relationship model, thereby establishing a one-to-one correspondence between any pixel point in the surveillance video and the actual geographic coordinates. On this basis, the actual geographic coordinates of the target of interest can be derived using the temporal dynamic information of the target in the video.

[0045] According to the approximate location of the camera, the surveillance video shooting scene is transformed in perspective to achieve mapping matching with the satellite image of the location to obtain the real geographic coordinates of the target in the image. At present, most image coordinates to geographic coordinates conversion uses the internal and external parameters of the camera to establish a conversion relationship, which requires complex and cumbersome steps such as camera calibration and pose estimation. However, in practical applications, although the surveillance video of the camera can be obtained, it is difficult to obtain its internal and external parameters. Therefore, in the embodiment of this case, the method of establishing a mapping relationship model using the perspective transformation principle does not need to obtain the internal and external parameters of the camera. It only needs to provide 4 pairs of points before and after the transformation to perform an orthographic perspective transformation on the image.

[0046] Suppose the coordinates of the pixel point on the original image are (x, y), and the corresponding homogeneous coordinates are (x, y, z); (X, Y, Z) are the homogeneous coordinates of the pixel point of the image after transformation. There is a perspective projection relationship as shown in the formula:

[0047]

[0048] in represents the linear transformation of the image, Used to generate image perspective transformation, T3 = [a 31 a 32 ] indicates image translation.

[0049] Normalize Z (X ′ =X / Z,Y ′ =Y / Z,Z ′ =Z / Z), and obtain the normalized homogeneous coordinates (X ′ ,Y ′ ,1), the two-dimensional plane coordinates corresponding to the homogeneous coordinates (X,Y,Z) of the transformed image pixel point are (X ′ ,Y ′ ), that is, when the homogeneous coordinate Z ′ =1, point (X ′ ,Y ′ ) is the two-dimensional plane coordinates of the pixel point corresponding to the original image after the transformation.

[0050]

[0051] Generally speaking, let a 33 =1, expand the above formula (2) to get the affine transformation of a point:

[0052]

[0053] Among them, a ij is the parameter to be determined. There are 8 unknowns in total, so 4 pairs of corresponding points are needed. The matrix is ​​the mapping matrix that changes the perspective of the original image pixels to the new viewing plane.

[0054] S102, detecting and tracking the target in the surveillance video and extracting the target motion trajectory, mapping the target motion trajectory to the geographic space according to the mapping relationship model to determine the coordinates of the target in the geographic space, and visually displaying the target motion trajectory in the geographic space.

[0055] Specifically, the detection and tracking of targets in surveillance videos and the extraction of target motion trajectories can be set to include:

[0056] Detect and identify the target in the monitoring identification using the pre-trained target recognition detection model, and output the target detection and recognition result, which includes the target detection frame, target category and target confidence;

[0057] The target detection and recognition results are input into the pre-trained multi-target tracker to track the target and extract the target motion trajectory using the multi-target tracker.

[0058] In surveillance videos, decision makers usually focus on moving dynamic foreground targets rather than fixed background information. With background information as a reference, the focus is on studying moving targets. Therefore, it is necessary to identify and detect the target and extract its movement trajectory. Since Yolov5+DeepSort achieves a good balance between real-time and accuracy, this algorithm is used in the embodiment of this case to identify, detect and track the target of interest in the surveillance video.

[0059] like Figure 2 As shown in the figure, the input video frame sequence is processed by perspective transformation technology to establish a mapping relationship model from image space to geographic space; the Yolov5+DeepSort technology is used to detect and track the target of interest in the video, extract its motion trajectory, and map its trajectory to the geographic space through the established mapping relationship model, so as to determine the precise coordinates of the target in the actual geographic space; finally, the surveillance video target tracking visualization prototype system is used to visualize its tracking trajectory.

[0060] Considering the real-time performance, the Yolov5s version with a smaller model is used to identify and detect the target. The target detection box, category, confidence and other information output by Yolov5 detection are input into DeepSort multi-target tracker for tracking. Figure 3 The specific process can be described as follows:

[0061] First, after passing through the yolov5 detection model, the Kalman filter predicts the target's motion state and obtains information describing the target's position and state; at the same time, the Mahalanobis distance is used for the motion features and the target's appearance features are added; finally, the Hungarian algorithm is used to match the predicted tracking and the detection box in the current frame for cascade matching and IOU matching, and updated through the Kalman filter.

[0062] The output of video processing using yolov5s+DeepSort is the position of the tracked target in the video image, identity recognition information, etc. At the same time, in order to facilitate more intuitive understanding, its motion trajectory is also visualized when tracking the target. The pixel coordinates of the target in the video image, that is, the point set of the track nodes of the tracked target in the image space in the video image, combined with the perspective transformation matrix calculated in the previous article, can be used to obtain the point set corresponding to the track nodes of the target under the orthographic perspective, and then the four pairs of point mapping relationships corresponding to the image after perspective transformation and the actual satellite image can be used to obtain the latitude and longitude coordinates of the target track node point set in the actual geographic space through interpolation.

[0063] The visualization of the target movement trajectory in the geographic space may include:

[0064] The target tracking visualization monitoring platform is used to visualize the movement trajectory of the tracked target. The target tracking visualization monitoring platform adopts a B / S architecture and provides front-end display and back-end management of integrated monitoring video and two-dimensional map based on the star map online map and monitoring video.

[0065] There are some problems with the traditional method of mapping surveillance video to geographic space: simply superimposing surveillance video on a geographic base map will cause the video content to overlap with the geographic map information, resulting in confusing visual effects and information redundancy. Especially in large-scale and complex scenes, it is difficult for humans to quickly and accurately determine the geographic location of the video target and highlight the key surveillance objects. This practice deviates from the original intention of combining surveillance video with geographic information to achieve key target monitoring management and geographic information analysis. Based on the mapping model from image space to geographic space and the research on target detection, tracking and trajectory extraction, the surveillance video target tracking visualization prototype system can be used to provide monitoring personnel with a global and macroscopic observation perspective to break through the dilemma of traditional monitoring being limited to a single perspective, and can help them efficiently grasp the overall situation of a large-scale monitoring area.

[0066] The prototype system for visualizing surveillance video target tracking can adopt B / S architecture and be developed using HTML, CSS, and JavaScript. There are two main external access services, one is the online map, which uses the Star Map Earth service; the other is the surveillance video service, taking the video shot by the surveillance camera as an example. The functional design of the system can mainly include the front-end display system for the integrated surveillance video and two-dimensional map for ordinary users and the back-end management system for administrator users, aiming to provide users with a convenient and intuitive surveillance video and map integration experience. The front-end display system is mainly for ordinary users. The main interface integrates the Star Map Earth, providing basic map operation functions such as zooming in, zooming out, and panning, target trajectory display functions, and surveillance area supervision management functions. Users can realize dynamic perception of the overall situation of the large scene in the surveillance area by viewing the mapping display of the surveillance video tracking target on the two-dimensional map. The back-end management system is mainly for management personnel, mainly including system administrator management functions and surveillance camera management functions. Through the back-end management system, managers can add, delete, modify, and query surveillance camera information, manage surveillance videos, and realize effective management of surveillance videos.

[0067] Furthermore, based on the above method, an embodiment of the present invention also provides a surveillance video dynamic target tracking system coupled with geographic space information, comprising: a video conversion module and a target tracking module, wherein:

[0068] A video transformation module, used to perform perspective transformation processing on a surveillance video frame sequence to establish a mapping relationship model for conversion from image space to geographic space, wherein the perspective transformation processing is used to establish a mapping relationship model by performing orthographic perspective transformation on the surveillance video image;

[0069] The target tracking module is used to detect and track targets in surveillance videos and extract target motion trajectories, map target motion trajectories to geographic space based on a mapping relationship model to determine the coordinates of the target in geographic space, and visualize the target motion trajectory in geographic space.

[0070] In order to verify the effectiveness of this solution, the following is a further explanation based on experimental data:

[0071] To verify the feasibility and robustness of the solution, the experimental data uses road surveillance videos and four surveillance areas for verification. The computer hardware environment is Lenovo Y9000P 12th Gen Intel(R)Core(TM)i7-12700H2.70GHz, 16G memory, NVIDIA GeForce RTX 3060Laptop GPU 6GB graphics card.

[0072] Currently, the cameras installed for surveillance have a rough location, such as a certain intersection. Based on this information, the approximate longitude and latitude of the surveillance video can be obtained. At the same time, satellite images with longitude and latitude information near the location can be obtained. Then, through the above-mentioned image space to geographic space mapping relationship model, using the original video image and the corresponding 4 pairs of points on the orthographic viewing plane, the corresponding points and point numbers are annotated as follows Figure 4 As shown, taking the corresponding points in area 1 as an example, the perspective mapping transformation matrix is ​​obtained as shown in Table 1. Among them, the pixel coordinates and the longitude and latitude coordinates of the geographic space are manually selected in the image or map to correspond to the coordinates of the landmark objects or corner points.

[0073] Table 1 Image coordinates of region 1 and corresponding pixel coordinates of points in the new viewing plane

[0074]

[0075] The derived transformation matrix can be used to transform the video image into a new orthographic viewing plane, such as Figure 4 As shown, Figure 4 (a) and (b) are the original camera video image and the new view plane image after perspective transformation, respectively.

[0076] The pixel coordinates in the converted orthophoto plane image and the longitude and latitude coordinates of the corresponding points in the real map are used to interpolate and calculate the real longitude and latitude coordinates of other pixel points, that is, the real geographic longitude and latitude corresponding to the pixel coordinates of the target trajectory in the orthophoto plane. In the experiment, in addition to solving the corresponding points used in the mapping matrix, some more iconic corner points or significant ground points are selected as check points as evenly as possible to evaluate the accuracy of this scheme; at the same time, this scheme is compared with the direct linear transform (DLT); both methods are tested on area 1 as an example, and the check point coordinates and the mapped coordinates are shown in Table 2, and their mean square error is calculated. For the convenience of calculation, the mapped longitude and latitude are converted to the world coordinate system coordinates. The results show that the accuracy of the DLT result is relatively higher, but DLT requires 6 pairs of pixel coordinates and corresponding world coordinate points to solve the equation, while the perspective transformation only requires 4 pairs of two-dimensional control points to complete the solution. It is simple and convenient to use, and the accuracy is not much reduced, which can meet the relevant requirements.

[0077] Table 2 Perspective transformation and DLT mapping coordinates and error analysis

[0078]

[0079]

[0080] Figure 5The tracker can track multiple targets of multiple types or specify a target for tracking. Here, only some targets are tracked and output. The YOLOv5 algorithm assigns a calculated detection box bbox to each detected target. The number in the upper left corner of the bbox is used to uniquely identify the tracked target.

[0081] The actual latitude and longitude coordinates of the tracking trajectory are visualized through the surveillance video target tracking visualization prototype system as follows Figure 6 By monitoring the mapping relationship between the video and the map, the actual geographic coordinates of the target are obtained and then superimposed on the two-dimensional map. Therefore, the "ghosting" phenomenon can be effectively avoided and the expression effect of dynamic targets can be improved.

[0082] The above data show that compared with the traditional method of integrating video and geographic information, this solution can avoid the process of obtaining internal and external parameters of the camera and unifying the coordinate system, greatly reducing the amount of data calculation. At the same time, through the interactive surveillance video target tracking visualization prototype system, the surveillance target is visualized in the geographic space, which is convenient for monitoring personnel to understand and analyze the surveillance target, provide measurable and locatable functions to monitoring personnel, reduce the workload of monitoring personnel, and solve the limitation of traditional target tracking tasks that are only analyzed in image space. From the test results, the visualization effect of integrating the target of interest in the surveillance video into the geographic space is good, and different interactive functions can serve different application needs. It has good application prospects in the field of dynamic target recognition and tracking.

[0083] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0084] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0085] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.

[0086] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.

[0087] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for dynamic target tracking in surveillance video coupled with geographic spatial information, characterized in that: Include: Performing perspective transformation processing on the surveillance video frame sequence to establish a mapping relationship model for converting from image space to geographic space, wherein the perspective transformation processing is used to establish the mapping relationship model by performing orthographic perspective transformation on the surveillance video image; Detect and track targets in surveillance videos and extract target motion trajectories. Map target motion trajectories to geographic space based on a mapping relationship model to determine the coordinates of the target in geographic space, and visualize the target motion trajectory in geographic space.

2. The method for dynamic target tracking in surveillance video coupled with geographic spatial information according to claim 1, characterized in that: Perform perspective transformation on surveillance video frame sequences, including: The perspective transformation projection relationship is represented by using the two-dimensional coordinates of the pixels of the surveillance video image and the corresponding homogeneous coordinates and mapping parameters, wherein the mapping parameters include: image linear transformation parameters, image translation parameters, image perspective transformation parameters and constant parameters; The perspective transformation projection relationship is used to obtain the homogeneous coordinates of the pixel points of the surveillance video image after the perspective transformation, and the two-dimensional plane coordinates of the pixel points of the surveillance video image after the perspective transformation are obtained through normalization processing; Assign values ​​to constant parameters, and obtain the perspective transformation expression of image pixels according to the two-dimensional plane coordinates after perspective transformation; solve the perspective transformation expression using the four pairs of corresponding points specified before and after perspective transformation in the surveillance video image, and obtain the mapping matrix that transforms the image pixels to the new viewing plane according to the solution result, and use the mapping matrix as a mapping relationship model.

3. The method for dynamic target tracking in surveillance video coupled with geographic spatial information according to claim 2, characterized in that: The two-dimensional coordinates of the pixels of the surveillance video image after perspective transformation are obtained through normalization processing, including: Normalize the Z in the homogeneous coordinates (X, Y, Z) of the surveillance video image pixel point to obtain the normalized homogeneous coordinates (X', Y', 1) of the surveillance video image pixel point, where X' = X / Z and Y' = Y / Z; According to the normalized homogeneous coordinates (X', Y', 1) of the pixel point of the surveillance video image, the two-dimensional plane coordinates (X', Y') of the corresponding pixel point after perspective transformation are obtained.

4. The method for dynamic target tracking in surveillance video coupled with geographic spatial information according to claim 1, characterized in that: Detect and track targets in surveillance videos and extract target motion trajectories, including: Detect and identify the target in the monitoring identification using the pre-trained target recognition detection model, and output the target detection and recognition result, which includes the target detection frame, target category and target confidence; The target detection and recognition results are input into the pre-trained multi-target tracker to track the target and extract the target motion trajectory using the multi-target tracker.

5. The method for dynamic target tracking in surveillance video coupled with geographic spatial information according to claim 4, characterized in that: The target recognition and detection model is constructed using the yolov5 network, and the multi-target tracker is constructed using the DeepSort algorithm.

6. The method for dynamic target tracking in surveillance video coupled with geographic spatial information according to claim 4 or 5, characterized in that: Use a multi-target tracker to track the target and extract the target's motion trajectory, including: The Kalman filter is used to predict the target motion state and obtain the target motion characteristics, wherein the target motion characteristics include target position description information and target state description information; Based on the target motion features, target appearance features and target motion Mahalanobis distance, the Hungarian algorithm is used to perform cascade matching and IOU matching on the tracking target and the detection frame in the current video frame, and then updated through the Kalman filter to obtain the position and identity information of the tracking target in the video frame; The target trajectory node point set is obtained according to the pixel coordinates of the target in the video frame, and the target motion trajectory is drawn according to the trajectory node point set.

7. The method for dynamic target tracking in surveillance video coupled with geographic spatial information according to claim 1, characterized in that: Visualize the target movement trajectory in geographic space, including: The target tracking visualization monitoring platform is used to visualize the movement trajectory of the tracked target. The target tracking visualization monitoring platform adopts a B / S architecture and provides front-end display and back-end management of integrated monitoring video and two-dimensional map based on the star map online map and monitoring video.

8. A surveillance video dynamic target tracking system coupled with geographic space information, characterized in that: Contains: video transformation module and target tracking module, among which, A video transformation module, used to perform perspective transformation processing on a surveillance video frame sequence to establish a mapping relationship model for conversion from image space to geographic space, wherein the perspective transformation processing is used to establish a mapping relationship model by performing orthographic perspective transformation on the surveillance video image; The target tracking module is used to detect and track targets in surveillance videos and extract target motion trajectories, map target motion trajectories to geographic space based on a mapping relationship model to determine the coordinates of the target in geographic space, and visualize the target motion trajectory in geographic space.

9. An electronic device, characterized in that: include: at least one processor, and a memory coupled to the at least one processor; The memory stores a computer program, and the computer program can be executed by the at least one processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 can be implemented.

Images