Gun-ball linked target tracking method, device, electronic device and storage medium

The system addresses the issue of missed target tracking by using a ball camera to identify and convert coordinates for precise tracking with a gun camera, ensuring targets with specific details are tracked effectively.

CN114120165BActive Publication Date: 2024-12-03ZHEJIANG DAHUA TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111193535.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-13
Publication Date
2024-12-03
Estimated Expiration
2041-10-13

AI Technical Summary

Technical Problem

Existing gun-ball linkage systems fail to actively identify targets with specific details before tracking, leading to missed tracking of specific targets.

Method used

The system uses a ball camera to identify targets in a preset monitoring area, converts the ball camera's tracking information to panoramic coordinates, and employs a target matching algorithm for the gun camera to actively track the target in panoramic coordinates.

Benefits of technology

This approach allows for the active identification and tracking of targets with specific details, reducing the omission of target tracking by first identifying the target with the ball camera and then converting its coordinates for precise tracking by the gun camera.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114120165B_ABST
    Figure CN114120165B_ABST
Patent Text Reader

Abstract

The present application relates to a method, device, electronic device and storage medium for target tracking in a gun-ball linkage, wherein the method comprises: using a ball camera to identify a target in a preset monitoring area and track the target; based on the coordinate mapping relationship between the ball camera coordinates and the panoramic coordinates, converting the ball camera tracking information obtained by the ball camera tracking the target from the ball camera coordinates to the panoramic coordinates to obtain the panoramic tracking information of the target; controlling the gun camera to identify and track the target in the panoramic coordinates based on the panoramic tracking information using a target matching algorithm. Through the present application, the problem that the target with specific details cannot be found first and then tracked in the related art, and the tracking of the specific target is easily missed, is solved, and the target with specific details can be tracked when the gun camera cannot identify the specific details, thereby reducing the omission of target tracking.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of monitoring technology, and in particular to a target tracking method, device, electronic device and storage medium for gun-ball linkage. Background Art

[0002] In some scenes that require large-scale monitoring, such as viaducts and important roads, some multi-eye splicing (panoramic) cameras or gun cameras will be selected, and ball cameras, such as gun-ball integrated devices, will be used to take into account detailed observation. This type of equipment is installed at a relatively high height and has a wide monitoring range, so the gun-end algorithm cannot perform some detailed analysis, such as license plate detection, but is good at some behavior analysis, type detection, and statistical analysis. Therefore, a gun-ball linkage solution is needed. The gun-ball linkage solution usually includes a gun camera and at least one ball camera. Among them, the gun camera is responsible for capturing moving targets, and then according to the coordinate position of the moving target in the video obtained by the gun camera, it controls the full-scale movement of the ball camera and the lens magnification and zoom control (Pan / Tilt / Zoom, PTZ), so that the ball camera rotates to the moving target image tracked by the gun camera, and then the ball camera further tracks the moving target in detail. Therefore, the core of the ball camera linkage solution lies in the detection and tracking of moving targets.

[0003] At present, the target tracking scheme of the gun-ball linkage is to identify and track the target with the gun camera, convert the gun camera target coordinates into the ball camera coordinates, rotate the ball camera to observe the same target, and judge whether it is the same target by comparing the similarity of the gun camera and the ball camera target images, and then continue to control the ball camera to track according to the target coordinates tracked by the gun camera. The disadvantage of this scheme is that the ball camera tracking is passive tracking, that is, the ball camera pan / tilt is controlled to track in real time according to the tracking results of the gun camera. However, since the gun camera cannot identify the details of the target, it is impossible to first find the target with specific details and then track the target, which makes it easy to miss the tracking of specific targets.

[0004] There is no effective solution to the problem in related technologies that it is impossible to first discover a target with specific details and then track the target, which easily leads to missing the tracking of a specific target. Summary of the invention

[0005] In this embodiment, a gun-ball linked target tracking method, device, electronic device and storage medium are provided to solve the problem in the related art that it is impossible to first discover a target with specific details and then track the target, and it is easy to miss the tracking of a specific target.

[0006] In the first aspect, a target tracking method for gun-ball linkage is provided in this embodiment, comprising:

[0007] Using a ball camera to identify a target in a preset monitoring area and track the target;

[0008] Based on the coordinate mapping relationship between the ball camera coordinates and the panoramic coordinates, the ball camera tracking information obtained by the ball camera tracking the target is converted from the ball camera coordinates to the panoramic coordinates to obtain the panoramic tracking information of the target;

[0009] The gun is controlled to identify and track the target in the panoramic coordinates using a target matching algorithm based on the panoramic tracking information.

[0010] In some embodiments, the use of a ball camera to identify a target in a preset monitoring area and track the target includes:

[0011] Control the ball camera to obtain regional monitoring video frames of the preset monitoring area;

[0012] Matching the target image in the regional monitoring video frame with the target image in a preset target database to identify the target;

[0013] The ball camera is controlled to track the target using a tracking algorithm.

[0014] In some embodiments, the target image is a license plate image; and the target database is a license plate database.

[0015] In some embodiments, the use of a tracking algorithm to control the ball camera to track the target includes:

[0016] The tracking algorithm is used to control the pan / tilt of the ball camera to actively track the target and predict the target speed.

[0017] In some embodiments, the gun-ball linkage target tracking method provided in this embodiment further includes:

[0018] After using the ball camera to identify the target in the preset monitoring area and track the target, based on the coordinate conversion relationship between the ball camera coordinates at multiple magnifications and the ball camera coordinates at single magnifications, the ball camera coordinates of the target center point at the current magnification are converted to the ball camera coordinates of the target center point at single magnification.

[0019] In some embodiments, tracking the target in the panoramic coordinates includes:

[0020] In the panoramic monitoring video frame of the gun camera, an image of the target is superimposed and marked based on the panoramic tracking information;

[0021] The target is tracked in the panoramic coordinates based on the superimposed markers using a preset multiple algorithm strategy.

[0022] In some embodiments, the gun-ball linkage target tracking method provided in this embodiment further includes:

[0023] By using a panoramic intelligent algorithm, the gun is controlled to independently identify the panoramic object in each frame of the panoramic monitoring video frame, and track the panoramic object to cache the panoramic object information obtained by multiple frames of the panoramic intelligent algorithm.

[0024] In some embodiments, the control gun recognizes the target in the panoramic coordinates using a target matching algorithm based on the panoramic tracking information, including:

[0025] The gun camera is controlled to filter out the panoramic monitoring video frame that is closest to the current regional monitoring video frame of the ball camera in terms of acquisition time of the gun camera based on the panoramic tracking information, and determine the panoramic object information corresponding to the panoramic monitoring video frame;

[0026] The panoramic coordinates in the panoramic tracking information are position matched with each panoramic object in the filtered panoramic monitoring video frame to determine the target in the panoramic monitoring video frame and the corresponding identity information.

[0027] In some embodiments, when the target is a vehicle, the control gun camera screens out a panoramic monitoring video frame that is closest to the current regional monitoring video frame of the ball camera in terms of acquisition time of the gun camera based on the panoramic tracking information, and determines the panoramic object information corresponding to the panoramic monitoring video frame, including:

[0028] Comparing the target vehicle speed in the panoramic tracking information with a preset vehicle speed threshold;

[0029] If the target vehicle speed is less than the preset vehicle speed threshold, it is determined to be a low-speed scene, and based on the timestamp in the panoramic tracking information, the panoramic monitoring video frame closest to the current regional monitoring video frame of the ball camera in terms of the acquisition time of the gun camera is screened out, and the panoramic object information corresponding to the panoramic monitoring video frame is determined;

[0030] If the target vehicle speed is greater than or equal to the preset vehicle speed threshold, it is determined to be a high-speed scene. Based on the preset delay time value and the timestamp in the panoramic tracking information, the panoramic surveillance video frame that is closest to the current area surveillance video frame of the ball camera in terms of the acquisition time of the gun camera is screened out, and the panoramic object information corresponding to the panoramic surveillance video frame is determined.

[0031] In some embodiments, the step of matching the panoramic coordinates in the panoramic tracking information with the panoramic objects in the filtered panoramic monitoring video frame to determine the target in the panoramic monitoring video frame and the corresponding identity information includes:

[0032] Perform distance statistics between the panoramic coordinates in the panoramic tracking information and the center point coordinates of each panoramic object in the filtered panoramic monitoring video frame to obtain distance information and corresponding identity information of each panoramic object;

[0033] Each of the distance information is compared in pairs, and the panoramic object with the smallest distance information and the corresponding identity information are screened out to determine the target and the corresponding identity information.

[0034] In some embodiments, the gun-ball linkage target tracking method provided in this embodiment further includes:

[0035] After determining the target and the corresponding identity information in the panoramic monitoring video frame, multiple position matching is performed in consecutive frames to determine the final target and the corresponding identity information.

[0036] In some embodiments, the gun-ball linkage target tracking method provided in this embodiment further includes:

[0037] After determining the target and the corresponding identity information, the gun is controlled to track the target in the panoramic coordinates based on the identity information and using a preset multiple algorithm strategy.

[0038] In some embodiments, the gun-ball linkage target tracking method provided in this embodiment further includes:

[0039] After the target is tracked in the panoramic coordinates based on the identity information, the image of the target is superimposed, marked and displayed in the panoramic monitoring video frame of the gun camera based on the panoramic tracking information.

[0040] In a second aspect, a gun-ball linked target tracking device is provided in this embodiment, comprising: a first identification and tracking module, a conversion module, and a second identification and tracking module;

[0041] The first identification and tracking module is used to use the ball camera to identify the target in the preset monitoring area and track the target;

[0042] The conversion module is used to convert the ball camera tracking information obtained by the ball camera tracking the target from the ball camera coordinates to the panoramic coordinates based on the coordinate mapping relationship between the ball camera coordinates and the panoramic coordinates, so as to obtain the panoramic tracking information of the target;

[0043] The second identification and tracking module is used to control the gun to identify and track the target in the panoramic coordinates based on the panoramic tracking information and using a target matching algorithm.

[0044] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the gun-ball linked target tracking method described in the first aspect is implemented.

[0045] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored, and when the program is executed by a processor, the target tracking method of the gun-ball linkage described in the first aspect is implemented.

[0046] Compared with the related art, the target tracking method, device, electronic device and storage medium of the gun-ball linkage provided in this embodiment use the ball camera to identify the target in a preset monitoring area and track the target; based on the coordinate mapping relationship between the ball camera coordinates and the panoramic coordinates, the ball camera tracking information obtained by the ball camera tracking the target is converted from the ball camera coordinates to the panoramic coordinates to obtain the panoramic tracking information of the target; the gun camera is controlled to identify and track the target in the panoramic coordinates based on the panoramic tracking information using the target matching algorithm, which solves the problem in the related art that it is impossible to first find the target with specific details and then track the target, and it is easy to miss the tracking of the specific target. The specific target is first found by using the ball camera, and its ball camera coordinates are converted into panoramic coordinates, and the gun camera is controlled to identify and track the target, so that the target with specific details can be tracked when the gun camera cannot identify the specific details, thereby reducing the omission of target tracking.

[0047] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0049] Figure 1 This is a hardware structure block diagram of a terminal device of a target tracking method for gun-ball linkage provided by an embodiment of the present application;

[0050] Figure 2 is a flow chart of a target tracking method for gun-ball linkage provided by an embodiment of the present application;

[0051] Figure 3 yes Figure 2 Flow chart of step S210;

[0052] Figure 4 It is a comparative schematic diagram of obtaining surveillance video frame timestamps by a ball camera and a gun camera provided in an embodiment of the present application;

[0053] Figure 5 It is a schematic diagram of the time difference between the frames of a ball camera and a gun camera provided by an embodiment of the present application entering a relatively stable state;

[0054] Figure 6 It is a schematic diagram of the time difference between the frames of a ball camera and a gun camera entering a relatively stable state provided by another embodiment of the present application;

[0055] Figure 7 It is a flowchart of a target tracking method for gun-ball linkage provided by an embodiment of the present application;

[0056] Figure 8 It is a structural block diagram of a target tracking device for gun-ball linkage provided in one embodiment of the present application.

[0057] In the figure: 210, a first identification and tracking module; 220, a conversion module; 230, a second identification and tracking module. DETAILED DESCRIPTION

[0058] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0059] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. In this application, "one", "a", "the", "these" and similar words do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. Similar words such as "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. "Multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0060] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure block diagram of a terminal of the target tracking method of the gun-ball linkage of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0061] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the target tracking method of the gun-ball linkage in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0062] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0063] In this embodiment, a target tracking method for gun-ball linkage is provided. Figure 2 FIG. 1 is a flow chart of the target tracking method of the gun-ball linkage of the present embodiment. Figure 2 As shown, the process includes the following steps:

[0064] Step S210, using the ball camera to identify the target in the preset monitoring area and track the target;

[0065] Step S220, based on the coordinate mapping relationship between the ball camera coordinates and the panoramic coordinates, the ball camera tracking information obtained by the ball camera tracking the target is converted from the ball camera coordinates to the panoramic coordinates to obtain the panoramic tracking information of the target;

[0066] Step S230, controlling the gun to identify and track the target in panoramic coordinates using a target matching algorithm based on the panoramic tracking information.

[0067] It should be noted that the ball camera and the gun camera can be an integrated device, such as a gun-ball integrated device. The gun-ball integrated device can be set at a preset point, such as: on an overpass, above a main road, and in a tunnel, etc., where a large range of monitoring is required. The ball camera executes the ball camera intelligent algorithm at the preset point to monitor the monitoring area to identify targets with specific details in the monitoring area, and then executes the tracking algorithm to further track the moving targets in detail, and the ball camera tracking information will be obtained. Among them, for different targets, the specific details are also different. For example: if the target is a motor vehicle or a non-motor vehicle, the specific details can be a license plate, and a corresponding license plate library can be set up for easy identification; if the target is a person, the specific details can be facial features, and a corresponding facial feature library can be set up for easy identification. Here, the targets and the corresponding specific details are not given one by one.

[0068] The coordinate mapping relationship between the ball camera coordinates and the panoramic coordinates can be calibrated using a coordinate conversion algorithm. The calibration process is: controlling the ball camera to obtain regional surveillance video frames, controlling the gun camera to obtain panoramic surveillance video frames, and selecting the same target in the regional surveillance video frames and the panoramic surveillance video frames, such as: selecting a fixed building, a stationary motor vehicle, or a stationary person. Based on the coordinate mapping relationship, the coordinates and other related tracking information can be converted between the ball camera and the gun camera, that is, the ball camera tracking information can be converted into panoramic tracking information. The ball camera tracking information includes but is not limited to target coordinates, target speed, target direction, specific details, and timestamps. The timestamp can be added when the regional surveillance video frame is obtained, or when the regional surveillance video is encoded after the regional surveillance video is obtained.

[0069] The target matching algorithm is executed in the gun camera, and the target is identified and tracked in panoramic coordinates based on the panoramic tracking information. Using the above target matching algorithm, the gun camera completes the matching tracking of the target based on the panoramic tracking information, so that the target with specific details can be tracked, reducing the omission of target tracking. The ball camera and the gun camera are only synchronized in the early stage of target discovery and tracking, and then they track independently, and whether they are lost or not does not affect each other. In other embodiments, the ball camera and the gun camera can be independent devices, which can be implemented in the same way as the gun-ball all-in-one device.

[0070] Through the above steps, the ball camera is first used to find a specific target, the ball camera coordinates are converted into panoramic coordinates, and the gun camera is controlled to identify and track the target, so that the target with specific details can be tracked when the gun camera cannot identify the specific details, and the omission of target tracking is reduced. The problem that the target with specific details cannot be found first and then tracked in the related art, and the tracking of the specific target is easily missed is solved.

[0071] The following is a detailed description of each step:

[0072] In some embodiments, such as Figure 3 As shown, step S210 includes the following steps:

[0073] Step S211, controlling the ball camera to obtain a regional monitoring video frame of a preset monitoring area;

[0074] Step S212, matching the target image in the regional monitoring video frame with the target image in a preset target database to identify the target;

[0075] Step S213, using the tracking algorithm to control the ball camera to track the target.

[0076] Specifically, the ball camera can first obtain the regional monitoring video of the preset monitoring area, and use the encoding technology to extract the regional monitoring video frame from the regional monitoring video and add a timestamp. The ball camera executes the ball camera intelligent algorithm at the preset point to monitor the monitoring area, and matches the target image in the regional monitoring video frame with the target image in the preset target database to identify the target with specific details. Finally, the tracking algorithm is used to control the ball camera to track the target. Among them, the target image is a license plate image; the target database is a license plate database. Then the license plate image in the regional monitoring video frame is matched with the license plate image in the preset license plate database, and when the license plate image in the regional monitoring video frame matches the license plate image in the license plate database, the target is identified. Of course, the target image can also be a face, and the target database can be a face database, etc., and there is no limitation on this.

[0077] The ball camera intelligent algorithm may be a CNN-based video structured detection algorithm, which can simultaneously detect motor vehicles, non-motor vehicles, people and their features, and can also meet the requirements of motor vehicle license plate detection of a certain pixel size. In one embodiment, the CNN-based video structured detection algorithm recognizes the target by first training a CNN video structured detection model.

[0078] Among them, the tracking algorithm can automatically select the target to be tracked and realize long-term variable magnification tracking. It is widely used in many monitoring places such as stations, subways, communities and schools. Tracking algorithms generally include the following:

[0079] The first algorithm: tracking algorithm based on model matching, such as particle filtering method, mean-shift method, etc. Specifically, first extract the target feature as a template, then determine the position information closest to the feature in the next frame based on the feature template, and then use the determined position information as the target's position information again, and update the feature template.

[0080] The second algorithm is the Lucas-Kanade feature tracking algorithm. The specific processing flow is: first extract the feature pixels on the target, and then track each feature pixel (also called feature pixel). Optionally, the Lucas (lk, Lucas-Kanade) optical flow method can be used to calculate the speed of the feature pixel.

[0081] The third algorithm: Tracking learning detection tracking algorithm, this algorithm is a strategy to improve target tracking by detection. Specifically: first, you need to set up a short-time tracker, and use the set short-time tracker to continuously track the target, and generate an online detector. The online detector uses the tracking result of the short-time tracker as the output of the trainer, and updates the tracking target of the short-time tracker according to the output generated. When the short-time tracker loses the target, the target can be retrieved through the online detector, thereby achieving the purpose of extending the tracking time and anti-occlusion.

[0082] Specifically, when tracking a target, the tracking algorithm is used to control the pan / tilt of the ball camera to actively track the target and predict the target speed. If the target is a motor vehicle or non-motor vehicle, the target speed is the vehicle speed; if the target is a person, the target speed is the person's moving speed.

[0083] In some of the embodiments, in order to improve the recognition and tracking rate, after using the ball camera to recognize the target in the preset monitoring area and track the target, the following steps are also included:

[0084] Based on the coordinate conversion relationship between the ball camera coordinates of multiple magnifications and the ball camera coordinates of single magnifications, the ball camera coordinates of the target center point of the ball camera at the current magnification are converted to the ball camera coordinates of the target center point at single magnification.

[0085] The coordinate conversion relationship between the multi-magnification ball camera coordinates and the single-magnification ball camera coordinates can be pre-calibrated. Select a fixed building, a stationary motor vehicle or a stationary person; convert the relationship between the ball camera coordinates at different magnifications to obtain the coordinate conversion relationship. In other words, the conversion between the ball camera coordinates of any multiple and the single-magnification ball camera coordinates can be achieved. Thus, the ball camera coordinates of the target center point at the current magnification can be converted to the ball camera coordinates of the target center point at the single magnification.

[0086] In some embodiments, tracking the target in panoramic coordinates in step S230 includes the following steps:

[0087] In the panoramic monitoring video frame of the gun camera, the target image is superimposed and marked based on the panoramic tracking information;

[0088] Using preset multiple algorithm strategies, the target is tracked in panoramic coordinates based on superimposed markers.

[0089] Specifically, the multiple algorithm strategies refer to the ball camera intelligent algorithm and tracking algorithm in the ball camera; the panoramic intelligent algorithm and coordinate conversion algorithm in the gun camera. Through the coordination of these algorithm strategies, the ball camera first detects the target with specific details and tracks it, and then links the gun camera to superimpose the superimposed mark on the target in the gun camera monitoring video to track the target and reduce the omission of target tracking. Among them, the superimposed mark refers to the warning color and panoramic tracking information (vehicle speed, size, direction, license plate, etc.). The use of superimposed marks can be to use the warning color to distinguish from the ordinary track box color when displaying the track box in the monitoring video, so as to intuitively obtain the tracking details of the target captured by the ball camera in the panoramic monitoring video.

[0090] In some embodiments, the gun-ball linkage target tracking method provided in this embodiment further includes the following steps:

[0091] Using the panoramic intelligent algorithm, the gun is controlled to independently identify the panoramic objects in each frame of the panoramic monitoring video frame, and track the panoramic objects to cache the panoramic object information obtained by the multi-frame panoramic intelligent algorithm.

[0092] Among them, the panoramic intelligent algorithm is a video structured detection algorithm based on CNN, which can independently identify the panoramic object to be tracked in each frame of the panoramic surveillance video frame and realize long-term tracking. The panoramic object refers to the target obtained without specific detail recognition in the panoramic surveillance video frame. For example, the target in the ball camera refers to the vehicle with a specific license plate. Then the panoramic object is the vehicle in the panoramic surveillance video frame. Since both the gun camera and the ball camera monitor in real time and obtain the surveillance video. Moreover, the monitoring range of the gun camera is wider than that of the ball camera. After the target with specific details is found in the ball camera, the gun camera must also have the target, but the gun camera cannot recognize the specific details, so it is unknown which of the panoramic objects the target of the specific details is. Therefore, the panoramic object is tracked in the gun camera to cache the panoramic object information obtained by the multi-frame panoramic intelligent algorithm, so as to prepare for the subsequent target matching of the specific detail target, so that the gun camera can also recognize the specific detail target.

[0093] In some embodiments, step S230 includes the following steps:

[0094] Step S231, controlling the gun camera to filter out the panoramic monitoring video frame that is closest to the current area monitoring video frame of the ball camera in terms of gun camera acquisition time based on the panoramic tracking information, and determining the panoramic object information corresponding to the panoramic monitoring video frame.

[0095] Specifically, since the ball camera and the gun camera are two relatively independent devices, there is a time difference (denoted by Δtr) between the camera of the ball camera and the camera of the gun camera when they acquire the same image; in addition, there is a processing time from the camera acquiring the image to encoding it into a surveillance video frame in YUV format, and the time consumption varies depending on the camera model, image size, and ISP processing flow (denoted by Δtb for the ball camera and Δtg for the gun camera). That is, the timestamp in the surveillance video frame in YUV format lags behind the actual image acquisition time, and the delay time of the ball camera and the gun camera is different. Therefore, it is necessary to filter out the panoramic surveillance video frame that is closest to the current regional surveillance video frame of the ball camera in terms of the acquisition time of the gun camera, and then determine the panoramic object information corresponding to the panoramic surveillance video frame. Among them, the panoramic object information includes but is not limited to the center coordinates of each panoramic object, the corresponding identity information, and the timestamp in the current panoramic surveillance video frame.

[0096] like Figure 4As shown in the figure, the bare frame of the ball camera at time t2 corresponds to the regional monitoring video frame at time t5, that is, the timestamp of the nth frame of the regional monitoring video frame of the ball camera is t5; the bare frame of the gun camera at time t1 corresponds to the panoramic monitoring video frame at time t9, that is, the timestamp of the mth frame of the panoramic monitoring video frame of the gun camera is t9. The two bare frames at time t1 and t2 are the closest in actual time, that is, the regional monitoring video frame of the ball camera at time t5 and the panoramic monitoring video frame at time t9 are the closest in time; if the above delay problem is not considered, it will be mistakenly believed that the regional monitoring video frame of the ball camera at time t8 and the panoramic monitoring video frame at time t9 are the closest in acquisition time.

[0097] For low-speed scenes, the above time difference and processing delay have little effect on the matching of the targets of the ball camera and the gun camera, and can be ignored. However, in high-speed scenes, when there are many targets, it cannot be ignored; otherwise, matching errors may occur, resulting in tracking errors. For example: the vehicle speed is 90km / h (25m / s), the algorithm frame rate is 10, the difference between the ball camera delay and the panoramic delay is 1 frame (0.1s), then the distance error is 2.5 meters. The higher the target speed and the greater the delay, the greater the distance error. Therefore, ensure that the video format of the ball camera and the gun camera is the same (PAL or NTSC, the same frame rate); ensure that the frame rate of the ball camera intelligent algorithm and the frame rate of the panoramic intelligent algorithm are the same; so that the time difference between the frames of the ball camera and the gun camera can enter a relatively stable state (such as Figure 5 As shown in the figure, Δt≈Δti (i=1,2,3...)), Δt is the relatively stable time difference; Δti is the time difference between the i-th adjacent frames; if the video frame rate and the algorithm frame rate are different, the difficulty of processing the time difference between the ball camera and the gun camera frames into a relatively stable state increases (such as Figure 6 As shown, the dotted line indicates that the frame expires and is replaced by a new frame), and the matching time increases.

[0098] Specifically, when the target is a vehicle, step S231 includes the following steps:

[0099] Comparing the target vehicle speed in the panoramic tracking information with a preset vehicle speed threshold;

[0100] If the target vehicle speed is less than the preset vehicle speed threshold, it is determined as a low-speed scene. Based on the timestamp in the panoramic tracking information, the panoramic surveillance video frame closest to the current area surveillance video frame of the ball camera in terms of the gun camera acquisition time is selected, and the panoramic object information corresponding to the panoramic surveillance video frame is determined; in the low-speed scene: the distance error caused by the time difference can be ignored, and the closest (min(tg i -tb n )) Compare two frames. tg i is the panoramic surveillance video frame with timestamp ti; tb n Monitor video frames for the region with timestamp tn.

[0101] If the target vehicle speed is greater than or equal to the preset vehicle speed threshold, it is determined to be a high-speed scene. Based on the preset delay time value and the timestamp in the panoramic tracking information, the panoramic monitoring video frame closest to the current regional monitoring video frame of the dome camera in terms of the gun camera acquisition time is selected, and the panoramic object information corresponding to the panoramic monitoring video frame is determined. In high-speed scenes: Since the hardware model of the gun camera dome camera is fixed, and the image resolution and image processing configuration are usually fixed during operation, the delay time value (i.e. Δtb, Δtg) from image acquisition to YUV format monitoring video frame encoding can be obtained through device performance testing; according to the current panoramic monitoring video frame Fb transmitted to the panorama by the dome camera n Timestamp tb n , minus the ball camera delay time value Δtb, to get the approximate acquisition time tb n '; Cache the processing results of the multi-frame panoramic intelligent algorithm, and monitor the timestamp of each YUV format video frame cached (remember the i-th frame Fg i Timestamp tg i ) minus the panoramic delay time value Δtg to obtain the approximate acquisition time tg i ', will tb n 'With tg i 'Compare and get the panoramic surveillance video frame with the smallest time difference (recorded as the mth frame, the error is recorded as Δt), so as to select the closest (min(tg i -tb n -Δtb+Δtg)) The panoramic surveillance video frame at the moment the gun camera acquires the video.

[0102] Step S232 , position matching is performed between the panoramic coordinates in the panoramic tracking information and each panoramic object in the filtered panoramic monitoring video frame to determine the target in the panoramic monitoring video frame and the corresponding identity information.

[0103] Specifically, position matching is a distance comparison process between the panoramic coordinates and the coordinates of the center points of each panoramic object. The panoramic coordinates in the panoramic tracking information and the coordinates of the center points of each panoramic object in the selected panoramic monitoring video frame are statistically analyzed to obtain the distance information of each panoramic object and the corresponding identity information; each distance information is compared pairwise to select the panoramic object with the minimum distance information and the corresponding identity information to determine the target and the corresponding identity information.

[0104] The panoramic coordinates in the panoramic tracking information are denoted as: A0; the center point coordinates of each panoramic object are denoted as: Bi, where i is the target ID value. Calculate the distance between the panoramic coordinates in the panoramic tracking information and the center point targets of each panoramic object in the filtered panoramic surveillance video frame, denoted as di. The minimum value (min(di) = min(|A0 Bi|)) is considered to be the same target, and the identity information ID of this target is recorded and counted, and the count is denoted as ci.

[0105] Since there may be errors in the targets and corresponding identity information determined by a single position match, in order to improve the accuracy of target matching, after determining the targets and corresponding identity information in the panoramic surveillance video frame, multiple position matches are performed within consecutive frames to determine the final targets and corresponding identity information.

[0106] Specifically, perform N (frames) of comparisons continuously, that is, repeat the position match N times, and find the one with the largest calculated identity information ID (max(ci)), or the one with the same identity information ID for k consecutive times (k < N), as the final determined target that is the same as the PTZ camera. Among them, N can be 3 - 40 times, for example, N is 30 times, and the algorithm frame rate is 10. Taking the time unit as the limit, compare for 3 seconds. For the number of comparisons, a too short match may not be accurate, and too long may increase accidental risk factors such as the target disappearing, and also affect the user experience.

[0107] In some embodiments, for a target that is a vehicle, after determining the target and corresponding identity information, the following steps are further included: controlling the PTZ camera to track the target under the panoramic coordinates based on the identity information using a preset multiple algorithm strategy.

[0108] Due to factors such as installation height, monitoring distance and viewing angle, splicing distortion, light brightness, and partial occlusion of the gun camera, the panoramic intelligent algorithm not only has poor results in analyzing target details (such as license plates, faces, etc.), but also may lose the target (reflected in the continuity of the target identity information ID, that is, the identity information ID changes), especially at the edge of the picture (long distance) The target pixel size (width and height) is at the minimum recognition capability boundary of the algorithm, and the probability of its identity information ID changing is higher. As a result, when the panoramic intelligent algorithm selects the algorithm information ID, when the prompt is marked on the interface, the target may be marked incorrectly or lost due to the change of the identity information ID. Therefore, the target's identity information ID is configured to the panoramic intelligent algorithm in the multiple algorithm strategy to track the target under panoramic coordinates, ensuring that the identity information ID of the single target remains unchanged during tracking, or the identity information ID is lost and no longer reused for a period of time. The multiple algorithm strategy refers to the ball camera intelligent algorithm and tracking algorithm in the ball camera; the panoramic intelligent algorithm and coordinate conversion algorithm in the gun camera. Through the coordination of these algorithm strategies, the ball camera can first detect and track the target with specific details, and then link the gun camera to track the target with specific details based on the identity information.

[0109] In some of the embodiments, after the target is tracked in panoramic coordinates based on the identity information, the image of the target is superimposed, marked and displayed in the panoramic monitoring video frame of the gun camera based on the panoramic tracking information.

[0110] Specifically, the superimposed mark is superimposed on the target of the gun camera surveillance video to track the target and reduce the omission of target tracking. The superimposed mark refers to the warning color and panoramic tracking information (vehicle speed, size, direction, license plate, etc.). The superimposed mark can be used to distinguish the warning color from the ordinary track box color when displaying the track box in the surveillance video, so as to intuitively obtain the tracking details of the target captured by the ball camera on the panoramic surveillance video.

[0111] like Figure 7 As shown, the identification and tracking process of a target vehicle with a specific license plate is described as follows:

[0112] Step S71, synchronizing the time of the ball camera and the gun camera, and adding system timestamps to the video encoding stages of the ball camera and the gun camera respectively;

[0113] In this step, the coordinate mapping relationship between the dome camera coordinates at one magnification and the panoramic coordinates can also be calibrated; the coordinate conversion relationship between the dome camera coordinates at any magnification to the dome camera coordinates at one magnification can be calibrated; the dome camera executes the dome camera intelligent algorithm to analyze the license plate of the passing vehicle, and compares it with the license plate in the license plate database, and enables the tracking algorithm after the comparison; the gun camera runs the panoramic intelligent algorithm to analyze all panoramic objects in the panoramic monitoring video frame.

[0114] Step S72, configure the license plate database of the license plate name and configure the speed threshold (note v c ), configure the ball camera encoding delay time value (Δtb), configure the gun camera encoding delay time value (Δtg), configure the video frame rate f, configure the frame rate fa of the panoramic intelligent algorithm, configure the ball camera preset points and the associated ball camera intelligent algorithm.

[0115] Step S73, the tracking algorithm calculates the vehicle speed (denoted as v), controls the PTZ of the ball camera to track (the ball camera intelligent algorithm is still executed during the tracking process); converts the center point coordinates (A) of the target at the current magnification of the ball camera to the center point coordinates (A1) of the target at 1 magnification, and obtains the ball camera coordinates (A1), and determines the ball camera tracking information; the ball camera tracking information includes the ball camera coordinates (A1), vehicle speed (v), direction, license plate, timestamp tb n wait.

[0116] Step S74: The ball camera tracking information is transmitted to the gun camera. The gun camera converts the ball camera coordinates (A1) in the ball camera tracking information into panoramic coordinates (A0) based on the coordinate mapping relationship. The gun camera caches the panoramic object information of the multi-frame panoramic intelligent algorithm processing results; the panoramic object information includes the vehicle coordinates, identity information ID and the timestamp of the corresponding frame (tg i ).

[0117] Step S75, comparing the vehicle speed (v) with a preset vehicle speed threshold (v c ).

[0118] Step S76: If the vehicle speed is less than the speed threshold, find the (min(tg i -tb n ))Compare the processing results in one frame.

[0119] Step S77: If the vehicle speed is greater than or equal to the speed threshold, the (min(tg i -tb n -Δtb+Δtg)) in one frame.

[0120] Step S78, after selecting and filtering out the panoramic surveillance video frame that is closest to the current regional surveillance video frame of the ball camera in terms of the time of acquisition by the gun camera, calculate the coordinates of the center point of each panoramic object in the processing result of the frame (B i , i is the target’s identity information ID) and the distance (d i ), the smallest value (min(d i )=min(|A0 B i |)) is considered to be the same target, and the cumulative number of matches (c i ).

[0121] Step S79, repeat steps S74-S78 N times, where the target identity information ID appears the most times (max(c i ), record the number of times the identity information ID appears k, and consider it to be the same target; configure the matching identity information ID to the panoramic intelligent algorithm.

[0122] In step S80, the panoramic intelligent algorithm enables multiple algorithm strategies to track the single target to prevent the target's identity information ID from changing to the greatest extent possible; if the tracking fails, the identity information ID is placed in a delayed recycling reuse cache and reused after a long delay.

[0123] Step S81, in the panoramic monitoring video frame of the monitoring video, the trajectory box corresponding to the identity information ID is marked with a special color to distinguish it from other common targets, and the license plate information is superimposed, so as to track the target set in the license plate database on the panoramic monitoring video.

[0124] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0125] In this embodiment, a target tracking device for gun-ball linkage is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. The terms "module", "unit", "subunit", etc. used below may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0126] Figure 8 : is a structural block diagram of the target tracking device of the gun-ball linkage of the present embodiment. Figure 8 As shown, the device includes: a first identification and tracking module 210, a conversion module 220 and a second identification and tracking module 230;

[0127] The first identification and tracking module 210 is used to identify and track a target in a preset monitoring area using a ball camera;

[0128] The conversion module 220 is used to convert the ball camera tracking information obtained by the ball camera tracking the target from the ball camera coordinates to the panoramic coordinates based on the coordinate mapping relationship between the ball camera coordinates and the panoramic coordinates to obtain the panoramic tracking information of the target;

[0129] The second identification and tracking module 230 is used to control the gun to identify and track the target in panoramic coordinates based on the panoramic tracking information and using a target matching algorithm.

[0130] Through the above device, the ball camera is first used to find a specific target, and its ball camera coordinates are converted into panoramic coordinates, and the gun camera is controlled to identify and track the target, so that the target with specific details can be tracked when the gun camera cannot identify specific details, and the omission of target tracking is reduced. The problem that the target with specific details cannot be found first and then tracked in the related art, and the tracking of specific targets is easily missed.

[0131] In some embodiments, the first identification and tracking module 210 includes an acquisition unit, a first identification unit, and a first tracking unit;

[0132] An acquisition unit, used for controlling the ball camera to acquire regional monitoring video frames of a preset monitoring area;

[0133] A first recognition unit is used to match a target image in a regional monitoring video frame with a target image in a preset target database to recognize the target;

[0134] The first tracking unit is used to control the ball camera to track the target by using a tracking algorithm.

[0135] In some of the embodiments, the target image is a license plate image; and the target database is a license plate database.

[0136] In some of the embodiments, the first tracking unit is further used to control the pan / tilt of the ball camera to actively track the target and predict the target speed by using a tracking algorithm.

[0137] In some embodiments, the target tracking device for gun-ball linkage provided by this embodiment is Figure 8 On the basis of , it also includes a coordinate conversion module;

[0138] The coordinate conversion module is used to use the ball camera to identify the target in the preset monitoring area, track the target, and then convert the ball camera coordinates of the target center point at the current magnification to the ball camera coordinates of the target center point at the single magnification based on the coordinate conversion relationship between the ball camera coordinates at multiple magnifications and the ball camera coordinates at the single magnification.

[0139] In some of the embodiments, the second identification and tracking module 230 is further used to overlay a mark on the image of the target based on the panoramic tracking information in the panoramic monitoring video frame of the gun camera;

[0140] Using preset multiple algorithm strategies, the target is tracked in panoramic coordinates based on superimposed markers.

[0141] In some embodiments, the target tracking device for gun-ball linkage provided by this embodiment is Figure 8 Based on it, it also includes a cache module;

[0142] The cache module is used to use the panoramic intelligent algorithm to control the gun to independently identify the panoramic object in each frame of the panoramic monitoring video frame, and track the panoramic object to cache the panoramic object information obtained by the multi-frame panoramic intelligent algorithm.

[0143] In some of the embodiments, the second identification and tracking module 230 based on the embodiment with the cache module includes a screening unit and a second identification unit;

[0144] A screening unit is used to control the gun camera to screen out the panoramic monitoring video frame that is closest to the current regional monitoring video frame of the ball camera in terms of the gun camera acquisition time based on the panoramic tracking information, and determine the panoramic object information corresponding to the panoramic monitoring video frame;

[0145] The second recognition unit is used to match the panoramic coordinates in the panoramic tracking information with the panoramic objects in the screened panoramic monitoring video frame to determine the target in the panoramic monitoring video frame and the corresponding identity information.

[0146] In some of the embodiments, when the target is a vehicle, the screening unit is further configured to compare the target vehicle speed in the panoramic tracking information with a preset vehicle speed threshold;

[0147] If the target vehicle speed is less than the preset vehicle speed threshold, it is determined as a low-speed scene. Based on the timestamp in the panoramic tracking information, the panoramic surveillance video frame closest to the current regional surveillance video frame of the ball camera in terms of the gun camera acquisition time is screened out, and the panoramic object information corresponding to the panoramic surveillance video frame is determined;

[0148] If the target vehicle speed is greater than or equal to the preset vehicle speed threshold, it is judged as a high-speed scene. Based on the preset delay time value and the timestamp in the panoramic tracking information, the panoramic surveillance video frame that is closest to the current area surveillance video frame of the ball camera in terms of the gun camera acquisition time is screened out, and the panoramic object information corresponding to the panoramic surveillance video frame is determined.

[0149] In some of the embodiments, the second identification unit is further used to perform distance statistics between the panoramic coordinates in the panoramic tracking information and the center point coordinates of each panoramic object in the filtered panoramic monitoring video frame to obtain the distance information of each panoramic object and the corresponding identity information;

[0150] Each distance information is compared pairwise to screen out the panoramic object with the minimum distance information and the corresponding identity information to determine the target and the corresponding identity information.

[0151] In some of the embodiments, the target tracking device for gun-ball linkage of the present embodiment further includes a position matching unit;

[0152] The position matching unit is used to perform multiple position matching in consecutive frames after determining the target and the corresponding identity information in the current panoramic monitoring video frame to determine the final target and the corresponding identity information.

[0153] In some of the embodiments, the target tracking device for gun-ball linkage of the present embodiment further includes a second tracking unit;

[0154] The second tracking unit is used to control the gun to track the target in panoramic coordinates based on the identity information after determining the target and the corresponding identity information using a preset multiple algorithm strategy.

[0155] In some of the embodiments, the gun-ball linkage target tracking device of the embodiment further includes a superposition unit;

[0156] The superimposition unit is used to, after tracking the target in panoramic coordinates based on identity information, overlay mark and display the image of the target based on the panoramic tracking information in the panoramic monitoring video frame of the gun camera.

[0157] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0158] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0159] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0160] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0161] S1, using the ball camera to identify the target in the preset monitoring area and track the target;

[0162] S2, based on the coordinate mapping relationship between the ball camera coordinates and the panoramic coordinates, converting the ball camera tracking information obtained by the ball camera tracking the target from the ball camera coordinates to the panoramic coordinates to obtain the panoramic tracking information of the target;

[0163] S3, controls the gun to identify and track the target in panoramic coordinates using a target matching algorithm based on the panoramic tracking information.

[0164] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0165] In addition, in combination with the gun-ball linkage target tracking method provided in the above embodiments, a storage medium can also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any gun-ball linkage target tracking method in the above embodiments is implemented.

[0166] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0167] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0168] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0169] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A target tracking method for gun-ball linkage, characterized in that: include: Using a ball camera to identify a target in a preset monitoring area and track the target; Based on the coordinate mapping relationship between the ball camera coordinates and the panoramic coordinates, the ball camera tracking information obtained by the ball camera tracking the target is converted from the ball camera coordinates to the panoramic coordinates to obtain the panoramic tracking information of the target; The gun is controlled to identify and track the target in the panoramic coordinates by using a target matching algorithm based on the panoramic tracking information; The control gun recognizes and tracks the target in the panoramic coordinates by using a target matching algorithm based on the panoramic tracking information, including: The gun camera is controlled to filter out a panoramic monitoring video frame that is closest to the current regional monitoring video frame of the ball camera in terms of acquisition time of the gun camera based on the panoramic tracking information, and determine the panoramic object information corresponding to the panoramic monitoring video frame; The panoramic coordinates in the panoramic tracking information are position matched with each panoramic object in the filtered panoramic monitoring video frame to determine the target in the panoramic monitoring video frame and the corresponding identity information.

2. The target tracking method of gun-ball linkage according to claim 1, characterized in that: The method of using a ball camera to identify a target in a preset monitoring area and tracking the target includes: Control the ball camera to obtain regional monitoring video frames of the preset monitoring area; Matching the target image in the regional monitoring video frame with the target image in a preset target database to identify the target; The ball camera is controlled to track the target using a tracking algorithm.

3. The target tracking method of gun-ball linkage according to claim 2, characterized in that: The target image is a license plate image; the target database is a license plate database.

4. The target tracking method of gun-ball linkage according to claim 2, characterized in that: The method of controlling the ball camera to track the target by using a tracking algorithm includes: The tracking algorithm is used to control the pan / tilt of the ball camera to actively track the target and predict the target speed.

5. The target tracking method of gun-ball linkage according to claim 1, characterized in that: Also includes: After using the ball camera to identify the target in the preset monitoring area and track the target, based on the coordinate conversion relationship between the ball camera coordinates at multiple magnifications and the ball camera coordinates at single magnifications, the ball camera coordinates of the target center point at the current magnification are converted to the ball camera coordinates of the target center point at single magnification.

6. The target tracking method of gun-ball linkage according to claim 1, characterized in that: The tracking of the target in the panoramic coordinates includes: In the panoramic monitoring video frame of the gun camera, an image of the target is superimposed and marked based on the panoramic tracking information; The target is tracked in the panoramic coordinates based on the superimposed markers using a preset multiple algorithm strategy.

7. The target tracking method of gun-ball linkage according to claim 1, characterized in that: Also includes: By using a panoramic intelligent algorithm, the gun is controlled to independently identify the panoramic object in each frame of the panoramic monitoring video frame, and track the panoramic object to cache the panoramic object information obtained by multiple frames of the panoramic intelligent algorithm.

8. The target tracking method of gun-ball linkage according to claim 1, characterized in that: When the target is a vehicle, the control gun camera screens out a panoramic monitoring video frame that is closest to the current regional monitoring video frame of the ball camera in terms of acquisition time of the gun camera based on the panoramic tracking information, and determines the panoramic object information corresponding to the panoramic monitoring video frame, including: Comparing the target vehicle speed in the panoramic tracking information with a preset vehicle speed threshold; If the target vehicle speed is less than the preset vehicle speed threshold, it is determined to be a low-speed scene, and based on the timestamp in the panoramic tracking information, the panoramic monitoring video frame closest to the current regional monitoring video frame of the ball camera in terms of the acquisition time of the gun camera is screened out, and the panoramic object information corresponding to the panoramic monitoring video frame is determined; If the target vehicle speed is greater than or equal to the preset vehicle speed threshold, it is determined to be a high-speed scene. Based on the preset delay time value and the timestamp in the panoramic tracking information, the panoramic surveillance video frame that is closest to the current area surveillance video frame of the ball camera in terms of the acquisition time of the gun camera is screened out, and the panoramic object information corresponding to the panoramic surveillance video frame is determined.

9. The target tracking method of gun-ball linkage according to claim 1, characterized in that: The step of matching the panoramic coordinates in the panoramic tracking information with the panoramic objects in the filtered panoramic monitoring video frame to determine the target in the panoramic monitoring video frame and the corresponding identity information includes: Perform distance statistics between the panoramic coordinates in the panoramic tracking information and the center point coordinates of each panoramic object in the filtered panoramic monitoring video frame to obtain distance information and corresponding identity information of each panoramic object; Each of the distance information is compared in pairs, and the panoramic object with the smallest distance information and the corresponding identity information are screened out to determine the target and the corresponding identity information.

10. The target tracking method of gun-ball linkage according to claim 1, characterized in that: Also includes: After determining the target and the corresponding identity information in the panoramic monitoring video frame, multiple position matching is performed in consecutive frames to determine the final target and the corresponding identity information.

11. The target tracking method of gun-ball linkage according to claim 1, characterized in that: Also includes: After determining the target and the corresponding identity information, the gun is controlled to track the target in the panoramic coordinates based on the identity information and using a preset multiple algorithm strategy.

12. The target tracking method of gun-ball linkage according to claim 11, characterized in that: Also includes: After the target is tracked in the panoramic coordinates based on the identity information, the image of the target is superimposed, marked and displayed in the panoramic monitoring video frame of the gun camera based on the panoramic tracking information.

13. A target tracking device for gun-ball linkage, characterized in that: include: A first identification and tracking module, a conversion module, and a second identification and tracking module; The first identification and tracking module is used to use the ball camera to identify the target in the preset monitoring area and track the target; The conversion module is used to convert the ball camera tracking information obtained by the ball camera tracking the target from the ball camera coordinates to the panoramic coordinates based on the coordinate mapping relationship between the ball camera coordinates and the panoramic coordinates, so as to obtain the panoramic tracking information of the target; The second identification and tracking module is used to control the gun to identify and track the target in the panoramic coordinates using a target matching algorithm based on the panoramic tracking information; The second identification and tracking module is further used to control the gun camera to filter out the panoramic monitoring video frame that is closest to the current regional monitoring video frame of the ball camera in terms of the acquisition time of the gun camera based on the panoramic tracking information, and determine the panoramic object information corresponding to the panoramic monitoring video frame; The panoramic coordinates in the panoramic tracking information are position matched with each panoramic object in the filtered panoramic monitoring video frame to determine the target in the panoramic monitoring video frame and the corresponding identity information.

14. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the gun-ball linked target tracking method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the gun-ball linked target tracking method according to any one of claims 1 to 12 are implemented.

Citation Information

Patent Citations

  • Target object tracking method and device and network video recording equipment

    CN111402304A

  • Tracking shooting method and device and monitoring system

    CN112492261A