Cooperative tracking method and device of gun and ball machine, computer device and medium

Abstract

The application relates to the security monitoring technical field and provides a cooperative tracking method and device of a gun camera and a ball camera, computer equipment and a medium. The method comprises the following steps: acquiring touch operation information of a panoramic video picture collected by the gun camera; converting the touch operation information into a normalized coordinate instruction to lock a target object corresponding to the normalized coordinate instruction in the panoramic video picture; performing electronic cutting on the panoramic video picture so that the target object is always located at a center position of the panoramic video picture; calculating object parameters corresponding to the target object in the panoramic video picture; driving the ball camera to rotate according to the object parameters, acquiring a locking picture output by the gun camera and a zoomed-in picture collected by the ball camera; and realizing cooperative tracking of the target object according to the locking picture and the zoomed-in picture. According to the application, the professional equipment is not needed, the target locking and tracking can be realized through the touch operation of a mobile terminal, one-point tracking and one-frame zooming are realized, and remote control at any time and any place is supported.

Description

Technical Field

[0001] This application relates to the field of security monitoring technology, and in particular to a method, device, computer equipment, and medium for collaborative tracking of bullet cameras and PTZ cameras. Background Technology

[0002] Currently, PTZ (Pocket Camera and Ball Control) systems are widely used in various security monitoring scenarios. However, existing PTZ systems rely on dedicated hardware keyboards or computer clients for operation, and cannot be remotely controlled anytime and anywhere via mobile terminals, resulting in cumbersome operation procedures and affecting the user experience. Summary of the Invention

[0003] This application provides a method, device, computer equipment, and medium for collaborative tracking of bullet and PTZ cameras, aiming to solve the problem that existing bullet-PTZ linkage systems rely on dedicated hardware keyboards or computer clients for operation, and cannot be remotely controlled anytime and anywhere via mobile terminals, resulting in cumbersome operation steps and affecting the user experience.

[0004] In a first aspect, embodiments of this application provide a collaborative tracking method for bullet cameras and PTZ cameras, the method comprising: Acquire touch operation information from the panoramic video footage captured by the camera; The touch operation information is converted into normalized coordinate instructions to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame; The panoramic video image is electronically cropped so that the target object is always in the center of the panoramic video image; The object parameters corresponding to the target object are calculated in the panoramic video frame; the object parameters include at least one or more of the PTZ camera's horizontal angle, vertical angle, and zoom ratio. The PTZ camera is driven to rotate according to the object parameters, and the locked image output by the bullet camera and the magnified image captured by the PTZ camera are obtained. The target object is tracked collaboratively based on the locked and zoomed-in views.

[0005] In some embodiments, converting the touch operation information into normalized coordinate instructions to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame includes: parsing the touch operation information to obtain the operation type corresponding to the touch operation information and the corresponding display coordinates on the panoramic video frame; converting the display coordinates into normalized coordinates of the original frame corresponding to the panoramic video frame; and generating the normalized coordinate instructions according to the operation type and the normalized coordinates to lock the target area corresponding to the normalized coordinate instructions in the panoramic video frame.

[0006] In some embodiments, electronically cropping the panoramic video frame to ensure that the target object is always at the center of the panoramic video frame includes: obtaining the center pixel coordinates of the target object in the panoramic video frame; calculating the cropping start coordinates of the panoramic video frame based on the center pixel coordinates; performing smoothing filtering on the cropping start coordinates; cropping the panoramic video frame based on the processed cropping start coordinates; and outputting a locked frame centered on the target object.

[0007] In some embodiments, the step of calculating the object parameters corresponding to the target object in the panoramic video frame includes: obtaining pre-calibrated spatial mapping parameters of the bullet camera and the PTZ camera; converting the pixel coordinates of the target object in the panoramic video frame into a direction vector in the PTZ camera coordinate system; calculating the horizontal and vertical angles corresponding to the PTZ camera based on the direction vector; and calculating the zoom ratio corresponding to the PTZ camera based on the size of the target object in the panoramic video frame.

[0008] In some embodiments, driving the PTZ camera to rotate according to the object parameters and acquiring the locked image output by the bullet camera and the magnified image captured by the PTZ camera includes: generating a PTZ camera drive command based on the horizontal angle, vertical angle, and zoom ratio; sending the PTZ camera drive command to the PTZ camera to control the PTZ camera to rotate according to the horizontal angle and vertical angle, and to adjust according to the zoom ratio; receiving the magnified image captured by the PTZ camera and the locked image output by the bullet camera, and performing time synchronization processing on the locked image and the magnified image.

[0009] In some embodiments, the collaborative tracking of the target object based on the locked screen and the zoomed-in screen includes: tracking the target object in the locked screen and continuously acquiring the real-time position information of the target object; calculating the position change information of the target object based on the continuous real-time position information; when the position change exceeds a preset change range, recalculating the object parameters corresponding to the PTZ camera; and adjusting the horizontal angle, vertical angle, and zoom ratio of the PTZ camera based on the recalculated object parameters to maintain continuous tracking of the target object.

[0010] In some embodiments, before the collaborative tracking of the target object is performed based on the locked screen and the magnified screen, the method further includes: if fine-tuning operation information is received on the locked screen and / or the magnified screen, adjusting the angle and / or zoom of the PTZ camera in response to the fine-tuning operation information; and after the fine-tuning operation information is responded to, performing collaborative tracking of the target object based on the locked screen and the magnified screen.

[0011] Secondly, this application provides a collaborative tracking device for bullet and PTZ cameras, comprising: The information acquisition unit is used to acquire touch operation information of the panoramic video images captured by the camera. An object locking unit is used to convert the touch operation information into normalized coordinate instructions in order to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame. The image cropping unit is used to electronically crop the panoramic video image so that the target object is always in the center of the panoramic video image. The parameter calculation unit is used to calculate the object parameters corresponding to the target object in the panoramic video frame; the object parameters include at least one or more of the PTZ camera's horizontal angle, vertical angle, and zoom ratio. The PTZ camera drive unit is used to drive the PTZ camera to rotate according to the object parameters, and to acquire the locked image output by the bullet camera and the magnified image captured by the PTZ camera; The collaborative tracking unit is used to achieve collaborative tracking of the target object based on the locked screen and the zoomed-in screen.

[0012] Thirdly, this application provides a computer device, including a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the method provided in any embodiment of this application.

[0013] Fourthly, this application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to implement the method provided in any embodiment of this application.

[0014] This application eliminates the need for specialized equipment, enabling target locking and tracking via mobile terminal touch control. It allows for point-to-point tracking and frame zooming, and supports remote control anytime, anywhere.

[0015] Meanwhile, the camera uses electronic cropping to keep the target always centered, preserving the overall field of view while ensuring the target is not lost, and the PTZ camera automatically turns and zooms to provide clear and detailed images; By normalizing coordinate transformation and pre-calibrating spatial mapping, the accurate calculation of the bullet camera pixel coordinates to the PTZ camera control parameters is achieved, significantly reducing the linkage response delay. The provided method supports seamless switching between automatic intelligent tracking and manual fine-tuning, and the dual-screen synchronous display facilitates real-time monitoring and intervention by users, improving tracking stability in complex scenarios.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart illustrating the steps of a collaborative tracking method for bullet and PTZ cameras provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the principle of a collaborative tracking method between a bullet camera and a PTZ camera according to an embodiment of this application; Figure 3 This is a schematic block diagram of the structure of a collaborative tracking device for bullet and PTZ cameras provided in one embodiment of this application; Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0022] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0023] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] Currently, PTZ (Pocket Camera and Ball Control) systems are widely used in various security monitoring scenarios. However, existing PTZ systems rely on dedicated hardware keyboards or computer clients for operation, and cannot be remotely controlled anytime and anywhere via mobile terminals, resulting in cumbersome operation procedures and affecting the user experience.

[0026] Please refer to Figure 1 This application provides a collaborative tracking method for bullet and PTZ cameras, applied to computer equipment. The computer equipment can be deployed on a single server or server cluster, or on a handheld terminal, laptop, wearable device, or robot, etc. It should be noted that all information involved in the method provided in this application is extracted with the authorization of the relevant user and in accordance with relevant regulations, and will not infringe on user privacy.

[0027] The provided method for collaborative tracking of bullet and PTZ cameras includes steps S101 to S106. Details are as follows: Step S101. Obtain touch operation information of the panoramic video image captured by the camera.

[0028] Specifically, this step is implemented through a native application running on the mobile terminal. This application retrieves the main stream video data from the fixed camera via Real-time Streaming Protocol (RTSP) or Web Real-time Communication Protocol (WebRTC), and decodes the video data using a video decoding module. The decoded panoramic video frames are then rendered onto the terminal's touch display surface via a graphics rendering interface. The canvas layer of the touch display surface completely overlaps with the video rendering layer to ensure a one-to-one correspondence between the coordinates of the touch operation and the pixel coordinates of the video image.

[0029] The application continuously listens for user touch operations through the touch event capture interface provided by the operating system. When a user touch action is detected, it obtains the operation type corresponding to the touch event, the coordinates of the touch point in the canvas coordinate system, and the timestamp of the touch event. The operation types include single-point click, rectangular selection, and drag-and-drop fine-tuning.

[0030] For example, in this step, the application simultaneously receives and displays the sub-stream video data output by the high-speed PTZ camera in the form of a floating window. The sub-stream video has a resolution of 1280×720 and a frame rate of 25 frames per second, to ensure that the user can observe the global scene and target details at the same time during operation.

[0031] Step S102. Convert the touch operation information into normalized coordinate instructions to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame.

[0032] Specifically, this step first parses the touch operation information obtained in step S101, extracting the operation type and the display coordinates of the touch point in the canvas coordinate system. Since the terminal display resolution and the original video resolution of the camera are usually inconsistent, it is necessary to establish a mapping relationship from display coordinates to original pixel coordinates. Specifically, assuming the original video resolution of the camera is W×H and the display resolution of the terminal canvas is Wc×Hc, the scaling factor s=(Wc / W,Hc / H) is calculated, and the display coordinates (xc,yc) are converted into the pixel coordinates (xp,yp)=(xc / sx,yc / sy) of the original frame of the camera.

[0033] Furthermore, the original pixel coordinates are normalized to the 0~1 interval, resulting in normalized coordinates (xn, yn) = (xp / (W-1), yp / (H-1)). Based on different operation types, corresponding normalized coordinate instructions are generated: For single-point click operations, a command containing the instruction type identifier "click" and the normalized target point (xn, yn) is generated. For rectangular selection operations, the center point (xc, yc) = ((xs+xe) / 2, (ys+ye) / 2) and the width and height (w, h) of the rectangular area are calculated, where (xs, ys) are the coordinates of the starting point of the rectangle and (xe, ye) are the coordinates of the ending point of the rectangle. After normalizing the center point and the width and height, a command containing the instruction type identifier "rect_select", the normalized center point (xn, yn), and the normalized area ratio (wn, hn) = (w / Wc, h / Hc) is generated. For drag-and-tune operations, the displacement increment (dx, dy) of the touch point and the scaling factor are calculated, and an incremental command containing the instruction type identifier "fine_tune", the horizontal displacement increment dx, the vertical displacement increment dy, and the scaling factor is generated.

[0034] The generated normalized coordinate command is sent to the edge master control unit, which locks the corresponding target object in the panoramic video frame according to the normalized coordinate in the command.

[0035] Step S103. Electronically crop the panoramic video frame to ensure that the target object is always in the center of the panoramic video frame.

[0036] Specifically, this step leverages the high resolution of the camera module to achieve target centering and locking using electronic pan-tilt-zoom (ePTZ) technology. Specifically, the edge control unit continuously acquires raw YUV420 format video frames from the camera module's video stream. After target locking, the tracker continuously outputs the target's bounding box (bbox) in the raw frames and calculates the target's center pixel coordinates (Cx, Cy).

[0037] The ePTZ output resolution is set to W_out × H_out. To ensure the target remains geometrically centered in the output image, the desired top-left corner coordinates of the crop are calculated as (left_target, top_target) = (Cx - W_out / 2, Cy - H_out / 2). To prevent image jitter caused by target movement, the crop coordinates are smoothed using a second-order low-pass filter with a cutoff frequency of 3Hz and a smoothing factor α = 0.15. The filtering formula is as follows: left_new=α×left_target+(1-α)×left_current; top_new=α×top_target+(1-α)×top_current; Where left_current and top_current are the coordinates of the top left corner of the cropped frame in the previous frame. The filtered cropped coordinates are constrained to the effective range: left_new∈[0,W-W_out], top_new∈[0,H-H_out].

[0038] The edge control unit crops the original video frames according to the processed cropping coordinates, generates a locked screen centered on the target, and pushes the locked screen to the mobile terminal application for display.

[0039] Step S104. Calculate the object parameters corresponding to the target object in the panoramic video frame; the object parameters include at least one or more of the PTZ camera's horizontal angle, vertical angle, and zoom ratio.

[0040] Specifically, this step, based on pre-calibrated spatial mapping parameters between the bullet camera and the PTZ camera, converts the pixel coordinates of the target in the bullet camera's view into the absolute position parameters of the PTZ camera. The spatial mapping parameters include the bullet camera's intrinsic parameter matrix K, lens distortion coefficients, and the extrinsic parameter rotation matrix R and translation vector T between the bullet and PTZ cameras. The specific calculation process includes converting the target's pixel coordinates P_raw=(u,v) in the bullet camera's original frame into a normalized three-dimensional direction vector in the bullet camera's coordinate system, including: V_gun = K_inv × [u,v,1] T ; Where K_inv is the inverse of the gun's intrinsic parameter matrix.

[0041] Based on the extrinsic parameters between the camera and the PTZ camera, the direction vector is transformed to the PTZ camera coordinate system, including: V_dome = R × V_gun + T; The horizontal angle Pan(θ) and vertical angle Tilt(φ) of the PTZ camera are calculated from the direction vector in the PTZ camera coordinate system: θ=atan2(V_dome.x,V_dome.z); φ = asin(V_dome.y / |V_dome|); The calculated radian value is converted into an angle system and then adjusted according to the quadrant of the PTZ camera's mechanical coordinate system.

[0042] The zoom ratio of the PTZ camera is calculated by dynamically setting the magnification ratio based on the initial height H_obj of the target bounding box for single-point click operations, so that the target occupies 15% to 25% of the height of the PTZ camera image. Specifically, a pixel-magnification lookup table is pre-established, mapping the target's height pixel value to the corresponding zoom value, so that the height of the target's bounding box within the PTZ camera image is 200 to 280 pixels.

[0043] For rectangular selection operations, calculate the zoom ratio required to make the rectangular area exactly fill the PTZ camera's image. A fast algorithm is used: ZF = Z_max × min(1.0, a / wn), where Z_max is the PTZ camera's maximum zoom ratio, a is an empirical coefficient of 0.3, and wn is the normalized width of the rectangular area, while limiting the zoom ratio to no more than 20x.

[0044] Step S105. Drive the PTZ camera to rotate according to the object parameters, and acquire the locked image output by the bullet camera and the magnified image captured by the PTZ camera.

[0045] Specifically, this step first encapsulates the horizontal angle, vertical angle, and zoom ratio of the PTZ camera obtained in step S104 into PTZ camera drive commands. The drive commands adopt the AbsoluteMove command in the ONVIF standard protocol, setting the Pan value to θ°, the Tilt value to φ°, the movement speed to 80% of the PTZ camera's maximum speed, and the Zoom value to the calculated zoom ratio.

[0046] The packaged drive commands are sent to the high-speed PTZ camera via the network. Upon receiving the commands, the PTZ camera quickly rotates to the specified angle and adjusts the zoom level according to the command parameters. Simultaneously, the edge control unit continuously receives the locked image output from the bullet camera and the magnified image captured by the PTZ camera, and performs time synchronization processing on the two video streams.

[0047] To ensure video synchronization accuracy, both the mobile terminal and the edge control unit synchronize their clocks via the Network Time Protocol (NTP). Both video streams carry precise timestamps, and the application renders the locked and zoomed-in frames synchronously based on the timestamps, ensuring that the latency difference between the two frames is less than 200ms.

[0048] Step S106. Implement collaborative tracking of the target object based on the locked screen and the zoomed-in screen.

[0049] Specifically, this step achieves continuous coordinated tracking of the target by continuously tracking changes in its position and dynamically adjusting the electronic cropping parameters of the camera and the PTZ parameters of the PTZ camera. Specifically: The edge control unit maintains the target tracker running at a rate of 25 frames per second, continuously outputting the target's bounding box and center coordinates in the original gun frame. The tracker employs a scale-adaptive kernel correlation filter (DSST) to adapt to changes in target scale.

[0050] Every 300ms or when the target center displacement exceeds 5 pixels, the absolute target position of the PTZ camera is recalculated, and the PTZ camera's motion is adjusted using a proportional-derivative (PID) controller. The PID controller has a proportional coefficient P=0.8 and a derivative coefficient D=0.2. When the position error |e_pan|>0.3°, variable speed motion is triggered to ensure smooth tracking of the PTZ camera without overshoot.

[0051] Meanwhile, the ePTZ locking module continuously calculates new cropping coordinates based on the target's real-time position, updates and pushes the locking screen.

[0052] When the peak sidelobe ratio of the tracking response is less than 5.0, the target is determined to be lost, and a re-detection mechanism is initiated: the search area is expanded to 1.5 times the range of the original frame, the target detection network is called to search for similar targets again, and if the target is detected again within 3 seconds, tracking is resumed; if the target is not detected, tracking is stopped and a target loss prompt is sent to the application.

[0053] When a new target tracking instruction is received from the user, the old tracking trajectory is cleared and the tracking is switched to the new target.

[0054] In some embodiments, converting the touch operation information into normalized coordinate instructions to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame includes: parsing the touch operation information to obtain the operation type corresponding to the touch operation information and the corresponding display coordinates on the panoramic video frame; converting the display coordinates into normalized coordinates of the original frame corresponding to the panoramic video frame; and generating the normalized coordinate instructions according to the operation type and the normalized coordinates to lock the target area corresponding to the normalized coordinate instructions in the panoramic video frame.

[0055] This embodiment details the specific implementation of step S102, "converting the touch operation information into normalized coordinate instructions to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame".

[0056] First, the touch operation information is parsed to obtain the operation type and corresponding display coordinates on the panoramic video screen. The application obtains a MotionEvent object through the operating system's touch event interface, and extracts the touch action type (ACTION_DOWN, ACTION_MOVE, ACTION_UP), the number of touch points, and the coordinates (xc, yc) of each touch point from this object. Based on the different touch actions, the corresponding operation type is identified. A quick press and release action on a single touch point is recognized as a single-point click operation; The action of pressing a single touch point, sliding a certain distance, and then lifting it is recognized as a rectangular selection operation, and the coordinates of the start and end points of the sliding process are recorded; In PTZ camera tracking mode, a sliding motion of a single touch point is identified as a translation fine-tuning operation, while a pinching or opening motion of two touch points is identified as a zoom fine-tuning operation.

[0057] Next, the display coordinates are converted into normalized coordinates of the original frame corresponding to the panoramic video image. Assuming the original video resolution of the camera is 3840×2160 and the display resolution of the terminal canvas is 1920×1080, then the scaling factor s = (1920 / 3840, 1080 / 2160) = (0.5, 0.5). For the display coordinates (xc, yc), this is converted to the original pixel coordinates (xp, yp) = (xc / 0.5, yc / 0.5) = (2xc, 2yc). Further normalization yields (xn, yn) = (2xc / 3839, 2yc / 2159).

[0058] Finally, a normalized coordinate instruction is generated based on the operation type and the normalized coordinates to lock the target area corresponding to the normalized coordinate instruction in the panoramic video frame. The generated instruction is structured data in JSON format, containing a unique message number, instruction type, target parameters, and timestamp. After receiving the instruction, the edge control unit performs the corresponding target locking operation according to the instruction type: for example, for the "click" instruction, a 128×128 pixel region of interest (ROI) is created around the position corresponding to the normalized coordinates in the original frame and sent to the target detection network for detection. The target bounding box closest to the click position and with a confidence score greater than 0.7 is selected as the tracking target; if no target meeting the conditions is detected, a 200×200 pixel rectangle is generated centered on the click position as the tracking area.

[0059] For the "rect_select" instruction, the target region is directly delineated in the original frame based on the normalized center coordinates and region width and height. The detector is not started, and the tracker is initialized directly with this region.

[0060] In some embodiments, electronically cropping the panoramic video frame to ensure that the target object is always at the center of the panoramic video frame includes: obtaining the center pixel coordinates of the target object in the panoramic video frame; calculating the cropping start coordinates of the panoramic video frame based on the center pixel coordinates; performing smoothing filtering on the cropping start coordinates; cropping the panoramic video frame based on the processed cropping start coordinates; and outputting a locked frame centered on the target object.

[0061] This embodiment details the specific implementation of step S103, "electronically cropping the panoramic video image so that the target object is always in the center of the panoramic video image".

[0062] First, the center pixel coordinates of the target object in the panoramic video frame are obtained. The tracker of the edge control unit outputs the bounding box information of the target in each frame, in the format (x1, y1, x2, y2), where (x1, y1) are the coordinates of the top-left corner of the bounding box, and (x2, y2) are the coordinates of the bottom-right corner of the bounding box. The center pixel coordinates (Cx, Cy) of the target are calculated as follows: Cx = (x1 + x2) / 2; Cy = (y1 + y2) / 2; Secondly, the cropping start coordinates of the panoramic video image are calculated based on the center pixel coordinates, and then smoothed using a filtering process. Assuming the ePTZ output resolution is 1920×1080, the desired top-left corner coordinates for cropping are: left_target=Cx-960; top_target=Cy-540; To avoid image jitter, a second-order low-pass filter is used to smooth the crop coordinates. During initialization, `left_current` and `top_current` are set to 0. During each frame update: left_new=0.15×left_target+0.85×left_current; top_new=0.15×top_target+0.85×top_current; Constrain left_new between 0 and 1920, and top_new between 0 and 1080 to ensure that the cropped area does not exceed the original image area.

[0063] Finally, the panoramic video image is cropped according to the processed cropping start coordinates, outputting a locked image centered on the target object. The edge control unit uses the cv::Rect class of the OpenCV library to create the cropping region and calls the cv::Mat::operator() function to crop the original YUV420 frames. The cropped frames are encoded in H.264 format and pushed to the mobile terminal application via the RTSP protocol. After receiving the locked stream, the application replaces the original panoramic stream for display, and the user will see that the target always remains in the center of the image, and the background smoothly shifts as the target moves.

[0064] In some embodiments, the step of calculating the object parameters corresponding to the target object in the panoramic video frame includes: obtaining pre-calibrated spatial mapping parameters of the bullet camera and the PTZ camera; converting the pixel coordinates of the target object in the panoramic video frame into a direction vector in the PTZ camera coordinate system; calculating the horizontal and vertical angles corresponding to the PTZ camera based on the direction vector; and calculating the zoom ratio corresponding to the PTZ camera based on the size of the target object in the panoramic video frame.

[0065] This embodiment details the specific implementation of step S104, "calculating the object parameters corresponding to the target object in the panoramic video frame".

[0066] First, the pre-calibrated spatial mapping parameters of the bullet camera and the PTZ camera are obtained. During system initialization, the edge control unit broadcasts the calibration parameters of the bullet camera and the PTZ camera, including the bullet camera's intrinsic parameter matrix K, distortion coefficients, extrinsic parameter rotation matrix R, and translation vector T. If precise calibration is not performed, a manual reference point calibration method is used: the user specifies at least four pairs of bullet camera screen coordinates and corresponding PTZ values ​​on the application, and the homography matrix H is calculated using the least squares method for subsequent coordinate mapping.

[0067] Next, the pixel coordinates of the target object in the panoramic video frame are converted into a direction vector in the PTZ camera's coordinate system. For the target's original pixel coordinates P_raw=(u,v), distortion correction is first performed using the camera's distortion coefficients to obtain the corrected pixel coordinates (u',v'). Then, the normalized direction vector is calculated: x = (u' - cx) / fx; y = (v' - cy) / fy; V_gun=[x,y,1] T ; Where (cx, cy) are the principal points of the bolt carrier, and (fx, fy) are the focal lengths of the bolt carrier.

[0068] Transform the direction vector to the PTZ camera coordinate system using the extrinsic parameter matrix: V_dome = R × V_gun + T; Third, calculate the horizontal and vertical angles corresponding to the PTZ camera based on the stated direction vector. Calculate the azimuth angle θ and pitch angle φ of the PTZ camera: θ=atan2(V_dome.x,V_dome.z)×180 / π; φ=asin(V_dome.y / sqrt(V_dome.x²+V_dome.y²+V_dome.z²))×180 / π; Adjust the angle range according to the mechanical coordinate system of the PTZ camera, for example, adjust θ to 0°~360° and φ to -90°~90°.

[0069] Finally, the zoom ratio corresponding to the PTZ camera is calculated based on the size of the target object in the panoramic video frame. For a single-point click operation, the height H_obj of the target frame is obtained as y2-y1. A pre-established pixel-zoom ratio lookup table is consulted to find the zoom ratio Z corresponding to a target height of 240 pixels in the PTZ camera frame. For example, when H_obj = 40 pixels, the corresponding zoom ratio is 6x; when H_obj = 20 pixels, the corresponding zoom ratio is 12x.

[0070] For rectangular selection, the normalized width wn = w / 1920 is calculated, where w is the display width of the rectangle. A fast algorithm is used to calculate the zoom ratio. ZF = 20 × min(1.0, 0.3 / wn); If the calculated result is greater than 20, then 20 will be taken as the final zoom ratio.

[0071] In some embodiments, driving the PTZ camera to rotate according to the object parameters and acquiring the locked image output by the bullet camera and the magnified image captured by the PTZ camera includes: generating a PTZ camera drive command based on the horizontal angle, vertical angle, and zoom ratio; sending the PTZ camera drive command to the PTZ camera to control the PTZ camera to rotate according to the horizontal angle and vertical angle, and to adjust according to the zoom ratio; receiving the magnified image captured by the PTZ camera and the locked image output by the bullet camera, and performing time synchronization processing on the locked image and the magnified image.

[0072] This embodiment details the specific implementation of step S105, "driving the PTZ camera to rotate according to the object parameters, and acquiring the locked image output by the bullet camera and the magnified image captured by the PTZ camera."

[0073] First, the PTZ camera drive commands are generated based on the horizontal angle, vertical angle, and zoom ratio. The application encapsulates the calculated PTZ parameters into ONVIF standard AbsoluteMove commands, which include: ProfileToken: the PTZ camera's media configuration identifier; Position: containing the absolute position values ​​of Pan, Tilt, and Zoom; Speed: containing the movement speed values ​​of Pan, Tilt, and Zoom.

[0074] Next, the PTZ camera drive command is sent to the PTZ camera to control its rotation according to the horizontal and vertical angles, and to adjust according to the zoom ratio. The command is published to the topic / camera / ctrl / dome01 via the MQTT protocol, with a QoS level set to 1 to ensure reliable transmission. To handle a large number of instantaneous fine-tuning commands, the edge control unit maintains a command buffer queue with a deduplication mechanism, retaining only the latest fine-tuning commands for transmission to avoid cumulative latency.

[0075] Finally, the system receives the magnified image captured by the PTZ camera and the locked image output by the bullet camera, and performs time synchronization processing on the locked image and the magnified image. The edge control unit pulls the locked stream from the bullet camera and the magnified stream from the PTZ camera via the RTSP protocol, and adds an NTP synchronization timestamp to each frame of video. After receiving the two video streams, the mobile terminal application performs synchronous rendering based on the timestamps. When the time difference between the two frames is less than 200ms, both frames are displayed simultaneously; if the time difference is greater than 200ms, the application waits for the later frame to arrive before displaying it.

[0076] In some embodiments, the collaborative tracking of the target object based on the locked screen and the zoomed-in screen includes: tracking the target object in the locked screen and continuously acquiring the real-time position information of the target object; calculating the position change information of the target object based on the continuous real-time position information; when the position change exceeds a preset change range, recalculating the object parameters corresponding to the PTZ camera; and adjusting the horizontal angle, vertical angle, and zoom ratio of the PTZ camera based on the recalculated object parameters to maintain continuous tracking of the target object.

[0077] This embodiment details the specific implementation of step S106, "achieving collaborative tracking of the target object based on the locked screen and the zoomed-in screen".

[0078] First, the target object in the locked frame is tracked, and its real-time position information is continuously acquired. The edge control unit initializes the DSST tracker, extracting the features of the initial target bounding box as a tracking template. The tracker runs at a rate of 25 frames per second, performing feature matching on the current frame each frame and outputting the target's bounding box and response value.

[0079] Secondly, the position change information of the target object is calculated based on the continuous real-time position information. The center coordinates (Cx_prev, Cy_prev) of the target are recorded when the PTZ camera parameters were last updated. The displacement of the target center in the current frame relative to the last update is calculated, including: dx = Cx_current - Cx_prev; dy = Cy_current - Cy_prev; displacement magnitude d = sqrt(dx 2 +dy 2 ).

[0080] Third, when the position change exceeds a preset range, the object parameters corresponding to the PTZ camera are recalculated. The preset update conditions are: the time since the last update exceeds 300ms, or the target displacement d exceeds 5 pixels. When either condition is met, the recalculation of the PTZ camera parameters is triggered.

[0081] Finally, the horizontal angle, vertical angle, and zoom ratio of the PTZ camera are adjusted based on the recalculated object parameters to maintain continuous tracking of the target object. A PID controller is used to calculate the PTZ camera's movement speed. v_pan=Kp×e_pan+Kd×(e_pan-e_pan_prev) / dt; Where e_pan = θ_desired - θ_current, Kp = 0.8, Kd = 0.2, and dt is the frame interval. When |e_pan| > 0.3°, a PTZ relative motion command is sent to adjust the PTZ camera position. Simultaneously, the ePTZ module continuously updates the cropping coordinates to ensure the target remains centered on the locked image.

[0082] When the peak sidelobe ratio of the tracking response is less than 5.0, the target is determined to be lost. At this time, the edge control unit expands the search area to 1.5 times the size of the original frame and calls the YOLOv5s target detection network to perform a full-image search. If a target of the same category as the initial target and with a confidence score greater than 0.7 is detected within 3 seconds, the target is taken as the new tracking target and tracking is resumed; if no target is detected within 3 seconds, tracking stops and a target loss notification is sent to the application.

[0083] In some embodiments, before the collaborative tracking of the target object is performed based on the locked screen and the magnified screen, the method further includes: if fine-tuning operation information is received on the locked screen and / or the magnified screen, adjusting the angle and / or zoom of the PTZ camera in response to the fine-tuning operation information; and after the fine-tuning operation information is responded to, performing collaborative tracking of the target object based on the locked screen and the magnified screen.

[0084] In PTZ camera tracking mode, the application continuously monitors touch events in the small area of ​​the PTZ camera's screen. Single-finger swipe operation: Calculate the swipe vector (dx, dy), which is mapped to the PTZ camera's angle increment. delta_pan = dx × 0.02°; delta_tilt = -dy × 0.02°; A PTZ relative motion command is generated and sent to the edge master control unit. The master control unit immediately executes the PTZRelativeMove operation, adding an incremental value to the current angle.

[0085] Two-finger pinch operation: Calculate the scale_factor, the percentage change in distance between the two touch points, and map it to the zoom increment. delta_z=(scale_factor-1)×2.0; If delta_z > 0, zoom in; if delta_z < 0, zoom out. The zoom ratio is limited to between 1x and 20x.

[0086] During manual fine-tuning, the edge control unit pauses the angle calculation and update of automatic tracking, but continues to run the tracker and calculate the target's coordinates. When the user raises their finger (ACTION_UP event), a 0.5-second delay timer is started. After the timer expires, automatic tracking mode is automatically resumed, the absolute position of the PTZ camera is recalculated based on the target's current coordinates output by the tracker, and the PTZ camera is moved to the target position in a smooth transition to avoid screen jumps.

[0087] In some embodiments, such as Figure 2 As shown, the system of this application includes, from left to right, a user operation terminal, a coordinate transformation unit, a bullet processing unit, a parameter calculation unit, bullet and PTZ camera hardware devices, and a dual-screen output module.

[0088] The user terminal, located on the far left, is a mobile terminal with a touchscreen. The screen displays a panoramic monitoring view captured by the camera, showing a moving target object. The diagram illustrates a finger tapping the screen, corresponding to step 1: acquiring touch operation information from the camera's panoramic video view. The user initiates a tracking request by tapping the target object on the screen.

[0089] The coordinate transformation unit is connected to the user's operating terminal and receives touch operation information. This corresponds to step 2: converting to normalized coordinate instructions → locking the target object in the panoramic view. This unit converts the displayed coordinates clicked by the user into normalized coordinates of the original frame of the camera and generates structured instructions. The target object is enclosed in a solid line in the diagram, indicating the target lock status.

[0090] The camera processing unit is connected to the coordinate transformation unit and receives normalized coordinate commands. This corresponds to step 3: electronic cropping of the panoramic video to keep the target centered. This unit electronically crops the panoramic video based on the target's real-time position, outputting a locked image centered on the target. The image shows the cropped image, with the target positioned in the exact center.

[0091] The parameter calculation unit is connected to the camera module processing unit and receives the target's pixel coordinate information. This corresponds to step 4: calculating target object parameters → outputting the PTZ camera's horizontal / vertical angle / zoom ratio. This unit converts the target's pixel coordinates into the PTZ camera's parameters based on pre-calibrated spatial mapping parameters.

[0092] The bullet camera and PTZ camera hardware are connected to the parameter calculation unit. The bullet camera is fixedly installed and continuously captures panoramic video; the PTZ camera is a high-speed PTZ camera that can rotate and adjust focus. Based on the PTZ parameters output by the parameter calculation unit, it adjusts its angle and zoom level to capture magnified details of the target. This corresponds to step 5: driving the PTZ camera to rotate and adjust focus according to the parameters, outputting dual images.

[0093] The dual-view output module is located on the far right and is divided into two screens. The left screen displays a small panoramic view from the camera, showing the overall scene and target position; the right screen displays a large close-up view from the PTZ camera, showing detailed information about the target. This corresponds to step 6: Implementing collaborative target tracking based on the dual screens.

[0094] The entire system forms a closed loop: user operation → coordinate transformation → target locking → electronic cropping → parameter calculation → PTZ camera drive → dual-screen display → continuous tracking. The modules are connected by arrows, clearly demonstrating the flow of data and commands.

[0095] In some embodiments, after the application starts, it connects to the edge control unit via Wi-Fi or a cellular network. The edge control unit discovers the bullet camera and PTZ camera devices via the local area network and establishes a communication connection. The application pulls the main stream (3840×2160@25fps) of the bullet camera and the sub-stream (1280×720@25fps) of the PTZ camera via the RTSP protocol. The edge control unit broadcasts the calibration parameters of the bullet camera and PTZ camera, including the bullet camera's intrinsic parameter matrix K=[[1800,0,1920],[0,1800,1080],[0,0,1]], distortion coefficients [0.1,-0.05,0,0,0], and the extrinsic parameter rotation matrix R and translation vector T=[0.5,0,0.2] (unit: meters) of the PTZ camera relative to the bullet camera.

[0096] If precise calibration is not performed, the system enters manual calibration mode: the user clicks on the four reference points in the camera's view on the application and manually adjusts the camera to align with each reference point, recording the corresponding PTZ value. The edge control unit calculates the homography matrix H based on these four pairs of coordinates for subsequent coordinate mapping.

[0097] The user taps on a walking humanoid target in the application's panoramic view. The application captures this single-point tap and obtains the display coordinates of the touch point (672, 670).

[0098] The application calculates the scaling factor s = (1920 / 3840, 1080 / 2160) = (0.5, 0.5), converting the display coordinates to raw pixel coordinates (1344, 1340). Further normalization yields (xn, yn) = (1344 / 3839, 1340 / 2159) ≈ (0.35, 0.62). A structured instruction containing the instruction type "click" and the normalized coordinates is generated and published via MQTT to the topic / camera / ctrl / gun01 with QoS=1.

[0099] After receiving the instruction, the edge control unit creates a 128×128 pixel Region of Interest (ROI) around the (0.35, 0.62) position in the original bolt image frame, and sends it to the YOLOv5s target detection network for detection. A humanoid target bounding box with a confidence score of 0.85 is detected, with a bounding box of (1280, 1260, 1408, 1420). The DSST tracker is initialized with this target bounding box, and target features are extracted as the tracking template.

[0100] Electronic cropping of the gun: The target center coordinates (Cx, Cy) = (1344, 1340) are calculated. The desired cropping top-left corner coordinates are (1344-960, 1340-540) = (384, 800). After second-order low-pass filtering, the actual cropping coordinates are (57.6, 120). The edge control unit crops the original frame, generating a 1920×1080 locked image, and pushes it to the application.

[0101] PTZ camera parameter calculation: The target pixel coordinates (1344, 1340) are converted to a direction vector V_gun in the camera coordinate system: V_gun = [(1344-1920) / 1800, (1340-1080) / 1800, 1]^T ≈ [-0.32, 0.144, 1]^T. Transformed to the PTZ coordinate system using the extrinsic parameter matrix, V_dome ≈ [-0.32, 0.144, 1.5]^T is obtained. The calculated horizontal angle θ = atan2(-0.32, 1.5) × 180 / π ≈ -12°, and the vertical angle φ = asin(0.144 / sqrt(0.32² + 0.144² + 1.5²)) × 180 / π ≈ 5.4°. The target frame height is 160 pixels. Looking up the pixel-magnification lookup table, the corresponding zoom magnification is found to be 6x.

[0102] The edge control unit generates an ONVIFAbsoluteMove command, setting Pan = -12°, Tilt = 5.4°, Zoom = 6x, and movement speed to 80% of maximum speed. The command is sent to the PTZ camera, which quickly rotates to the designated position and adjusts the zoom level. The application simultaneously receives the locked view from the bullet camera and the zoomed-in view from the PTZ camera, synchronizing them via NTP timestamps to ensure a latency difference of less than 200ms between the two views.

[0103] The tracker operates at 25 frames per second, continuously outputting the target's bounding box. Every 300ms, the edge control unit recalculates the PTZ camera parameters based on the target's latest position and adjusts the PTZ camera's motion via a PID controller. Simultaneously, the ePTZ module continuously updates the cropping coordinates to keep the target centered in the locked frame.

[0104] When the target moves, the PTZ camera follows its rotation, and the bullet camera's locked view remains centered on the target. Users can adjust the angle by sliding a single finger or pinching two fingers to adjust the zoom on the PTZ camera's small screen. After 0.5 seconds, the system automatically resumes automatic tracking mode.

[0105] The edge control unit packages the PTZ value of the PTZ camera, the target center coordinates, and the tracking confidence level into status data five times per second and sends it to the application. The application updates the UI display based on the status data, including information such as the tracking frame, zoom level, and signal strength. The entire system forms a complete closed loop of "interaction-command-tracking-feedback-fine-tuning," with end-to-end latency controlled within 600ms.

[0106] When a user clicks on a new target, the system clears the old tracking trajectory and switches to the new target for tracking. When there is no user interaction, the PTZ camera can cruise and scan according to a preset trajectory. Once a tracking command is received, cruise is immediately interrupted and tracking mode is entered.

[0107] Please see Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of the collaborative tracking device 200 for bullet and PTZ cameras provided in this application embodiment. The collaborative tracking device 200 for bullet and PTZ cameras is used to execute the steps of the collaborative tracking method for bullet and PTZ cameras shown in the above embodiments. The collaborative tracking device 200 for bullet and PTZ cameras can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, a laptop computer, a wearable device, or a robot.

[0108] like Figure 3 As shown, the collaborative tracking device 200 for both the camera and the PTZ camera includes: The information acquisition unit 201 is used to acquire touch operation information of the panoramic video image captured by the gun. The object locking unit 202 is used to convert the touch operation information into normalized coordinate instructions in order to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame; The image cropping unit 203 is used to electronically crop the panoramic video image so that the target object is always in the center of the panoramic video image. The parameter calculation unit 204 is used to calculate the object parameters corresponding to the target object in the panoramic video frame; the object parameters include at least one or more of the PTZ camera's horizontal angle, vertical angle, and zoom ratio. The PTZ camera drive unit 205 is used to drive the PTZ camera to rotate according to the object parameters, and to acquire the locked image output by the bullet camera and the magnified image captured by the PTZ camera. The collaborative tracking unit 206 is used to achieve collaborative tracking of the target object based on the locked screen and the zoomed-in screen.

[0109] In some embodiments, converting the touch operation information into normalized coordinate instructions to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame includes: parsing the touch operation information to obtain the operation type corresponding to the touch operation information and the corresponding display coordinates on the panoramic video frame; converting the display coordinates into normalized coordinates of the original frame corresponding to the panoramic video frame; and generating the normalized coordinate instructions according to the operation type and the normalized coordinates to lock the target area corresponding to the normalized coordinate instructions in the panoramic video frame.

[0110] In some embodiments, electronically cropping the panoramic video frame to ensure that the target object is always at the center of the panoramic video frame includes: obtaining the center pixel coordinates of the target object in the panoramic video frame; calculating the cropping start coordinates of the panoramic video frame based on the center pixel coordinates; performing smoothing filtering on the cropping start coordinates; cropping the panoramic video frame based on the processed cropping start coordinates; and outputting a locked frame centered on the target object.

[0111] In some embodiments, the step of calculating the object parameters corresponding to the target object in the panoramic video frame includes: obtaining pre-calibrated spatial mapping parameters of the bullet camera and the PTZ camera; converting the pixel coordinates of the target object in the panoramic video frame into a direction vector in the PTZ camera coordinate system; calculating the horizontal and vertical angles corresponding to the PTZ camera based on the direction vector; and calculating the zoom ratio corresponding to the PTZ camera based on the size of the target object in the panoramic video frame.

[0112] In some embodiments, driving the PTZ camera to rotate according to the object parameters and acquiring the locked image output by the bullet camera and the magnified image captured by the PTZ camera includes: generating a PTZ camera drive command based on the horizontal angle, vertical angle, and zoom ratio; sending the PTZ camera drive command to the PTZ camera to control the PTZ camera to rotate according to the horizontal angle and vertical angle, and to adjust according to the zoom ratio; receiving the magnified image captured by the PTZ camera and the locked image output by the bullet camera, and performing time synchronization processing on the locked image and the magnified image.

[0113] In some embodiments, the collaborative tracking of the target object based on the locked screen and the zoomed-in screen includes: tracking the target object in the locked screen and continuously acquiring the real-time position information of the target object; calculating the position change information of the target object based on the continuous real-time position information; when the position change exceeds a preset change range, recalculating the object parameters corresponding to the PTZ camera; and adjusting the horizontal angle, vertical angle, and zoom ratio of the PTZ camera based on the recalculated object parameters to maintain continuous tracking of the target object.

[0114] In some embodiments, before the collaborative tracking of the target object is performed based on the locked screen and the magnified screen, the method further includes: if fine-tuning operation information is received on the locked screen and / or the magnified screen, adjusting the angle and / or zoom of the PTZ camera in response to the fine-tuning operation information; and after the fine-tuning operation information is responded to, performing collaborative tracking of the target object based on the locked screen and the magnified screen.

[0115] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-described collaborative tracking device for bullet and PTZ cameras and each module can be referred to the corresponding content in the various embodiments of the above-described collaborative tracking method for bullet and PTZ cameras, and will not be repeated here.

[0116] The aforementioned method for coordinated tracking of the camera and PTZ camera can be implemented as a computer program, which can be used in various ways, such as... Figure 3 It runs on the system shown.

[0117] Please see Figure 4 , Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. The computer device includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0118] The storage medium can store operating devices and computer programs. The computer program includes program instructions that, when executed, cause the processor to perform any cooperative tracking method for both bullet and PTZ cameras.

[0119] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0120] Internal memory provides an environment for the execution of computer programs stored in non-volatile storage media. When executed by a processor, the computer program enables the processor to perform any cooperative tracking method for bullet and PTZ cameras.

[0121] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0122] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0123] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Acquire touch operation information from the panoramic video footage captured by the camera; The touch operation information is converted into normalized coordinate instructions to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame; The panoramic video image is electronically cropped so that the target object is always in the center of the panoramic video image; The object parameters corresponding to the target object are calculated in the panoramic video frame; the object parameters include at least one or more of the PTZ camera's horizontal angle, vertical angle, and zoom ratio. The PTZ camera is driven to rotate according to the object parameters, and the locked image output by the bullet camera and the magnified image captured by the PTZ camera are obtained. The target object is tracked collaboratively based on the locked and zoomed-in views.

[0124] In some embodiments, converting the touch operation information into normalized coordinate instructions to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame includes: parsing the touch operation information to obtain the operation type corresponding to the touch operation information and the corresponding display coordinates on the panoramic video frame; converting the display coordinates into normalized coordinates of the original frame corresponding to the panoramic video frame; and generating the normalized coordinate instructions according to the operation type and the normalized coordinates to lock the target area corresponding to the normalized coordinate instructions in the panoramic video frame.

[0125] In some embodiments, electronically cropping the panoramic video frame to ensure that the target object is always at the center of the panoramic video frame includes: obtaining the center pixel coordinates of the target object in the panoramic video frame; calculating the cropping start coordinates of the panoramic video frame based on the center pixel coordinates; performing smoothing filtering on the cropping start coordinates; cropping the panoramic video frame based on the processed cropping start coordinates; and outputting a locked frame centered on the target object.

[0126] In some embodiments, the step of calculating the object parameters corresponding to the target object in the panoramic video frame includes: obtaining pre-calibrated spatial mapping parameters of the bullet camera and the PTZ camera; converting the pixel coordinates of the target object in the panoramic video frame into a direction vector in the PTZ camera coordinate system; calculating the horizontal and vertical angles corresponding to the PTZ camera based on the direction vector; and calculating the zoom ratio corresponding to the PTZ camera based on the size of the target object in the panoramic video frame.

[0127] In some embodiments, driving the PTZ camera to rotate according to the object parameters and acquiring the locked image output by the bullet camera and the magnified image captured by the PTZ camera includes: generating a PTZ camera drive command based on the horizontal angle, vertical angle, and zoom ratio; sending the PTZ camera drive command to the PTZ camera to control the PTZ camera to rotate according to the horizontal angle and vertical angle, and to adjust according to the zoom ratio; receiving the magnified image captured by the PTZ camera and the locked image output by the bullet camera, and performing time synchronization processing on the locked image and the magnified image.

[0128] In some embodiments, the collaborative tracking of the target object based on the locked screen and the zoomed-in screen includes: tracking the target object in the locked screen and continuously acquiring the real-time position information of the target object; calculating the position change information of the target object based on the continuous real-time position information; when the position change exceeds a preset change range, recalculating the object parameters corresponding to the PTZ camera; and adjusting the horizontal angle, vertical angle, and zoom ratio of the PTZ camera based on the recalculated object parameters to maintain continuous tracking of the target object.

[0129] In some embodiments, before the collaborative tracking of the target object is performed based on the locked screen and the magnified screen, the method further includes: if fine-tuning operation information is received on the locked screen and / or the magnified screen, adjusting the angle and / or zoom of the PTZ camera in response to the fine-tuning operation information; and after the fine-tuning operation information is responded to, performing collaborative tracking of the target object based on the locked screen and the magnified screen.

[0130] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the cooperative tracking method for bullet and PTZ cameras provided in any embodiment of this application.

[0131] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0132] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for coordinated tracking of a bullet camera and a PTZ camera, characterized in that, The method includes: Acquire touch operation information from the panoramic video footage captured by the camera; The touch operation information is converted into normalized coordinate instructions to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame; The panoramic video image is electronically cropped so that the target object is always in the center of the panoramic video image; The object parameters corresponding to the target object are calculated in the panoramic video frame; the object parameters include at least one or more of the PTZ camera's horizontal angle, vertical angle, and zoom ratio. The PTZ camera is driven to rotate according to the object parameters, and the locked image output by the bullet camera and the magnified image captured by the PTZ camera are obtained. The target object is tracked collaboratively based on the locked and zoomed-in views.

2. The method of claim 1, wherein, The step of converting the touch operation information into normalized coordinate instructions to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame includes: The touch operation information is analyzed to obtain the operation type corresponding to the touch operation information and the corresponding display coordinates on the panoramic video screen; The display coordinates are converted into normalized coordinates of the original frame corresponding to the panoramic video image; The normalized coordinate instruction is generated based on the operation type and the normalized coordinates to lock the target area corresponding to the normalized coordinate instruction in the panoramic video frame.

3. The method of claim 1, wherein, The step of electronically cropping the panoramic video frame to ensure that the target object is always centered in the panoramic video frame includes: Obtain the center pixel coordinates of the target object in the panoramic video frame; The cropping start coordinates of the panoramic video frame are calculated based on the center pixel coordinates, and the cropping start coordinates are then subjected to smoothing filtering. The panoramic video image is cropped based on the processed cropping start coordinates, and a locked image centered on the target object is output.

4. The method of claim 1, wherein, The step of calculating the object parameters corresponding to the target object in the panoramic video frame includes: Obtain the pre-calibrated spatial mapping parameters between the bullet camera and the PTZ camera; The pixel coordinates of the target object in the panoramic video frame are converted into a direction vector in the PTZ camera's coordinate system. The horizontal and vertical angles corresponding to the PTZ camera are calculated based on the direction vector. The zoom ratio corresponding to the PTZ camera is calculated based on the size of the target object in the panoramic video frame.

5. The method of claim 4, wherein, The step of driving the PTZ camera to rotate according to the object parameters and acquiring the locked image output by the bullet camera and the magnified image captured by the PTZ camera includes: Generate PTZ camera drive commands based on the horizontal angle, vertical angle, and zoom ratio; The PTZ camera drive command is sent to the PTZ camera to control the PTZ camera to rotate according to the horizontal and vertical angles, and to adjust according to the zoom ratio; The system receives the magnified image captured by the PTZ camera and the locked image output by the bullet camera, and performs time synchronization processing on the locked image and the magnified image.

6. The method according to claim 1, characterized in that, The method of collaboratively tracking the target object based on the locked view and the zoomed view includes: Track the target object in the locked screen and continuously acquire the real-time location information of the target object; The position change information of the target object is calculated based on the continuous real-time position information; When the position change exceeds a preset range, the object parameters corresponding to the PTZ camera are recalculated. The horizontal angle, vertical angle, and zoom ratio of the PTZ camera are adjusted according to the recalculated object parameters to maintain continuous tracking of the target object.

7. The method according to any one of claims 1-6, characterized in that, Before the collaborative tracking of the target object based on the locked view and the zoomed view, the method further includes: If a fine-tuning operation information is received on the locked screen and / or the magnified screen, the angle and / or zoom of the PTZ camera are adjusted in response to the fine-tuning operation information. After the fine-tuning operation information response is completed, the target object is tracked collaboratively based on the locked screen and the zoomed-in screen.

8. A collaborative tracking device for a bullet camera and a PTZ camera, characterized in that, The angle adjustment device is used to implement the method as described in any one of claims 1-7, including: The information acquisition unit is used to acquire touch operation information of the panoramic video images captured by the camera. An object locking unit is used to convert the touch operation information into normalized coordinate instructions in order to lock the target object corresponding to the normalized coordinate instructions in the panoramic video frame. The image cropping unit is used to electronically crop the panoramic video image so that the target object is always in the center of the panoramic video image. The parameter calculation unit is used to calculate the object parameters corresponding to the target object in the panoramic video frame; the object parameters include at least one or more of the PTZ camera's horizontal angle, vertical angle, and zoom ratio. The PTZ camera drive unit is used to drive the PTZ camera to rotate according to the object parameters, and to acquire the locked image output by the bullet camera and the magnified image captured by the PTZ camera; The collaborative tracking unit is used to achieve collaborative tracking of the target object based on the locked screen and the zoomed-in screen.

9. A computer device, comprising: Including memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the method as described in any one of claims 1 to 7.

Images