A method and system for two-way linkage interaction between a spliced panoramic preview window and a PTZ detail window

Abstract

This invention relates to the field of video surveillance and image processing technology, and particularly to a method and system for bidirectional interactive linkage between a stitched panoramic preview window and a PTZ detail window. The method includes: displaying a panoramic preview window containing a stitched panoramic image and a PTZ detail window containing a partially cropped and enlarged image; overlaying a field-of-view indicator box in the panoramic preview window, the field-of-view indicator box indicating the current area of ​​the PTZ detail window within the panoramic image; detecting user input operations on the panoramic preview window or the PTZ detail window; calculating the target state of the field-of-view indicator box and performing boundary constraints based on the type and location of the input operation to obtain an updated field-of-view state; synchronously refreshing the field-of-view indicator box in the panoramic preview window and the cropped area in the PTZ detail window based on the updated field-of-view state; and rendering and outputting the refreshed panoramic preview window and PTZ detail window to a display device. This solves the orientation loss problem of traditional PTZ.

Description

Technical Field

[0001] This invention relates to the field of video surveillance and image processing technology, and in particular to a method and system for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window. Background Technology

[0002] With the rapid development of video surveillance technology, the demand for wide-angle, high-resolution video acquisition is increasing in scenarios such as security monitoring, remote conferencing, and intelligent inspection. Traditional PTZ cameras achieve horizontal rotation (Pan), vertical tilt (Tilt), and zoom through mechanical structures, providing flexible viewing angle control capabilities.

[0003] In recent years, with the maturity of image stitching technology, technical solutions based on binocular or multi-camera stitching of panoramic images have gradually emerged, generating 180° or even 360° ultra-wide-angle panoramic images. Compared with traditional monocular wide-angle lenses, multi-camera stitching solutions maintain high resolution while avoiding the problems of severe edge distortion and significant image quality degradation associated with ultra-wide-angle lenses.

[0004] However, in existing technologies, the interaction methods of panoramic stitching cameras mostly follow the traditional PTZ gimbal operation paradigm, that is, controlling the panning of the virtual viewpoint through joystick controls or directional keys. The industry defaults to equating "viewpoint switching" with "mechanical rotation of the gimbal". This method has the following problems:

[0005] Loss of global orientation: Traditional PTZ only displays a local view and has no global reference; although existing panoramic cameras have panoramic thumbnails, they do not mark the precise location of the current field of view in the panoramic image, and users are prone to losing their way when zooming in to view details.

[0006] The operation is not intuitive and inefficient: Traditional PTZ cameras use a joystick to control the up and down and left and right directions, requiring multiple steps to switch from viewpoint A to viewpoint B; existing panoramic cameras only support unidirectional dragging from panoramic to detail, and do not support reverse linkage. Moreover, there is a delay of hundreds of milliseconds to several seconds in the entire process from receiving a command to the motor starting, rotating to the correct position, and the image stabilizing, which can easily create blind spots during the rotation.

[0007] Setting the cruise path is cumbersome: Traditional PTZ cruise requires setting each preset position one by one, manually rotating the gimbal to each target position, saving the preset position, and repeating this process many times to complete a cruise path, which takes a long time.

[0008] The panoramic window and the detail window lack two-way linkage: Even if the existing products provide a panoramic thumbnail window, they do not indicate the current field of view in the panoramic window, and there is no visual connection or two-way linkage between the two windows. Summary of the Invention

[0009] The purpose of this invention is to provide a method and system for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window, thereby solving at least one of the problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] The first aspect of this invention proposes a method for bidirectional interactive linkage between a stitched panoramic preview window and a PTZ detail window, comprising the following steps:

[0012] Displays a panoramic preview window containing the stitched panoramic image and a PTZ detail window containing a cropped and magnified image;

[0013] A field of view indicator box is overlaid in the panoramic preview window. The field of view indicator box is used to indicate the current area range of the PTZ detail window in the panoramic image.

[0014] Detect user input on the panoramic preview window or the PTZ detail window;

[0015] Based on the type and location of the input operation, calculate the target state of the field of view indicator box and perform boundary constraints to obtain the updated field of view state.

[0016] Based on the updated field of view state, the field of view indicator box in the panoramic preview window and the cropping area in the PTZ detail window are refreshed simultaneously.

[0017] The refreshed panoramic preview window and PTZ detail window are rendered and output to the display device.

[0018] A further improvement is that the field of view indicator frame is any one of a rectangular frame, an elliptical frame, a polygonal frame, a highlight mask, or a semi-transparent color block; the border color of the field of view indicator frame is a highlight color.

[0019] A further improvement is that the specific method for calculating the target state of the field of view indicator and executing boundary constraints based on the type and position of the input operation includes: when the input operation is dragging the field of view indicator in the panoramic preview window, obtaining the displacement of the drag operation; updating the position of the field of view indicator based on the displacement; determining whether the updated field of view indicator exceeds the boundary of the panoramic image; and when it exceeds the boundary, restricting the field of view indicator within the boundary of the panoramic image.

[0020] A further improvement is that the specific method for calculating the target state of the field of view indicator and executing boundary constraints based on the type and position of the input operation includes: when the input operation is dragging the image in the PTZ detail window, obtaining the displacement of the drag operation; mapping the displacement inversely to the displacement of the field of view indicator in the panoramic image according to the scaling factor of the PTZ detail window; updating the position of the field of view indicator based on the inversely mapped displacement; determining whether the updated field of view indicator exceeds the boundary of the panoramic image; and when it exceeds the boundary, restricting the field of view indicator within the boundary of the panoramic image.

[0021] A further improvement is that the specific method for calculating the target state of the field of view indicator and executing boundary constraints based on the type and location of the input operation includes: when the input operation is a zoom operation, obtaining the zoom center point and the zoom ratio change; using the zoom center point as an anchor point, calculating a new field of view indicator size based on the zoom ratio change; determining whether the new field of view indicator size exceeds a preset minimum and maximum size range; and when it exceeds the range, limiting the field of view indicator size to the preset range.

[0022] A further improvement is that the specific method for calculating the target state of the field of view indicator and executing boundary constraints based on the type and location of the input operation includes: when the input operation is a click operation in the panoramic preview window, obtaining the coordinate position of the click operation in the panoramic image; moving the center of the field of view indicator to the coordinate position; determining whether the moved field of view indicator exceeds the boundary of the panoramic image; and when it exceeds the boundary, restricting the field of view indicator to within the boundary of the panoramic image.

[0023] A further improvement is that the method also includes a cruise path setting step:

[0024] In response to a command to enter cruise setting mode, the system accepts the user's route input in the panoramic preview window;

[0025] When the path input is a continuous drag trajectory, the drag trajectory is sampled at a preset sampling interval, and a path point sequence is generated after smoothing and filtering.

[0026] When the path input is a sequence of key points clicked sequentially, a smooth curve path is generated between adjacent key points through Catmull-Rom spline interpolation;

[0027] The generated path is displayed as a highlighted line overlaid in the panoramic preview window, and key points are displayed as draggable control points.

[0028] The path is updated in real time in response to the user's drag operation on the control point, and the path is stored as a cruise path in response to the save command.

[0029] A further improvement is that the method also includes a cruise execution step:

[0030] In response to the command to start cruise, load the target cruise path and parameter configuration;

[0031] The field of view indicator box is driven to move frame by frame along the cruise path. The movement step size of the current frame is calculated and the position of the field of view indicator box is updated in each frame.

[0032] When the field of view indicator box reaches a key point, it pauses movement and stays for a preset time; when it reaches the end of the path, it performs one of the following actions according to the loop mode: one-way stop, round-trip reverse, or loop reset.

[0033] The panoramic preview window and PTZ detail window are refreshed synchronously with each frame; in response to manual operation by the user during cruise, cruise is paused and switched to manual mode; in response to the resume command, cruise continues from the interrupted position.

[0034] A further improvement is that the cruise execution steps also include adaptive speed adjustment: detecting the density of moving targets in the area near the field of view indicator box at preset evaluation intervals; when the density of moving targets exceeds a first preset threshold, reducing the cruise speed to a first speed level; when the density of moving targets is lower than a second preset threshold, restoring the cruise speed to a normal speed level; wherein, the speed switching adopts a frame-by-frame smooth transition.

[0035] A second aspect of this invention proposes a two-way interactive system for a stitched panoramic preview window and a PTZ detail window, comprising:

[0036] The dual-window display module is used to display a panoramic preview window containing a stitched panoramic image and a PTZ detail window containing a locally cropped and magnified image;

[0037] A field of view indicator module is used to overlay a field of view indicator box in the panoramic preview window, the field of view indicator box being used to indicate the current area range of the PTZ detail window in the panoramic image;

[0038] The input detection module is used to detect user input operations on the panoramic preview window or the PTZ detail window;

[0039] The linkage control module is used to calculate the target state of the field of view indicator box and execute boundary constraints according to the type and position of the input operation to obtain the updated field of view state.

[0040] The synchronous refresh module is used to synchronously refresh the field of view indicator box in the panoramic preview window and the cropping area in the PTZ detail window based on the updated field of view state.

[0041] The rendering output module is used to render and output the refreshed panoramic preview window and PTZ detail window to the display device.

[0042] The beneficial effects of this invention are as follows:

[0043] This invention employs a dual-window linkage architecture of a panoramic preview window and a PTZ detail window. In the panoramic preview window, a field of view indicator box continuously marks the precise spatial position of the current PTZ field of view. Regardless of which local area is zoomed in to view, the user can understand its location in the entire monitoring scene in real time, thus solving the orientation loss problem of traditional PTZ.

[0044] This invention supports multiple interaction methods, such as dragging the view frame in the panoramic window, dragging the image in the detail window, zooming, and clicking in the panoramic window, to achieve a two-way linkage and WYSIWYG operation experience. Compared with the traditional PTZ split-axis joystick operation, the operation efficiency is significantly improved.

[0045] This invention achieves perspective switching through digital cropping, eliminating mechanical rotation delays and blind spots in monitoring during rotation, resulting in near-instantaneous perspective switching. Furthermore, this invention supports real-time storage of panoramic images, and during playback, it can reproduce the dual-window interactive function, allowing users to view the content from any direction during recording, achieving a post-recording retrospective capability that is physically impossible with traditional PTZ. Attached Figure Description

[0046] Figure 1 This is a flowchart of a two-way interactive method for stitching a panoramic preview window and a PTZ detail window according to the present invention;

[0047] Figure 2 This is a flowchart of the cruise path setting steps in this invention;

[0048] Figure 3 This is an architecture diagram of a two-way interactive system for a stitched panoramic preview window and a PTZ detail window according to the present invention. Detailed Implementation

[0049] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. The described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort should fall within the scope of protection of the present invention.

[0050] Please refer to the attached document. Figure 1The first aspect of this invention proposes a method for bidirectional interactive linkage between a stitched panoramic preview window and a PTZ detail window, which is applied to a bidirectional interactive linkage system between a stitched panoramic preview window and a PTZ detail window.

[0051] The method includes the following steps:

[0052] Step S1: Display a panoramic preview window containing the stitched panoramic image and a PTZ detail window containing a partially cropped and enlarged image.

[0053] Step S2: Overlay a field of view indicator box in the panoramic preview window. The field of view indicator box is used to indicate the current area range of the PTZ detail window in the panoramic image. The current state of the field of view indicator box includes its position and size.

[0054] Step S3: Detect the user's input operation on the panoramic preview window or the PTZ detail window.

[0055] Step S4: Calculate the target state of the field of view indicator box and perform boundary constraints according to the type and location of the input operation to obtain the updated field of view state, wherein the target state includes at least position and size.

[0056] Step S5: Based on the updated field of view state, synchronously refresh the field of view indicator box in the panoramic preview window and the cropping area in the PTZ detail window.

[0057] Step S6: Render and output the refreshed panoramic preview window and PTZ detail window to the display device.

[0058] The method steps of this embodiment of the invention will be described in more detail below:

[0059] Specifically, in step S1, two windows are presented to the user through a display device: a panoramic preview window and a PTZ detail window.

[0060] Understandably, the panoramic preview window and PTZ detail window provide users with a visual interface that allows them to observe both the overall scene and local details simultaneously. The panoramic preview window presents a complete wide-angle view, giving users a clear overview of the entire monitoring scene; the PTZ detail window presents a magnified view of a local area, allowing users to observe detailed information.

[0061] Specifically, the panoramic preview window is used to display a stitched panoramic image, which is generated by combining left and right eye images acquired by a binocular camera after distortion correction, image registration, and stitching. The panoramic preview window displays the panoramic image at a reduced scale. The PTZ detail window is used to display a locally cropped and magnified image, i.e., the area corresponding to the field of view indicator box is cropped from the panoramic frame buffer, scaled to the target resolution, and then rendered and output. The two windows can be displayed side by side, top to bottom, or in a picture-in-picture format.

[0062] In this embodiment, the panoramic frame buffer is a circular buffer located in memory, used to cache the panoramic image data of the current frame in real time for subsequent cropping operations. The presence of the panoramic frame buffer allows the PTZ detail window renderer to quickly read pixel data of any region in the panoramic image at any time without having to re-acquire the image from the camera, thus achieving zero-latency viewpoint switching. The panoramic frame buffer stores the complete panoramic image after distortion correction, image registration, and stitching fusion.

[0063] Specifically, in step S2, the system overlays a field of view indicator box in the panoramic preview window, thereby establishing a visual connection between the panoramic preview window and the PTZ detail window. By observing the field of view indicator box in the panoramic preview window, the user can intuitively understand "where the currently magnified detail is located in the panoramic image", thus solving the problem of disorientation caused by traditional PTZ which only displays a local image.

[0064] In a preferred embodiment, the field of view indicator frame is any one of a rectangular frame, an elliptical frame, a polygonal frame, a highlight mask, or a semi-transparent color block; the border color of the field of view indicator frame is a highlight color, making it visually salient in the panoramic image.

[0065] Specifically, the position and size of the field of view indicator box are maintained by the field of view state manager. The field of view state manager is a data structure maintenance unit running in system memory, used to store and manage the current state parameters of the field of view indicator box, including its center x-coordinate. Central ordinate ,width and height The field-of-view state manager provides state reading and state update interfaces for use by the panoramic preview window renderer, PTZ detail window renderer, and linkage synchronization controller. Initially, the field-of-view indicator box is located in the center of the panoramic image, and its size corresponds to the field of view at the default zoom level.

[0066] For example, when the panoramic image resolution is 2560×720 pixels, the PTZ detail window resolution is 1024×576 pixels, and the default zoom level is 1x, the field of view indicator box is 1024×576 pixels in size and located at the center (1280, 360) of the panoramic image. When the user zooms in to 2x, the field of view indicator box shrinks to 512×288 pixels, while its center position remains unchanged. The field of view state manager records these parameter changes in real time, ensuring that both windows are always rendered based on the same set of state data.

[0067] Specifically, in step S3, the user's input is continuously monitored through an input event handler. The input event handler is a monitoring module running in the system background, used to capture all user operation events on the interactive interface, including operation type, operation location, and related parameters.

[0068] Understandably, by capturing the user's interactive intent as a trigger signal to drive the dual-window linkage, every drag, zoom, or click by the user means that the user wants to adjust the current viewing angle or zoom level.

[0069] Specifically, the types of input operations include: dragging the field of view indicator box in the panoramic preview window, dragging the image in the PTZ detail window, zooming, and clicking in the panoramic preview window. These input operations can be performed using a mouse (click, drag, scroll wheel) or a touchscreen (single-finger drag, two-finger pinch). The input event handler captures the type of operation, the operation location, and related parameters (such as displacement and zoom level changes), and passes them to the linkage synchronization controller for further processing.

[0070] Specifically, in step S4, the linkage synchronization controller executes different calculation logics according to the type of input operation. The linkage synchronization controller is the system's calculation unit, responsible for converting user input events into state update instructions for the view indicator box. By converting the user's operational intent into specific state changes in the view indicator box—such as how far the user dragged, how large they zoomed, or which location they clicked—the system needs to accurately calculate the new position and size of the view indicator box and ensure that the calculation result does not exceed the effective range of the panoramic image. After performing the position or size update, a boundary constraint check is performed to ensure that the view indicator box does not exceed the effective area of ​​the panoramic image.

[0071] The following describes the four different input operations using four different branching steps:

[0072] Branch A: When the input operation is to drag the field of view indicator box in the panoramic preview window, the specific method for calculating the target state of the field of view indicator box and performing boundary constraints according to the type and position of the input operation includes: obtaining the displacement of the drag operation; updating the position of the field of view indicator box according to the displacement; determining whether the updated field of view indicator box exceeds the boundary of the panoramic image; when it exceeds the boundary, restricting the field of view indicator box within the boundary of the panoramic image.

[0073] For example, suppose a user drags the field of view indicator box horizontally in the panoramic preview window by an amount of... The longitudinal displacement is The linkage synchronization controller reads the horizontal coordinate of the current view indicator box from the view status manager. and ordinate . This indicates the x-coordinate of the center of the field of view indicator box in the panoramic image before the update. This indicates the vertical coordinate of the center of the field-of-view indicator box in the panoramic image before the update. The synchronized controller calculates the new coordinates after the update.

[0074]

[0075]

[0076] in, This indicates the x-coordinate of the updated field-of-view indicator center in the panoramic image. This indicates the ordinate of the updated view indicator box center in the panoramic image, and then the synchronization controller executes boundary constraints: if Then let ;like Then let ,in The width of the panoramic image. This represents the current width of the field of view indicator box. Similarly, boundary constraints are applied to the vertical axis, and the height of the panoramic image is set. , The current height of the view indicator box, if Then let ;like Then let After the boundary constraints are completed, the linkage synchronization controller writes the updated state to the view state manager.

[0077] Branch B: When the input operation is dragging the image in the PTZ detail window, the specific method for calculating the target state of the view indicator box and performing boundary constraints according to the type and position of the input operation includes: obtaining the displacement of the drag operation; mapping the displacement inversely to the displacement of the view indicator box in the panoramic image according to the scaling factor of the PTZ detail window; updating the position of the view indicator box according to the inversely mapped displacement; determining whether the updated view indicator box exceeds the boundary of the panoramic image; and restricting the view indicator box within the boundary of the panoramic image when it exceeds the boundary.

[0078] For example, suppose the horizontal movement of the image when a user drags it in the PTZ detail window is... The longitudinal displacement is The current scaling factor is ( A larger value indicates a higher magnification and a smaller viewport size. This step ensures that when the user drags the image from the detail window, the viewport indicator in the panoramic window moves accordingly, achieving reverse linkage "from detail to the whole." When the user drags the image in the detail window, it means they want to see a magnified view of the adjacent area. The system needs to calculate the position of this adjacent area in the panoramic image and update the viewport indicator accordingly. The linkage synchronization controller maps the drag displacement of the detail window back to the displacement in the panoramic image according to the scaling factor.

[0079]

[0080]

[0081] in, This indicates the amount of lateral shift of the field of view indicator box within the panoramic image. This indicates the vertical shift of the field of view indicator box within the panoramic image. The synchronized controller then performs the same boundary constraint steps as in branch A.

[0082] Branch C: When the input operation is a zoom operation, the specific method for calculating the target state of the view indicator box and executing boundary constraints based on the type and location of the input operation includes: obtaining the zoom center point and the zoom ratio change of the zoom operation. Using the zoom center point as the anchor point, calculate the new field of view indicator frame size based on the zoom ratio change:

[0083]

[0084]

[0085] in, To scale the width of the foreground view indicator box, To scale the height of the foreground view indicator box. This represents the scaling factor (greater than 1 indicates magnification, less than 1 indicates reduction). This is the width of the zoomed-out view indicator box. The height of the zoomed-out view indicator frame is set; it is determined whether the new view indicator frame size exceeds the preset minimum and maximum size range; if it exceeds the range, the view indicator frame size is limited to the preset range. The preset minimum size corresponds to the maximum zoom ratio, and the preset maximum size corresponds to the minimum zoom ratio (i.e., panoramic full-screen display).

[0086] Understandably, the purpose of branch C is to allow users to zoom in or out of the field of view using the scroll wheel or pinch with two fingers. The zoom operation changes the size of the field of view indicator box (the smaller the box, the higher the magnification, and the larger the box, the lower the magnification). The system needs to ensure that the zoom result does not exceed the preset limit range to prevent the field of view box from being too small, resulting in too little content being displayed, or too large, resulting in a loss of meaning for detailed observation.

[0087] Branch D: When the input operation is a click operation in the panoramic preview window, the specific method for calculating the target state of the field of view indicator box and performing boundary constraints according to the type and location of the input operation includes: obtaining the coordinate position of the click operation in the panoramic image. Move the center of the field of view indicator to the coordinate position; determine whether the moved field of view indicator exceeds the boundary of the panoramic image; when it exceeds the boundary, restrict the field of view indicator to the boundary of the panoramic image.

[0088] Understandably, the purpose of branch D is to enable a one-step operation of "clicking wherever you want to see". When a user clicks on any location in the panoramic preview window, the field of view indicator box immediately jumps to that location, and the PTZ detail window simultaneously displays a magnified view of that location, greatly improving the efficiency of perspective switching.

[0089] This invention covers the main interactive operations of users on the dual-window platform through the aforementioned four branches, ensuring that any operation is responded to correctly and that the legitimacy of the field of view is maintained. The boundary constraints of all branches guarantee that the field of view indicator box will never exceed the effective area of ​​the panoramic image under any circumstances, avoiding the display of blank areas.

[0090] Specifically, in step S5, after the linkage synchronization controller completes the view state update and writes it to the view state manager, it simultaneously triggers the refresh operation of both renderers. The purpose of this step is to ensure that the content of the two windows always maintains spatial consistency, and that the position of the view indicator box in the panoramic preview window corresponds exactly to the image area displayed in the PTZ detail window, without any visual misalignment or delay.

[0091] The two renderers are a panoramic preview window renderer and a PTZ detail window renderer.

[0092] The panoramic preview window renderer is a module responsible for rendering the panoramic preview window image. Its workflow is as follows: read the current frame panoramic image from the panoramic frame buffer, scale it according to the display size of the panoramic preview window, then read the position and size of the current field of view indicator from the field of view status manager, and overlay the border and fill layer of the field of view indicator on the scaled panoramic image.

[0093] The PTZ detail window renderer is the module responsible for rendering the PTZ detail window image. Its workflow is as follows: read the current position of the view indicator box from the view state manager. and size Cropped from the panoramic frame buffer Centered on, with dimensions of The rectangular area is then cropped and scaled to the target resolution of the PTZ detail window (e.g., 1024×576 pixels) before being output. The relationship between the PTZ detail window renderer and the panoramic frame buffer is as follows: the panoramic frame buffer provides a complete panoramic image data source, and the PTZ detail window renderer, as a data consumer, extracts local areas from it. The two interact through a standard memory read / write interface, without the need for disk or network transmission.

[0094] Understandably, the two renderers, coordinated by the linkage synchronization controller, complete the refresh within the same refresh cycle, ensuring that the view indicator box in the panoramic preview window and the image displayed in the PTZ detail window are spatially completely corresponding. The refresh process is completed within one frame, achieving instantaneous response to view switching.

[0095] Specifically, in step S6, the panoramic preview image (including the panoramic image and the view indicator box overlay) output by the panoramic preview window renderer and the detail image (including the cropped and scaled local image) output by the PTZ detail window renderer are combined into a complete display frame and output to the display device to be presented to the user. The whole process is completed in a single frame time, achieving a delay-free interactive experience.

[0096] The complete workflow for the two-way interactive linkage between the panoramic preview window and the PTZ detail window is as follows:

[0097] After system initialization, the binocular cameras begin acquiring images. The acquired left and right eye images are then processed through distortion correction, image registration, and stitching to generate a panoramic image, which is then written to the panoramic frame buffer. The field-of-view state manager initializes the default field-of-view state, setting the field-of-view indicator box to the center of the panoramic image, with a size corresponding to the field of view at the default scaling factor. The panoramic preview window renderer reads the panoramic image from the panoramic frame buffer and scales it for display. Simultaneously, it reads the field-of-view indicator box status from the field-of-view state manager and overlays and draws the field-of-view indicator box in the panoramic preview window. The PTZ detail window renderer crops the area corresponding to the field-of-view indicator box from the panoramic frame buffer and scales it for display.

[0098] The input event handler continuously listens for user actions. When it detects that the user is dragging the view indicator box in the panoramic preview window, the linkage synchronization controller records the drag displacement and updates the view state. After performing boundary constraint checks, it notifies the panoramic preview window renderer and the PTZ detail window renderer to refresh synchronously. When it detects that the user is dragging the image in the PTZ detail window, the linkage synchronization controller calculates the input displacement and maps it inversely to the displacement of the view indicator box in the panoramic image. It then updates the view state and notifies both renderers to refresh synchronously. When a zoom operation is detected, the linkage synchronization controller calculates the new view indicator box size using the zoom center point as the anchor point. After performing size constraints, it notifies both renderers to refresh synchronously. When it detects a click operation by the user in the panoramic preview window, the linkage synchronization controller moves the center of the view indicator box to the click coordinates and notifies both renderers to refresh synchronously.

[0099] In a preferred embodiment of this method, the method further includes a cruise path setting step. This step allows users to set the cruise path intuitively, without having to manually rotate the gimbal and save preset positions. The path can be generated simply by drawing lines on the panoramic view or selecting points of interest, thus improving cruise setting efficiency from the traditional 5 to 10 minutes to approximately 15 seconds.

[0100] Specifically, in this embodiment, such as Figure 2 As shown, the specific steps for setting the cruise route include:

[0101] Step A1: In response to the command to enter cruise setting mode, the system accepts the user's route input in the panoramic preview window. Specifically, the user can enter cruise setting mode by clicking the "Cruise Setting" button or by long-pressing the panoramic preview window. After entering, the border of the panoramic preview window switches to a highlighted color to indicate to the user that they are currently in setting mode.

[0102] Step A2: When the path input is a continuous drag trajectory, the drag trajectory is sampled at a preset sampling interval (e.g., 50 milliseconds), and a path point sequence is generated after removing hand shaking noise through Gaussian smoothing filtering. .

[0103] Step A3: When the path input is a sequence of key points clicked sequentially, record the key point sequence. A smooth curve path is generated between adjacent keypoints using Catmull-Rom spline interpolation. Given four consecutive control points... , , , ,exist arrive Interpolation points between (in (where ∈[0,1] is the interpolation parameter) is calculated using the following formula:

[0104]

[0105] in, The interpolation parameter, ranging from 0 to 1, determines the position of the interpolation point between two key points. When =0, the interpolation point is located at , When =1, the interpolation point is located at ); , , , The coordinate vector of four consecutive control points (each control point contains an x-coordinate). and ordinate ); The calculated interpolation point coordinate vector.

[0106] Understandably, Catmull-Rom splines possess two superior characteristics suitable for this invention: First, the curve precisely passes through each set key point, ensuring that the field of view indicator frame always passes through the user-specified area of ​​interest during cruise; second, computational complexity is low. Suitable for real-time processing in embedded devices. Boundary condition handling method: the first end uses... As virtual The tail end with As virtual The interpolation step size is adaptively selected based on the speed to ensure that the spacing between adjacent interpolation points is within the range of 1 to 5 pixels.

[0107] Preferably, the cruise route setting step further includes:

[0108] Step A4: Display the generated path as a highlighted line overlay in the panoramic preview window, and display key points as draggable control points.

[0109] Step A5: Update the path in real time in response to the user's drag operation on the control point; save the path as the cruise path in response to the save command.

[0110] Specifically, users can visually edit the path, supporting operations such as dragging control points, double-clicking to add control points, long-pressing to delete control points, and undoing / undoing. The path updates in real time. Users can also configure cruise parameters, including cruise speed (slow / normal / fast and a custom slider), key point dwell time (0 to 30 seconds), loop mode (one-way / round-trip / loop), adaptive speed switch, and time scheduling. Path data is stored serially and supports custom names and import / export.

[0111] In a preferred embodiment of this method, the method further includes a cruise execution step. This step enables fully automated field-of-view inspection, with the field-of-view indicator frame automatically moving along a preset path, and the PTZ detail window simultaneously displaying magnified views of each location along the route. This allows for the cyclical viewing of multiple areas of interest within the panoramic image without manual intervention.

[0112] Specifically, in this embodiment, the cruise execution steps include the following methods:

[0113] In response to the command to start cruise, load the target cruise path and parameter configuration.

[0114] Specifically, the user selects the target path from the path list and clicks "Start Cruise" (or the system automatically triggers it based on a time schedule). The cruise execution engine then loads the path data and parameter configuration. Initialization parameters include: path point index. , loop counting Status = "Moving", Dwell Timer = 0, Cruise Speed =User-defined speed.

[0115] The view indicator box is moved frame by frame along the cruise path. The movement step size for each frame is calculated and the view indicator box position is updated. The calculation formula is:

[0116]

[0117] in, The step size (in pixels) by which the current frame's view indicator box moves along the path. The cruising speed of the current frame (unit: pixels per second). The time interval for each frame (in seconds). The view indicator box moves along the path to the next path point. distance.

[0118] When the field of view indicator reaches a key point, movement pauses and remains there for a preset duration. Upon reaching the end of the path, one of the following actions is executed based on the loop mode: unidirectional stop, round-trip reversal, or loop reset. The panoramic preview window and PTZ detail window are refreshed synchronously every frame.

[0119] In response to manual actions performed by the user during cruise (dragging or zooming in the panoramic preview window or PTZ detail window), cruise is paused and switched to manual mode; in response to a resume command, cruise resumes from the interrupted position.

[0120] Understandably, the purpose of this mechanism is to give users the authority to take over control at any time. When a suspicious target is detected during the cruise, the user can immediately manually pause the cruise and view the details. After viewing, the cruise can be resumed with one click without having to reset the route.

[0121] In a preferred embodiment, the cruise execution step further includes adaptive speed adjustment. The function of adaptive speed adjustment is to dynamically adjust the cruise speed according to the density of moving targets in the image, automatically decelerating in densely populated or frequently active areas to observe details, and automatically restoring normal speed in open areas to improve inspection efficiency.

[0122] Adaptive speed adjustment specifically includes: detecting the density of moving targets within a region near the field of view indicator (e.g., 1.5 to 2 times the field of view) at preset evaluation intervals (e.g., every 30 frames or every 1 second). Moving target detection can employ inter-frame differencing or background subtraction methods, and the proportion of moving pixels is statistically analyzed. .

[0123] When the density of moving targets exceeds a first preset threshold, the cruising speed is reduced to the first speed level. For example, a high density threshold is set. ,when At that time, the target speed Calculate using the following formula:

[0124]

[0125] in, Target cruising speed (unit: pixels per second). The normal cruise speed (in pixels per second) is set by the user, with a deceleration coefficient of 0.3. When the density of moving targets falls below a second preset threshold, the cruise speed is restored to the normal speed setting. For example, a low-density threshold is set. ,when hour, When the density of moving targets is between the two thresholds, the current speed is maintained.

[0126] Speed ​​switching uses a smooth frame-by-frame transition:

[0127]

[0128] in, The cruise speed before the update (unit: pixels / second). The updated cruise speed (unit: pixels per second). Target cruising speed (unit: pixels per second). The smoothing coefficient (with a value of 0.1) is the updated... As the next frame use.

[0129] It is understandable that by moving closer to the target speed in each frame... The 10% limit ensures smooth rather than abrupt speed changes, while also restricting the speed range. Smooth transitions avoid screen jitter caused by sudden speed changes, enhancing the viewing experience.

[0130] Specifically, in this embodiment, the system operates in three states: idle mode, cruise setting mode, and cruise execution mode. Idle mode is the default state and supports full-featured dual-window interaction (drag / zoom / click, etc.). In cruise setting mode, the interaction of the panoramic preview window is taken over by the cruise setting function (drag = draw path, click = add key points). In cruise execution mode, the field of view indicator box moves automatically along a preset path.

[0131] The state transition relationship is as follows: In idle mode, clicking "Cruise Setting" enters the cruise setting mode; selecting a path and clicking "Start Cruise" enters the cruise execution mode. In cruise setting mode, saving or canceling returns to idle mode. In cruise execution mode, pausing or stopping returns to idle mode. Any manual operation during cruise execution triggers a pause, switching to manual idle mode. In manual idle mode, clicking "Resume Cruise" returns to cruise execution mode. This ensures the system has clear behavioral boundaries in different operating modes, avoiding operational conflicts between cruise settings and dual-window linkage.

[0132] In a preferred embodiment, the system timestamps and stores the panoramic image sequence as a video file. The panoramic video recording contains complete panoramic information without losing any content from any direction. During playback, the system can reproduce the dual-window interactive interface and virtual PTZ function. Users can drag the field-of-view indicator box in the panoramic preview window or drag the image in the PTZ detail window to view content from any direction during recording. This function allows for the observation of permanently lost areas that were not oriented during shooting, as seen in traditional PTZ recordings.

[0133] like Figure 3 As shown, the second aspect of this invention proposes a two-way interactive system for a stitched panoramic preview window and a PTZ detail window, used to execute the two-way interactive method for a stitched panoramic preview window and a PTZ detail window proposed in the first aspect. The system includes the following functional modules:

[0134] The dual-window display module is used to display a panoramic preview window containing a stitched panoramic image and a PTZ detail window containing a locally cropped and magnified image.

[0135] A field of view indicator module is used to overlay a field of view indicator box in the panoramic preview window, the field of view indicator box being used to indicate the area range of the PTZ detail window in the panoramic image.

[0136] The input detection module is used to detect user input operations on the panoramic preview window or the PTZ detail window.

[0137] The linkage control module is used to calculate the target state of the field of view indicator box and execute boundary constraints according to the type and position of the input operation to obtain the updated field of view state.

[0138] The synchronous refresh module is used to synchronously refresh the field of view indicator box in the panoramic preview window and the cropping area in the PTZ detail window based on the updated field of view state.

[0139] The rendering output module is used to render and output the refreshed panoramic preview window and PTZ detail window to the display device.

[0140] The present invention has the following beneficial effects:

[0141] This invention employs a dual-window linkage architecture of a panoramic preview window and a PTZ detail window. In the panoramic preview window, a field of view indicator box continuously marks the precise spatial position of the current PTZ field of view, enabling users to understand the location of any local area in real time within the entire monitoring scene, regardless of which area is zoomed in on, thus solving the orientation loss problem of traditional PTZ.

[0142] This invention supports multiple interaction methods, including dragging the view frame in the panoramic window, dragging the image in the detail window, zooming, and clicking in the panoramic window, achieving a two-way interactive and WYSIWYG (What You See Is What You Get) user experience. Users can drag or click wherever they want to look, achieving this in one step. Compared to traditional PTZ split-axis joystick operation, which requires multiple adjustments in the horizontal and vertical directions, this significantly improves operational efficiency.

[0143] This invention achieves perspective switching through digital cropping, eliminating mechanical rotation delays and blind spots in monitoring during rotation; perspective switching is completed almost instantaneously. It also eliminates issues of mechanical wear, rotational noise, and motor lifespan limitations, significantly improving the long-term reliability of the product.

[0144] This invention supports full-time storage of panoramic images, and during playback, it can reproduce the dual-window interactive function, allowing users to view the content of the image from any direction during recording, thus achieving a post-event retrospective capability that is physically impossible with traditional PTZ.

[0145] Based on the same panoramic view, this invention can simultaneously generate multiple independent virtual PTZ output streams, which is equivalent to replacing multiple traditional PTZ cameras with one stitching camera, significantly reducing the hardware and cabling costs of multi-view monitoring.

[0146] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window, characterized in that, Includes the following steps: Displays a panoramic preview window containing the stitched panoramic image and a PTZ detail window containing a cropped and magnified image; A field of view indicator box is overlaid in the panoramic preview window. The field of view indicator box is used to indicate the current area range of the PTZ detail window in the panoramic image. Detect user input on the panoramic preview window or the PTZ detail window; Based on the type and location of the input operation, calculate the target state of the field of view indicator box and perform boundary constraints to obtain the updated field of view state. Based on the updated field of view state, the field of view indicator box in the panoramic preview window and the cropping area in the PTZ detail window are refreshed simultaneously. The refreshed panoramic preview window and PTZ detail window are rendered and output to the display device.

2. The method for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window according to claim 1, characterized in that, The field of view indicator frame is any one of a rectangular frame, an elliptical frame, a polygonal frame, a highlight mask, or a semi-transparent color block; the border color of the field of view indicator frame is a highlight color.

3. The method for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window according to claim 1, characterized in that, The specific method for calculating the target state of the field of view indicator and executing boundary constraints based on the type and position of the input operation includes: when the input operation is dragging the field of view indicator in the panoramic preview window, obtaining the displacement of the drag operation; updating the position of the field of view indicator based on the displacement; determining whether the updated field of view indicator exceeds the boundary of the panoramic image; and when it exceeds the boundary, restricting the field of view indicator within the boundary of the panoramic image.

4. The method for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window according to claim 1, characterized in that, The specific method for calculating the target state of the field of view indicator and executing boundary constraints based on the type and location of the input operation includes: when the input operation is dragging the image in the PTZ detail window, obtaining the displacement of the drag operation; mapping the displacement inversely to the displacement of the field of view indicator in the panoramic image according to the scaling factor of the PTZ detail window; updating the position of the field of view indicator based on the inversely mapped displacement; determining whether the updated field of view indicator exceeds the boundary of the panoramic image; and restricting the field of view indicator within the boundary of the panoramic image when it exceeds the boundary.

5. The method for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window according to claim 1, characterized in that, The specific method for calculating the target state of the field of view indicator and executing boundary constraints based on the type and location of the input operation includes: when the input operation is a zoom operation, obtaining the zoom center point and the zoom ratio change; using the zoom center point as an anchor point, calculating a new field of view indicator size based on the zoom ratio change; determining whether the new field of view indicator size exceeds a preset minimum and maximum size range; and when it exceeds the range, limiting the field of view indicator size to the preset range.

6. The method for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window according to claim 1, characterized in that, The specific method for calculating the target state of the field of view indicator and executing boundary constraints based on the type and location of the input operation includes: when the input operation is a click operation in the panoramic preview window, obtaining the coordinate position of the click operation in the panoramic image; moving the center of the field of view indicator to the coordinate position; determining whether the moved field of view indicator exceeds the boundary of the panoramic image; and when it exceeds the boundary, restricting the field of view indicator to within the boundary of the panoramic image.

7. The method for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window according to claim 1, characterized in that, The method also includes a cruise path setting step: In response to a command to enter cruise setting mode, the system accepts the user's route input in the panoramic preview window; When the path input is a continuous drag trajectory, the drag trajectory is sampled at a preset sampling interval, and a path point sequence is generated after smoothing and filtering. When the path input is a sequence of key points clicked sequentially, a smooth curve path is generated between adjacent key points through Catmull-Rom spline interpolation; The generated path is displayed as a highlighted line overlaid in the panoramic preview window, and key points are displayed as draggable control points. The path is updated in real time in response to the user's drag operation on the control point, and the path is stored as a cruise path in response to the save command.

8. The method for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window according to claim 7, characterized in that, The method also includes a cruise execution step: In response to the command to start cruise, load the target cruise path and parameter configuration; The field of view indicator box is driven to move frame by frame along the cruise path. The movement step size of the current frame is calculated and the position of the field of view indicator box is updated in each frame. When the field of view indicator box reaches a key point, the movement is paused and the frame remains there for a preset duration. Upon reaching the end of the path, one of the following actions is performed based on the loop pattern: one-way stop, round-trip reversal, or loop reset. The panoramic preview window and PTZ detail window are refreshed synchronously with each frame; in response to manual operation by the user during cruise, cruise is paused and switched to manual mode; in response to the resume command, cruise continues from the interrupted position.

9. The method for bidirectional interactive linkage between a panoramic preview window and a PTZ detail window according to claim 8, characterized in that, The cruise execution steps also include adaptive speed adjustment: detecting the density of moving targets in the area near the field of view indicator box at preset evaluation intervals; when the density of moving targets exceeds a first preset threshold, reducing the cruise speed to a first speed level; when the density of moving targets is lower than a second preset threshold, restoring the cruise speed to a normal speed level; wherein, the speed switching adopts a frame-by-frame smooth transition.

10. A two-way interactive system for a stitched panoramic preview window and a PTZ detail window, characterized in that, include: The dual-window display module is used to display a panoramic preview window containing a stitched panoramic image and a PTZ detail window containing a locally cropped and magnified image; A field of view indicator module is used to overlay a field of view indicator box in the panoramic preview window, the field of view indicator box being used to indicate the current area range of the PTZ detail window in the panoramic image; The input detection module is used to detect user input operations on the panoramic preview window or the PTZ detail window; The linkage control module is used to calculate the target state of the field of view indicator box and execute boundary constraints according to the type and position of the input operation to obtain the updated field of view state. The synchronous refresh module is used to synchronously refresh the field of view indicator box in the panoramic preview window and the cropping area in the PTZ detail window based on the updated field of view state. The rendering output module is used to render and output the refreshed panoramic preview window and PTZ detail window to the display device.

Images