Virtual PTZ control method, device, electronic device, and storage medium

Abstract

The present application proposes a control method, device, electronic device, and storage medium for a virtual pan-tilt head. The method includes: obtaining an imaging plane of an imaging module at a first angle and a fisheye image thereof captured by a fisheye lens, wherein the height value of the imaging plane is greater than the width value; determining a correction area of ​​the fisheye image in the imaging plane, performing distortion correction on the correction area, and obtaining a corrected image; performing a frame selection based on the corrected image with a first aspect ratio to obtain a first image to be processed, wherein the width value of the first image to be processed is greater than the height value; rotating the first image to be processed by 90 degrees to obtain a second image to be processed; performing a frame selection based on the second image to be processed with a second aspect ratio to obtain a first display image under a display window; and achieving a display effect of a virtual pan-tilt head perspective change by adjusting the coordinate offset of the center point of the display window on the calibration plane. The virtual pan-tilt head of the present application can achieve the effect of pan-tilt head change without requiring a pan-tilt head mechanical structure.

Description

Technical Field

[0001] The present application relates to the technical field of virtual pan-tilt platforms, and in particular to a control method, device, electronic device, and storage medium for a virtual pan-tilt platform. Background Art

[0002] The pan / tilt head is a supporting device for installing and fixing cameras. With the rapid development of science and technology, higher requirements are placed on the pan / tilt head.

[0003] Traditional gimbals typically utilize a mechanical structure coupled with a highly distorted fisheye lens. However, mechanical gimbals are limited by their physical structure and mechanical transmission components, presenting bottlenecks in terms of rotation speed, precision, flexibility, lifespan, and functionality. For example: 1) High-speed rotation can lead to mechanical wear and increased vibration, making ultra-fast, ultra-precise instantaneous angle adjustment difficult to achieve. 2) Mechanical gimbals require physical installation, which can be challenging in specialized environments or scenarios with limited space, such as small drones and portable surveillance equipment. Maintenance and upgrades are also complex. 3) During rotation, friction and transmission between mechanical components generate noise. This can interfere with the surrounding environment or affect audio quality in applications with high noise requirements, such as quiet indoor surveillance environments or those requiring audio capture. 4) Traditional mechanical gimbals primarily focus on basic angle control and device load-bearing capabilities. Implementing more complex features, such as simultaneous multi-target tracking and intelligent scene analysis, requires extensive hardware modifications, which are both costly and challenging to implement. Summary of the Invention

[0004] In order to solve the above technical problems, the present application proposes a control method, device, electronic device and storage medium for a virtual pan-tilt head, which can achieve the effect of pan-tilt head movement without the need for a pan-tilt head mechanical structure. Compared with the traditional mechanical structure, it can be small in size, low in cost, fast in response, noiseless, and has stronger functional scalability.

[0005] According to a first aspect of the present application, a method for controlling a virtual pan / tilt platform is proposed, comprising:

[0006] Acquire an imaging plane of the imaging module at a first angle and a fisheye image thereof captured by a fisheye lens, wherein a height value of the imaging plane is greater than a width value;

[0007] determining a correction region of the fisheye image in the imaging plane, and performing distortion correction on the correction region to obtain a corrected image;

[0008] Performing a frame selection based on the rectified image with a first aspect ratio to obtain a first image to be processed, wherein a width value of the first image to be processed is greater than a height value;

[0009] Rotating the first image to be processed by 90 degrees to obtain a second image to be processed;

[0010] Based on the second image to be processed, a frame is selected with a second aspect ratio to obtain a display window, and a mapping relationship of pixels of the second image to be processed in the display window is obtained to obtain a first display image;

[0011] By adjusting the coordinate offset of the center point of the display window on the calibration plane, a virtual pan-tilt viewing angle change display effect is achieved.

[0012] Preferably, the method further comprises:

[0013] Pixel compensation is performed on the first display image in the height and / or width direction to obtain a second display image with a target resolution.

[0014] Preferably, performing pixel compensation on the first display image in the height and / or width direction to obtain a second display image of target resolution includes:

[0015] An interpolation algorithm is used to perform interpolation compensation on the pixels of the first display image in the height and / or width direction to obtain a second display image of target resolution.

[0016] Preferably, the step of adjusting the coordinate offset of the center point of the display window on the calibration plane to achieve a display effect of a virtual pan-tilt platform viewing angle change includes:

[0017] The coordinate offset trajectory of the center point of the display window on the calibration plane is controlled by an S-shaped speed curve algorithm, so that the virtual pan-tilt viewing angle change display presents a smooth transition.

[0018] Preferably, the coordinate offset trajectory of the center point of the display window on the calibration plane includes three stages: acceleration, uniform speed and deceleration. By presetting the acceleration threshold parameters, the acceleration, velocity and displacement of the three stages are calculated respectively, thereby obtaining the coordinate offset trajectory.

[0019] Preferably, the acceleration equations of the three stages are expressed as follows:

[0020]

[0021] Integrating the acceleration gives the velocity equation as follows:

[0022]

[0023] Integrating the velocity equation yields the displacement equation as follows:

[0024]

[0025] in, is the acceleration, For speed, is the displacement, is the acceleration threshold parameter, t is the current moment, 0≤t< T 1 is the acceleration section, T 1≤t< T 2 is the uniform speed segment, T 2≤t≤ T It is the deceleration section.

[0026] Preferably, determining the correction area of ​​the fisheye image in the imaging plane includes:

[0027] Using the inscribed rectangular area of ​​the fisheye image as the correction area;

[0028] The step of performing frame selection based on the rectified image with a first aspect ratio to obtain a first image to be processed includes:

[0029] A first image to be processed is obtained by performing a maximum frame selection in a width direction based on the corrected image with a first aspect ratio.

[0030] According to a second aspect of the present application, a control device for a virtual pan / tilt platform is proposed, comprising:

[0031] an acquisition unit configured to acquire an imaging plane of the imaging module at a first angle and a fisheye image thereof captured by the fisheye lens, wherein a height value of the imaging plane is greater than a width value;

[0032] a correction unit configured to determine a correction area of ​​the fisheye image in the imaging plane, perform distortion correction on the correction area, and obtain a corrected image;

[0033] a frame selection unit configured to perform frame selection based on the rectified image with a first aspect ratio to obtain a first image to be processed, wherein a width value of the first image to be processed is greater than a height value;

[0034] a rotation unit configured to rotate the first image to be processed by 90 degrees to obtain a second image to be processed;

[0035] a display unit configured to perform a frame selection based on the second image to be processed with a second aspect ratio to obtain a display window, obtain a mapping relationship of pixels of the second image to be processed in the display window, and obtain a first display image;

[0036] The virtual pan-tilt control unit is configured to achieve a virtual pan-tilt viewing angle change display effect by adjusting the coordinate offset of the center point of the display window on the calibration plane.

[0037] According to the third aspect of the present application, an electronic device is proposed, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the control method of the virtual pan-tilt head provided in any embodiment of the first aspect above.

[0038] According to a fourth aspect of the present application, a computer-readable storage medium is proposed, on which a computer program is stored. When the program is executed by a processor, the control method of the virtual pan-tilt head provided in any embodiment of the first aspect above is implemented.

[0039] This application proposes a control method, device, electronic device and storage medium for a virtual pan-tilt head, which can achieve the effect of pan-tilt head movement without the need for a pan-tilt head mechanical structure. Compared with traditional mechanical structures, it can be small in size, low in cost, fast in response and noiseless.

[0040] Furthermore, by rotating the imaging module 90 degrees for imaging, the height of the imaging plane is made greater than the width, thereby retaining a larger field of view in the vertical direction of the fisheye image. Distortion correction is then performed on the selected correction area to obtain a corrected image, which is then cropped to the maximum width at the target ratio. The cropped image is then rotated 90 degrees to restore it to a normal viewing angle. In this way, without increasing the physical size of the lens, the virtual gimbal can obtain a larger field of view and viewing angle adjustment space in the vertical direction. As a result, the virtual gimbal has a larger rotation angle in the vertical direction, which can better realize the simulation of perspective translation / pitching motion.

[0041] Furthermore, an S-shaped speed curve algorithm is used to control the coordinate offset of the center point of the display window on the calibration plane, and the coordinate offset trajectory is optimized so that the change of the virtual gimbal's viewing angle presents a smooth transition, so that there will be no obvious lag in the process of rotating the virtual gimbal's viewing angle, and the response time is faster. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and, together with the description, serve to explain the principles of the present invention. Other embodiments and many of the expected advantages of the embodiments will be readily apparent as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with respect to each other. Like reference numerals designate corresponding similar parts.

[0043] Figure 1is a flow chart of a control method for a virtual PTZ according to a specific embodiment of the present application;

[0044] Figure 2 This is a comparison diagram of imaging at different placement angles of an imaging module according to a specific embodiment of the present application;

[0045] Figure 3 is a schematic diagram of a fisheye image of a calibration plane according to a specific embodiment of the present application;

[0046] Figure 4 is a schematic diagram of fisheye image correction according to a specific embodiment of the present application;

[0047] Figure 5 is a schematic diagram of a frame selection of a first image to be processed according to a specific embodiment of the present application;

[0048] Figure 6 is a schematic diagram of generating a second image to be processed according to a specific embodiment of the present application;

[0049] Figure 7 is a schematic diagram of display image generation according to a specific embodiment of the present application;

[0050] Figure 8 is a schematic diagram of display image generation according to another specific embodiment of the present application;

[0051] Figure 9 is a schematic diagram of display image generation according to another specific embodiment of the present application;

[0052] Figure 10 is a schematic diagram showing a change in viewing angle of a virtual PTZ according to a specific embodiment of the present application;

[0053] Figure 11 2 is a schematic diagram of a coordinate shift process of a center point of a display window on a calibration plane according to a specific embodiment of the present application;

[0054] Figure 12 is a structural diagram of a control device for a virtual PTZ according to a specific embodiment of the present application;

[0055] Figure 13 is a schematic diagram of an electronic device according to a specific embodiment of the present application. DETAILED DESCRIPTION

[0056] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the objects, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the present invention.

[0057] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of additional identical elements in the process, method, article, or device comprising the elements.

[0058] This application proposes a control method for a virtual pan-tilt platform. Figure 1 A flow chart of a method for controlling a virtual PTZ according to a specific embodiment of the present application is shown. Figure 1 As shown, the method includes the following steps:

[0059] Step S1: obtaining an imaging plane of the imaging module at a first angle and a fisheye image thereof captured by a fisheye lens, wherein a height value of the imaging plane is greater than a width value.

[0060] Generally speaking, the aspect ratio of an imaging module (Sensor) is usually 4:3, meaning the horizontal field of view of the imaging module is "4" and the vertical field of view is "3". However, for a virtual gimbal, a larger viewing angle adjustment range is required in the vertical direction, and therefore a larger field of view is required. Therefore, in this embodiment, the imaging module with a 4:3 aspect ratio is rotated 90 degrees (a first angle) for imaging. This changes the imaging plane of the imaging module from the original 4:3 aspect ratio to a 3:4 aspect ratio, achieving a larger vertical field of view.

[0061] Figure 2 FIG shows a comparison diagram of imaging at different placement angles of an imaging module according to a specific embodiment of the present application. Figure 2For ease of illustration, using an imaging module with an aspect ratio of 4:3 as an example, in a horizontal position, the upper left corner of imaging plane 10 is defined as point 0, the row direction of imaging plane 10 is defined as the X direction, and the column direction of imaging plane 10 is defined as the Y direction, forming the imaging module's scanning coordinate system X0Y. The imaging module scans and images in the X direction, resulting in a vertical field of view of "3" (in the Y direction) and a horizontal field of view of "4" (in the X direction) for the actual scene. In the vertical position, the reference coordinate system X0Y is rotated 90 degrees, and the imaging module still scans and images in the X direction. Therefore, the vertical field of view of the actual scene is "4" (in the X direction) and the horizontal field of view of "3" (in the X direction) for the actual scene. The angle of the actual scene in the imaging module's actual imaging remains unchanged, but the final image output from the imaging module appears to be a 90-degree rotation of the actual scene.

[0062] Based on the above principles, it can be understood that this embodiment obtains a larger field of view in the vertical direction by rotating the imaging module with an aspect ratio of 4:3 by 90 degrees (first angle) for imaging, but the image obtained from the imaging module is rotated by 90 degrees compared with the actual scene.

[0063] Figure 3 FIG. 1 shows a schematic diagram of a fisheye image of a calibration plane according to a specific embodiment of the present application. Figure 3 As shown, the resolution of the imaging plane 10 is N×M, where M>N. For example, N×M can be 480×640, 768×1024, 1200×1600, 1536×2048, and so on. N×M can be determined based on the usage scenario of the fisheye lens. N×M can be set or modified based on usage requirements. Specifically, N×M can be set or modified based on the size of the effective area in the fisheye image 20. N×M can cover the effective area in the fisheye image 20 or be larger than the effective area in the fisheye image 20.

[0064] Step S2: determining a correction area of ​​the fisheye image in the imaging plane, performing distortion correction on the correction area, and obtaining a corrected image.

[0065] Figure 4 FIG. 4 shows a schematic diagram of fisheye image correction according to a specific embodiment of the present application. Figure 4 As shown, in a specific embodiment, the inscribed rectangular area abcd of the fisheye image 20 is selected as the correction area, and then the GDC (Geometry Distortion Correction) module of the HiSilicon 35 series chip is used to perform distortion correction on the correction area abcd to obtain the corrected image 30.

[0066] In this embodiment, the GDC module of the HiSilicon 3519 chip is specifically used to perform barrel distortion correction on the correction area abcd to eliminate the barrel distortion and obtain a corrected image 30.

[0067] It should be noted that the inscribed rectangular area abcd of the fisheye image 20 can be a rectangle or a square.

[0068] Step S3: performing frame selection based on the rectified image with a first aspect ratio to obtain a first image to be processed, wherein the width value of the first image to be processed is greater than the height value.

[0069] Figure 5 FIG. 4 shows a schematic diagram of a frame selection of a first image to be processed according to a specific embodiment of the present application, as shown in FIG. Figure 5 As shown, in a specific embodiment, the corrected image 30 is subjected to maximum frame selection in the width direction with a first aspect ratio to obtain a first image to be processed 40 .

[0070] In this embodiment, the first aspect ratio is 4: 3. In other embodiments, the first aspect ratio may also be 16: 9 or other ratios, which are not limited here.

[0071] Step S4: Rotate the first image to be processed by 90 degrees to obtain a second image to be processed.

[0072] Figure 6 FIG. 4 shows a schematic diagram of generating a second image to be processed according to a specific embodiment of the present application, as shown in FIG. Figure 6 As shown, as previously described, the imaging module of this embodiment achieves a larger vertical field of view. However, the image obtained from the imaging module is rotated 90 degrees compared to the actual scene. Therefore, the first image to be processed 40 is rotated 90 degrees to obtain the second image to be processed 50, and the actual scene in the second image to be processed 50 is restored to a normal viewing angle. Furthermore, the height of the second image to be processed 50 is greater than its width, so the second image to be processed 50 has a larger vertical field of view, thereby providing greater rotation adjustment space.

[0073] Step S5: Based on the second image to be processed, a frame is selected with a second aspect ratio to obtain a display window, and a mapping relationship between pixels of the second image to be processed in the display window is obtained to obtain a first display image.

[0074] Figure 7 FIG. 4 shows a schematic diagram of display image generation according to a specific embodiment of the present application. Figure 7 As shown, in a specific embodiment, the second image to be processed 50 is framed with a second aspect ratio to obtain a first display image 60 to be displayed, which is directly displayed on the virtual pan-tilt platform.

[0075] In this embodiment, the second aspect ratio is 16:9. In other embodiments, the second aspect ratio may also be 4:3 or other ratios, which are not limited here.

[0076] In a preferred embodiment, step S5 further includes: performing pixel compensation on the first display image in the height and / or width direction to obtain a second display image of target resolution.

[0077] In a possible embodiment, an interpolation algorithm is used to perform interpolation compensation on the pixels of the first display image 60 in the height and / or width direction to obtain the second display image 70 of the target resolution.

[0078] In this embodiment, a bilinear interpolation algorithm is specifically used to interpolate and compensate the pixels of the first display image 60 in the height and / or width directions. In other embodiments, bicubic interpolation, nearest neighbor interpolation, etc. may also be used to interpolate and compensate the pixels of the first display image in the height and / or width directions, which is not limited here.

[0079] Figure 8 FIG. 4 shows a schematic diagram of display image generation according to another specific embodiment of the present application. Figure 8 As shown, in another specific embodiment, the second image to be processed 50 is framed with a second aspect ratio to obtain a first display image 60, and then the first display image 60 is interpolated and compensated in the height direction to obtain a second display image 70 of the target resolution, and is displayed on a virtual gimbal.

[0080] Figure 9 FIG. 4 shows a schematic diagram of display image generation according to another specific embodiment of the present application. Figure 9 As shown, in another specific embodiment, the second image to be processed 50 is framed with a second aspect ratio to obtain a first display image 60, and then the first display image 60 is interpolated and compensated in the width direction to obtain a second display image 70 of the target resolution, and is displayed on a virtual gimbal.

[0081] It is understandable that, in other embodiments, interpolation compensation may be performed on the first display image simultaneously in the height direction and the width direction to obtain a second display image of the target resolution.

[0082] Step S6: Achieve a display effect of a virtual pan-tilt viewing angle change by adjusting the coordinate offset of the center point of the display window on the calibration plane.

[0083] Figure 10 A schematic diagram showing a display of a virtual PTZ viewing angle change according to a specific embodiment of the present application is shown in FIG. Figure 10As shown, for ease of demonstration, the center point of the second image to be processed 50 is defined as point 0, the width of the second image to be processed 50 is defined as the X direction, and the height of the second image to be processed 50 is defined as the Y direction, establishing a calibration plane coordinate system X0Y. After selecting the display window, by adjusting the center point coordinates of the display window (i.e., the first display image 60) from (x1, y1) to (x2, y2), the display window can be slid horizontally or vertically, thereby simulating the pan / tilt motion of the virtual gimbal.

[0084] In a specific embodiment, the coordinate offset trajectory of the center point of the display window in the second image to be processed 50 is controlled by an S-shaped speed curve algorithm, so that the change in the virtual pan-tilt perspective is displayed to present a smooth transition.

[0085] Figure 11 FIG. 1 shows a schematic diagram of a coordinate shift process of the center point of a display window on a calibration plane according to a specific embodiment of the present application. Figure 11 As shown, the coordinate offset trajectory of the center point of the display window in the second image to be processed 50 includes three stages: acceleration stage 0-T1, uniform speed stage T1-T2, and deceleration stage T2-T. By presetting the acceleration threshold parameter, in the acceleration stage, the acceleration increases from 0 to the acceleration threshold, and the speed reaches the maximum value; in the uniform speed stage, the acceleration is 0 and the speed remains constant at the maximum value; in the deceleration stage, the acceleration decreases from the negative acceleration threshold to 0 until the speed decelerates to 0. The coordinate offset trajectory is obtained by calculating the acceleration, velocity, and displacement of the three stages.

[0086] Among them, the acceleration equation expressions of the three stages are as follows:

[0087]

[0088] Integrating the acceleration yields the following velocity equations for the three stages:

[0089]

[0090] Integrating the velocity equations yields the following three-stage displacement equations:

[0091]

[0092] in, is the acceleration, For speed, is the displacement, is the acceleration threshold parameter, t is the current moment, 0≤t< T 1 is the acceleration section, T 1≤t< T 2 is the uniform speed segment,T 2≤t≤ T It is the deceleration section.

[0093] In this way, by using the S-shaped speed curve algorithm to control the coordinate offset of the center point of the display window on the calibration plane, the coordinate offset trajectory is optimized, so that the change of the virtual gimbal's viewing angle presents a smooth transition, so that the virtual gimbal will not have obvious lag in the process of rotating the viewing angle, and the response time is faster.

[0094] In this embodiment, the response time of the virtual PTZ viewing angle change can be shortened to 50ms.

[0095] It should be noted that, in other implementations, other algorithms such as Bezier curves may also be used to achieve smooth transition of the display of the virtual pan-tilt viewing angle change, which is not limited here.

[0096] In summary, the present application provides a control method for a virtual pan-tilt head, which achieves the effect of pan-tilt head movement without the need for a pan-tilt head mechanical structure. Compared with traditional mechanical structures, it can be small in size, low in cost, fast in response, and noiseless. Specifically, the present application rotates the imaging module 90 degrees for imaging, so that the height value of the imaging plane is greater than the width value, so that the fisheye image retains a larger field of view in the vertical direction. Then, the correction area selected on the fisheye image is subjected to distortion correction to obtain a corrected image, and then the corrected image is cropped at the target ratio, and then the cropped image is rotated 90 degrees to restore it to a normal viewing angle. In this way, without increasing the physical size of the lens, the virtual pan-tilt head can obtain a larger field of view and viewing angle adjustment space in the vertical direction. Thus, the virtual pan-tilt head has a larger rotation angle in the vertical direction, which can better realize the simulation of perspective translation / pitching motion. Furthermore, an S-shaped speed curve algorithm is used to control the coordinate offset of the center point of the display window on the calibration plane, and the coordinate offset trajectory is optimized so that the change of the virtual gimbal's viewing angle presents a smooth transition, so that there will be no obvious lag in the process of rotating the virtual gimbal's viewing angle, and the response time is faster.

[0097] According to the above-mentioned virtual PTZ control method, based on the same inventive concept, the present application also proposes a virtual PTZ control device. Figure 12 FIG. 1 shows a structure diagram of a control device for a virtual PTZ according to a specific embodiment of the present application. Figure 12 As shown, the device includes:

[0098] An acquisition unit 100 is configured to acquire an imaging plane of the imaging module at a first angle and a fisheye image thereof captured by a fisheye lens, wherein a height value of the imaging plane is greater than a width value;

[0099] The correction unit 200 is configured to determine a correction area of ​​the fisheye image in the imaging plane, perform distortion correction on the correction area, and obtain a corrected image;

[0100] A frame selection unit 300 is configured to perform frame selection based on the rectified image with a first aspect ratio to obtain a first image to be processed, wherein a width value of the first image to be processed is greater than a height value;

[0101] The rotation unit 400 is configured to rotate the first image to be processed by 90 degrees to obtain a second image to be processed;

[0102] The display unit 500 is configured to perform a frame selection based on the second image to be processed with a second aspect ratio to obtain a display window, obtain a mapping relationship between pixels of the second image to be processed in the display window, and obtain a first display image;

[0103] The virtual pan-tilt control unit 600 is configured to achieve a virtual pan-tilt viewing angle change display effect by adjusting the coordinate offset of the center point of the display window on the calibration plane.

[0104] In a specific embodiment, the display unit 500 is further configured to perform pixel compensation on the first display image in the height and / or width direction to obtain a second display image of the target resolution.

[0105] In a specific embodiment, the virtual gimbal control unit 600 is further configured to control the coordinate offset trajectory of the center point of the display window on the calibration plane through an S-shaped speed curve algorithm, so that the virtual gimbal viewing angle change display presents a smooth transition.

[0106] According to the above-mentioned control method of the virtual pan-tilt head, based on the same inventive concept, the present application also proposes an electronic device. Figure 13 A schematic diagram of an electronic device according to a specific embodiment of the present application is shown. Figure 13 As shown, the electronic device includes: one or more processors 701, memory 702, a bus 703, and a communication interface 704. The one or more processors 701, memory 702, and communication interface 704 are connected via bus 703. The memory 702 is used to store one or more programs. When the one or more programs are executed by the one or more processors 701, the electronic device implements the virtual pan-tilt control method provided in any of the above embodiments.

[0107] According to the above-mentioned virtual pan-tilt control method, based on the same inventive concept, the present application also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the virtual pan-tilt control method provided by any of the above-mentioned embodiments is implemented.

[0108] In the embodiments of the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device / system / method embodiments described above are merely schematic. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0109] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0110] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0111] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0112] Obviously, those skilled in the art can make various modifications and changes to the embodiments of the present invention without departing from the spirit and scope of the present invention. In this way, if these modifications and changes are within the scope of the claims of the present invention and their equivalents, the present invention is also intended to cover these modifications and changes. The word "comprising" does not exclude the presence of other elements or steps not listed in the claims. The simple fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be considered as limiting the scope.

Claims

1. A method for controlling a virtual PTZ, characterized in that: include: Acquire an imaging plane of the imaging module at a first angle and a fisheye image thereof captured by a fisheye lens, wherein a height value of the imaging plane is greater than a width value; determining a correction region of the fisheye image in the imaging plane, and performing distortion correction on the correction region to obtain a corrected image; Performing a frame selection based on the rectified image with a first aspect ratio to obtain a first image to be processed, wherein a width value of the first image to be processed is greater than a height value; Rotating the first image to be processed by 90 degrees to obtain a second image to be processed; Based on the second image to be processed, a frame is selected with a second aspect ratio to obtain a display window, a mapping relationship of pixels of the second image to be processed in the display window is obtained to obtain a first display image, and pixel compensation is performed on the first display image in a height and / or width direction to obtain a second display image of a target resolution; The center point of the second image to be processed is defined as a circle, the width direction is defined as the X direction, and the height direction is defined as the Y direction. A calibration plane coordinate system is established. By adjusting the coordinate offset of the center point of the display window on the calibration plane, a virtual pan-tilt perspective change display effect is achieved.

2. The method according to claim 1, characterized in that The performing pixel compensation on the first display image in a height and / or width direction to obtain a second display image with a target resolution includes: An interpolation algorithm is used to perform interpolation compensation on the pixels of the first display image in the height and / or width direction to obtain a second display image of target resolution.

3. The method according to claim 1, characterized in that The method of adjusting the coordinate offset of the center point of the display window on the calibration plane to achieve a display effect of a virtual pan-tilt platform viewing angle change includes: The coordinate offset trajectory of the center point of the display window on the calibration plane is controlled by an S-shaped speed curve algorithm, so that the virtual pan-tilt viewing angle change display presents a smooth transition.

4. The method according to claim 3, characterized in that The coordinate offset trajectory of the center point of the display window on the calibration plane includes three stages: acceleration, uniform speed, and deceleration. By presetting the acceleration threshold parameters, the acceleration, velocity, and displacement of the three stages are calculated respectively, thereby obtaining the coordinate offset trajectory.

5. The method according to claim 4, characterized in that The acceleration equations of the three stages are as follows: Integrating the acceleration gives the velocity equation as follows: Integrating the velocity equation yields the displacement equation as follows: in, is the acceleration, For speed, is the displacement, is the acceleration threshold parameter, t is the current moment, 0≤t< T 1 is the acceleration section, T 1≤t< T 2 is the uniform speed section, T 2≤t≤ T It is the deceleration section.

6. The method according to claim 1, characterized in that Determining a correction area of ​​the fisheye image in the imaging plane includes: Using the inscribed rectangular area of ​​the fisheye image as the correction area; The step of performing frame selection based on the rectified image with a first aspect ratio to obtain a first image to be processed includes: A first image to be processed is obtained by performing a maximum frame selection in a width direction based on the corrected image with a first aspect ratio.

7. A virtual PTZ control device, characterized in that: include: an acquisition unit configured to acquire an imaging plane of the imaging module at a first angle and a fisheye image thereof captured by a fisheye lens, wherein a height value of the imaging plane is greater than a width value; a correction unit configured to determine a correction area of ​​the fisheye image in the imaging plane, perform distortion correction on the correction area, and obtain a corrected image; a frame selection unit configured to perform frame selection based on the rectified image with a first aspect ratio to obtain a first image to be processed, wherein a width value of the first image to be processed is greater than a height value; a rotation unit configured to rotate the first image to be processed by 90 degrees to obtain a second image to be processed; a display unit configured to frame the second image to be processed at a second aspect ratio to obtain a display window, obtain a mapping relationship between pixels of the second image to be processed in the display window to obtain a first display image, and perform pixel compensation on the first display image in a height and / or width direction to obtain a second display image of a target resolution; A virtual pan-tilt control unit is configured to define the center point of the second image to be processed as a circle, the width direction as the X direction, and the height direction as the Y direction. It is recommended to calibrate the plane coordinate system and achieve a virtual pan-tilt perspective change display effect by adjusting the coordinate offset of the center point of the display window on the calibration plane.

8. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images