Virtual holder control method and device, electronic equipment and storage medium

Abstract

The invention provides a virtual holder control method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an imaging plane of an imaging module at a first angle and a fisheye image shot by a fisheye lens, and enabling the height value of the imaging plane to be greater than the width value; determining a correction area of the fisheye image in the imaging plane, and performing distortion correction on the correction area to obtain a corrected image; frame selection is carried out based on the corrected image according to the first aspect ratio, a first to-be-processed image is obtained, and the width value of the first to-be-processed image is larger than the height value; rotating the first to-be-processed image by 90 degrees to obtain a second to-be-processed image; based on the second to-be-processed image, frame selection is carried out according to a second aspect ratio, and a first display image under the display window is obtained; by adjusting the coordinate offset of the central point of the display window on the calibration plane, the visual angle change display effect of the virtual holder is realized. According to the virtual holder, on the premise that a holder mechanical structure is not needed, the holder changing effect can be achieved.

Description

Technical Field

[0001] This application relates to the technical field of virtual pan-tilt, and specifically relates to a control method, device, electronic device, and storage medium for a virtual pan-tilt. Background Art

[0002] A pan-tilt is a support device for installing and fixing a camera. With the rapid development of science and technology, higher requirements are also put forward for pan-tilts.

[0003] Traditional pan-tilts generally use a mechanical structure + a fisheye lens with large distortion. However, mechanical pan-tilts are limited by physical structures and mechanical transmission components, and there are bottlenecks in terms of rotation speed, accuracy, flexibility, service life, and usage functions. For example: 1) Problems such as mechanical wear and increased jitter may occur during high-speed rotation, and it is difficult to achieve ultra-high-speed and ultra-precise instant angle adjustment; 2) Mechanical pan-tilts need to be installed in a physical space. For some special environments or scenarios with limited space, such as small unmanned aerial vehicles and portable monitoring devices, installation and wiring may face many difficulties, and later maintenance and upgrading are also relatively complex; 3) During the rotation of a mechanical pan-tilt, friction and transmission between mechanical components will generate a certain amount of noise, which may interfere with the surrounding environment or affect the quality of audio acquisition in some occasions with high noise requirements, such as a quiet indoor monitoring environment and scenarios where audio acquisition is required; 4) The functions of traditional mechanical pan-tilts mainly focus on basic angle control and equipment bearing. To achieve more complex functions, such as simultaneous tracking of multiple targets and intelligent scene analysis, large-scale transformation of hardware is required, which is costly and difficult to implement. Summary of the Invention

[0004] To solve the above technical problems, this application proposes a control method, device, electronic device, and storage medium for a virtual pan-tilt, which can achieve the effect of pan-tilt movement without the need for a pan-tilt mechanical structure. Compared with traditional mechanical structures, it can be small in size, low in cost, fast in response, noise-free, and has stronger function expandability.

[0005] According to the first aspect of this application, a control method for a virtual pan-tilt is proposed, including: Obtain the imaging plane of the imaging module at a first angle and the fisheye image obtained by shooting with a fisheye lens, where the height value of the imaging plane is greater than the width value; Determine the correction area of the fisheye image in the imaging plane, and perform distortion correction on the correction area to obtain a corrected image; Based on the corrected image, perform a frame selection with a first aspect ratio to obtain a first image to be processed, where the width value of the first image to be processed is greater than the height value; Rotate the first image to be processed by 90 degrees to obtain a second image to be processed; Perform a box selection on the basis of the second image to be processed with a second aspect ratio to obtain a display window, and acquire the mapping relationship of the pixel points of the second image to be processed in the display window to obtain a first display image; Implement the virtual pan-tilt view change display effect by adjusting the coordinate offset of the center point of the display window on the calibration plane.

[0006] Preferably, the method further includes: Perform pixel compensation on the first display image in the height and / or width direction to obtain a second display image with a target resolution.

[0007] Preferably, the performing pixel compensation on the first display image in the height and / or width direction to obtain a second display image with a target resolution includes: Use an interpolation algorithm to perform interpolation compensation on the pixel points of the first display image in the height and / or width direction to obtain a second display image with a target resolution.

[0008] Preferably, the implementing the virtual pan-tilt view change display effect by adjusting the coordinate offset of the center point of the display window on the calibration plane includes: Control the coordinate offset trajectory of the center point of the display window on the calibration plane through an S-shaped speed curve algorithm, so that the virtual pan-tilt view change display presents a smooth transition.

[0009] Preferably, the coordinate offset trajectory of the center point of the display window on the calibration plane includes three stages: an acceleration stage, a constant-speed stage, and a deceleration stage. By presetting an acceleration threshold parameter, calculate the acceleration, speed, and displacement of the three stages respectively, so as to obtain the coordinate offset trajectory.

[0010] Preferably, the acceleration equation expressions of the three stages are as follows: Integrate the acceleration to obtain the speed equation expression as follows: Integrate the speed equation to obtain the displacement equation expression as follows: Wherein, is the acceleration, is the speed, is the displacement, is the acceleration threshold parameter, t is the current time, 0 ≤ t < T 1 is the acceleration stage, T 1 ≤ t < T 2 is the constant-speed stage,T 2 ≤ t ≤ T is the deceleration section.

[0011] Preferably, determining the correction area of the fisheye image in the imaging plane includes: Using the inscribed rectangular area of the fisheye image as the correction area; The step of obtaining the first image to be processed by cropping based on the corrected image with a first aspect ratio includes: Performing maximum cropping in the width direction on the corrected image with a first aspect ratio to obtain the first image to be processed.

[0012] According to a second aspect of the present application, a control device for a virtual pan-tilt is provided, including: An acquisition unit configured to acquire the imaging plane of the imaging module at a first angle and the fisheye image captured by the fisheye lens, wherein the height value of the imaging plane is greater than the width value; A correction unit configured to determine the correction area of the fisheye image in the imaging plane, perform distortion correction on the correction area to obtain a corrected image; A cropping unit configured to crop the corrected image with a first aspect ratio to obtain a first image to be processed, wherein the width value of the first image to be processed is greater than the height value; A rotation unit configured to rotate the first image to be processed by 90 degrees to obtain a second image to be processed; A display unit configured to crop the second image to be processed with a second aspect ratio to obtain a display window, and obtain the mapping relationship of the pixel points of the second image to be processed in the display window to obtain a first display image; A virtual pan-tilt control unit configured to achieve the display effect of the virtual pan-tilt view change by adjusting the coordinate offset of the center point of the display window on the calibration plane.

[0013] According to a third aspect of the present application, an electronic device is provided, including: one or more processors; a memory for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the control method of the virtual pan-tilt provided in any one of the above first aspects.

[0014] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, it implements the control method of the virtual pan-tilt provided in any one of the above first aspects.

[0015] The present application provides a control method, apparatus, electronic device, and storage medium for a virtual pan-tilt head, which can achieve the effect of pan-tilt movement without the need for a mechanical structure of the pan-tilt head. Compared with the traditional mechanical structure, it can be small in size, low in cost, fast in response, and noise-free.

[0016] Further, by rotating the imaging module by 90 degrees for imaging, the height value of the imaging plane is made greater than the width value, so that a larger field of view range is retained in the vertical direction of the fisheye image. Then, distortion correction is performed on the selected correction area to obtain a corrected image, and then the corrected image is cropped maximally in the width direction at a target ratio, and then the cropped image is rotated by 90 degrees to restore to a normal viewing angle. In this way, without increasing the physical size of the lens, the virtual pan-tilt head can obtain a larger field of view range and viewing angle adjustment space in the vertical direction. Thus, the virtual pan-tilt head has a larger rotation angle in the vertical direction and can better simulate the viewing angle translation / pitching movement.

[0017] Further, the S-shaped speed curve algorithm is used to control the coordinate offset of the center point of the display window on the calibration plane, and the coordinate offset trajectory is optimized, so that the viewing angle change of the virtual pan-tilt head shows a smooth transition, so that there is no obvious lag during the process of the virtual pan-tilt head rotating the viewing angle, and the response time is faster. Description of the Drawings

[0018] The drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The drawings illustrate the embodiments and, together with the description, are used to explain the principles of the present invention. Other embodiments and many of the intended advantages of the embodiments will be readily apparent as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with each other. Like reference numerals refer to corresponding like parts.

[0019] Figure 1 is a flowchart of a control method for a virtual pan-tilt head according to a specific embodiment of the present application; Figure 2 is a comparison diagram of imaging of an imaging module at different placement angles according to a specific embodiment of the present application; Figure 3 is a schematic diagram of a fisheye image of a calibration plane according to a specific embodiment of the present application; Figure 4 is a schematic diagram of distortion correction of a fisheye image according to a specific embodiment of the present application; Figure 5 is a schematic diagram of frame selection of a first image to be processed according to a specific embodiment of the present application; Figure 6 is a schematic diagram of generation of a second image to be processed according to a specific embodiment of the present application; Figure 7 It is a schematic diagram of generating a display image according to a specific embodiment of the present application; Figure 8 It is a schematic diagram of generating a display image according to another specific embodiment of the present application; Figure 9 It is a schematic diagram of generating a display image according to yet another specific embodiment of the present application; Figure 10 It is a schematic diagram of the change in the perspective of a virtual pan-tilt head according to a specific embodiment of the present application; Figure 11 It is a schematic diagram of the coordinate offset process of the center point of a display window on a calibration plane according to a specific embodiment of the present application; Figure 12 It is a structural diagram of a control device of a virtual pan-tilt head according to a specific embodiment of the present application; Figure 13 It is a schematic diagram of an electronic device according to a specific embodiment of the present application. Detailed implementation manners

[0020] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without some of these specific details. The following description of the embodiments is only provided to provide a better understanding of the present invention by showing examples of the present invention.

[0021] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, the elements defined by the statement "comprising......" do not exclude the presence of additional identical elements in the process, method, article or device comprising the elements.

[0022] The present application proposes a control method for a virtual pan-tilt head. Figure 1 The flowchart of the control method for a virtual pan-tilt head according to a specific embodiment of the present application is shown, as Figure 1 shown, the method includes the following steps: Step S1: Obtain the imaging plane of the imaging module at the first angle and the fisheye image obtained by the fisheye lens shooting, where the height value of the imaging plane is greater than the width value.

[0023] Generally speaking, the aspect ratio of the imaging module (Sensor) is usually 4:3, that is, the field of view range of the imaging module in the horizontal direction is "4", and the field of view range in the vertical direction is "3". However, for a virtual pan-tilt, a larger viewing angle adjustment space is required in the vertical direction, so a larger field of view range is needed. Therefore, in this embodiment, the imaging module with an aspect ratio of 4:3 is rotated 90 degrees (the first angle) for imaging, so that the imaging plane of the imaging module changes from the original 4:3 aspect ratio to 3:4 aspect ratio, and a larger field of view range is obtained in the vertical direction.

[0024] Figure 2 Shows a comparison diagram of the imaging of the imaging module at different placement angles according to a specific embodiment of the present application. As Figure 2 shown, for the convenience of display, taking the imaging module with an aspect ratio of 4:3 as an example, in the horizontal placement state, the upper left corner of the imaging plane 10 is defined as the origin 0, the row direction of the imaging plane 10 is defined as the X direction, and the column direction of the imaging plane 10 is defined as the Y direction, forming the scanning coordinate system X0Y of the imaging module. The imaging module scans and images in the X direction. Therefore, the field of view range obtained in the vertical direction (Y direction) for the actual scene is "3", and the field of view range obtained in the horizontal direction (X direction) is "4". In the vertical placement state, the reference coordinate system X0Y rotates 90 degrees accordingly, and the imaging module still scans and images in the X direction. Therefore, the field of view range obtained in the vertical direction (X direction) for the actual scene is "4", and the field of view range obtained in the horizontal direction (X direction) is "3". The angle of the actual scene in the real imaging of the imaging module remains unchanged, but the final image obtained from the imaging module is that the actual scene is rotated 90 degrees.

[0025] Based on the above principle, it can be understood that in this embodiment, by rotating the imaging module with an aspect ratio of 4:3 by 90 degrees (the first angle) for imaging, a larger field of view range is obtained in the vertical direction, but the image obtained from the imaging module is rotated 90 degrees compared with the actual scene.

[0026] Figure 3 Shows a schematic diagram of the fisheye image of the calibration plane according to a specific embodiment of the present application. As Figure 3As shown, the resolution of the imaging plane 10 is N×M, where M > N. For example, N×M can be 480×640, 768×1024, 1200×1600, 1536×2048, etc. N×M can be determined based on the usage scenario of the fisheye lens. N×M can be set or changed based on usage requirements. Specifically, N×M can be set or changed based on the size of the valid area in the fisheye image 20. N×M can cover the valid area in the fisheye image 20 or be larger than the valid area in the fisheye image 20.

[0027] Step S2: Determine the correction area of the fisheye image in the imaging plane, and perform distortion correction on the correction area to obtain a corrected image.

[0028] Figure 4 The following shows a schematic diagram of the correction of a fisheye image according to a specific embodiment of the present application. As Figure 4 shown, in a specific embodiment, the inscribed rectangular area abcd of the fisheye image 20 is selected as the correction area, and then the GDC (Geometry Distortion Correction) module of the Hisilicon 35 series chip is used to perform distortion correction on the correction area abcd to obtain a corrected image 30.

[0029] In this embodiment, the GDC module of the Hisilicon 3519 chip is specifically used to perform barrel distortion correction on the correction area abcd to eliminate barrel distortion and obtain a corrected image 30.

[0030] It should be noted that the inscribed rectangular area abcd of the fisheye image 20 can be a rectangle or a square.

[0031] Step S3: Based on the corrected image, perform a frame selection with a first aspect ratio to obtain a first image to be processed, where the width value of the first image to be processed is greater than the height value.

[0032] Figure 5 The following shows a schematic diagram of the frame selection of the first image to be processed according to a specific embodiment of the present application. As Figure 5 shown, in a specific embodiment, the maximum frame selection in the width direction is performed on the corrected image 30 with a first aspect ratio to obtain a first image to be processed 40.

[0033] In this embodiment, the first aspect ratio is 4:3. In other embodiments, the first aspect ratio can also be other ratios such as 16:9, and there is no limitation here.

[0034] Step S4: Rotate the first image to be processed by 90 degrees to obtain a second image to be processed.

[0035] Figure 6Shows a schematic diagram of the generation of a second image to be processed according to a specific embodiment of the present application, as Figure 6 shown. As described above, the imaging module of this embodiment obtains a larger field of view in the vertical direction, but the image obtained from the imaging module is rotated 90 degrees compared to the actual scene. Therefore, the first image to be processed 40 is rotated 90 degrees to obtain the second image to be processed 50, and the actual scene in the second image to be processed 50 is restored to the normal perspective. Moreover, the height value of the second image to be processed 50 is greater than the width value, and the second image to be processed 50 has a larger field of view in the vertical direction, thus having a larger rotation adjustment space.

[0036] Step S5: Based on the second image to be processed, perform a frame selection with a second aspect ratio to obtain a display window, and obtain the mapping relationship of the pixel points of the second image to be processed in the display window to obtain the first display image.

[0037] Figure 7 Shows a schematic diagram of the generation of a display image according to a specific embodiment of the present application, as Figure 7 shown. In a specific embodiment, the second image to be processed 50 is frame-selected with a second aspect ratio to obtain the first display image 60 to be displayed, and it is directly displayed on the virtual cloud platform.

[0038] In this embodiment, the second aspect ratio is 16:9. In other embodiments, the second aspect ratio can also be other ratios such as 4:3, and no limitation is made here.

[0039] In a preferred embodiment, step S5 further includes: performing pixel compensation on the first display image in the height and / or width direction to obtain a second display image with a target resolution.

[0040] In a possible embodiment, an interpolation algorithm is used to perform interpolation compensation on the pixel points of the first display image 60 in the height and / or width direction to obtain a second display image 70 with a target resolution.

[0041] In this embodiment, a bilinear interpolation algorithm is specifically used to perform interpolation compensation on the pixel points of the first display image 60 in the height and / or width direction. In other embodiments, bicubic interpolation, nearest neighbor interpolation, etc. can also be used to perform interpolation compensation on the pixel points of the first display image in the height and / or width direction, and no limitation is made here.

[0042] Figure 8 Shows a schematic diagram of the generation of a display image according to another specific embodiment of the present application, as Figure 8As shown, in another specific embodiment, the second image to be processed 50 is framed with a second aspect ratio to obtain a first display image 60. Then, interpolation compensation is performed on the first display image 60 in the height direction to obtain a second display image 70 with a target resolution, which is then displayed on the virtual cloud platform.

[0043] Figure 9 FIG. shows a schematic diagram of generating a display image according to another specific embodiment of the present application. As Figure 9 shown, in yet another specific embodiment, the second image to be processed 50 is framed with a second aspect ratio to obtain a first display image 60. Then, interpolation compensation is performed on the first display image 60 in the width direction to obtain a second display image 70 with a target resolution, which is then displayed on the virtual cloud platform.

[0044] It can be understood that in other embodiments, interpolation compensation can also be performed on the first display image in both the height direction and the width direction to obtain a second display image with a target resolution.

[0045] Step S6: By adjusting the coordinate offset of the center point of the display window on the calibration plane, the virtual cloud platform viewing angle change display effect is achieved.

[0046] Figure 10 FIG. shows a schematic diagram of the virtual cloud platform viewing angle change display according to a specific embodiment of the present application. As Figure 10 shown, for the convenience of display, the center point of the second image to be processed 50 is defined as the origin 0, the width direction of the second image to be processed 50 is defined as the X direction, and the height direction of the second image to be processed 50 is defined as the Y direction, to establish a calibration plane coordinate system X0Y. After the display window is framed, by adjusting the center point coordinates of the display window (i.e., the first display image 60) from (x1, y1) to (x2, y2), the display window can be slid in the horizontal / vertical direction, thereby realizing the simulation of the virtual cloud platform viewing angle translation / pitching motion.

[0047] In a specific embodiment, the S-shaped speed curve algorithm is used to control the coordinate offset trajectory of the center point of the display window in the second image to be processed 50, so that the virtual cloud platform viewing angle change display presents a smooth transition.

[0048] Figure 11 FIG. shows a schematic diagram of the coordinate offset process of the center point of the display window on the calibration plane according to a specific embodiment of the present application. As Figure 11As shown in the figure, the coordinate offset trajectory of the center point of the display window in the second image to be processed 50 includes an acceleration stage 0 - T1, a constant velocity stage T1 - T2, and a deceleration stage T2 - T. By presetting the acceleration threshold parameter, in the acceleration stage, the acceleration increases from 0 to the acceleration threshold, and the velocity reaches the maximum value; in the constant velocity stage, the acceleration is 0, and the velocity remains constant at the maximum value; in the deceleration stage, the acceleration decreases from the negative acceleration threshold to 0 until the velocity decelerates to 0. By calculating the acceleration, velocity, and displacement of the three stages respectively, the coordinate offset trajectory can be obtained.

[0049] Among them, the acceleration equation expressions of the three stages are as follows: Integrating the acceleration gives the velocity equation expressions of the three stages as follows: Integrating the velocity equation gives the displacement equation expressions of the three stages as follows: Among them, is the acceleration, is the velocity, is the displacement, is the acceleration threshold parameter, t is the current time, 0 ≤ t < T 1 is the acceleration stage, T 1 ≤ t < T 2 is the constant velocity stage, T 2 ≤ t ≤ T is the deceleration stage.

[0050] In this way, by using the S-shaped velocity curve algorithm to control the coordinate offset of the center point of the display window on the calibration plane, the coordinate offset trajectory is optimized, so that the virtual pan-tilt view change shows a smooth transition, and thus there will be no obvious jamming during the process of the virtual pan-tilt rotating the view, and the response time is faster.

[0051] In this embodiment, the response time of the virtual pan-tilt view change can be shortened to 50 ms.

[0052] It should be noted that in other embodiments, the smooth transition of the virtual pan-tilt view change can also be achieved through other algorithms such as Bezier curves, which is not limited here.

[0053] In summary, the control method of a virtual pan-tilt provided in this application can achieve the effect of pan-tilt movement without the need for a pan-tilt mechanical structure. Compared with the traditional mechanical structure, it can be smaller in size, lower in cost, faster in response, and noise-free. Specifically, in this application, the imaging module is rotated by 90 degrees for imaging, so that the height value of the imaging plane is greater than the width value, and thus a larger field of view range is retained in the vertical direction of the fisheye image. Then, the distortion correction is performed on the selected correction area in the fisheye image to obtain a corrected image, and then the corrected image is cropped at a target ratio, and then the cropped image is rotated by 90 degrees to restore to a normal viewing angle. In this way, without increasing the physical size of the lens, the virtual pan-tilt can obtain a larger field of view range and viewing angle adjustment space in the vertical direction. Therefore, the virtual pan-tilt has a larger rotation angle in the vertical direction and can better simulate the perspective translation / pitching movement. Further, the S-shaped speed curve algorithm is used to control the coordinate offset of the center point of the display window on the calibration plane, and the coordinate offset trajectory is optimized, so that the perspective change of the virtual pan-tilt presents a smooth transition, and thus there will be no obvious jitter during the rotation of the virtual pan-tilt perspective, and the response time is faster.

[0054] Based on the above control method of the virtual pan-tilt, and based on the same inventive concept, this application also proposes a control device for a virtual pan-tilt. Figure 12 The structural diagram of the control device for the virtual pan-tilt according to a specific embodiment of this application is shown in Figure 12 As shown, the device includes: An acquisition unit 100 configured to acquire the imaging plane of the imaging module at a first angle and the fisheye image obtained by the fisheye lens shooting, wherein the height value of the imaging plane is greater than the width value; A correction unit 200 configured to determine the correction area of the fisheye image in the imaging plane, and perform distortion correction on the correction area to obtain a corrected image; A cropping unit 300 configured to crop based on the corrected image at a first aspect ratio to obtain a first image to be processed, wherein the width value of the first image to be processed is greater than the height value; A rotation unit 400 configured to rotate the first image to be processed by 90 degrees to obtain a second image to be processed; A display unit 500 configured to crop based on the second image to be processed at a second aspect ratio to obtain a display window, and obtain the mapping relationship of the pixel points of the second image to be processed in the display window to obtain a first display image; A virtual pan-tilt control unit 600 configured to achieve the display effect of the virtual pan-tilt perspective change by adjusting the coordinate offset of the center point of the display window on the calibration plane.

[0055] In a specific embodiment, the display unit 500 is further configured to perform pixel compensation on the first display image in the height and / or width directions to obtain a second display image with a target resolution.

[0056] In a specific embodiment, the virtual pan-tilt control unit 600 is further configured to control the coordinate offset trajectory of the center point of the display window on the calibration plane through an S-shaped speed curve algorithm, so that the display of the virtual pan-tilt view change presents a smooth transition.

[0057] Based on the above control method of the virtual pan-tilt, and based on the same inventive concept, the present application also proposes an electronic device. Figure 13 The schematic diagram of an electronic device according to a specific embodiment of the present application is shown, as Figure 13 shown, the electronic device includes: one or more processors 701, a memory 702, a bus 703, and a communication interface 704. Among them, the one or more processors 701, the memory 702, and the communication interface 704 are connected through the bus 703. The memory 702 is used to store one or more programs. When the one or more programs are executed by the one or more processors 701, the electronic device implements the control method of the virtual pan-tilt provided in any of the above embodiments.

[0058] Based on the above control method of the virtual pan-tilt, and based on the same inventive concept, the present application also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the control method of the virtual pan-tilt provided in any of the above embodiments.

[0059] In the embodiments of the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the above-described device / system / method embodiments are merely illustrative. For example, the division of the units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in an electrical or other form.

[0060] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0061] In addition, in each embodiment of the present invention, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0062] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs, etc., which can store program codes.

[0063] Obviously, those skilled in the art can make various modifications and changes to the embodiments of the present invention without departing from the spirit and scope of the present invention. In this way, if these modifications and changes are within the scope of the claims of the present invention and their equivalent forms, the present invention also aims to cover these modifications and changes. The word "comprising" does not exclude the presence of other elements or steps not listed in the claims. The simple fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A method for controlling a virtual PTZ, characterized in that: include: Acquire an imaging plane of the imaging module at a first angle and a fisheye image thereof captured by a fisheye lens, wherein a height value of the imaging plane is greater than a width value; Determine a correction region of the fisheye image in the imaging plane, perform distortion correction on the correction region, and obtain a corrected image; Performing frame selection based on the corrected image with a first aspect ratio to obtain a first image to be processed, wherein a width value of the first image to be processed is greater than a height value; Rotate the first image to be processed by 90 degrees to obtain a second image to be processed; Based on the second image to be processed, a frame is selected with a second aspect ratio to obtain a display window, and a mapping relationship of pixels of the second image to be processed in the display window is acquired to obtain a first display image; By adjusting the coordinate offset of the center point of the display window on the calibration plane, the display effect of the virtual pan-tilt viewing angle change is achieved.

2. The method according to claim 1, characterized in that The method further comprises: Pixel compensation is performed on the first display image in the height and / or width direction to obtain a second display image with a target resolution.

3. The method according to claim 2, characterized in that The performing pixel compensation on the first display image in the height and / or width direction to obtain a second display image with a target resolution includes: An interpolation algorithm is used to perform interpolation compensation on the pixels of the first display image in the height and / or width direction to obtain a second display image with a target resolution.

4. The method according to claim 1, characterized in that: The method of adjusting the coordinate offset of the center point of the display window on the calibration plane to achieve a display effect of a virtual PTZ viewing angle change includes: The coordinate offset trajectory of the center point of the display window on the calibration plane is controlled by an S-shaped speed curve algorithm, so that the change of the virtual pan / tilt viewing angle is displayed with a smooth transition.

5. The method according to claim 4, characterized in that The coordinate offset trajectory of the center point of the display window on the calibration plane includes three stages: acceleration stage, uniform speed stage and deceleration stage. By presetting the acceleration threshold parameters, the acceleration, velocity and displacement of the three stages are calculated respectively, thereby obtaining the coordinate offset trajectory.

6. The method according to claim 5, characterized in that The acceleration equations of the three stages are as follows: Integrating the acceleration gives the velocity equation as follows: Integrating the velocity equation yields the displacement equation as follows: in, is the acceleration, For speed, is the displacement, is the acceleration threshold parameter, t is the current moment, 0≤t< T 1 is the acceleration stage, T 1≤t< T 2 is the uniform speed section, T 2≤t≤ T It is the deceleration stage.

7. The method according to claim 1, characterized in that The determining a correction area of ​​the fisheye image in the imaging plane comprises: Using the inscribed rectangular area of ​​the fisheye image as the correction area; The step of performing frame selection based on the rectified image with a first aspect ratio to obtain a first image to be processed includes: Based on the corrected image, a maximum frame selection is performed in a width direction with a first aspect ratio to obtain a first image to be processed.

8. A virtual PTZ control device, characterized in that: include: an acquisition unit configured to acquire an imaging plane of the imaging module at a first angle and a fisheye image thereof photographed by a fisheye lens, wherein a height value of the imaging plane is greater than a width value; A correction unit configured to determine a correction area of ​​the fisheye image in the imaging plane, perform distortion correction on the correction area, and obtain a corrected image; a frame selection unit configured to perform frame selection based on the rectified image with a first aspect ratio to obtain a first image to be processed, wherein a width value of the first image to be processed is greater than a height value; A rotation unit, configured to rotate the first image to be processed by 90 degrees to obtain a second image to be processed; A display unit is configured to select a frame based on the second image to be processed with a second aspect ratio to obtain a display window, obtain a mapping relationship of pixels of the second image to be processed in the display window, and obtain a first display image; The virtual pan-tilt control unit is configured to achieve a virtual pan-tilt viewing angle change display effect by adjusting the coordinate offset of the center point of the display window on the calibration plane.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images