Virtual optical zoom pan-tilt apparatus and method

Abstract

The invention discloses virtual optical zoom holder equipment and a virtual optical zoom holder method, and belongs to the field of power transmission line monitoring. Comprising a plurality of high-pixel fixed-focus cameras, the high-pixel fixed-focus cameras are arranged in the horizontal direction and the vertical direction, all the high-pixel fixed-focus cameras in the horizontal direction achieve a 360-degree visual field in the horizontal direction, and all the high-pixel fixed-focus cameras in the vertical direction achieve a 360-degree visual field in the vertical direction. According to the invention, a plurality of cameras are used for replacing a holder to rotate, and the high-pixel cameras are low in cost, so that the transmission mechanical cost of the holder is saved, and the equipment cost is greatly reduced; the problems that optical zoom starts self-inspection, the zoom response speed is low, focusing is not clear and the like are effectively solved, and the cost of an optical zoom machine core is saved. The problem that the visual angle is small under the condition of large zooming is also solved, the visual angle can be amplified at will under the condition that the resolution capability is guaranteed, and more details can be seen at the same angle; and the equipment cost and the user experience are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of power transmission line monitoring, and in particular to a virtual optical zoom pan / tilt device and method. Background Art

[0002] With the increasing construction of visualized intelligent lines for the State Grid's transmission lines, as the basic equipment for intelligent line construction, the demand for the functions and indicators of transmission visualization equipment is increasing. Real-time video, omnidirectional pan / tilt, speed of viewing real-time video, pan / tilt rotation speed, speed of optical zoom, and preset position accuracy all put forward very high requirements.

[0003] To solve the above problems, most cameras on the market use optical zoom cameras equipped with mechanical high-speed omnidirectional pan / tilts, which to a certain extent solve the problems of omnidirectional and zoom. However, due to the linkage of the mechanical structure of the multi-processing system, there are problems such as high power consumption, long processing timeliness, large deviation in preset positions, and technical problems such as video jitter caused by the accumulation of camera jitter in large zoom scenes. Summary of the invention

[0004] The present invention provides a virtual optical zoom pan-tilt device, which uses multiple cameras to replace pan-tilt rotation, reduces device cost and power consumption, and improves the accuracy and speed of zoom and pan-tilt rotation.

[0005] The present invention provides the following technical solutions:

[0006] A virtual optical zoom pan-tilt device includes a plurality of high-pixel fixed-focus cameras, wherein:

[0007] Multiple high-pixel fixed-focus cameras are arranged in the horizontal and vertical directions. All high-pixel fixed-focus cameras in the horizontal direction achieve a 360-degree field of view in the horizontal direction, and all high-pixel fixed-focus cameras in the vertical direction achieve a 360-degree field of view in the vertical direction.

[0008] Furthermore, the multiple high-pixel fixed-focus cameras include a front camera, a rear camera, a left camera, a right camera, an upper camera and a lower camera;

[0009] The front camera, left camera, rear camera and right camera are in the same horizontal plane and are arranged at 90 degrees with a set center point as the center of the circle. The fields of view of the front camera, rear camera, left camera and right camera are respectively facing the front, rear, left and right directions; the central axes of the upper camera and the lower camera pass through the center point, and the fields of view of the upper camera and the lower camera are respectively facing directly upward and directly downward.

[0010] Furthermore, the field of view angles of the front camera, rear camera, left camera, right camera, upper camera and lower camera are greater than or equal to 90 degrees and less than 180 degrees.

[0011] Furthermore, the front camera, rear camera, left camera, right camera, upper camera and lower camera are located in the same housing, and the front camera, rear camera, left camera, right camera, upper camera and lower camera are fixed on a housing bracket.

[0012] Furthermore, the front camera and the rear camera are connected to the first ISP module of the main control unit through a first selection switch, the left camera and the right camera are connected to the second ISP module of the main control unit through a second selection switch, and the upper camera and the lower camera are connected to the third ISP module of the main control unit through a third selection switch.

[0013] Furthermore, the resolution of the high-pixel fixed-focus camera is 50M pixel.

[0014] A method for a virtual optical zoom pan / tilt device, the method comprising:

[0015] S10: acquiring a full-field original image of a selected high-pixel fixed-focus camera;

[0016] S11: compressing the full-field original image according to a default compression ratio to obtain a default image;

[0017] S12: when it is necessary to simulate variable magnification of the selected high-pixel fixed-focus camera, an image to be magnified is intercepted on the full-view original image according to the magnification to be magnified;

[0018] The center of the image to be magnified coincides with the center of the full-view original image, the width and height of the image to be magnified are respectively 1 / N of the full-view original image, and N is the magnification to be magnified;

[0019] S13: calculating a compression ratio according to the magnification ratio to be enlarged and a default compression ratio, compressing the image to be enlarged according to the compression ratio to be enlarged, and obtaining an image after variable magnification;

[0020] The compression ratio to be compressed is 1 / N of the default compression ratio.

[0021] Furthermore, the method further comprises:

[0022] S20: when a panoramic image needs to be synthesized, full-field original images of all high-pixel fixed-focus cameras participating in the synthesis in the horizontal direction or the vertical direction are obtained;

[0023] S21: stitching all the acquired full-view original images to obtain a panoramic image;

[0024] When there is an overlapping area in the fields of view of two adjacent full-field original images, the images are spliced ​​according to the overlapping area.

[0025] Furthermore, the method further comprises:

[0026] S30: when it is necessary to simulate the pan / tilt rotation, obtaining a full-view original image or a stitched panoramic image of a selected high-pixel fixed-focus camera;

[0027] S31: setting an initial position on the acquired full-view original image or panoramic image, and outputting an image of a set size with the initial position as the center as the initial position image;

[0028] S32: Calculate the field of view movement amplitude according to the set pan / tilt rotation angle, and translate the initial position to the target position according to the field of view movement amplitude;

[0029] S33: With the target position as the center, output an image of a set size as the image after the pan / tilt head is rotated.

[0030] The present invention has the following beneficial effects:

[0031] The present invention uses multiple cameras to replace the pan-tilt rotation, and the high-pixel camera has a low cost, which saves the transmission mechanical cost of the pan-tilt, greatly reducing the equipment cost; it effectively solves the problems of optical zoom startup self-inspection, slow zoom response speed, and unclear focusing, and saves the cost of the optical zoom movement; it also solves the problem of small viewing angle in a large zoom situation. While ensuring the resolution capability, the viewing angle can be enlarged at will, and more details can be seen at the same angle; there are great improvements in equipment cost and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a structural schematic diagram of the virtual optical zoom pan-tilt device of the present invention;

[0033] Figure 2 It is a logical schematic diagram of the virtual optical zoom pan-tilt device of the present invention;

[0034] Figure 3 A schematic diagram of a virtual optical zoom pan / tilt device of the present invention performing virtual zoom;

[0035] Figure 4 A schematic diagram of a virtual optical zoom pan-tilt device of the present invention performing panoramic stitching;

[0036] Figure 5 It is a schematic diagram of the virtual optical zoom pan-tilt device of the present invention performing virtual pan-tilt rotation. DETAILED DESCRIPTION

[0037] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0038] The embodiment of the present invention provides a virtual optical zoom pan / tilt device, such as Figure 1-2 As shown, it includes a plurality of high-pixel fixed-focus cameras 100, wherein:

[0039] Multiple high-pixel fixed-focus cameras 100 are arranged in the horizontal and vertical directions. All high-pixel fixed-focus cameras in the horizontal direction achieve a 360-degree field of view in the horizontal direction, and all high-pixel fixed-focus cameras in the vertical direction achieve a 360-degree field of view in the vertical direction.

[0040] In the present invention, a 360-degree field of view in the horizontal direction refers to a 360-degree field of view that surrounds the horizontal circle in the front, left, back, and right directions. A 360-degree field of view in the vertical direction refers to a 360-degree field of view that surrounds the vertical circle in the vertical direction. For example, a 360-degree field of view consisting of a circle in the front, bottom, back, and top directions (of course, fields of view that surround a circle in other directions in the vertical direction are also the 360-degree field of view described in the present invention, such as the left, bottom, right, and top directions). In the vertical direction, in actual situations, generally only one side of the field of view is concerned at the same time. For example, for a 360-degree field of view in the front, bottom, back, and top directions, generally only the 180-degree field of view in the front (i.e., a 90-degree field of view in each pitch and elevation of the front side) or the 180-degree field of view in the back (i.e., a 90-degree field of view in each pitch and elevation of the back side) is concerned at the same time. Nevertheless, a 360-degree vertical field of view is necessary for the present invention, because the high-pixel fixed-focus camera is fixed, and a 180-degree front field of view and a 180-degree rear field of view are both required.

[0041] The present invention realizes a 360-degree horizontal and 360-degree vertical field of view through the setting of multiple high-pixel fixed-focus cameras. Using this as a virtual optical zoom pan-tilt device, it can realize simulated optical zoom, simulated pan-tilt rotation and panoramic photography in any one or more directions (see below for specific methods).

[0042] In one example, the plurality of high-pixel fixed-focus cameras 100 include a front camera 1 , a rear camera 2 , a left camera 3 , a right camera 4 , an upper camera 5 , and a lower camera 6 .

[0043] The front camera 1, the left camera 3, the rear camera 2 and the right camera 4 are in the same horizontal plane to ensure that the horizontal angle error does not exceed 1 degree. They are arranged at 90 degrees with the set center point as the center of the circle. The fields of view of the front camera 1, the left camera 3, the rear camera 2 and the right camera 4 are respectively facing the front, left, rear and right directions to achieve a 360-degree field of view along a horizontal circle.

[0044] The central axes of the upper camera 5 and the lower camera 6 pass through the center point, and the fields of view of the upper camera 5 and the lower camera 6 are oriented toward directly upward and directly downward, respectively. The front camera 1, the lower camera 6, the rear camera 2, and the upper camera 5 realize a 360-degree field of view along a vertical circle in the front and rear directions, that is, a 90-degree field of view in the front side and a 90-degree field of view in the rear side. The left camera 3, the lower camera 6, the right camera 4, and the upper camera 5 realize a 360-degree field of view along a vertical circle in the left and right directions, that is, a 90-degree field of view in the left side and a 90-degree field of view in the right side.

[0045] In the present invention, the front camera 1 and the rear camera 2 will not appear in the same field of view at the same time, the left camera 3 and the right camera 4 will not appear in the same field of view at the same time, and the upper camera 5 and the lower camera 6 will not appear in the same field of view at the same time. That is, the field of view angles of the front camera 1, the rear camera 2, the left camera 3, the right camera 4, the upper camera 5 and the lower camera 6 are greater than or equal to 90 degrees and less than 180 degrees. Exemplarily, the horizontal field of view of each camera is 110 degrees, and the vertical field of view is 90 degrees.

[0046] Each camera needs to be connected to the main control unit 7. In order to reduce costs, a main control unit 7 with three ISP modules can be selected. The six cameras are combined in pairs and connected to the three independent ISP modules of the main control unit 7 after switching through the selection switch. The independent processing of three video streams can be realized at the same time. After the processed three video streams are spliced ​​into the processing module, three fused panoramic video streams are output. The switching direction of the three-way selection switch is determined by the positioning position of the current video stream and the rotation direction of the pan / tilt.

[0047] Adjacent cameras have a splicing function and may be turned on at the same time, so adjacent cameras cannot be switched with switches. It is imperative to ensure that adjacent cameras independently occupy separate ISP resources at the same time. Specifically, the front camera 1 and the rear camera 2 are connected to the first ISP module 9 of the main control unit 7 through the first selection switch 8, the left camera 3 and the right camera 4 are connected to the second ISP module 11 of the main control unit 7 through the second selection switch 10, and the upper camera 5 and the lower camera 6 are connected to the third ISP module 13 of the main control unit 7 through the third selection switch 12. The first selection switch 8, the second selection switch 10 and the third selection switch 12 can be analog switches.

[0048] The aforementioned front camera 1, rear camera 2, left camera 3, right camera 4, upper camera 5 and lower camera 6 are located in the same housing 14, and the front camera 1, rear camera 2, left camera 3, right camera 4, upper camera 5 and lower camera 6 are fixed on the housing bracket 15.

[0049] In the present invention, the resolution of the high-pixel fixed-focus camera 100 is 50M pixel.

[0050] For the aforementioned virtual optical zoom pan / tilt device, the method of simulating zooming, pan / tilt rotation and foreground camera is as follows:

[0051] 1. Use the video resolution compression conversion technology of a high-pixel fixed-focus camera to achieve simulated optical zoom. The specific methods include:

[0052] S10: Acquire a full-field original image of the selected high-pixel fixed-focus camera.

[0053] S11: compressing the full-field original image according to a default compression ratio to obtain a default image, that is, an unmagnified image.

[0054] S12: When it is necessary to simulate zooming in on the selected high-pixel fixed-focus camera, an image to be magnified is captured on the full-view original image according to the magnification to be magnified.

[0055] The center of the image to be magnified coincides with the center of the full-view original image, the width and height of the image to be magnified are 1 / N of the full-view original image respectively, and N is the magnification to be magnified.

[0056] S13: Calculate the compression ratio according to the magnification ratio and the default compression ratio, compress the image to be enlarged according to the compression ratio, and obtain the image after variable magnification;

[0057] The compression ratio to be compressed is 1 / N of the default compression ratio.

[0058] With the continuous improvement of ISP and CPU capabilities, image processing chips have become increasingly capable of processing high-definition camera data. The central processing chip used in the present invention directly increases the image camera resolution to 50M pixel, while the high-definition video resolution currently defined by the security industry is generally 720p (1M pixel). Based on this theory and using software processing methods, the theoretical zoom value can be obtained while ensuring that the resolution is not affected and a high resolution can still be obtained: 50M / 1M=50 times; even if the security industry demands 1080p (2 million pixels), the theoretical zoom value is still 25 times optical zoom.

[0059] The present invention adopts an algorithm for compressing high-pixel video into low-pixel video, which in principle ensures that the resolution of the low-pixel video is not affected and continuously maintains the original resolution set by the user.

[0060] like Figure 3As shown in the figure, the original pixel of the full-view original image is 50M pixel. In the original state, the processor compresses the resolution of the full-view original image video stream by 50 times (50M / 1M) and transmits it to the ISP for processing. After the ISP processing, the user sees a full-view HD video stream with a resolution of 720p. When the user needs to enlarge the video stream by 4 times to see more details, the processor intercepts the original video to the content of the rectangular frame (1 / 4 of the full-view original image) and compresses the resolution by 12.5 times (50M / 4 / 1M) and transmits it to the ISP for processing. After the ISP processing, the user sees a HD video stream with a resolution of 720p within the rectangular frame. The clarity is not reduced, but the viewing angle is smaller, so more details can be seen, thereby achieving the effect of simulated optical zoom.

[0061] The current PTZ equipment basically meets the requirements of omnidirectional and optical zoom. Its hardware device adopts the solution of main control board + optical zoom camera + PTZ control mainboard + omnidirectional PTZ. The system adopts the solution of main control and communication system + PTZ control system + optical zoom system. The optical zoom is the overall zoom of mechanical and optical structure, and the zoom time is about 10S. The optical zoom is a mechanical action, which has the problem of extra power consumption and video jitter. The zoom situation will amplify the jitter situation in the 1x situation by a corresponding multiple.

[0062] The virtual optical zoom effect of the present invention is realized by using a CPU for processing, and the video resolution compression conversion technology is used to simulate the magnification scheme to replace the mechanical zoom system. It does not require the linkage of the mechanical structure, has a fast zoom speed, high zoom accuracy, and saves the power consumption caused by the mechanical structure action. It has great advantages in cost, power consumption, and response time. In terms of video anti-shake, since the video is panoramic output and chip processing, there will be no problem of video jitter amplification, and the edge data of the viewing angle will not be lost during the video anti-shake processing.

[0063] 2. Use image stitching technology to achieve panoramic function. The specific methods include:

[0064] S20: When a panoramic image needs to be synthesized, full-field original images of all high-pixel fixed-focus cameras participating in the synthesis in the horizontal direction or the vertical direction are obtained.

[0065] For example, the full-view original images of the front camera, rear camera, left camera, and right camera in the horizontal direction are obtained. For another example, the full-view original images of the front camera, upper camera, and lower camera in the vertical direction are obtained.

[0066] S21: stitching all the acquired full-view original images to obtain a panoramic image.

[0067] When stitching, when there is an overlap in the field of view of two adjacent full-view original images, the images are stitched according to the overlap, such as Figure 4 shown.

[0068] For example, the horizontal angle of a single camera is 110 degrees, which cannot cover the 360-degree angle of the whole scene. Therefore, one camera is added to the back, left and right sides of the front camera, and a total of four cameras are used to achieve 360-degree scene stitching. Since the front and rear camera angles do not overlap, and the left and right camera angles do not overlap, each video angle processes no more than two video streams in the horizontal direction, and no more than three video streams in the vertical direction. The current ISP chip can easily process multi-image stitching, thereby achieving a horizontal 360-degree viewing angle. The vertical angle camera uses the same principle to achieve a panoramic gimbal viewing angle of -90 degrees to +90 degrees.

[0069] The present invention uses multiple groups of high-definition cameras to realize the full-image stitching function to simulate the horizontal 360-degree and vertical 180-degree panoramic coverage of the pan-tilt product, thereby realizing an omnidirectional pan-tilt product in the power transmission operation and maintenance scenario.

[0070] 3. Use the regional positioning technology in the video stream after panoramic stitching of multiple cameras or the video stream of a single camera to simulate the pan-tilt rotation. The specific methods include:

[0071] S30: when it is necessary to simulate the pan / tilt rotation, a full-view original image or a stitched panoramic image of a selected high-pixel fixed-focus camera is obtained.

[0072] S31: setting an initial position on the acquired full-view original image or panoramic image, and outputting an image of a set size with the initial position as the center as the initial position image.

[0073] S32: Calculate the field of view movement range according to the set pan / tilt rotation angle, and translate the initial position to the target position according to the field of view movement range.

[0074] S33: With the target position as the center, output an image of a set size as the image after the pan / tilt head is rotated.

[0075] like Figure 5 As shown in the figure, the coordinates of the center point of the original image are set to (0, 0), the horizontal angle of the entire image is 110 degrees, the vertical resolution is 90 degrees, the maximum horizontal angle of a single image can be rotated from -55 degrees to +55 degrees, and the vertical rotation angle is -45 degrees to +45 degrees. Assuming that the simulated gimbal rotates from (0, 0) to (x, y), the processor only needs to translate the center point of the image to the (x, y) coordinates.

[0076] The existing technology uses a mechanical structure for bidirectional pan-tilt rotation. There is a time delay for 360-degree horizontal rotation, and the mechanical structure has position errors. The preset position accuracy is generally 0.1 degrees. In a large zoom optical system, the preset position accuracy is poor and the desired scene cannot be seen. The pan-tilt rotation is a mechanical action, which consumes extra power, up to 10W or more.

[0077] The present invention uses the regional positioning technology in the video stream to replace the traditional mechanical transmission pan-tilt solution, which has great advantages in cost, power consumption, response time, and positioning accuracy, and the preset position accuracy is theoretically error-free.

[0078] To sum up, the present invention uses multiple cameras to replace the pan-tilt rotation, and the high-pixel camera has a low cost, which saves the transmission mechanical cost of the pan-tilt, greatly reducing the equipment cost; it effectively solves the problems of optical magnification startup self-inspection, slow magnification response speed, and unclear focusing, and saves the cost of the optical magnification movement; it also solves the problem of small viewing angle in the case of large magnification. While ensuring the resolution capability, the viewing angle can be enlarged at will, and more details can be seen at the same angle; there are great improvements in equipment cost and user experience.

[0079] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A virtual optical zoom pan / tilt device, characterized in that: Includes multiple high-pixel fixed-focus cameras, including: Multiple high-pixel fixed-focus cameras are arranged in the horizontal and vertical directions. All high-pixel fixed-focus cameras in the horizontal direction achieve a 360-degree field of view in the horizontal direction, and all high-pixel fixed-focus cameras in the vertical direction achieve a 360-degree field of view in the vertical direction.

2. The virtual optical zoom pan / tilt device according to claim 1, characterized in that: The multiple high-pixel fixed-focus cameras include a front camera, a rear camera, a left camera, a right camera, an upper camera and a lower camera; The front camera, left camera, rear camera and right camera are in the same horizontal plane and are arranged at 90 degrees with a set center point as the center of the circle. The fields of view of the front camera, rear camera, left camera and right camera are respectively facing the front, rear, left and right directions; the central axes of the upper camera and the lower camera pass through the center point, and the fields of view of the upper camera and the lower camera are respectively facing directly upward and directly downward.

3. The virtual optical zoom pan / tilt device according to claim 2, characterized in that: The field of view angles of the front camera, rear camera, left camera, right camera, upper camera and lower camera are greater than or equal to 90 degrees and less than 180 degrees.

4. The virtual optical zoom pan / tilt device according to claim 3, characterized in that: The front camera, rear camera, left camera, right camera, upper camera and lower camera are located in the same housing, and the front camera, rear camera, left camera, right camera, upper camera and lower camera are fixed on a housing bracket.

5. The virtual optical zoom pan / tilt device according to claim 3, characterized in that: The front camera and the rear camera are connected to the first ISP module of the main control unit through a first selection switch, the left camera and the right camera are connected to the second ISP module of the main control unit through a second selection switch, and the upper camera and the lower camera are connected to the third ISP module of the main control unit through a third selection switch.

6. The virtual optical zoom pan / tilt device according to any one of claims 1 to 5, characterized in that: The resolution of the high-pixel fixed-focus camera is 50M pixel.

7. A method for a virtual optical zoom pan / tilt device according to any one of claims 1 to 6, characterized in that: The method comprises: S10: acquiring a full-field original image of a selected high-pixel fixed-focus camera; S11: compressing the full-field original image according to a default compression ratio to obtain a default image; S12: when it is necessary to simulate variable magnification of the selected high-pixel fixed-focus camera, an image to be magnified is intercepted on the full-view original image according to the magnification to be magnified; The center of the image to be magnified coincides with the center of the full-view original image, the width and height of the image to be magnified are respectively 1 / N of the full-view original image, and N is the magnification to be magnified; S13: calculating a compression ratio according to the magnification ratio to be enlarged and a default compression ratio, compressing the image to be enlarged according to the compression ratio to be enlarged, and obtaining an image after variable magnification; The compression ratio to be compressed is 1 / N of the default compression ratio.

8. The method according to claim 7, characterized in that The method further comprises: S20: when a panoramic image needs to be synthesized, full-field original images of all high-pixel fixed-focus cameras participating in the synthesis in the horizontal direction or the vertical direction are obtained; S21: stitching all the acquired full-view original images to obtain a panoramic image; When there is an overlapping area in the fields of view of two adjacent full-field original images, the images are spliced ​​according to the overlapping area.

9. The method according to claim 8, characterized in that The method further comprises: S30: when it is necessary to simulate the pan / tilt rotation, obtaining a full-view original image or a stitched panoramic image of a selected high-pixel fixed-focus camera; S31: setting an initial position on the acquired full-view original image or panoramic image, and outputting an image of a set size with the initial position as the center as the initial position image; S32: Calculate the field of view movement amplitude according to the set pan / tilt rotation angle, and translate the initial position to the target position according to the field of view movement amplitude; S33: With the target position as the center, output an image of a set size as the image after the pan / tilt head is rotated.

Images