Auxiliary lens-based gimbal camera system and control method

Abstract

The present application discloses an auxiliary lens-based gimbal camera system, comprising a gimbal, a camera provided on the gimbal, at least one auxiliary lens, and a main control assembly. The gimbal is used for adjusting the steering angle of the camera and / or the auxiliary lens. The main control assembly is connected to the gimbal, the camera, and the auxiliary lens, and is used for driving, on the basis of an external instruction, the gimbal to adjust the steering angle of the camera and / or the auxiliary lens, controlling the camera and the auxiliary lens to work so as to respectively capture a main picture and corresponding panoramic pictures, splicing and combining the panoramic pictures captured by the auxiliary lenses to obtain a spliced video stream, inputting, on the basis of the external instruction, the spliced video stream, the main picture, and the panoramic pictures into an algorithm model for processing, performing video mixing on the spliced video stream, the main picture, and the panoramic pictures to generate a corresponding video and outputting the corresponding video. Also disclosed is an auxiliary lens-based gimbal camera control method.

Description

Gimbal camera system based on auxiliary lens and control method

[0001] This application is based on and claims priority to Chinese Patent Application No. 202411057718.4, filed on August 2, 2024, the entire contents of which are incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of cameras, and more particularly to a gimbal camera system based on auxiliary lens and a control method. BACKGROUND

[0003] In a traditional gimbal camera, as the camera constantly adjusts with the change of the angle of the gimbal, and its field of view also changes due to zooming (such as optical zooming or digital zooming, etc.), the content and range of the picture are constantly changing. For an image acquisition system, the picture output by the camera is constantly changing according to the user's needs, such as outputting a zoomed or regionally cropped picture, or a picture of a different position. However, for a gimbal camera or a gimbal camera head equipped with multiple intelligent functions, the AI algorithms of the intelligent functions are also based on the image of the camera to achieve, such as common target tracking, switching of the shooting target, human-computer interaction, scene analysis, and picture content understanding, etc. These AI functions often require a large field of view range, which is often contradictory to the needs of the camera's picture output, making the AI algorithm analysis in the intelligent gimbal camera system mostly limited by the camera imaging needs. Moreover, in many scene image output scenarios, close-up and wide-angle picture information needs to be output simultaneously, and picture switching needs to be performed. Although a traditional PTZ camera can perform close-up on a target or scene through optical zooming or other zooming forms, it cannot output wide-angle picture information at the same time. When wide-angle information is output, there is no way to provide high-quality close-up pictures. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a gimbal camera system based on auxiliary lens and a control method to increase the perception range of the gimbal camera system so that its intelligent functions are not limited by the camera's own imaging needs, and at the same time, the output video content is more valuable.

[0005] To solve the above technical problems, according to an aspect of the present application, a gimbal camera system based on auxiliary lenses is provided, comprising a gimbal, a camera disposed on the gimbal, at least one auxiliary lens, and a master control component, wherein the gimbal is used to adjust the turning angle of the camera and / or auxiliary lens; the master control component is connected with the gimbal, camera and auxiliary lens, and is used to drive the gimbal to adjust the turning angle of the camera and / or auxiliary lens according to an external instruction, while controlling the camera and auxiliary lens to work so as to respectively capture a main picture and a corresponding panoramic picture, and to splice and merge the panoramic pictures captured by each auxiliary lens to obtain a spliced video stream, and according to the external instruction, the spliced video stream, the main picture captured by the camera and the panoramic pictures captured by each auxiliary lens are input into an algorithm model for processing, while the spliced video stream, the main picture and each panoramic picture are video mixed to generate a corresponding video and output.

[0006] To solve the above technical problems, according to another aspect of the present application, a gimbal camera control method based on auxiliary lenses is provided, comprising: acquiring the turning angle of a camera and auxiliary lenses disposed on a gimbal, wherein the number of auxiliary lenses is configured to be at least one; driving the gimbal to adjust the turning angle of the camera and / or auxiliary lenses according to an external instruction, while controlling the camera and auxiliary lenses to work so as to respectively capture a main picture and a corresponding panoramic picture; splicing and merging the panoramic pictures captured by each auxiliary lens to obtain a spliced video stream; according to the external instruction, inputting the spliced video stream, the main picture captured by the camera and the panoramic pictures captured by each auxiliary lens into an algorithm model for processing; according to the external instruction, video mixing the spliced video stream, the main picture and each panoramic picture to generate a corresponding video and output.

[0007] The beneficial technical effects of the present application are that, compared with the prior art, the gimbal camera system based on auxiliary lenses of the present application is provided with a camera and at least one auxiliary lens on the gimbal thereof, a main control assembly can drive the gimbal to adjust the turning angles of the camera and auxiliary lenses according to external instructions, control the camera and auxiliary lenses to work to respectively capture a main picture and corresponding panoramic pictures, splice and combine the panoramic pictures captured by each auxiliary lens to obtain a spliced video stream, and input the spliced video stream, the main picture captured by the camera and each panoramic picture into an algorithm model for processing according to the external instructions, and video stream the spliced video stream, the main picture and each panoramic picture to generate corresponding videos and output. It can be seen that, the present application can obtain a wider visual perception range through the auxiliary lenses, so as to balance the accuracy of intelligent function algorithm analysis (such as target tracking, switching of a shooting target and human-computer interaction) and the value of the output of the shooting picture in the gimbal camera system, that is, the panoramic pictures captured by the auxiliary lenses and the spliced video stream after splicing the panoramic pictures captured by the auxiliary lenses and the main picture captured by the camera are used as the input of the intelligent function module, so that the algorithm analysis of the intelligent function module is not affected by the change of the main lens (i.e. the camera) to cause the perception range to be reduced or unable to be perceived to affect the analysis accuracy, and is not limited by the imaging requirement of the camera itself, and the spliced video stream, the main picture and each panoramic picture are mixed and output after the splicing, which enriches the video content of the output, that is, additional scene perception content (the panoramic picture captured by the auxiliary lens) can be provided outside the close-up picture (the main picture captured by the camera) when the close-up picture is provided, and more valuable output content is provided. BRIEF DESCRIPTION OF DRAWINGS

[0008] FIG. 1 is a structural block diagram of the gimbal camera system based on auxiliary lenses of the present application.

[0009] FIG. 2 is a physical diagram of the first embodiment of the gimbal camera system based on auxiliary lenses of the present application.

[0010] FIG. 3 is a diagram of the connection between the auxiliary lens and the gimbal in the gimbal camera system based on auxiliary lenses shown in FIG. 2.

[0011] FIG. 4 is a diagram of the second embodiment of the gimbal camera system based on auxiliary lenses of the present application.

[0012] FIG. 5 is a diagram of the third embodiment of the gimbal camera system based on auxiliary lenses of the present application.

[0013] FIG. 6 is a diagram of the fourth embodiment of the gimbal camera system based on auxiliary lenses of the present application.

[0014] FIG. 7 is a diagram of the fifth embodiment of the gimbal camera system based on auxiliary lenses of the present application.

[0015] Fig. 8 is a schematic diagram of a sixth embodiment of the gimbal camera system based on an auxiliary lens according to the present application.

[0016] Fig. 9 is a schematic diagram of a seventh embodiment of the gimbal camera system based on an auxiliary lens according to the present application.

[0017] Fig. 10 is a flowchart of a specific embodiment of a control method of the gimbal camera based on an auxiliary lens according to the present application. DETAILED DESCRIPTION

[0018] In order to make the person skilled in the art more clearly understand the purpose, technical solution and advantages of the present application, the present application is further described below in conjunction with the drawings and examples.

[0019] Referring to FIG. 1, FIG. 1 is a structural block diagram of a gimbal camera system 100 based on auxiliary lenses according to the present application. In the embodiment shown in the drawing, the gimbal camera system 100 based on auxiliary lenses includes a gimbal 11, a camera 12 arranged on the gimbal 11, at least one auxiliary lens 14, and a master control assembly 13. In the present application, the camera 12 and the auxiliary lens 14 are arranged on the gimbal 11, and the gimbal 11 can be used to adjust the turning angle of the camera 12 and / or the auxiliary lens 14. In the present application, the gimbal 11 can be a single-axis gimbal, a dual-axis gimbal, or a three-axis gimbal. Specifically, referring to FIG. 2, the gimbal 11 includes a base 111, a gimbal control board 112, and a movement shaft 113 connected to the base 111. The gimbal control board 112 and the master control assembly 13 can be arranged in the base 111 or placed at other positions of the gimbal 11, and the auxiliary lens 14 can be arranged on the movement shaft 113 or the base 111. When the auxiliary lens 14 is arranged on the movement shaft 113, the camera 12 and the auxiliary lens 14 can be driven to rotate by the movement shaft 113 to adjust the turning angle. When the auxiliary lens 14 is arranged on the base 111, the auxiliary lens 14 does not rotate. The master control assembly 13 is connected to the gimbal 11, the camera 12, and the auxiliary lens 14, and is used to drive the gimbal 11 to adjust the turning angle of the camera 12 and / or the auxiliary lens 14 according to an external instruction, control the camera 12 and the auxiliary lens 14 to work to respectively capture a main picture and a corresponding panoramic picture, splice and merge the panoramic pictures captured by each auxiliary lens 14 to obtain a spliced video stream, and input the spliced video stream, the main picture captured by the camera 12, and the panoramic pictures captured by each auxiliary lens 14 into an algorithm model for processing according to the external instruction, and perform video mixing of the spliced video stream, the main picture, and each panoramic picture to generate a corresponding video and output the video. It can be seen that, by arranging the auxiliary lens 14 on the gimbal 11, the present application expands the perception range, optimizes the images input into the algorithm model of the intelligent function, so that the algorithm analysis of the intelligent function is not affected by the change of the main lens (i.e., the camera 12) to cause the perception range to be reduced or unable to be perceived, thereby affecting the analysis accuracy, and is not limited by the imaging requirements of the camera itself. In addition to the output of the main picture, the present application also increases the output of the wide-angle picture, and takes into account the accuracy of the algorithm analysis (such as target tracking, switching of a shooting target, and human-computer interaction) of the algorithm model of the intelligent function in the gimbal camera system 100 and the value of the output of the shooting picture.

[0020] As shown in FIG. 2 and FIG. 3, FIG. 2 and FIG. 3 show the physical schematic diagram of the first embodiment of the gimbal camera system 100 based on auxiliary lens of the present application. In this embodiment, the number of auxiliary lenses 14 is configured as one, which can be a wide-angle lens or a fisheye lens, and the gimbal 11 is a three-axis gimbal, the movement axis 113 of which includes a yaw axis (responsible for horizontal rotation), a pitch axis (responsible for vertical rotation) and a roll axis (responsible for the rotation direction of the picture clockwise / counterclockwise rotation direction), while in some other embodiments, the gimbal 11 can be a two-axis gimbal (the movement axis 113 includes the yaw axis and the pitch axis), or a single-axis gimbal (the movement axis 113 includes the yaw axis). Specifically, in the embodiment shown in the drawings, the auxiliary lens 14 is arranged on the yaw axis rotating mechanism 1131 of the gimbal 11, and the included angle a in the figure is the field of view angle of the auxiliary lens 14. Understandably, the yaw axis rotating mechanism 1131 represents the rotating part relative to the base 111, and the schematic diagram in the drawings uses a circle for illustrative representation, which is actually related to the actual appearance of product design, and is only used for reference here. And in the embodiment shown, the number of auxiliary lenses 14 is one, then in this embodiment, the main control component 13 on the one hand inputs the main picture taken by the camera 12 and the panoramic picture taken by the auxiliary lens 14 into the algorithm model for processing, and on the other hand directly mixes the main picture taken by the camera 12 and the panoramic picture taken by the auxiliary lens 14 to generate the corresponding video and output. Based on the above design, the auxiliary lens 14 is arranged on the yaw axis rotating mechanism 1131, and the perception picture range only changes in the horizontal direction, when the angle position of the camera 12 (i.e. the main lens) changes, the angle range of the auxiliary lens 14 in the vertical and plane rotation directions does not change, and a large enough field of view angle range different from the camera 12 can be obtained.

[0021] It can be understood that the main control component 13 can mix the main picture taken by the camera 12 and the panoramic picture taken by the auxiliary lens 14 according to the corresponding positional relationship between the auxiliary lens 14 and the camera 12 and the layout of the picture positions to generate a corresponding video and output; specifically, the pixels of the main picture and each panoramic picture at each moment are mixed and calculated into one pixel according to the corresponding positional relationship between the auxiliary lens 14 and the camera 12 and the layout of the picture positions, so as to perform operations such as switching, cropping, affine or perspective transformation of the picture, or superposition of multiple video sources, thereby generating a corresponding video. For example, the output video or image can simultaneously provide a close-up picture (the main picture taken by the camera 12) and a wide-angle picture (scene perception content corresponding to the close-up picture, i.e., the panoramic picture taken by the auxiliary lens 14). In this embodiment, the main control component 13 includes a main controller, and the main controller is electrically connected with the holder 11, the camera 12 and the auxiliary lens 14, that is, the camera 12 and the auxiliary lens 14 share the main controller for processing; in some other embodiments, the main control component 13 can include a main controller and a secondary controller, the main controller is electrically connected with the camera 12, and the secondary controller is electrically connected with the auxiliary lens 14, and the holder 11 can be connected with the main controller or the secondary controller. Specifically, the main controller can use a chip with image / video processing capability, including but not limited to x86, ARM, RISC V, GPU, NPU, ISP and other chips that can be used for processing images or videos. In this embodiment, the video stream of the auxiliary lens 14 can be processed using a separate secondary controller, the main controller can be used to drive the holder 11 to adjust the turning angle of the camera 12 and the auxiliary lens 14 according to external instructions, and simultaneously control the camera 12 to work to take a main picture, and the secondary controller can be used to control the auxiliary lens 14 to work to take a corresponding panoramic picture according to external instructions.

[0022] Further, in some embodiments, the auxiliary-lens-based holder camera system 100 can further include at least one pitch driving mechanism 15 connected with the main control component 13 and the auxiliary lens 14, for adjusting the pitch turning angle of the auxiliary lens 14 according to the instruction from the main control component 13. In this application, the pitch driving mechanism 15 can be a motor. Based on the above design, the pitch driving mechanism 15 can adjust and control the pitch angle of the auxiliary lens 14 to further increase the perception range of the pitch angle of the auxiliary lens 14 and the output picture range. In this embodiment, the number of pitch driving mechanisms 15 corresponds to one, and one pitch driving mechanism 15 controls one auxiliary lens 14. If there are multiple auxiliary lenses 14, one pitch driving mechanism 15 can be used to control multiple auxiliary lenses 14, or one pitch driving mechanism 15 can be used to control each auxiliary lens 14, which can be set according to actual needs.

[0023] Referring to FIG. 4, FIG. 4 is a schematic diagram of a second embodiment of the gimbal camera system 100 based on auxiliary lenses according to the present application. The difference between the present embodiment and the first embodiment described above is that the number of auxiliary lenses 14 is different. In the present embodiment, the number of auxiliary lenses 14 is configured to be two, and the two auxiliary lenses 14 are arranged around the yaw axis rotation mechanism 1131 of the gimbal 11, and together form a visual angle perception range greater than or equal to 180°. As shown in FIG. 4, the visual angle perception range formed by the two auxiliary lenses 14 is about 180° (the included angle a in the figure). That is, in the present embodiment, by arranging two auxiliary lenses 14 with a slightly smaller field of view, a visual angle perception range of about 180° in front is obtained by splicing. Then, the main control assembly 13 obtains a spliced video stream by splicing and merging the panoramic pictures taken by the two auxiliary lenses 14. Understandably, if the main control assembly 13 includes a sub-controller, the two auxiliary lenses 14 can share one sub-controller. At this time, the sub-controller can be used to control the auxiliary lenses 14 to work according to external instructions to take corresponding panoramic pictures, and can splice and merge the panoramic pictures taken by each auxiliary lens 14 to obtain a spliced video stream. Alternatively, one auxiliary lens 14 uses one sub-controller, and if multiple sub-controllers are used for processing, high-speed low-latency video data interfaces (such as HDMI, MIPI, BT series (such as 656 / 1120 / 2020 or similar interfaces) interfaces, etc.), high-speed stream data interfaces (such as Ethernet, USB, WIFI, etc.), metadata interfaces (such as serial data bus such as serial port, SPI, I2C, CAN, etc.), and general-purpose GPIO interfaces can be used for communication between the sub-controllers. Specifically, the two sub-controllers control the two auxiliary lenses 14 to work, and the main controller can splice the two panoramic pictures according to the positional relationship of the two auxiliary lenses 14 to obtain a spliced video stream. Then, on the one hand, the spliced video stream, the main picture taken by the camera 12, and the panoramic pictures taken by the two auxiliary lenses 14 are input into an algorithm model for processing. On the other hand, the spliced video stream, the main picture taken by the camera 12, and the panoramic pictures taken by the two auxiliary lenses 14 are directly video mixed to generate a corresponding video and output. Understandably, according to the actual computing power of the chips of the main controller and the sub-controller, in some other embodiments, the processing of splicing and merging each panoramic picture and the processing of video mixing the spliced video stream, the main picture, and each panoramic picture can also be performed in the sub-controller. Based on the above design, the present embodiment uses two auxiliary lenses 14 with a smaller field of view, so the distortion of the picture can be very small, the low distortion advantage of the small field of view lens can be fully utilized, the picture can be more natural and realistic, the performance advantage of the intelligent algorithm can be more fully utilized, and high-quality video output can be maintained.

[0024] Referring to FIG. 5, FIG. 5 is a schematic diagram of a third embodiment of the gimbal camera system 100 based on auxiliary lenses according to the present application. The difference between the present embodiment and the second embodiment described above is that the specific positions of the auxiliary lenses 14 are different. In the present embodiment, the number of auxiliary lenses 14 is also configured to be two, and the two auxiliary lenses 14 are arranged around the center of the yaw axis rotation mechanism 1131 of the gimbal 11 to obtain a 360° perceived visual angle. As shown in FIG. 5, the two auxiliary lenses 14 are symmetrically arranged on the yaw axis rotation mechanism 1131, the field angles partially overlap, and the field angles of the two auxiliary lenses 14 together constitute a 360° visual angle perception range (the included angle a in the figure). This increases the perceived picture of the gimbal camera system 100, optimizes the image input into the intelligent function algorithm model, and makes the algorithm analysis of the intelligent function module not affected by the change of the main lens (i.e., the camera 12) to cause the perception range to be reduced or unable to be perceived, thereby affecting the analysis accuracy. At the same time, the availability of the picture output is increased, and the user is provided with diversified visual angles to generate a picture that is more in line with the user's needs, so as to better meet the customer's demand for the output picture.

[0025] Referring to FIG. 6, FIG. 6 is a schematic diagram of a fourth embodiment of the gimbal camera system 100 based on auxiliary lenses according to the present application. The difference between the present embodiment and the third embodiment described above is that the number of auxiliary lenses 14 and the specific positions of the auxiliary lenses 14 are different. In the present embodiment, the number of auxiliary lenses 14 is configured to be four, and the four auxiliary lenses 14 are arranged around the center of the yaw axis rotation mechanism 1131 of the gimbal 11 and are uniformly spaced, and together constitute a 360° visual angle perception range (the included angle a shown in FIG. 6). In the present embodiment, the field angle of the auxiliary lens 14 is smaller, and in the horizontal direction, a 360° visual angle perception range is obtained while the low distortion advantage of the small field angle lens is fully utilized, so that the picture is more natural and realistic, which is beneficial to improve the analysis accuracy of the intelligent function algorithm model and the output value of the shooting picture.

[0026] Referring to FIG. 7, FIG. 7 is a schematic diagram of a fifth embodiment of the gimbal camera system 100 based on the auxiliary lens of the present application. The difference between the present embodiment and the first embodiment described above is that the auxiliary lens 14 is arranged at a different position. In the present embodiment, one auxiliary lens 14 is arranged on the base 111 of the gimbal 11, so that even when the angle position of the camera 12 (i.e. the main lens) changes, the auxiliary lens 14 does not change accordingly, and a sufficiently large field of view range different from the camera 12 can still be obtained. In the present embodiment, the arrangement of the auxiliary lens 14 is such that the sensing image of the auxiliary lens 14 does not rotate, so that no matter how the camera 12 (i.e. the main lens) moves, the image sensed by the auxiliary lens 14 remains unchanged, which is suitable for applications that require the wide-angle image or the panoramic image to remain unchanged. Understandably, in some other embodiments, the auxiliary lens 14 on the base 111 can also be arranged in multiple, for example, two or four, as in the second embodiment, the third embodiment and the fourth embodiment, which can also improve the analysis accuracy of the intelligent function algorithm model and the output value of the captured image.

[0027] Referring to FIG. 8, FIG. 8 is a schematic diagram of a sixth embodiment of the gimbal camera system 100 based on the auxiliary lens of the present application. The difference between the present embodiment and the first embodiment described above is that the auxiliary lens 14 is arranged at a different position. In the present embodiment, one auxiliary lens 14 is arranged on one side of the lens of the camera 12 and close to the lens of the camera 12, and the viewing angle direction of the auxiliary lens 14 is arranged to be the same as the viewing angle direction of the lens of the camera 12. In the present embodiment, as shown in FIG. 8, the field of view angle (the angle a in the figure) of the auxiliary lens 14 is larger than the field of view angle (the angle b in the figure) of the camera 12, the auxiliary lens 14 and the camera 12 are arranged together and move with the camera 12, and the viewing angle directions are the same. When the camera 12 moves under the drive of the movement axis 113 of the gimbal 11, the image captured by the auxiliary lens 14 also changes accordingly. Video mixing can be performed according to the relative geometric relationship between the main image captured by the camera 12 and the panoramic image captured by the auxiliary lens 14, which can expand the sensing range of the gimbal camera system 100, i.e. in addition to the output of the main image, the output of the wide-angle image is also increased, which takes into account the accuracy of the algorithm analysis (such as target tracking, switching of the captured target, and human-computer interaction) of the intelligent function algorithm model and the value of the output of the captured image in the gimbal camera system 100. Moreover, the auxiliary lens 14 and the camera 12 are placed together, which is convenient for overall appearance design and can ensure the consistency of the appearance design as much as possible.

[0028] Referring to FIG. 9, FIG. 9 is a schematic diagram of a seventh embodiment of the gimbal camera system 100 based on auxiliary lenses according to the present application. The difference between the present embodiment and the sixth embodiment described above is that the number of auxiliary lenses 14 is different. In the present embodiment, the number of auxiliary lenses 14 is configured to be two. One auxiliary lens 14 is located on one side of the lens of the camera 12 and is close to the lens of the camera 12. The viewing angle direction of the auxiliary lens 14 is the same as that of the lens of the camera 12. The other auxiliary lens 14 is located on the side of the camera 12 away from the lens. As shown in FIG. 9, the viewing angles of the two auxiliary lenses 14 together form a viewing angle perception range of about 360°, which can achieve full-range perception and further increase the perceived picture of the gimbal camera system 100.

[0029] As can be seen from the above, the gimbal camera system 100 based on auxiliary lenses according to the present application can obtain a relatively wide viewing angle perception range through the auxiliary lenses 14 arranged thereon. The images input into the intelligent function algorithm model are optimized, so that the algorithm analysis of the intelligent function module is not affected by the change of the main lens (i.e., the camera 12) to cause the perception range to be reduced or unable to be perceived, thereby affecting the analysis accuracy. The present application is not limited by the imaging requirements of the camera itself. At the same time, the output picture is enriched, i.e., in addition to the main picture output, wide-angle picture output is also increased. The accuracy of the algorithm analysis (such as target tracking, switching of the shooting target, and human-computer interaction) of the intelligent function algorithm model in the gimbal camera system 100 and the value of the shooting picture output are taken into account. The problem that the intelligent function AI algorithm analysis in the conventional camera is limited by the imaging function of the main lens of the camera and the output picture cannot well meet the customer's requirements is effectively solved.

[0030] Referring to FIG. 10, FIG. 10 is a flowchart of a gimbal camera control method based on auxiliary lenses according to the present application. The gimbal camera control method based on auxiliary lenses according to the present application can be applied to the gimbal camera system based on auxiliary lenses described in all the embodiments above. In the embodiment shown in the drawing, the gimbal camera control method based on auxiliary lenses comprises the following steps.

[0031] S101, acquiring the turning angle of the camera and the auxiliary lens arranged on the gimbal.

[0032] In the present application, the gimbal can be a single-axis gimbal, a dual-axis gimbal, or a three-axis gimbal. Specifically, the gimbal comprises a base, a gimbal control board, and a motion shaft connected to the base and electrically connected to the gimbal control board. The number of auxiliary lenses can be at least one. The auxiliary lenses can be arranged on the camera, on the motion shaft, or on the base of the gimbal, or on the motion shaft (such as a yaw shaft rotating mechanism) of the gimbal. When the auxiliary lenses are arranged on the motion shaft, the camera and the auxiliary lenses can be rotated by the motion shaft to adjust the turning angle. When the auxiliary lenses are arranged on the base, the auxiliary lenses do not rotate.

[0033] S102, according to the external instruction driving the holder to adjust the turning angle of the camera and / or auxiliary lens, and controlling the camera and auxiliary lens to work to respectively capture the main picture and the corresponding panoramic picture.

[0034] S103, splicing and merging the panoramic pictures captured by each auxiliary lens to obtain a spliced video stream.

[0035] S104, according to the external instruction, inputting the spliced video stream, the main picture captured by the camera and the panoramic pictures captured by each auxiliary lens into an algorithm model for processing.

[0036] S105, according to the external instruction, video mixing the spliced video stream, the main picture and each panoramic picture to generate a corresponding video and output.

[0037] In the embodiment, the external instruction can be target tracking and close-up and panoramic picture output during stage performance. The auxiliary lens provided in the application can improve the perception ability of the holder camera system to the surrounding environment, so that the intelligent function algorithm analysis is not limited by the imaging ability of the camera (i.e. the main lens), and at the same time, the corresponding scene picture can be supplemented on the basis of the close-up shooting of the camera, providing users with diversified visual angles to generate pictures more in line with user needs.

[0038] It should be noted that the skilled in the art can clearly understand that the specific implementation process of the above-mentioned holder camera control method based on auxiliary lens can refer to the corresponding description in the foregoing system embodiments, and for the convenience and brevity of description, it will not be repeated here.

[0039] The above only describes the preferred embodiments of the present application, and does not make any form of limitation on the present application. The skilled in the art can make various equivalent changes and improvements on the basis of the above-mentioned embodiments, and any equivalent changes or modifications made within the scope of claims shall fall within the protection scope of the present application.

Claims

1. A gimbal camera system based on an auxiliary lens, characterized in that, The gimbal camera system comprises a gimbal, a camera disposed on the gimbal, at least one auxiliary lens, and a main control assembly, wherein The gimbal is used to adjust the turning angle of the camera and / or auxiliary lens; The main control assembly is connected with the gimbal, camera and auxiliary lens, and is used to drive the gimbal to adjust the turning angle of the camera and / or auxiliary lens according to an external instruction, control the camera and auxiliary lens to work so as to respectively capture a main picture and a corresponding panoramic picture, splice and merge the panoramic pictures captured by each auxiliary lens to obtain a spliced video stream, and input the spliced video stream, the main picture captured by the camera and the panoramic pictures captured by each auxiliary lens into an algorithm model for processing according to the external instruction, and perform video mixing on the spliced video stream, the main picture and each panoramic picture to generate a corresponding video and output the video.

2. The auxiliary-lens-based gimbal camera system of claim 1, wherein, The auxiliary lens is a wide-angle lens or a fisheye lens, and the auxiliary lens is disposed on a yaw axis rotation mechanism of the gimbal.

3. The auxiliary-lens-based gimbal camera system of claim 2, wherein, The number of auxiliary lenses is configured to be one, and one auxiliary lens is disposed on the yaw axis rotation mechanism of the gimbal. Alternatively, the number of auxiliary lenses is configured to be multiple, and multiple auxiliary lenses are disposed around the yaw axis rotation mechanism of the gimbal.

4. The auxiliary-lens-based gimbal camera system of claim 1, wherein, The auxiliary lens is a wide-angle lens or a fisheye lens, and the auxiliary lens is disposed on a base of the gimbal.

5. The auxiliary-lens-based gimbal camera system of claim 4, wherein, The number of auxiliary lenses is configured to be one, and one auxiliary lens is disposed on the base of the gimbal. Alternatively, the number of auxiliary lenses is configured to be multiple, and multiple auxiliary lenses are disposed around the base.

6. The auxiliary-lens-based gimbal camera system of claim 2 or 4, wherein, The gimbal camera system further comprises at least one pitch driving mechanism connected with the main control assembly and the auxiliary lens, and used to adjust the pitch turning angle of the auxiliary lens according to an instruction from the main control assembly.

7. The auxiliary-lens-based gimbal camera system of claim 1, wherein, The auxiliary lens is a wide-angle lens or a fisheye lens, the number of auxiliary lenses is configured to be one, and the auxiliary lens is located on one side of a lens in the camera, and the viewing angle direction of the auxiliary lens is configured to be the same as the viewing angle direction of the lens in the camera.

8. The auxiliary-lens-based gimbal camera system of claim 1, wherein, The number of auxiliary lenses is two, one auxiliary lens is located on one side of the lens in the camera, and the viewing angle direction of the auxiliary lens is the same as the viewing angle direction of the lens in the camera, and the other auxiliary lens is located on the side of the camera away from the lens.

9. The auxiliary-lens-based gimbal camera system of claim 1, wherein, The main control assembly comprises a main controller, and the main controller is electrically connected with the gimbal, camera and auxiliary lens; or the main control assembly comprises a main controller and at least one auxiliary controller, the main controller is electrically connected with the camera and / or gimbal, and the auxiliary controller is electrically connected with the auxiliary lens and / or gimbal. 10.A gimbal camera control method based on an auxiliary lens, characterized in that, The gimbal camera system comprises: obtaining the turning angle of the camera and auxiliary lens disposed on the gimbal, wherein the number of auxiliary lenses is configured to be at least one; driving the gimbal to adjust the turning angle of the camera and / or auxiliary lens according to an external instruction, and controlling the camera and auxiliary lens to work so as to respectively capture a main picture and a corresponding panoramic picture; splicing and merging the panoramic pictures captured by each auxiliary lens to obtain a spliced video stream; According to the external instruction, the spliced video stream, the main picture taken by the camera and the panoramic pictures taken by the auxiliary cameras are input into an algorithm model for processing; According to the external instruction, the spliced video stream, the main picture and the panoramic pictures are video mixed to generate corresponding videos and output.

Images