A handheld anti-shake gimbal camera and an operating system thereof

Abstract

The application discloses a handheld anti-shake gimbal camera and an operating system thereof, and belongs to the field of gimbal cameras, which comprises a handheld body, a three-axis gimbal mechanism, a camera and a main control hardware board, the main control hardware board adopts a dual-chip architecture of an Android application processor and an independent gimbal MCU, an Android operating system comprises audio and video virtualization routing middleware arranged above an Android hardware abstraction layer, a plurality of virtual device nodes are created for different applications to be bound, a stream data distribution engine distributes audio and video data according to independent parameters of each node, an audio routing virtualization module and an intelligent audio mixing scheduling module realize independent routing and audio mixing output of multiple audio, a resource arbitration management module reduces priority when concurrent overrunning, a system bypass control module is used for bypassing the middleware when a single application is used. The application realizes concurrent calling of the camera, the microphone and the loudspeaker by multiple applications at the same time, and improves the multi-task creation capability and the hardware resource utilization rate of the handheld gimbal camera.

Description

Technical Field

[0001] This invention relates to the field of gimbal cameras, and more particularly to a handheld image-stabilized gimbal camera and its operating system. Background Technology

[0002] With the explosive growth in demand for short videos, outdoor travel photography, and solo vlog creation, integrated three-axis gimbal cameras have become mainstream portable shooting devices. Existing products integrate three-axis gimbal motors, touchscreens, joysticks, dual microphones, Type-C interfaces, and expansion threads. However, they all use manufacturer-locked embedded systems with only the original shooting software built in, making it impossible to install third-party applications. The system does not grant file management permissions, resulting in the following drawbacks: the complete creation process must be completed with a mobile phone, creating a fragmented workflow; material transfer only supports proprietary protocols, lacking bidirectional batch management and automatic cloud backup; it cannot perform live streaming on a standalone device; and the 4GB of memory, high-performance chip, and full-featured Type-C interface are only used for gimbal control and framing, resulting in a serious waste of hardware resources.

[0003] A few existing solutions propose running the Android operating system on handheld gimbal cameras to support the installation of third-party applications. However, the camera service and audio strategy of the Android operating system adopt a hardware resource exclusivity mechanism, allowing only a single foreground application to exclusively access the camera module and microphone at any given time. In multi-task creation scenarios that require simultaneous operation of preview, live streaming, recording, AI analysis, etc., users cannot record 4K backup footage while live streaming, nor can they play reference audio in the background and add synchronized dubbing during editing. The lack of resource scheduling and arbitration mechanisms when multiple applications request hardware at the same time can easily lead to hardware overload or system lag. Users also cannot intuitively perceive the usage status of each application on the camera, microphone, and speaker, and the permission control is opaque. These defects severely restrict the multi-task creation potential of the Android operating system on handheld gimbal cameras. Therefore, there is an urgent need to design a handheld image-stabilized gimbal camera and its operating system. Summary of the Invention

[0004] The purpose of this invention is to address the deficiencies in the existing technology by proposing a handheld gimbal camera with image stabilization and its operating system.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A handheld gimbal camera with image stabilization and its operating system, comprising: The handheld camera body, the three-axis gimbal mechanism located on the top of the handheld camera body, the camera mounted on the three-axis gimbal mechanism, and the main control hardware board located inside the handheld camera body; The three-axis gimbal mechanism includes a yaw axis motor, a pitch axis motor, and a roll axis motor. The main control hardware board includes an Android application processor and an independent gimbal MCU. The Android application processor and the independent gimbal MCU are connected via serial communication, and the Android application processor runs on the Android operating system. The front of the handheld device features a 1.95-inch touchscreen and a five-way joystick, arranged from top to bottom. The Android operating system connects to camera 1 through a standard driver interface, directly outputting raw images and audio to third-party applications, so that users can directly install and run editing, live streaming, or media management software.

[0006] As a further embodiment of the present invention, the five-dimensional joystick is used to control the movement of the gimbal and trigger the shooting operation, and the front of the handheld device is provided with a power button and a status indicator light, which is used to distinguish between standby, photo taking and video recording states.

[0007] As a further embodiment of the present invention, the handheld device is also provided with dual microphones, including microphone I and microphone II, and a speaker is provided at the bottom of the handheld device. Microphone I and microphone II are integrated with the speaker to form a noise reduction algorithm for sound pickup and sound quality improvement.

[0008] As a further embodiment of the present invention, the bottom of the handheld device integrates a full-function Type-C interface and a 1 / 4-20" threaded hole. The bottom side of the handheld device also has a quick-release slot and a lanyard hole. The Type-C interface enables OTG function for direct connection to external storage devices, Android phones or computers, to realize bidirectional batch transmission of materials and file management.

[0009] As a further embodiment of the present invention, the camera lens supports quick assembly of magnetic filters, and the camera is a 48-megapixel camera with a built-in 1 / 2.0-inch sensor and an F1.8 aperture lens, supporting 4K high-definition video recording, slow motion and time-lapse photography. The handheld body has a power supply module inside, and the power supply module has a built-in 3500mAh lithium battery.

[0010] An operating system for a handheld gimbal camera with image stabilization includes an audio / video virtualization routing middleware set above the Android hardware abstraction layer, the audio / video virtualization routing middleware comprising: The physical device interface layer is used to acquire raw YUV video streams and PCM audio streams; The data acquisition buffer module is used to align the original YUV video stream and PCM audio stream according to time slice buffering to generate a multimedia data frame queue; The virtual device management module is used to create multiple virtual camera device nodes and virtual audio device nodes, which can be bound to different applications. The streaming data distribution engine is used to copy video frames in the data frame queue into multiple copies, and then encode, trim, or scale them according to the independent parameters of each virtual device node before distributing them. The encoding parameter adaptation module is used to independently maintain imaging parameters for each virtual Camera device node and notify the streaming data distribution engine when the parameters change; The audio routing virtualization module is used to create multiple virtual audio output channels, each of which can be independently configured with volume, channel mapping and sound effect parameters; The intelligent mixing and scheduling module is used to mix multiple audio streams according to weights and output them, and to copy and distribute the audio streams captured by dual microphones to multiple applications; The resource arbitration management module is used to monitor hardware load and degrade streaming specifications according to priority strategies when concurrency limits are exceeded. The priority strategy configuration module provides a user configuration interface to set the priority order and degradation strategy of each application. The system bypass control module is used to bypass the virtual device management module and the streaming data distribution engine when a single application accesses the system, allowing the application to communicate directly with the hardware abstraction layer. The Media Center module in the notification bar is used to generate a control panel in the notification bar, displaying a list of applications consuming resources and providing mute, pause, and toggle controls; The hardware status monitoring module is used to periodically collect status data on gimbal motor load, battery level, body temperature, and remaining storage capacity.

[0011] As a further aspect of the present invention, the virtual camera device node created by the virtual device management module includes a foreground preview node, a background recording node, and an AI analysis node, which are respectively bound to different applications; the streaming data distribution engine outputs a low-latency video stream to the foreground preview node, an encoded and compressed storage-level video stream to the background recording node, and a scaled inference-level video stream to the AI ​​analysis node.

[0012] As a further aspect of the present invention, the intelligent mixing scheduling module is equipped with audio scene priority rules. When a live streaming application is in the foreground, the notification sounds of other applications are automatically attenuated to a preset percentage, and the microphone main channel gain and background sound gain are adjusted.

[0013] As a further aspect of the present invention, the data collection dimensions of the hardware status monitoring module include the drive current, PWM duty cycle, winding temperature of each axis motor in the three-axis gimbal mechanism, battery voltage, charging and discharging current, remaining power, cycle count, multi-point temperature of the device body, and remaining storage capacity and write speed.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Users control the gimbal movement with a five-way joystick and trigger the camera to complete the shooting. The shot footage is directly stored in the device's storage. Through the application installation function opened by the Android operating system, users can directly open the editing APP to complete post-production, open the live streaming APP to push and publish, or upload through the Internet via social APP without the need for an external mobile phone. This enables the entire creative process of shooting, post-production, live streaming and publishing to be completed independently with a single device, greatly reducing the burden of carrying it when going out, and adapting to the creative needs of multiple scenarios such as Vlog, store visits and outdoor live streaming.

[0015] 2. During shooting or live streaming, the Android operating system connects to the camera through a standard driver interface to output the original image and audio to third-party applications without loss. At the same time, when the Android operating system runs the gimbal control program in the background, it sends the gimbal control commands to the independent gimbal MCU through the serial port. The independent gimbal MCU calculates in real time and drives the yaw axis motor, pitch axis motor and roll axis motor to perform image stabilization. The gimbal control and the Android application do not interfere with each other. Even when the live streaming software is running under high load, the image stabilization remains stable and without delay, ensuring the smoothness and reliability of the camera movement.

[0016] 3. When users conduct outdoor live streaming in multi-application concurrent scenarios, the audio and video virtualization routing middleware binds the live streaming APP and the background recording APP to different virtual camera device nodes through the virtual device management module. The streaming data distribution engine copies the original video stream captured by the camera into multiple copies and processes them according to the independent parameters of each node before distributing them. The intelligent mixing and scheduling module copies and distributes the microphone audio stream to each application and mixes the multiple audio streams according to their weights before outputting them. This allows one application to live stream while another application records a 4K backup of the original footage in the background. Moreover, each application can independently adjust the volume and imaging parameters, which greatly improves the efficiency of multi-task creation and the utilization rate of hardware resources.

[0017] 4. When multiple concurrent requests from applications cause the hardware load to exceed the limit, the resource arbitration management module automatically performs degradation processing on low-priority applications according to the priority strategy, such as reducing the frame rate, reducing the resolution, or pausing the video stream, to ensure that the operation of high-priority applications is smooth and unaffected, thereby ensuring the stability and reliability of core creation tasks.

[0018] 5. After shooting, users can enable the OTG function through the Type-C interface to directly connect to external storage devices, Android phones or computers. They can batch copy and move 4K high-definition materials through the system's native file manager. They can also connect to USB flash drives or external hard drives to back up large-capacity videos to external storage. No special software is required to complete bidirectional material transfer. If the transfer is interrupted, it can be resumed. This solves the problems of slow export and difficult backup of existing devices. The operation logic is completely consistent with that of general Android terminals.

[0019] 6. When shooting in high-temperature outdoor environments for extended periods, the priority strategy configuration module works in conjunction with the resource arbitration management module to dynamically adjust the running frequency of third-party background applications to reduce overall device heat generation and prevent overheating shutdowns that interrupt recording. At the same time, the Android operating system removes redundant background services such as calls, SMS, and cellular networks to reduce system memory usage. It also performs UI adaptation for small screens and automatically hides the top status bar for full-screen framing when entering shooting mode, thereby effectively extending the device's battery life and balancing the multi-application running capabilities of Android with the heat dissipation requirements of handheld devices. Attached Figure Description The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Figure 1 This is a schematic diagram of the overall structure of a handheld gimbal camera with image stabilization proposed in this invention. Figure 2 This is a rear view schematic diagram of a handheld gimbal camera with image stabilization proposed in this invention. Figure 3 This is a bottom schematic diagram of a handheld gimbal camera with image stabilization proposed in this invention. Figure 4 This is a system block diagram of the operating system for a handheld gimbal camera with image stabilization proposed in this invention. Legend: 1. Camera; 2. Yaw axis motor; 3. Touch screen; 4. Status indicator light; 5. Five-way joystick; 6. Speaker; 7. Quick release slot; 8. Lanyard hole; 9. Power button; 10. Microphone I; 11. Pitch axis motor; 12. Roll axis motor; 13. Microphone II; 14. Type-C interface; 15. 1 / 4-20" threaded hole. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0021] Example 1: Reference Figure 4 The Android operating system, from bottom to top, comprises the physical hardware layer, hardware abstraction layer, audio / video virtualization routing middleware, Android system service layer, and application layer; the audio / video virtualization routing middleware includes: At the physical device interface layer, the raw YUV video stream from the camera is acquired via the V4L2 driver, and the dual-microphone PCM audio stream is acquired via the ALSA driver. The data acquisition buffer module aligns the video and audio streams by timestamp and packages them into a multimedia data frame queue. The virtual device management module creates multiple virtual Camera and virtual Audio device nodes above the hardware abstraction layer for different applications to bind to. The streaming data distribution engine copies the video frames into multiple replicas, encodes, trims, or scales them according to the independent parameters of each virtual device, and then distributes them. The encoding parameter adaptation module independently maintains parameters such as resolution, frame rate, and exposure compensation for each virtual Camera node, and notifies the distribution engine to adjust its strategy when parameters change. The audio routing virtualization module intercepts AudioFrames. Linger's audio output creates multiple virtual audio channels, each with independently configured volume, channel mapping, and sound effect parameters. The intelligent mixing and scheduling module mixes multiple audio streams according to weights and outputs them to the speakers, while simultaneously copying and distributing the microphone audio stream to multiple applications. The resource arbitration management module monitors indicators such as ISP load and CPU usage, and degrades applications based on priority when concurrency exceeds limits. The priority policy configuration module provides a user interface to set the priority and degradation policies for each application. The system bypass control module bypasses the middleware and connects directly to the hardware abstraction layer when a single application accesses the system. The notification bar media center module generates a resident control panel that displays a resource usage list and provides mute and pause controls. The hardware status monitoring module periodically collects data such as motor current, battery level, and body temperature.

[0022] Example 2: After the device is powered on, the user opens the live streaming app for outdoor live streaming. The system bypass control module detects that only one application is accessing the camera, maintaining bypass status with a latency of less than 60ms. Subsequently, the user opens the "Background Recording" app to start 4K recording, and the system completes the switch within 100ms: the virtual device management module binds the live streaming app to the foreground preview node, and the background recording app to the background recording node. The streaming data distribution engine copies the original YUV video stream into two copies. The foreground node only performs color space conversion and outputs directly to ensure low latency, while the background node calls the hardware ISP to encode and write to storage. The encoding parameter adaptation module maintains parameters independently for both nodes. The intelligent mixing scheduling module copies and distributes the microphone audio to both applications after noise reduction. When the live streaming app is set to "Live Streaming Priority," the background application notification sound is automatically attenuated to 1%. The resource arbitration management module monitors the ISP load rate at 78% and does not intervene. The media center module in the notification bar displays the occupancy status of both applications, and the user can independently mute the audio of either application.

[0023] Example 3: Based on the concurrent operation of live streaming and recording in Example 2, when a user opens a third application, "Editing APP," to preview the footage, the ISP load rate exceeds the 85% threshold. The resource arbitration management module queries the priority list (live streaming APP has the highest priority 1, background recording has the highest priority 2, and editing APP has the lowest priority 3), and demotes the editing APP accordingly. The live streaming and background recording are not affected. After closing the editing APP, the background recording automatically resumes full-specification output.

[0024] Example 4: When a user only uses the system camera app to take pictures, the system bypass control module switches the system to bypass mode, the middleware is completely dormant, and the YUV video stream is directly transmitted to the camera app through the hardware abstraction layer without additional delay, power consumption is reduced by about 30%, and battery life is extended; when the user opens a second application to request the camera, the system automatically switches from bypass to virtualization state within 100ms.

[0025] The overall workflow is as follows: After powering on, the Android system and the independent gimbal MCU initialize synchronously, and the touchscreen loads the Android desktop. Users control the gimbal to adjust the angle using the five-way joystick 5, click the joystick to take a photo, and long press to record 4K video. The captured footage is automatically saved to local storage. Users can open an editing app on the device to complete post-production, or open a live streaming app to push and publish the video. Alternatively, the device can connect directly to a USB flash drive, mobile phone, or computer via the Type-C interface 14OTG to export footage in batches. The entire process is completed on a single device without the need for an external mobile phone.

Claims

1. A handheld gimbal-stabilized camera, characterized in that, include: The handheld camera body, the three-axis gimbal mechanism set on the top of the handheld camera body, the camera (1) mounted on the three-axis gimbal mechanism, and the main control hardware board set inside the handheld camera body; The three-axis gimbal mechanism includes a yaw axis motor (2), a pitch axis motor (11), and a roll axis motor (12). The main control hardware board includes an Android application processor and an independent gimbal MCU. The Android application processor and the independent gimbal MCU are connected via serial communication, and the Android application processor runs on the Android operating system. The handheld device has a 1.95-inch touchscreen (3) and a five-way joystick (5) arranged from top to bottom on the front. The Android operating system connects to the camera (1) through a standard driver interface, and outputs the original image and audio directly to third-party applications to support users to directly install and run editing, live streaming or material management software.

2. A handheld gimbal camera with image stabilization according to claim 1, characterized in that, The five-dimensional joystick (5) is used to control the movement of the gimbal and trigger the shooting operation. The front of the handheld device is equipped with a power button (9) and a status indicator light (4). The status indicator light (4) is used to distinguish between standby, photo taking and video recording states.

3. A handheld gimbal camera with image stabilization according to claim 1, characterized in that, The handheld device is also equipped with dual microphones, including microphone I (10) and microphone II (13). A speaker (6) is provided at the bottom of the handheld device. Microphone I (10) and microphone II (13) are integrated with the speaker (6) to form a noise reduction algorithm for sound reception and sound quality improvement.

4. A handheld gimbal camera with image stabilization according to claim 1, characterized in that, The bottom of the handheld device is equipped with a full-function Type-C interface (14) and a 1 / 4-20" threaded hole (15). The bottom side of the handheld device is also provided with a quick-release slot (7) and a lanyard hole (8). The Type-C interface (14) enables the OTG function to directly connect to external storage devices, Android phones or computers to realize bidirectional batch transmission of materials and file management.

5. A handheld gimbal camera with image stabilization according to claim 1, characterized in that, The camera (1) has a lens end that supports quick assembly of magnetic filters, and the camera (1) is a 48-megapixel camera with a built-in 1 / 2.0-inch sensor and an F1.8 aperture lens. It supports 4K high-definition material recording, slow motion and time-lapse photography. The handheld body has a power supply module inside, and the power supply module has a built-in 3500mAh lithium battery.

6. An operating system for a handheld gimbal camera with image stabilization according to any one of claims 1-5, characterized in that, This includes an audio / video virtualization routing middleware configured above the Android Hardware Abstraction Layer, wherein the audio / video virtualization routing middleware includes: The physical device interface layer is used to acquire raw YUV video streams and PCM audio streams; The data acquisition buffer module is used to align the original YUV video stream and PCM audio stream according to time slice buffering to generate a multimedia data frame queue; The virtual device management module is used to create multiple virtual camera device nodes and virtual audio device nodes, which can be bound to different applications. The streaming data distribution engine is used to copy video frames in the data frame queue into multiple copies, and then encode, trim, or scale them according to the independent parameters of each virtual device node before distributing them. The encoding parameter adaptation module is used to independently maintain imaging parameters for each virtual Camera device node and notify the streaming data distribution engine when the parameters change; The audio routing virtualization module is used to create multiple virtual audio output channels, each of which can be independently configured with volume, channel mapping and sound effect parameters; The intelligent mixing and scheduling module is used to mix multiple audio streams according to weights and output them, and to copy and distribute the audio streams captured by dual microphones to multiple applications; The resource arbitration management module is used to monitor hardware load and degrade streaming specifications according to priority strategies when concurrency limits are exceeded. The priority strategy configuration module provides a user configuration interface to set the priority order and degradation strategy of each application. The system bypass control module is used to bypass the virtual device management module and the streaming data distribution engine when a single application accesses the system, allowing the application to communicate directly with the hardware abstraction layer. The Media Center module in the notification bar is used to generate a control panel in the notification bar, displaying a list of applications consuming resources and providing mute, pause, and toggle controls; The hardware status monitoring module is used to periodically collect status data on gimbal motor load, battery level, body temperature, and remaining storage capacity.

7. The operating system of the handheld gimbal camera according to claim 6, characterized in that, The virtual camera device nodes created by the virtual device management module include a foreground preview node, a background recording node, and an AI analysis node, which are bound to different applications respectively; the streaming data distribution engine outputs a low-latency video stream to the foreground preview node, an encoded and compressed storage-level video stream to the background recording node, and a scaled inference-level video stream to the AI ​​analysis node.

8. The operating system of the handheld gimbal camera according to claim 6, characterized in that, The intelligent mixing and scheduling module has audio scene priority rules. When a live streaming application is in the foreground, it automatically attenuates the notification sounds of other applications to a preset percentage and adjusts the microphone's main channel gain and background sound gain.

9. The operating system of the handheld gimbal camera according to claim 6, characterized in that, The hardware status monitoring module collects data on the following dimensions: drive current, PWM duty cycle, winding temperature of each axis motor in the three-axis gimbal mechanism, battery voltage, charging and discharging current, remaining power, cycle count, multi-point temperature of the device body, and remaining storage capacity and write speed.

Images