A live panoramic and close-up synchronous acquisition device and system based on binocular vision

Abstract

The application discloses a kind of live broadcast panorama and close-up synchronous acquisition device and system based on binocular vision, it is related to video acquisition and live broadcast technical field.For single lens cannot give consideration to panorama and close-up, multi-position switching cost is high problem, the application sets up: binocular camera module is by wide-angle panoramic lens and long-focus close-up lens synchronous acquisition two-way video stream;Adjusting module is according to live broadcast scene real-time control long-focus lens zoom focusing, first scene keeps panorama, second scene captures detail;Main control processing unit is to two-way video stream pixel level alignment and synchronous coding generation composite video stream;Network communication module is directly exported to live broadcast push stream equipment or monitoring terminal via network port.The application fuses panorama and close-up in the same time axis, picture information quantity and availability significantly improve;Adjusting module supports manual knob and event automatic identification two modes, give consideration to low delay composition and automatic capture;Multiple picture synthesis mode dynamic switching, adapt to live broadcast and monitoring multi-scene, low in cost, convenient to deploy.

Description

Technical Field

[0001] This invention relates to the field of video acquisition and live streaming technology, specifically to a device and system for simultaneous acquisition of live panoramic and close-up shots based on binocular vision. Background Technology

[0002] With the popularization of online live streaming, outdoor live streaming has placed higher demands on the amount of information in the footage: viewers need to understand the panoramic environment on site, and also hope to see key details clearly. However, traditional single-lens solutions have inherent limitations. If a wide-angle lens is used, distant details are lost; if a telephoto lens is used, the overall scene cannot be captured. A single camera position cannot simultaneously meet the viewing needs of panoramic and close-up views on the same timeline, making it difficult to achieve both usability and information content.

[0003] To address these issues, existing solutions primarily employ multi-camera manual switching or binocular / multi-camera equipment. While multi-camera shooting combined with manual switching by the director can achieve panoramic and close-up shots, it is costly, complex to set up, and requires dedicated personnel, making it difficult for solo outdoor creators to implement. Furthermore, there is often insufficient time to switch cameras when crucial shots appear. Some binocular intelligent PTZ cameras utilize AI algorithms to automatically track moving targets, with the machine autonomously determining the tracking object and driving the gimbal to turn. While this achieves automatic tracking, panoramic shots are discarded during gimbal turning, and AI decision-making deprives the anchor of subjective compositional freedom. The algorithms and servo systems also introduce high hardware costs and outdoor reliability risks.

[0004] In summary, existing live streaming and monitoring acquisition equipment struggles to achieve real-time, subjective control and simultaneous acquisition of panoramic and close-up footage by a single operator without interrupting their own activities, while maintaining low cost and portability, and to provide a flexible, multi-mode synthesized video stream for real-time streaming. Therefore, there is an urgent need for a highly integrated, intuitive device and system capable of simultaneously acquiring and outputting panoramic and close-up footage. Summary of the Invention

[0005] The purpose of this invention is to solve the problems in existing live streaming and monitoring acquisition equipment where a single camera cannot simultaneously capture panoramic and close-up shots on the same timeline, resulting in a difficulty in achieving both the amount of information and usability of the footage. Furthermore, the high cost of manual switching between multiple cameras and the inability to respond to critical shots in time make it difficult for a single person to achieve low-cost, portable, real-time subjective control and synchronous acquisition of panoramic and close-up shots. Therefore, this invention provides a live streaming panoramic and close-up synchronous acquisition device and system based on binocular vision.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A live panoramic and close-up synchronous acquisition device based on binocular vision, characterized in that it includes a binocular camera module, an adjustment module, a main control processing unit and a network communication module; The binocular camera module includes a first lens and a second lens, which simultaneously acquire a first video stream and a second video stream. The adjustment module is connected to the main control processing unit and controls the zoom magnification and focus distance of the second lens in real time. The zoom magnification and focus distance are adjusted according to the live broadcast scene. When in the first scene, the zoom magnification and focus distance of the second lens are not adjusted. When in the second scene, the zoom magnification and focus distance are adjusted to capture the details of the second scene. The main control processing unit is connected to the binocular camera module and the adjustment module respectively. It receives the first video stream and the second video stream, adjusts the second video stream according to the control signal output by the adjustment module, and performs pixel-level alignment and synchronous encoding on the first video stream and the adjusted second video stream to generate a composite video stream. The network communication module is connected to the main control processing unit and transmits the composite video stream to the live streaming device or monitoring terminal in real time.

[0007] By using binocular synchronous acquisition and pixel-level alignment and synchronous encoding, panoramic views and close-ups are integrated into the same timeline, fundamentally solving the contradiction that a single lens cannot capture both the overall situation and local details. The main control unit uniformly schedules the two video streams, achieving consistency in temporal synchronization and spatial alignment, which significantly improves the information density and usability of live broadcasts and monitoring footage.

[0008] The first lens is a wide-angle panoramic lens, and the second lens is a telephoto close-up lens. The first lens and the second lens are arranged side by side to ensure the consistency of the two lenses in the horizontal and vertical directions.

[0009] The combination of a wide-angle lens and a telephoto lens is naturally suited to the needs of panoramic and close-up shooting. The side-by-side layout keeps the optical axes of the two lenses consistent in the horizontal and vertical directions, minimizing parallax, reducing the algorithmic complexity of subsequent pixel-level alignment and correction, and improving alignment accuracy.

[0010] The adjustment module includes a physical manual adjustment knob integrated on the device body. The physical manual adjustment knob is electrically connected to the main control processing unit and responds to the operator's physical rotation operation to generate the control signal to control the zoom magnification and focus distance of the second lens in real time. The physical manual adjustment knob is located in the top area of ​​the device body, and the knob surface is provided with scale markings to indicate the current zoom magnification.

[0011] The physical manual adjustment knob is located on the top of the device, allowing for easy one-handed operation. The knob surface features graduated markings indicating the current zoom level, providing the operator with intuitive tactile and visual feedback.

[0012] The physical knob provides low-latency manual control without relying on external computing power and network. Operators can directly and intuitively adjust the close-up parameters, which is suitable for scenarios where network is limited or where manual image composition is required. The surface scale markings facilitate quick reading and reproduction of zoom status, improving on-site operation efficiency.

[0013] The first scenario is a waiting monitoring state where no target event is detected, and the second lens maintains a preset zoom level and focus distance; the second scenario is a detail capture state triggered by the target event, and the adjustment module responds to the target event by automatically adjusting the zoom level and focus distance of the second lens to capture the detailed image of the target event; the target event includes a preset state change event of the monitored object.

[0014] By clearly defining two working modes—idle monitoring and event capture—the lens is prevented from frequently zooming under meaningless normal conditions, thereby reducing mechanical wear and image shake, extending the life of optical components and stabilizing output, while allowing resources to be concentrated on detail capture when truly needed.

[0015] The adjustment module includes a target event recognition unit; when the target event recognition unit recognizes a target event, the adjustment module automatically generates the control signal to control the second lens to adjust the zoom level and focus distance to capture detailed images of the target event; when the target event is not recognized, the adjustment module does not generate the control signal, the second lens maintains the current zoom level and focus distance, and the live broadcast scene remains in the first scene.

[0016] It enables automatic close-up capture triggered by events, automatically focusing on details without human intervention the moment a critical event occurs, significantly reducing the rate of missed shots and significantly improving the intelligence level and event response speed of monitoring and live streaming.

[0017] The device further includes an audio acquisition unit and a vibration detection unit; the target event recognition unit receives the following multiple input signals: the first video stream, the second video stream, the audio signal acquired by the audio acquisition unit, and the vibration signal detected by the vibration detection unit; the target event recognition unit performs multimodal fusion discrimination on the multiple input signals, specifically including: performing moving target detection on the video stream to obtain video features, performing time-frequency analysis on the audio signal to obtain audio spectrum features, and performing amplitude and time-domain analysis on the vibration signal to obtain vibration features; when the joint criterion of at least two of the video features, audio spectrum features, and vibration features exceeds a preset threshold, a target event is determined to be identified, and the adjustment module automatically generates the control signal.

[0018] By fusing video, audio, and vibration modalities for discrimination, and using the joint determination of at least two feature streams as a trigger condition, multi-sensor cross-verification is achieved. It can accurately capture close-up moments, making identification more accurate and reliable, and does not rely on complex AI models, with low hardware cost and fast response.

[0019] The main control processing unit performs image compositing on the first video stream and the adjusted second video stream to generate the composite video stream; the image compositing mode includes picture-in-picture mode, split-screen mode, or single-screen mode; the picture-in-picture mode superimposes the adjusted second video stream as a sub-screen onto a preset area of ​​the first video stream; the split-screen mode displays the first video stream and the adjusted second video stream in different areas of the screen; the single-screen mode outputs either the first video stream or the adjusted second video stream separately; the main control processing unit dynamically switches between the picture-in-picture mode, the split-screen mode, and the single-screen mode according to the live streaming scenario.

[0020] Multiple synthesis modes can be dynamically switched to flexibly adapt to different viewing needs: picture-in-picture takes into account both the overall picture and details, split screen facilitates comparison, and single screen focuses on key information, thereby improving the terminal viewing experience and information transmission efficiency, and enabling the same composite stream to meet the needs of multiple scenarios such as live streaming and monitoring.

[0021] The binocular camera module is connected to the main control processing unit via an LVDS differential interface. The first video stream and the second video stream are transmitted to the main control processing unit via the LVDS differential interface. The LVDS differential interface uses differential signal pairs for data transmission. The main control processing unit is connected to the second lens via an I2C bus. The main control processing unit configures the initial parameters of the second lens via the I2C bus and transmits the control signals via the I2C bus to control the second lens to perform zoom and focus operations. The initial parameters include exposure time and white balance. The main control processing unit includes a hardware encoder, which performs H.265 or H.264 standard encoding compression on the composite video stream.

[0022] The LVDS differential interface has strong anti-interference capabilities and high bandwidth, ensuring low latency and low loss transmission of dual-channel video; the I2C bus completes lens parameter configuration and zoom / focus control with a simplified link, saving pins and wiring; the built-in hardware encoder directly completes H.265 / H.264 compression, reducing the main control CPU load, improving encoding efficiency and significantly saving transmission bandwidth.

[0023] The network communication module includes a network port, and the main control processing unit outputs the encoded composite video stream directly to the live streaming device or monitoring terminal via the network port to achieve low-latency remote viewing.

[0024] Composite video streams are output directly through the network port, eliminating the need for additional protocol conversion and intermediate caching, minimizing end-to-end latency, and meeting the remote low-latency viewing needs of live streaming and security monitoring in real-time sensitive scenarios.

[0025] The device also includes a power management unit, which receives external power input via a USB interface and converts the external power into the operating voltage required by the binocular camera module, the adjustment module, the main control processing unit, and the network communication module.

[0026] Unified power management draws power from a universal USB interface and converts it to the voltage required by each module, simplifying the overall power supply structure, improving the portability and deployment flexibility of the equipment, and facilitating rapid setup in environments without dedicated power supplies, such as outdoor live streaming and mobile monitoring.

[0027] A live streaming panoramic and close-up synchronous acquisition system based on binocular vision, characterized in that it includes: an acquisition device, which is a live streaming panoramic and close-up synchronous acquisition device based on binocular vision, used to synchronously acquire a first video stream and a second video stream, adjust the zoom factor and focus distance of the second lens according to the live streaming scene, perform pixel-level alignment and synchronous encoding on the first video stream and the adjusted second video stream to generate a composite video stream, and output the composite video stream through a network communication module; The live streaming device is communicatively connected to the network communication module of the acquisition device, receives the composite video stream, encapsulates the composite video stream using a streaming protocol, and outputs a live stream. The live streaming service platform is communicatively connected to the live streaming device, receives the live stream, and distributes it to multiple viewer terminals; The viewer terminal is connected to the live streaming service platform to receive and display the live stream.

[0028] This constitutes a complete end-to-end link from local data collection to streaming encapsulation, platform distribution, and terminal display. The responsibilities of each module are decoupled and their collaboration is clear, which facilitates large-scale deployment and supports one-to-many concurrent distribution to multiple terminals, thereby improving the availability, stability, and scalability of the live monitoring system as a whole. Attached Figure Description

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 Overall flow chart of the device Figure 2 Overall 3D structural diagram Figure 3 Physical manual adjustment flowchart Figure 4 Automatic Target Event Recognition Flowchart Figure 5 Main control schematic diagram Figure 6 Binocular camera sensor schematic diagram Detailed Implementation Example 1: like Figure 1 As shown, the present invention provides a live panoramic and close-up synchronous acquisition device based on binocular vision, characterized in that it includes a binocular camera module, an adjustment module, a main control processing unit and a network communication module; The binocular camera module includes a first lens and a second lens, which simultaneously acquire a first video stream and a second video stream. The adjustment module is connected to the main control processing unit and controls the zoom magnification and focus distance of the second lens in real time. The zoom magnification and focus distance are adjusted according to the live broadcast scene. When in the first scene, the zoom magnification and focus distance of the second lens are not adjusted. When in the second scene, the zoom magnification and focus distance are adjusted to capture the details of the second scene. The main control processing unit is connected to the binocular camera module and the adjustment module respectively. It receives the first video stream and the second video stream, adjusts the second video stream according to the control signal output by the adjustment module, and performs pixel-level alignment and synchronous encoding on the first video stream and the adjusted second video stream to generate a composite video stream. The network communication module is connected to the main control processing unit and transmits the composite video stream to the live streaming device or monitoring terminal in real time.

[0030] Figure 2 The overall design of the device is showcased, with its core components including a binocular camera module and an adjustment knob. The binocular camera module consists of a wide-angle panoramic lens and a telephoto close-up lens, arranged side-by-side to ensure consistency and stability in both horizontal and vertical directions. The wide-angle panoramic lens, located on the left, captures the panoramic view of the live broadcast environment; the telephoto close-up lens, located on the right, focuses on key action details during the live broadcast. Both lenses are dustproof and waterproof to withstand complex outdoor environments. The adjustment knob, located on the top of the device, allows for easy one-handed operation and features finely engraved scales to provide intuitive feedback on zoom levels. Furthermore, the device's casing is made of high-strength engineering plastic, offering excellent impact resistance and weather resistance, while optimized heat dissipation design ensures stability during extended operation.

[0031] like Figure 3As shown, the adjustment module includes a physical manual adjustment knob integrated on the device body. The physical manual adjustment knob is electrically connected to the main control processing unit, responds to the operator's physical rotation operation, and generates the control signal to control the zoom magnification and focus distance of the second lens in real time. The physical manual adjustment knob is located in the top area of ​​the device body, and the knob surface is provided with scale markings to indicate the current zoom magnification.

[0032] The physical manual adjustment knob is electrically connected to the main control processing unit. The operator physically rotates the knob, and the integrated rotary encoder (such as an incremental or absolute rotary encoder) converts the rotation direction and amount into electrical signals, generating control signals which are then output to the main control processing unit. The main control processing unit analyzes these control signals and sends zoom / focus control commands to the second lens (telephoto close-up) via the I2C bus, controlling the zoom magnification and focus distance of the second lens in real time.

[0033] The physical knob has tactile feedback, allowing operators to adjust the composition of close-up shots in real time without shifting their visual attention during live streaming. This is something that AI automatic tracking and remote control solutions cannot achieve. At the same time, the manual solution does not require expensive components such as AI chips and servo motors, significantly reducing hardware costs and failure rates.

[0034] The first scenario is a waiting monitoring state without any target event triggering. In this state, the second lens maintains the default zoom level and focus distance (e.g., the factory default panoramic viewing angle), the adjustment module does not generate adjustment signals, the first video stream and the second video stream are synchronously acquired and synthesized for output, and the live broadcast mainly displays a panoramic view.

[0035] The second scenario is the detail capture state triggered by the target event. In this state, the adjustment module adjusts the zoom level and focus distance of the second lens according to the target event in order to capture detailed images of the target event.

[0036] The target events include, but are not limited to, the state change events of the preset monitored objects; taking fishing live broadcast as an example: when no fish are hooked, the system is in the first scene, and the audience can watch the panoramic view of the fishing ground; when a fish bites the hook (the float sinks and the fishing line tightens), the system switches to the second scene, and the second camera is automatically or manually adjusted to the telephoto state, accurately locking the focus of the image on the position of the fish hook, capturing the details of the moment the fish is hooked, while the panoramic view of the first camera is maintained, so that the audience can see the panoramic environment and close-up details at the same time.

[0037] The main control processing unit performs pixel-level alignment between the first video stream and the adjusted second video stream, and then performs image compositing in picture-in-picture mode or split-screen mode to generate a composite video stream.

[0038] Picture-in-Picture mode: The first video stream serves as the main view, while the second video stream is embedded within it as a child window. The size and position of the child window are configurable (e.g., it defaults to the bottom right corner of the main view and can be shrunk / enlarged). Viewers can focus on embedded close-up details while enjoying the panoramic environment.

[0039] Split-screen mode: The first video stream and the second video stream each occupy a corresponding part of the display area (e.g., half the screen on the left and right or half the screen on the top and bottom), which is suitable for scenarios that require comparison and display.

[0040] The main control processing unit also supports single-screen output mode: based on operator instructions or scene switching signals, it can output only the first or second video stream. For example, while waiting for a fish to bite, it can output only a panoramic view, and when the fish is hooked, it can output only a close-up view.

[0041] The three modes can be dynamically switched according to the operator's instructions or scene switching signals, that is, to realize the dynamic switching between picture-in-picture mode, split-screen mode and single-screen output mode, so as to meet the diverse needs of different live streaming platforms and viewers.

[0042] like Figure 5 , Figure 6 As shown, in this embodiment, both the first and second lenses are connected to the main control processing unit via LVDS differential interfaces to achieve high-speed, low-noise video data transmission. The LVDS (Low Voltage Differential) interface employs differential signal transmission technology, featuring low voltage swing, high transmission rate, and low electromagnetic interference, effectively suppressing electromagnetic interference and ensuring the integrity of the video signal. In this embodiment, the two LVDS interfaces transmit the first and second video streams to the main control processing unit in real time via parallel data channels.

[0043] In addition, the main control processing unit is connected to the first and second lenses via the I2C bus to configure lens parameters (including exposure time, gain, and white balance) and control the zoom and focus operations of the second lens. The I2C bus is characterized by its simple structure and high reliability. It only requires two signal lines (clock line SCL and data line SDA) to complete communication between master and slave devices, and is widely used in peripheral control of embedded systems.

[0044] The main control processing unit integrates a hardware encoder that supports H.265 and / or H.264 encoding formats for real-time encoding and compression of composite video streams. Hardware encoding offers lower latency and CPU utilization compared to software encoding, meeting the low latency and high image quality requirements of live streaming scenarios. H.265 reduces the bitrate by approximately 50% compared to H.264 at the same image quality, making it suitable for bandwidth-constrained mobile live streaming scenarios. H.264 offers better compatibility, meeting the access requirements of various live streaming platforms. This embodiment can dynamically select the encoding format according to the requirements of the live streaming platform.

[0045] The network communication module includes a network port. The main control processing unit outputs the encoded composite video stream directly to the live streaming device or monitoring terminal via the network port to achieve low-latency remote viewing.

[0046] The power management unit includes a USB power interface that receives external power (such as from a power bank, USB adapter, or car power supply). After being connected via the USB power interface, the power management chip performs voltage step-down and regulation processing. The chip's power supply circuit then provides multiple stable operating voltages to the main control processing unit and core components such as the binocular camera module. For example, the power management chip converts a 5V input into multiple voltages such as 3.3V, 1.8V, and 1.1V, supplying different voltage domains to the main control processing unit's I / O interfaces, DDR memory, core logic, and the analog and digital circuits of the binocular camera module. The USB power supply solution allows the device to be powered by common portable power banks, facilitating long-term outdoor live streaming.

[0047] Example 2: like Figure 4 As shown, in this embodiment, the adjustment module adopts an automatic implementation method, specifically including a target event recognition unit; the device also includes an audio acquisition unit and / or a vibration detection unit.

[0048] The target event recognition unit is connected to the main control processing unit. It can be a dedicated hardware acceleration module integrated inside the main control processing unit, or it can be a processor chip independent of the main control processing unit.

[0049] The audio acquisition unit includes a MEMS microphone array, configured to acquire ambient audio signals and output digital audio signals. The ambient audio signals include the sound of fishing line taut, the sound of fishing line cutting water, the sound of water splashing, and wind noise. The vibration detection unit includes a piezoelectric vibration sensor, which is attached to the outer shell of the device body or fixed to the surface of the external fishing rod body by a clamp. It is configured to detect vibration signals and output digital vibration signals, including rod tip vibration, rod body bending vibration and environmental mechanical vibration. The target event recognition unit receives one or more of the following input signals: First video stream (panoramic view from the first camera); Second video stream (close-up footage from the second camera); The audio signal acquired by the audio acquisition unit; Vibration signals detected by the vibration detection unit.

[0050] Step S1: The binocular camera module begins synchronous acquisition, and the audio acquisition unit and / or vibration detection unit begin operation; Step S2: The target event recognition unit performs feature extraction on the multiple input signals, specifically including: (a) A video feature extraction subunit is configured to: extract the region of interest of the float in the first video stream and / or the second video stream; calculate the displacement vector sequence of the float using optical flow or inter-frame difference method; extract the video displacement amplitude features and displacement frequency features; perform frequency domain analysis on the displacement frequency features; and output a wind disturbance mark when the main frequency component of the displacement signal falls into a preset wind disturbance frequency band. (b) An audio spectrum feature extraction subunit is configured to: perform a short-time Fourier transform on the digital audio signal, calculate the frequency band energy density and spectral centroid within a preset target frequency band as audio spectrum features; and calculate the background noise energy in the low-frequency band outside the target frequency band for adaptive noise threshold adjustment. (c) Vibration feature extraction subunit, configured to: after bandpass filtering the digital vibration signal, calculate the root mean square amplitude, peak count and frequency centroid within the sliding window as vibration features; Step S3: The time alignment module timestamps the video features, audio spectrum features, and vibration features; Step S4: The joint discrimination module makes a judgment based on the aligned features: when at least two of the three features exceed their respective preset thresholds within a preset time window Δt, the target event is identified, and the adjustment module automatically generates the control signal to control the second lens to adjust the zoom and focus distance, locking the focus of the image on the area where the target event is located, so as to capture the details of the target event; when only one feature exceeds its preset threshold, or when the video displacement frequency feature carries the wind disturbance mark and the root mean square amplitude of the vibration feature does not exceed the vibration threshold, the joint discrimination module does not determine it as a target event; Step S5: When no target event is detected, the adjustment module does not generate a control signal, the second lens maintains the current zoom level and focus distance, and the system maintains the panoramic monitoring output in the first scene; Step S6: Repeat steps S2 to S5 to achieve continuous monitoring and automatic response to the target event.

[0051] Example 3: This invention also provides a live streaming panoramic and close-up synchronous acquisition system based on binocular vision, comprising: The binocular vision-based live panoramic and close-up synchronous acquisition device described in the embodiment is used to synchronously acquire the first video stream and the second video stream, adjust the zoom factor and focus distance of the second lens according to the live scene switching, and perform pixel-level alignment and synchronous encoding on the first video stream and the adjusted second video stream to generate a composite video stream. The live streaming device (such as a streaming box, encoder, or PC / mobile phone with streaming software installed) is connected to the network communication module of the acquisition device to receive the composite video stream and push it to the live streaming service platform. In this embodiment, the acquisition device outputs the composite video stream to the live streaming device via the network communication module using the RTSP / RTMP protocol, and the live streaming device then pushes the composite video stream to the live streaming service platform using the RTMP protocol. The live streaming service platform (such as Douyin, Kuaishou, Bilibili, etc., or the enterprise's self-built live streaming service) communicates with the live streaming push equipment to receive and distribute composite video streams; Viewer terminals (such as mobile phones, tablets, PCs, and smart TVs) communicate with the live streaming service platform to receive and display composite video streams, allowing remote viewers to watch synchronized panoramic and close-up shots in real time.

[0052] It should be noted that the live streaming device can be integrated into the acquisition device (i.e., directly pushed by the network communication module of the acquisition device), or it can be a standalone device; when it is a standalone device, the acquisition device and the live streaming device are connected via wired or wireless means. All the above-mentioned modifications are within the protection scope of this invention.

Claims

1. A device for simultaneous acquisition of live panoramic and close-up shots based on binocular vision, characterized in that, It includes a binocular camera module, an adjustment module, a main control processing unit, and a network communication module; The binocular camera module includes a first lens and a second lens, which simultaneously acquire a first video stream and a second video stream. The adjustment module is connected to the main control processing unit and controls the zoom magnification and focus distance of the second lens in real time. The zoom magnification and focus distance are adjusted according to the live broadcast scene. When in the first scene, the zoom magnification and focus distance of the second lens are not adjusted. When in the second scene, the zoom magnification and focus distance are adjusted to capture the details of the second scene. The main control processing unit is connected to the binocular camera module and the adjustment module respectively. It receives the first video stream and the second video stream, adjusts the second video stream according to the control signal output by the adjustment module, and performs pixel-level alignment and synchronization encapsulation on the first video stream and the adjusted second video stream to generate a composite video stream. The network communication module is connected to the main control processing unit and transmits the composite video stream to the live streaming device or monitoring terminal in real time.

2. The apparatus according to claim 1, characterized in that, The adjustment module includes a physical manual adjustment knob integrated on the device body. The physical manual adjustment knob is electrically connected to the main control processing unit and responds to the operator's physical rotation operation to generate the control signal to control the zoom magnification and focus distance of the second lens in real time. The physical manual adjustment knob is located in the top area of ​​the device body, and the knob surface is provided with scale markings to indicate the current zoom magnification.

3. The apparatus according to claim 1, characterized in that, The first scenario is a waiting monitoring state where no target event is detected, and the second lens maintains a preset zoom level and focus distance; the second scenario is a detail capture state triggered by the target event, and the adjustment module responds to the target event by automatically adjusting the zoom level and focus distance of the second lens to capture the detailed image of the target event; the target event includes a preset state change event of the monitored object.

4. The apparatus according to claim 3, characterized in that, The adjustment module includes a target event recognition unit; when the target event recognition unit recognizes a target event, the adjustment module automatically generates the control signal to control the second lens to adjust the zoom level and focus distance in order to capture detailed images of the target event. When the target event is not detected, the adjustment module does not generate the control signal, the second lens maintains the current zoom level and focus distance, and the live streaming scene remains in the first scene.

5. The apparatus according to claim 4, characterized in that, The device further includes an audio acquisition unit and a vibration detection unit; the target event recognition unit receives the following multiple input signals: the first video stream, the second video stream, the audio signal acquired by the audio acquisition unit, and the vibration signal detected by the vibration detection unit; The target event recognition unit performs multimodal fusion discrimination on the multi-channel input signals, specifically including: performing moving target detection on the video stream to obtain video features, performing time-frequency analysis on the audio signal to obtain audio spectrum features, and performing amplitude and time-domain analysis on the vibration signal to obtain vibration features; When the combined criteria of at least two of the video features, audio spectrum features, and vibration features exceed a preset threshold, a target event is identified, and the adjustment module automatically generates the control signal.

6. The apparatus according to claim 1, characterized in that, The main control processing unit performs image compositing on the first video stream and the adjusted second video stream to generate the composite video stream; the image compositing mode includes picture-in-picture mode, split-screen mode, or single-screen mode; the picture-in-picture mode superimposes the adjusted second video stream as a sub-screen onto a preset area of ​​the first video stream; the split-screen mode displays the first video stream and the adjusted second video stream in different areas of the screen; the single-screen mode outputs either the first video stream or the adjusted second video stream separately; the main control processing unit dynamically switches between the picture-in-picture mode, the split-screen mode, and the single-screen mode according to the live streaming scenario.

7. The apparatus according to claim 1, characterized in that, The binocular camera module is connected to the main control processing unit via an LVDS differential interface, and the first video stream and the second video stream are transmitted to the main control processing unit via the LVDS differential interface; The LVDS differential interface uses differential signal pairs for data transmission; the main control processing unit is connected to the second lens via an I2C bus, and configures the initial parameters of the second lens via the I2C bus, and transmits the control signals via the I2C bus to control the second lens to perform zoom and focus operations; the initial parameters include exposure time and white balance; the main control processing unit includes a hardware encoder, which performs H.265 or H.264 standard encoding and compression on the composite video stream.

8. The apparatus according to claim 1, characterized in that, The network communication module includes a network port, and the main control processing unit outputs the encoded composite video stream directly to the live streaming device or monitoring terminal via the network port to achieve low-latency remote viewing.

9. The apparatus according to claim 1, characterized in that, The device also includes a power management unit, which receives external power input via a USB interface and converts the external power into the operating voltage required by the binocular camera module, the adjustment module, the main control processing unit, and the network communication module.

10. A live streaming panoramic and close-up synchronous acquisition system based on binocular vision, characterized in that, include: The acquisition device is a live panoramic and close-up synchronous acquisition device based on binocular vision. It is used to synchronously acquire a first video stream and a second video stream, adjust the zoom factor and focus distance of the second lens according to the live scene, perform pixel-level alignment and synchronous encoding on the first video stream and the adjusted second video stream to generate a composite video stream, and output the composite video stream through a network communication module. The live streaming device is communicatively connected to the network communication module of the acquisition device, receives the composite video stream, encapsulates the composite video stream using a streaming protocol, and outputs a live stream. The live streaming service platform is communicatively connected to the live streaming device, receives the live stream, and distributes it to multiple viewer terminals; The viewer terminal is connected to the live streaming service platform to receive and display the live stream.

Images