Distributed camera system supporting automatic discovery and dynamic negotiation and interaction method

Abstract

The invention discloses a distributed camera system supporting automatic discovery and dynamic negotiation and an interaction method, and particularly relates to the technical field of distributed camera systems.An equipment discovery unit is integrated in a distributed engine module, and automatic identification and state management of equipment are achieved in a broadcast mode; a dynamic capability negotiation mechanism is introduced into the far-end camera connection unit, so that more flexible resource matching is realized; a refined control instruction set is expanded in a remote camera forwarding unit, and a streaming media packaging module is newly added to support seamless connection with a standard streaming media protocol.

Description

Technical Field

[0001] This invention relates to the field of distributed camera system technology, and more specifically to a distributed camera system and interaction method that supports automatic discovery and dynamic negotiation. Background Technology

[0002] Existing technologies (such as Virtio-based local device virtualization) are traditional methods for virtualizing hardware resources (e.g., cameras) within a single physical device. Virtual devices (e.g., virtio-camera) are created through a hypervisor layer, allowing different operating systems (e.g., host OS and guest OS) running on the same device to share access to the physical camera device. However, the main drawback is:

[0003] (1) Cross-device limitations: Virtio's core flaw lies in its design intent, which is virtualization within a single device. It cannot be extended to allow camera resource sharing between different physical devices on the same network. It lacks network communication capabilities and device discovery mechanisms.

[0004] (2) High system coupling: Specific modifications and adaptations are required in the host operating system driver layer, Hypervisor layer and guest operating system driver layer, which is complex to implement and has poor compatibility.

[0005] (3) Dedicated driver dependency: The guest operating system must use a specific virtio driver and cannot transparently use the standard camera operation interface.

[0006] Existing technology two (remote camera access based on common streaming media protocols such as RTSP) allows one device to request and receive video streams from a camera on another device over a network using standard streaming media protocols. For example, a client device requests a camera stream from a host device via the RTSP protocol. However, its main drawback is:

[0007] (1) Functional limitations (streaming only): Protocols such as RTSP mainly focus on the transmission and control of video streams (such as playback and pause). They lack a fine-grained mechanism for discovering, negotiating, and uniformly controlling the underlying capabilities of the camera (such as resolution, frame rate, focus, exposure mode, etc.). They cannot achieve a complete, fine-grained operation process similar to that of a local camera, such as "opening the camera device -> configuring parameters -> acquiring frame data".

[0008] (2) Inconsistent protocols and complex integration: The client and server sides need to implement specific streaming media server and client logic respectively, resulting in low integration with the device's own camera management module (such as Camera HAL / V4L2) and a large amount of development and adaptation work.

[0009] (3) Lack of device identity and capability abstraction: There is no unified "virtual device" abstraction for remote camera devices, and the client cannot operate the remote camera as transparently as operating the local camera.

[0010] The invention patent with publication number CN117278845A proposes a single-device distributed camera subsystem and a distributed remote camera system, but this distributed camera system has the following technical defects:

[0011] 1. Lack of an efficient automatic device discovery mechanism: Relying on manual configuration or static connection methods, it is impossible to achieve automatic identification, status awareness and role assignment of camera devices in the network environment, resulting in low system deployment and maintenance efficiency.

[0012] 2. Rigid device capability negotiation mechanism: The device lacks flexibility in the process of matching camera capabilities (such as resolution, frame rate, encoding format, etc.), and cannot be dynamically adapted according to the actual needs of the client and network conditions, which affects resource utilization efficiency and user experience.

[0013] 3. Insufficient granularity of camera control: Solutions based on streaming media protocols (such as RTSP) only support basic streaming operations and lack fine-grained control over low-level camera parameters, such as automatic exposure and focus mode selection, which limits the system's usability in high-level vision application scenarios.

[0014] 4. Poor compatibility with streaming media transmission protocols: The system does not fully consider compatibility with mainstream streaming media protocols (such as RTSP / RTP) in terms of video stream encapsulation and transmission, resulting in difficulties in cross-platform and cross-system integration.

[0015] Therefore, there is an urgent need to build a distributed camera system and interaction method that supports automatic device discovery, dynamic capability negotiation, and streaming media transmission. Summary of the Invention

[0016] To address these issues, this invention adds a device discovery unit to the distributed engine module and employs a broadcast message method, enabling the system to automatically discover devices, manage their status, and announce capabilities. Specifically, a dynamic capability negotiation mechanism is introduced in the remote camera connection unit, and the basic control commands are strengthened and a streaming media encapsulation module is added to the remote camera forwarding unit, thus resolving the problems mentioned in the background art.

[0017] To achieve the above objectives, the present invention provides the following technical solution: a distributed camera interaction method supporting automatic discovery and dynamic negotiation, applied to at least two devices in a network, each device including a single-device distributed camera subsystem, the subsystem including: a camera device management module, a remote camera module, a remote camera device module, a local camera device module, and a distributed engine module, the distributed engine module including a device discovery unit; the method includes the following steps:

[0018] Device discovery and role declaration: When the first device starts up as a host, the device discovery unit in its distributed engine module periodically sends broadcast messages on the network; when the second device starts up as a client, the device discovery unit in its distributed engine module listens for broadcast messages on the network; after receiving the broadcast message from the first device, it parses and obtains the broadcast message information, and presents the camera capability list of the first device and its broadcast message information on the user interface of the second device.

[0019] The remote camera connection and dynamic negotiation process responds to a second device user's operation of selecting a target camera resource on the first device via an application. The remote camera module of the second device dynamically selects a set of optimal capability configuration parameters from the camera capability list broadcast by the first device, based on the application's requirements, the second device's platform capabilities, and network conditions. The remote camera module of the second device generates a target camera connection request containing the optimal capability configuration parameters and sends the connection request to the server of the distributed engine module of the first device via the client of the second device's distributed engine module. The server of the first device's distributed engine module forwards the connection request to the remote camera module of the first device. The remote camera module of the first device processes the connection request and establishes a connection with the corresponding local camera device on the first device based on the optimal capability configuration parameters.

[0020] After a successful connection, the remote camera module of the first device notifies the camera device management module of the first device; at the same time, the connection success response is transmitted to the remote camera module of the second device through the server of the distributed engine module of the first device and the client of the distributed engine module of the second device.

[0021] Upon receiving a successful connection response, the remote camera module of the second device notifies the camera device management module of the second device to virtualize and generate a remote camera device corresponding to the target camera on the first device within the operating system of the second device, and mounts it, so that the remote camera device can coexist with the local camera device of the second device.

[0022] The present invention also discloses a distributed camera system that supports automatic discovery and dynamic negotiation. The system is based on the above-mentioned interaction method and includes a remote camera module, a camera device management module, a remote camera device module, a local camera device module, and a distributed engine module.

[0023] The remote camera module includes a remote camera forwarding unit and a remote camera connection unit;

[0024] The camera device management module includes a remote camera management unit and a local camera management unit;

[0025] The distributed engine module includes a client, a server, and a device discovery unit.

[0026] Preferably, the remote camera forwarding unit is extended with a refined control instruction set to respond to and forward client requests for adjusting the underlying parameters of the remote camera, including resolution selection, automatic exposure settings, and automatic focus settings.

[0027] Preferably, the remote camera forwarding unit adds a streaming media encapsulation module; when the client requests remote camera data, the host executes the following process:

[0028] The server sends a data request to the remote camera management unit of the camera device management module;

[0029] The remote camera management unit calls the local camera device module to acquire raw image data from the physical camera hardware;

[0030] The local camera device module transmits data to the remote camera forwarding unit on the host side;

[0031] The streaming media encapsulation module encapsulates the raw image data into standard streaming media protocol formats (including RTSP / RTP) and transmits it to the client through the distributed engine module.

[0032] Preferably, the remote camera connection unit adds a dynamic capability negotiation mechanism, the process of which includes:

[0033] Before initiating a connection, the client obtains the host's camera capability list by parsing the broadcast message of the device discovery unit;

[0034] Based on application requirements and platform processing capabilities, the client dynamically selects and requests a set of optimal configuration parameters from the capability list.

[0035] Preferably, the data structure of the broadcast message includes at least:

[0036] Device ID;

[0037] A list of camera capabilities, including the resolution, frame rate, and image format parameters supported by the device;

[0038] Device role information, used to declare a device as a host or client.

[0039] Compared with the prior art, the present invention has the following advantages:

[0040] 1. Integrate a device discovery unit into the distributed engine module to achieve automatic device identification and status management via broadcasting;

[0041] 2. Introduce a dynamic capability negotiation mechanism in the remote camera connection unit to achieve more flexible resource matching;

[0042] 3. The refined control instruction set is expanded in the remote camera forwarding unit, and a new streaming media encapsulation module is added to support seamless integration with standard streaming media protocols and enhance system compatibility. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the process of device discovery, networking, and connection.

[0044] Figure 2 This is a flowchart illustrating the process of refined control;

[0045] Figure 3 This is a schematic diagram of the streaming media data transmission process. Detailed Implementation

[0046] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] This invention proposes a distributed camera interaction method that supports automatic discovery and dynamic negotiation, comprising the following steps:

[0048] Step 1: Device Discovery and Role Declaration:

[0049] When a device boots up as a host, its device discovery unit periodically broadcasts its device ID, camera capability list, and device role information over the network.

[0050] When the device starts up as a client, its device discovery unit listens for broadcast messages in the network, parses the message content to automatically discover available hosts and their camera capabilities, and feeds the information back to the local application.

[0051] Device roles (host / client) are dynamically selected by the user through the application.

[0052] Step 2: Request Distribution and Operation Execution:

[0053] The application sends an operation request (such as "get local camera view" or "call remote camera to take a picture") to the client's distributed engine client.

[0054] Step 3: Dynamic Capability Negotiation

[0055] Before initiating a connection, the client dynamically selects the optimal configuration (such as resolution and frame rate) from the host's camera capability list based on application requirements.

[0056] The client's distributed engine client passes the request (including negotiation parameters) to the host's distributed engine server; the host's distributed engine server forwards the request to the remote camera management unit of the camera device management module; the remote camera management unit calls the local camera device module to operate the physical camera hardware.

[0057] Step 4: Streaming media data transmission:

[0058] The local camera device module on the host side acquires image data and transmits it to the remote camera forwarding unit via the camera device management module; the streaming media encapsulation module encapsulates the data into standard RTSP / RTP packets and transmits them to the client; the remote camera device module on the client submits the data frames to its camera device management module for use by the application.

[0059] Step 5: Status Maintenance

[0060] The camera device management module continuously manages the status and connection of all local physical cameras and virtual remote cameras.

[0061] The present invention also discloses a distributed camera system that supports automatic discovery and dynamic negotiation. The system is based on the above-mentioned interaction method and includes a remote camera module, a camera device management module, a remote camera device module, a local camera device module, and a distributed engine module.

[0062] The remote camera module includes a remote camera relay unit and a remote camera connection unit;

[0063] The camera device management module includes a remote camera management unit and a local camera management unit;

[0064] The distributed engine module includes a client, a server, and a device discovery unit.

[0065] The remote camera relay unit is expanded with a refined control instruction set to respond to and forward operations such as resolution selection, automatic exposure settings, and automatic focus settings from the client.

[0066] The remote camera forwarding unit has added a streaming media encapsulation module. The host uses this module to encapsulate the image data acquired by the physical camera into the standard RTSP / RTP format and transmit it to the client.

[0067] The remote camera connection unit adds a dynamic capability negotiation mechanism, allowing the client to dynamically select the optimal configuration from the host capability list based on application requirements before establishing a connection.

[0068] The broadcast message data structure must include at least the device ID, a list of camera capabilities (resolution / frame rate / image format), and device role information.

[0069] Example 1: Complete automatic discovery, negotiation, and control process:

[0070] like Figure 1 As shown, this embodiment demonstrates the workflow of the system of the present invention in a complete interaction, with a focus on how the newly enhanced modules work together.

[0071] S1: Device Discovery and Networking:

[0072] When the host device powers on and starts up, the device discovery unit in its distributed engine module begins to periodically send UDP broadcast messages within the local area network.

[0073] The broadcast message shall contain at least the following structured data:

[0074] deviceId: "Host_Device_001" (Device ID)

[0075] role: "host" (device role)

[0076] cameraCapabilities: [ {resolution: "1920x1080", frameRate: 30,format: "H.264"}, {resolution: "1280x720", frameRate: 60, format: "YUV"}] (List of camera capabilities)

[0077] After the client device starts up, its device discovery unit listens to the network and receives broadcast messages from the host. The client parses the messages, extracts the host's device ID and its camera capability list, and automatically presents this information to the user.

[0078] S2: Dynamic connectivity and capability negotiation:

[0079] Users see the remote camera with device ID "Host_Device_001" on the client application interface and select to connect.

[0080] When the client's remote camera connection unit is triggered, a dynamic capability negotiation mechanism is executed. Specifically, based on its own application requirements (such as the need for high-definition video calls) and the platform's decoding capabilities (such as only supporting the H.264 format), the client dynamically selects and requests an optimal configuration from the host's camera capability list, for example, {resolution: "1920x1080", frameRate: 30, format: "H.264"}, and initiates a connection request to the host.

[0081] S3: Refined Control:

[0082] like Figure 2 As shown, after the connection is established, if the user wants to adjust the image quality of the remote camera on the client, the client application will issue a fine-grained control command, such as {command: "set exposure", value: "auto"} (set automatic exposure).

[0083] The instruction is transmitted to the host via the client's distributed engine module.

[0084] The remote camera forwarding unit of the host receives this instruction and forwards it to the camera device management module of the local machine, which ultimately acts on the physical camera hardware.

[0085] S4: Streaming data transmission:

[0086] like Figure 3 As shown, users can preview the application through the client application.

[0087] Image data captured by the host's physical camera is processed by the camera device management module and then sent to the remote camera forwarding unit.

[0088] The streaming media encapsulation module within this unit begins operation, encapsulating the raw image frame data into standard data packets conforming to the RTSP / RTP protocol.

[0089] The encapsulated streaming media data is stably transmitted to the client through the network channel established by the distributed engine module.

[0090] After the client's remote camera device module receives the data, it submits it to its camera device management module, and the application can then display a smooth and clear remote video image.

[0091] The effect of this embodiment: This embodiment fully demonstrates the process from automatic "plug-and-play" device discovery, to flexible resource matching (dynamic negotiation), to professional parameter adjustment (fine-tuning), ultimately achieving high-quality streaming media transmission. The entire process requires no manual configuration of network addresses, and the client can operate the remote camera in all aspects, just like operating a local camera.

[0092] Example 2: Detailed Implementation of the Dynamic Capability Negotiation Mechanism:

[0093] This embodiment focuses on explaining the specific implementation logic of the dynamic capability negotiation mechanism, which is the key to achieving on-demand resource matching.

[0094] The host broadcast capability list includes: [ {res: "1920x1080", fps: 30, fmt: "H.264"}, {res: "1280x720", fps: 60, fmt: "YUV"}].

[0095] Client A, with its high screen resolution and sufficient network bandwidth, successfully requested and established a high-definition video connection in 1920x1080, H.264 format through the negotiation mechanism.

[0096] Client B, due to its high processor load, actively chooses to request a "1280x720, YUV" format connection from the host capability list to avoid decoding pressure, thus ensuring system smoothness.

[0097] Client C only requests connections in "640x480, YUV" format because its application only requires low-resolution images for face detection, even though the host supports higher capabilities.

[0098] The effect of this embodiment: This embodiment demonstrates that the present invention breaks through the limitation of traditional streaming media solutions where clients can only passively receive data. Through dynamic negotiation, different clients can intelligently select the most suitable capability configuration based on their actual needs, processing capabilities, and real-time network conditions, thereby achieving efficient resource utilization and overall system load balancing.

[0099] Example 3: Specific applications of fine-grained control and streaming media encapsulation:

[0100] This embodiment demonstrates how the present invention enables advanced control of a remote camera in a video conferencing scenario.

[0101] The client, as a participant in the video conference, discovered and connected to the host's conference room camera.

[0102] During the meeting, the client user noticed that the screen was too dark, so they adjusted the "exposure compensation" through the application.

[0103] This command {command: "set exposure compensation", value: "+1"} is sent to the host.

[0104] The remote camera forwarding unit of the host not only forwards the instruction, but also translates it into instructions that the camera manufacturer's SDK can understand, such as camera.setProperty(EXPOSURE_BIAS, +1).

[0105] Meanwhile, the streaming media encapsulation module in the remote camera forwarding unit of the host continuously encapsulates video data into standard RTP packets, ensuring perfect compatibility with third-party video conferencing software.

[0106] This embodiment demonstrates that the present invention can not only achieve basic video stream acquisition, but also provide a device-level, refined control experience, and has strong ecosystem integration capabilities with standard streaming media output.

[0107] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for distributed camera interaction supporting auto-discovery and dynamic negotiation, the method comprising: The application is applied to at least two devices in a network, each of which comprises a single-device distributed camera subsystem, the subsystem including a camera device management module, a remote camera module, a remote camera device module, a local camera device module and a distributed engine module, the distributed engine module comprising a device discovery unit; the method comprising the following steps: Device discovery and role declaration, when a first device starts as a host, the device discovery unit in the distributed engine module of the first device periodically sends a broadcast message on the network; when a second device starts as a client, the device discovery unit in the distributed engine module of the second device listens to the broadcast message on the network; after receiving the broadcast message of the first device, the broadcast message information is parsed and obtained, and the first device and the camera capability list in the broadcast message information are presented on the user interface of the second device; Remote camera connection and dynamic negotiation, in response to an operation of selecting a target camera resource on the first device by a user of the second device through an application, the remote camera module of the second device dynamically selects a set of optimal capability configuration parameters from the camera capability list broadcast by the first device according to the requirements of the application, the platform capability of the second device and the network status; the remote camera module of the second device generates a target camera connection request containing the optimal capability configuration parameters, and sends the connection request to the server of the distributed engine module of the first device through the client of the distributed engine module of the second device; the server of the distributed engine module of the first device forwards the connection request to the remote camera module of the first device; the remote camera module of the first device processes the connection request and establishes a connection with the corresponding local camera device on the first device according to the optimal capability configuration parameters; After the connection is successful, the remote camera module of the first device notifies the camera device management module of the first device; at the same time, the connection success response is transmitted to the remote camera module of the second device through the server of the distributed engine module of the first device and the client of the distributed engine module of the second device; The remote camera module of the second device notifies the camera device management module of the second device to virtually generate a remote camera device corresponding to the target camera on the first device in the operating system of the second device according to the received connection success response, and mounts the remote camera device, so that the remote camera device coexists with the local camera device of the second device.

2. A distributed camera system supporting automatic discovery and dynamic negotiation, characterized in that, The system is based on the interactive method of claim 1, comprising a remote camera module, a camera device management module, a remote camera device module, a local camera device module and a distributed engine module; The remote camera module comprises a remote camera forwarding unit and a remote camera connection unit; The camera device management module comprises a remote camera management unit and a local camera management unit; The distributed engine module comprises a client, a server and a device discovery unit.

3. The system of claim 2, wherein, The remote camera forwarding unit is extended with a set of fine control instructions, which is used to respond to and forward the underlying parameter adjustment request of the client to the remote camera, including resolution selection, automatic exposure setting and automatic focusing setting.

4. The system of claim 2, wherein, The remote camera forwarding unit adds a streaming media encapsulation module; when the client requests remote camera data, the host executes the following process: The server sends a data request to the remote camera management unit of the camera device management module; The remote camera management unit calls the local camera device module to collect raw image data from the physical camera hardware; The local camera device module transmits the data to the remote camera forwarding unit of the host; The streaming media encapsulation module encapsulates the raw image data into a standard streaming media protocol format and transmits it to the client through the distributed engine module.

5. The system of claim 2, wherein, The remote camera connection unit adds a dynamic capability negotiation mechanism, and the process includes: Before initiating the connection, the client obtains the camera capability list of the host by analyzing the broadcast packet of the device discovery unit; The client dynamically selects and requests a set of optimal configuration parameters from the capability list according to the application requirements and platform processing capabilities.

6. The system of claim 5, wherein, The data structure of the broadcast packet at least includes: Device ID; Camera capability list, including device supported resolution, frame rate and image format parameters; Device role information, used to declare the device as a host or client.

Images