Distributed multi-lens camera system and host service communication control method thereof

Abstract

The invention discloses a distributed multi-lens camera system and a host service communication control method thereof.The distributed multi-lens camera system comprises a main control computer system and a camera control system, and the main control computer module comprises a multi-language SDK module, a module scheduling module, a communication transmission module, an image processing module and a local storage module; the camera control system comprises a camera control module, communication transmission modules, a local cache module and an image preprocessing module, and the two communication transmission modules are connected. According to the distributed multi-lens camera system and the host service communication control method thereof, the system deployment and maintenance complexity is greatly reduced through the modular and multi-lens design, PoE + power supply simplifies wiring, each module only needs one network cable, hot plug and dynamic extension are supported, different scales of scenes are adapted, efficient communication is achieved, image synchronization is optimized, and the system performance is improved. Three-dimensional reconstruction is facilitated, a high-degree-of-freedom software interface improves the system integration efficiency, and rapid development and deployment are supported.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera systems, in particular to a distributed multi-lens camera system and a host service communication control method thereof. BACKGROUND

[0002] Currently, in three-dimensional scene modeling, panoramic acquisition, human body modeling and other scenes, multiple industrial cameras (such as Hikvision robots, Basler brand, etc.) are often used to surround the target to build a camera array. Such systems have good resolution and imaging quality, but usually have the following problems: high cost, each industrial camera has a high unit price, and dozens of cameras are needed to build a system; complex system deployment, mechanical structure, wiring, power supply and control all require professional engineers; lack of modular control, each camera needs to be connected to the host or use multiple acquisition cards and switches; it is difficult to synchronize shooting and unified scheduling, and there is a lack of open and high degree of freedom communication interface. In addition, current multi-lens modules (such as Ricoh GXR cameras and smartphone modules) only achieve small modular packaging in hardware structure and do not involve complete control systems and networking capabilities. With the continuous development of science and technology, people's requirements for camera systems are also getting higher and higher.

[0003] The existing camera system has certain drawbacks when in use, lacks modular control architecture, and has poor system expandability: the existing system usually adopts a centralized connection method, lacks a unified modular design, resulting in cumbersome wiring and control structure when the scale is expanded, and difficult maintenance; the granularity of the camera unit is too fine, and the deployment efficiency is low: the single-camera access method increases the complexity of wiring, power supply and configuration, which is not conducive to rapid deployment and flexible adaptation to scenes; the control method is closed and lacks flexible scheduling mechanism: some systems use manufacturer proprietary protocols or fixed control logic, which limits user-defined operations and upper system integration capabilities; interface standards are missing, and compatibility and expandability are insufficient: the protocols between systems are ambiguous and not unified, lack of open interface design, and difficult to seamlessly connect with existing software platforms or devices; the overall cost is high, and the investment in system expansion is large: industrial-grade devices and too fine granularity result in high unit cost, and large capital demand for system upgrade and horizontal expansion. Therefore, we propose a distributed multi-lens camera system and a host service communication control method thereof. SUMMARY

[0004] The technical problem solved by the present application is that, in view of the deficiencies of the prior art, the present application provides a distributed multi-lens camera system and a host service communication control method thereof, which greatly reduces the system deployment and maintenance complexity with modularization and multi-lens design, simplifies wiring with PoE+ power supply, supports hot plugging and dynamic expansion, adapts to different scale scenes, optimizes image synchronization with efficient communication, is beneficial to three-dimensional reconstruction, improves system integration efficiency with high degree of freedom software interface, and supports rapid development and deployment, which can effectively solve the problems in the background art.

[0005] The technical scheme is as follows: a distributed multi-lens camera system, comprising a host computer system and a camera control system, the host computer module comprising a multi-language SDK module, a module scheduling module, a communication transmission module, an image processing module and a local storage module, the camera control system comprising a camera control module, a communication transmission module, a local cache module and an image preprocessing module, the two communication transmission modules being connected, the communication transmission module comprising UDP, gRPC and RTSP, the UDP being a user datagram protocol, the gRPC being a remote procedure call protocol, and the RTSP being a real-time streaming protocol, the multi-language SDK module being called by a UI, the camera control module comprising V4L2 and OpenCV, the V4L2 being a driver framework of a kernel video device, and the OpenCV being a computer vision and machine learning software library, the multi-language SDK module being connected with a user interaction interface and calling an SDK to send instructions, and the SDK being a software development kit.

[0006] As a preferred technical scheme of the present application, the host computer system comprises a host computer, the host computer is connected with a POE network switch, the POE network switch is connected with a Raspberry Pi 5 single-board computer, the Raspberry Pi 5 single-board computer comprises a POE module and a host control module, and the host control module is connected with multiple groups of image sensors and cameras.

[0007] As a preferred technical scheme of the present application, the multi-language SDK module is bidirectionally signal-connected with the module scheduling module and the image processing module, the output end of the module scheduling module is signal-connected with the input end of the local storage module through the communication transmission module, the image processing module is bidirectionally signal-connected with the local storage module, the output end of the camera control module is signal-connected with the input end of the local cache module, and the local cache module is bidirectionally signal-connected with the image preprocessing module.

[0008] As a preferred technical scheme of the present application, the host computer is bidirectionally signal-connected with the POE network switch, the POE network switch is bidirectionally signal-connected with the Raspberry Pi 5 single-board computer, and the host control module is bidirectionally signal-connected with the multiple groups of image sensors and cameras.

[0009] As a preferred technical scheme of the present application, when the communication transmission module is gRPC, the user interaction interface clicks a function button and provides interface feedback, the host computer system constructs a gRPC request and processes accordingly, and the camera control system accepts instructions, executes instructions and returns responses.

[0010] As a preferred technical solution of the application, when the communication transmission module is UDP, the camera control system constructs a UDP broadcast, the host computer system listens to the UDP package, constructs a gRPC request, registers the module information and writes into the local configuration, and the user interaction interface displays the module list.

[0011] As a preferred technical solution of the application, when the communication transmission module is RTSP, the camera control system accepts the gRPC instruction, starts the RTSP service, the host computer system starts to pull the stream, the user interaction interface requests, the host computer system stops pulling the stream, and the camera control system accepts the gRPC instruction and stops the RTSP service.

[0012] A host service communication control method of a distributed multi-lens camera system, specifically comprising the following operation steps: S1: a gRPC server is run in the control unit to listen to remote instructions from the center host; S2: each module can connect information through a UDP local area network broadcast according to its serial number, realize dynamic discovery and centralized scheduling; S3: a synchronous control mechanism, the center controller can broadcast a synchronization instruction to multiple modules at the same time, the modules respond synchronously according to the set delay, ensure that the acquisition time error is controlled within ±10ms, support timestamp recording and result file uploading; S4: integration of a software development kit.

[0013] As a preferred technical solution of the application, the S2 step supports a photographing instruction, parameter configuration, heartbeat feedback and file transmission command set, and the center control machine runs a gRPC client program, which can be embedded into a client interface program.

[0014] As a preferred technical solution of the application, the S4 step provides a complete multi-language software interface for customizing a client and integrating a graphical interface, the interface encapsulates module management, command sending, image receiving and event notification functions, and is connected to an industrial application.

[0015] Beneficial effects: compared with the prior art, the application provides a distributed multi-lens camera system and a host service communication control method thereof, which has the following beneficial effects: the modularization and multi-lens design of the distributed multi-lens camera system and the host service communication control method thereof greatly reduce the system deployment and maintenance complexity, PoE+ power supply simplifies wiring, each module only needs one network cable, supports hot plug and dynamic expansion, adapts to different scale scenes, optimizes image synchronization through efficient communication, is beneficial to three-dimensional reconstruction, a high-degree software interface improves system integration efficiency, supports rapid development and deployment, the entire camera system structure is simple, convenient to operate, and the use effect is better than that of the traditional mode. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The whole structure schematic diagram of a distributed multi-lens camera system and a host service communication control method thereof.

[0017] Figure 2 The host computer schematic diagram in a distributed multi-lens camera system and a host service communication control method thereof.

[0018] Figure 3 The gRPC communication flow schematic diagram in a distributed multi-lens camera system and a host service communication control method thereof.

[0019] Figure 4 The UDP communication flow schematic diagram in a distributed multi-lens camera system and a host service communication control method thereof.

[0020] Figure 5 The RTSP communication flow schematic diagram in a distributed multi-lens camera system and a host service communication control method thereof. DETAILED DESCRIPTION

[0021] The technical solutions of the present application will be described clearly and completely in the following combined with the drawings and specific embodiments, but those skilled in the art will understand that the following described embodiments are part of the embodiments of the present application, not all the embodiments, and should not be regarded as limiting the scope of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application. The specific conditions are not specified in the embodiments, which are carried out according to the conventional conditions or the conditions recommended by the manufacturer. The reagents or instruments used are not specified by the manufacturer, which are conventional products that can be purchased on the market.

[0022] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0023] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0024] As shown in Figures 1-5 A distributed multi-lens camera system, including a host computer system and a camera control system, the host computer module includes a multi-language SDK module, a module scheduling module, a communication transmission module, an image processing module and a local storage module, the camera control system includes a camera control module, a communication transmission module, a local cache module and an image preprocessing module, the two groups of communication transmission modules are connected, the communication transmission module includes UDP, gRPC and RTSP, UDP is a user datagram protocol, gRPC is a remote procedure call protocol, RTSP is a real-time streaming protocol, the multi-language SDK module is called by UI, the camera control module includes V4L2 and OpenCV, V4L2 is a driver framework of kernel video device, OpenCV is a computer vision and machine learning software library, the multi-language SDK module connects the user interface and calls the SDK to send instructions, the SDK is a software development kit, the modularization and multi-lens design greatly reduce the system deployment and maintenance complexity, PoE+ power supply simplifies wiring, each module only needs one network cable, supports hot plug and dynamic expansion, adapts to different scale scenarios, efficient communication optimizes image synchronization, is beneficial to three-dimensional reconstruction, high degree of freedom software interface improves system integration efficiency, and supports rapid development and deployment.

[0025] Further, the host computer system includes a host computer, the host computer is connected with a POE network switch, the POE network switch is connected with a Raspberry Pi 5 single board computer, the Raspberry Pi 5 single board computer includes a POE module and a host module, the host module is connected with multiple groups of image sensors and cameras.

[0026] Further, the multi-language SDK module is bidirectionally signal connected with the module scheduling module and the image processing module, the output end of the module scheduling module is signal connected with the input end of the local storage module through the communication transmission module, the image processing module is bidirectionally signal connected with the local storage module, the output end of the camera control module is signal connected with the input end of the local cache module, and the local cache module is bidirectionally signal connected with the image preprocessing module.

[0027] Further, the main control computer is connected with the POE network switch in a bidirectional signal connection, the POE network switch is connected with the Raspberry Pi 5 single-board computer in a bidirectional signal connection, and the main control module is connected with multiple groups of image sensors and cameras in a bidirectional signal connection.

[0028] Further, when the communication transmission module is gRPC, the user interactive interface clicks a function button and provides interface feedback, the main control system constructs a gRPC request and processes the corresponding, the camera control system accepts the instruction, executes the instruction and returns a response.

[0029] Further, when the communication transmission module is UDP, the camera control system constructs a UDP broadcast, the main control system listens to a UDP packet, constructs a gRPC request, registers module information and writes the information into a local configuration, and the user interactive interface displays a module list.

[0030] Further, when the communication transmission module is RTSP, the camera control system accepts a gRPC instruction, starts an RTSP service, the main control system starts to pull a stream, the user interactive interface requests, the main control system stops pulling the stream, and the camera control system accepts a gRPC instruction and stops the RTSP service.

[0031] Module structure: Each module is controlled by a control unit (such as Raspberry Pi 5) and multiple industrial cameras (USB Video Class protocol, usually 4-6) connected through USB 3.0; The control unit is internally integrated with a central processing unit, a USB controller, a power management circuit and a network control chip, and the whole module is packaged into a unified shell.

[0032] All modules are connected to the center main control machine through a gigabit Ethernet and adopt a PoE+ single-wire power supply mode.

[0033] A host service communication control method of a distributed multi-lens camera system, specifically comprising the following operation steps: 1. A gRPC server is run in the control unit to listen to remote instructions from the center host 2. Each module can broadcast connection information through a UDP local area network according to its own serial number, to realize dynamic discovery and centralized scheduling; (1) Support for photographing instructions, parameter configuration, heartbeat feedback, file transmission and other command sets; (2) The center control machine runs a gRPC client program, which can be embedded into a client or other upper-layer graphical interface program.

[0034] 3. Synchronous control mechanism: (1) The center controller can broadcast synchronization instructions to multiple modules at the same time; (2) The module responds according to the set delay synchronization, and ensures that the acquisition time error is controlled within ±10ms; (3) Supports timestamp recording and result file uploading.

[0035] 4. SDK integration: (1) The system provides complete multilingual software interfaces for customizing clients and integrating graphical interfaces; (2) The interface encapsulates module management, command sending, image receiving, event notification, and other functions, facilitating industrial application access. Embodiment

[0036] Take Raspberry Pi 5 as the controller, connect four UVC industrial cameras to form a module, and deploy as follows: The Raspberry Pi is connected to the USB HUB and the camera through USB3.0; Use PoE+ expansion module to realize native network port power supply of Raspberry Pi, and connect gigabit Ethernet at the same time; Run the gRPC service program written in Python internally to listen to the center host command; The center control PC runs the back-end and image interface program, accesses the software interface, and can send "take a picture", "set parameters", and "transfer pictures" instructions; The module receives and caches the local images to be uploaded after acquisition; Multiple modules can respond synchronously to realize low-delay image acquisition.

[0037] Working principle: The gRPC server runs in the control unit to listen to remote instructions from the center host; each module can broadcast connection information through UDP local area network according to its serial number, realize dynamic discovery and centralized scheduling; synchronous control mechanism, the center controller can broadcast synchronous instructions to multiple modules at the same time, the module responds according to the set delay synchronization, ensures that the acquisition time error is controlled within ±10ms, supports timestamp recording and result file uploading; software development kit integration, modularization and multi-lens design greatly reduce the complexity of system deployment and maintenance, PoE+ power supply simplifies wiring, each module only needs one network cable, supports hot plug and dynamic expansion, adapts to different scale scenarios, optimizes image synchronization with efficient communication, is beneficial to three-dimensional reconstruction, high degree of freedom software interface improves system integration efficiency, supports rapid development and deployment.

[0038] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other present or future devices possess. That is, although specific embodiments have been disclosed herein, one of ordinary skill in the art will appreciate that other embodiments can be practiced under the scope of the disclosure. For example, some well-known structures have not been described in detail or at all in order to avoid obscuring the concepts of the present application. Like reference numerals can be used to denote like elements throughout the description and the drawings.

[0039] The foregoing is considered as illustrative only of the principles of the application. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the application to the exact construction and practice described. Accordingly, all such variations are intended to be included within the scope of the present application as defined in the following claims.

Claims

1. A distributed multi-lens camera system comprising a master control system and a camera control system, characterized in that: The host computer module includes a multi-language SDK module, a module scheduling module, a communication transmission module, an image processing module and a local storage module, the camera control system includes a camera control module, a communication transmission module, a local cache module and an image preprocessing module, the two groups of communication transmission modules are connected, the communication transmission module includes UDP, gRPC and RTSP, the UDP is a user datagram protocol, the gRPC is a remote procedure call protocol, the RTSP is a real-time streaming protocol, the multi-language SDK module is called by UI, the camera control module includes V4L2 and OpenCV, the V4L2 is a driver framework of kernel video device, the OpenCV is a computer vision and machine learning software library, the multi-language SDK module is connected with a user interface and calls an SDK to send instructions, and the SDK is a software development kit.

2. The distributed multi-lens camera system of claim 1, wherein: The host computer system includes a host computer, the host computer is connected with a POE network switch, the POE network switch is connected with a Raspberry Pi 5 single board computer, the Raspberry Pi 5 single board computer includes a POE module and a host module, and the host module is connected with multiple groups of image sensors and cameras.

3. The distributed multi-lens camera system of claim 1, wherein: The multi-language SDK module is bidirectionally and signal-connected with the module scheduling module and the image processing module, the output end of the module scheduling module is signal-connected with the input end of the local storage module through the communication transmission module, the image processing module is bidirectionally and signal-connected with the local storage module, the output end of the camera control module is signal-connected with the input end of the local cache module, and the local cache module is bidirectionally and signal-connected with the image preprocessing module.

4. The distributed multi-lens camera system of claim 2, wherein: The host computer is bidirectionally and signal-connected with the POE network switch, the POE network switch is bidirectionally and signal-connected with the Raspberry Pi 5 single board computer, and the host module is bidirectionally and signal-connected with multiple groups of image sensors and cameras.

5. The distributed multi-lens camera system of claim 1, wherein: When the communication transmission module is gRPC, the user interface clicks a function button and provides interface feedback, the host computer system constructs a gRPC request and processes accordingly, and the camera control system accepts instructions, executes instructions and returns responses.

6. The distributed multi-lens camera system of claim 1, wherein: When the communication transmission module is UDP, the camera control system constructs a UDP broadcast, the host computer system listens to a UDP package, constructs a gRPC request, registers module information and writes in a local configuration, and the user interface displays a module list.

7. The distributed multi-lens camera system of claim 1, wherein: When the communication transmission module is RTSP, the camera control system accepts a gRPC instruction, starts an RTSP service, the host computer system starts streaming, the user interface requests, the host computer system stops streaming, and the camera control system accepts a gRPC instruction and stops the RTSP service. 8.A method for controlling communication of a host service of a distributed multi-lens camera system, the method comprising: Specifically includes the following operation steps: S1: a control unit runs a gRPC server to listen to remote instructions from a central host; S2: each module can broadcast connection information through a UDP local area network according to a serial number, to realize dynamic discovery and centralized scheduling; S3: Synchronization control mechanism, the central controller can broadcast synchronization instructions to multiple modules at the same time, and the modules respond synchronously according to the set delay, ensuring that the acquisition time error is controlled within ±10ms, supporting timestamp recording and result file uploading; S4: Software development kit integration. 9.The method of claim 8, wherein the method further comprises: receiving, by the host server, a request for a service from the plurality of client devices; and providing, by the host server, the requested service to the plurality of client devices. The S2 step supports a set of commands including photographing instructions, parameter configuration, heartbeat feedback, and file transmission, and the central control machine runs a gRPC client program, which can be embedded into a client interface program. 10.The method of claim 8, wherein the method further comprises: receiving a request for a service from the plurality of slave cameras; and transmitting a response to the request for the service to the plurality of slave cameras. In the S4 step, the system provides complete multilingual software interfaces for customizing clients and integrating graphical interfaces, and the interfaces encapsulate module management, command sending, image receiving, and event notification functions, and are connected to industrial applications.

Images