A multi-camera system based on high impedance switching and method thereof

Abstract

The application provides a multi-camera system based on high-impedance switches and a method thereof, comprising: an SOC master control platform for controlling and managing data streams of multiple image sensors; the multiple image sensors are connected with the SOC master control platform through an MIPI shared bus; an I2C instruction control pin module for controlling switch flow operations of the image sensors through I2C instructions; a synchronization mechanism module for realizing synchronization operations of the multiple image sensors through a synchronization signal VSYNC and a clock signal Extclk; and ensuring that the multiple image sensors remain synchronized when outputting data in time-sharing rotation. The system hardware complexity is significantly reduced, the hardware failure rate is reduced, the instruction code timing control logic is used to replace an MIPI switch chip, the hardware cost of the multi-camera system is reduced, the performance is maintained, the architecture based on CMOS sensor native register control can be adapted to camera modules of different manufacturers, and the system expansion flexibility is improved.

Description

Technical Field

[0001] This invention belongs to the field of Linux embedded devices in multi-view photography, and specifically relates to a multi-camera system and method based on a high-impedance switch. Background Technology

[0002] In recent years, with the development and iteration of Linux embedded system electronic device technology in the fields of photography and videography, people have placed higher demands on high-resolution and multi-view images. High resolution and multi-view images enable clearer and more multi-angle detection of real-world scenes, thus providing better environmental detection performance for audio and video equipment. Data transmission between high-resolution image sensors and host computers requires a high-speed and interference-resistant MIPI data transmission protocol. Multi-view images refer to increasing the number of image sensors to capture images from different angles, thereby achieving the goal of multiple perspectives. Traditional multi-camera methods using the MIPI transmission protocol rely on a MIPI switch chip to connect two cameras. The host computer controls the MIPI bit selection pins to control the alternating transmission of image data from the two sensors connected to the MIPI (Mobile Industry Processor Interface) switch chip for time-division processing. Traditional multi-camera solutions rely on the data chip selection function of the MIPI switch chip to achieve the goal of multiple cameras. Generally, one MIPI switch chip can only connect two cameras, so the number of MIPI switch chips is N / 2+1 times the number of cameras. Adding more cameras also increases the cost of the MIPI switch chip.

[0003] Traditional multi-camera solutions utilize the data selection function of MIPI switch chips. MIPI switch chips are used to switch between multiple MIPI CSI-2 (Camera Serial Interface 2) interfaces, allowing the system to flexibly select data sources from different cameras or sensors. These chips are widely used in multi-camera solutions, such as smartphones, security monitoring equipment, and autonomous vehicles. The following are the working principles and some application scenarios of MIPI switch chips:

[0004] 1. MIPI CSI-2 protocol

[0005] MIPI CSI-2 is a high-speed serial interface standard used to connect image sensors (such as cameras) and processors (such as APUs and SoCs). It supports multi-lane transmission, with each lane operating independently, enabling high-bandwidth data transmission. A typical MIPI CSI-2 interface includes data lines (D0-D3) and a clock line (CLK), through which image data is transmitted.

[0006] 2. Functions of the MIPI Switch

[0007] Multiple Input / Output: The MIPI Switch chip can receive inputs from multiple MIPI CSI-2 interfaces and switch them to one or more output interfaces. This allows the system to select different cameras or sensors as data sources as needed.

[0008] Low-latency switching: The MIPI Switch chip is designed for fast switching, ensuring minimal data loss and latency during the switching process, which is crucial for real-time video streaming.

[0009] Transparent transmission: Under normal operating conditions, the MIPI switch chip is transparent to the data stream, meaning it does not modify or interfere with the data in transmission, and is only responsible for routing and switching.

[0010] Power Management: Some advanced MIPI switch chips also have power management capabilities, which can dynamically adjust power consumption based on the currently used channel to save energy.

[0011] 3. Work Mode

[0012] Bypass Mode: In this mode, the MIPI switch chip directly connects an input to the output without any processing or conversion. This is suitable for situations where a single camera or sensor is always the primary data source.

[0013] Switch Mode: In this mode, the MIPI switch chip selects different input sources based on external control signals (such as GPIO pins or I2C commands) and switches them to the output. This mode is suitable for multi-camera solutions, allowing the system to flexibly switch between different cameras. Multiplexing Mode: In this mode, the MIPI switch chip can simultaneously receive data from multiple input sources and multiplex them onto a single output channel. This method is suitable for applications that require merging data from multiple cameras.

[0014] 4. Control Method

[0015] Hardware control: The switching state of the MIPI switch chip is directly controlled via GPIO pins or other hardware signals. This method has a fast response time but lower flexibility.

[0016] Software control: The switching state of the MIPI switch chip is controlled by commands sent by the master control chip through communication interfaces such as I2C and SPI. This method is highly flexible and can dynamically adjust the switching logic according to the application's needs.

[0017] 5. Application Scenarios

[0018] Multi-camera phones: Modern smartphones are typically equipped with multiple cameras (such as wide-angle, telephoto, depth sensing, etc.). The MIPISwitch chip can quickly switch between these cameras, ensuring that users can seamlessly switch shooting modes.

[0019] Security monitoring: In multi-camera monitoring systems, the MIPI Switch chip can flexibly select cameras in different locations as data sources, improving the system's flexibility and reliability.

[0020] Autonomous driving: Autonomous vehicles are typically equipped with multiple cameras for environmental perception. The MIPI Switch chip can help the system quickly switch between different cameras to ensure that the most appropriate visual information is obtained in real time.

[0021] However, the existing technical solutions have the following drawbacks:

[0022] Traditional multi-camera solutions generally use MIPI switch chips to implement data chip selection. The number of chips increases by N / 2+1 times as the number of cameras increases, which not only increases the system complexity but also significantly increases the hardware cost. Summary of the Invention

[0023] To address the aforementioned issues, the purpose of this application is to provide a multi-camera method and system based on a high-impedance switch to reduce the hardware cost of the MIPI Switch chip. This method eliminates the hardware logic of the MIPI Switch chip and uses timing control logic based on I2C instruction codes to control the high-impedance switch register and outgoing current switch (PWDN) register built into the CMOS camera module, thereby achieving the data chip select function of the MIPI Switch chip.

[0024] This invention relates to Linux-embedded electronic devices used in multi-view photography and videography. Its core lies in controlling the high-impedance switch of a CMOS sensor via I2C command code through a main control platform, thereby replacing the hardware logic of a traditional MIPI switch chip and achieving an equivalent MIPI data chip select function.

[0025] Specifically, the present invention provides a multi-camera system based on a high-impedance switch, the system comprising:

[0026] SOC main control platform: used to control and manage the data streams of multiple image sensors; including external clock signal source Extclk, MIPI bus, and I2C instruction control pins;

[0027] Multiple image sensors: Multiple image sensors are connected to the SOC main control platform via a shared MIPI bus; including the main camera denoted as Master, secondary camera 0 denoted as Slaver0, and secondary camera 1 denoted as Slaver1;

[0028] MIPI bus: Used to transmit data streams from image sensors;

[0029] I2C instruction control pin module: used to control the switching operation of the image sensor via I2C instructions; including I2C addresses 0x80, 0x88 and 0x8c, corresponding to the main camera, secondary camera 0 and secondary camera 1 respectively;

[0030] Synchronization mechanism module: It realizes the synchronous operation of multiple image sensors through the synchronization signal VSYNC and the clock signal Extclk; and ensures that multiple image sensors maintain synchronization when outputting data in turn in a time-sharing manner.

[0031] The SOC main control platform further includes:

[0032] First, the main control platform controls the high-impedance switch register and outflow switch register of the CMOS sensor through I2C instructions, so that the data from the three cameras are output in turn in a time-sharing manner;

[0033] Secondly, the main control platform receives image data of the current scene in real time from the CMOS sensor camera and performs preprocessing on the image in a timely manner. The preprocessing includes optimization operations such as black level correction, demosaic, automatic white balance, automatic exposure, distortion correction, noise reduction, color space conversion, and sharpening. Then, the CMOS sensor camera is controlled in response to the image processing effect until the image achieves the expected effect.

[0034] Finally, after image processing is completed, the image data is encoded and compressed or displayed locally on the screen; the plurality of image sensors, i.e., CMOS sensors, further include:

[0035] The CMOS sensor is mainly responsible for acquiring image data of the current scene in real time. Its image data is transmitted to the image processing unit (ISP) in RAW format via the MIPI data transmission protocol for preprocessing. Assuming there are three CMOS sensors, i.e. three camera modules, they are labeled as Master, Slaver0, and Slaver1 respectively for easy differentiation.

[0036] The MIPI bus further includes:

[0037] MIPI is responsible for image data transmission between the CMOS sensor and the SOC main control platform. The three cameras and the SOC main control platform share a MIPI bus.

[0038] The synchronization mechanism module, i.e., the control synchronization signal, further includes:

[0039] The synchronization signal is a signal initiated by the main camera and simultaneously transmitted to the other secondary cameras. The synchronization signal lines of the three CMOS sensors are all connected to the synchronization signal VSYNC pin of the main camera. Using the synchronization signal as a reference and taking advantage of the controllability of sensor delay frame output, the MIPI data of the three CMOS sensors are output asynchronously in turn.

[0040] The clock signal is provided by the main control platform for the clock signal sources of the three CMOS sensors, that is, they are all connected to the clock pin of the main control, thus avoiding errors caused by clock signal differences;

[0041] The I2C command control pin module, i.e., the I2C control bus, further includes:

[0042] The configuration registers, high-impedance registers, and outflow registers of the three CMOS sensors are all controlled by the master controller via the I2C bus. At the same time, the three CMOS sensors need to have different I2C addresses. For example, the Master address is 0x80, the Slaver0 address is 0x88, and the Slaver1 address is 0x8c, in order to achieve the requirements of precise control.

[0043] The SOC main control platform controls the switching operation of the image sensor via I2C commands, specifically:

[0044] When the I2C address is 0x80, the switching flow of the main camera (Master) is controlled.

[0045] When the I2C address is 0x88, the switching flow of the secondary camera 0, i.e., Slaver0, is controlled.

[0046] When the I2C address is 0x8c, the switching flow of the secondary camera 1 (Slaver1) is controlled.

[0047] The MIPI bus uses the synchronization signal VSYNC and the clock signal Extclk to synchronize the operation of multiple image sensors, ensuring that only one image sensor outputs data at a time.

[0048] The MIPI bus performs data timing control, including the relationship between the data waveform of the MIPI bus composed of three camera systems and the I2C control logic, which consists of the following three parts: (1) Frame synchronization signal timing: The frame synchronization signal is issued by the Master. When Slaver0 and Slaver1 receive the falling edge of this signal, the sensor will be triggered to expose the frame. By utilizing the controllable characteristics of the sensor's delayed frame output, the MIPI data of the three CMOS sensors can be output asynchronously in turn. In order to control the three cameras to output 10 frames of data asynchronously in one cycle, the cycle of the synchronization signal needs to be controlled at 33.3ms to achieve the goal of each of the three cameras outputting 10 frames of data.

[0049] (2) MIPI bus data waveform: Three cameras share one MIPI bus. In order to ensure that only one sensor MIPI pin is connected to the main control MIPI pin at the same time, the CMOS sensor must have the function of high impedance switching of MIPI data, that is, to play the function of closing and opening the connection between the MIPI line and the main control.

[0050] (3) I2C instruction control timing: In the initial stage, the three sensors need to be initialized in sequence. After that, when the master controller detects the Frame done signal of the MIPI bus, it will trigger the hardware interrupt handling thread of Frame done. This thread will control the values ​​of bits 0x74[1] and 0x6c[7] so that Master, Slaver0 and Slaver1 will output MIPI data in turn in a time-sharing manner.

[0051] The selected CMOS sensor is model JXH63P. This sensor has a register for controlling the high-impedance switch. Below are the relevant control parameters for the high-impedance switch of model JXH63P:

[0052]

[0053] The system uses a high-impedance switch to enable time-sharing output from multiple image sensors, ensuring that the data streams of the MIPI bus do not conflict.

[0054] The system also includes a Framed hardware interrupt service routine and a Framed kernel processing thread, which are used to implement time-sharing output control of multiple image sensors through the synchronization mechanism semaphore fd_comp.

[0055] This application also relates to a multi-camera method based on a high-impedance switch, the method being applicable to any of the systems described above, the method comprising the following steps:

[0056] S1: Initialization Configuration:

[0057] Multiple CMOS sensors are initialized and configured via the I2C bus to enable them to function properly, and the variable Cmun is initialized to 0.

[0058] S2: Create a synchronization mechanism semaphore:

[0059] Create a semaphore fd_comp of type completion structure so that the main control platform's Frame done hardware interrupt service routine can trigger the semaphore fd_com, thereby starting the Frame done kernel processing thread;

[0060] S3: Create a Frame done kernel processing thread:

[0061] Create and run the Framedone kernel processing thread. When the master detects the Framedone signal of the MIPI bus, it wakes up the thread through the completion synchronization mechanism. Then, the thread will control the values ​​of bits 0x74[1] and 0x6c[7] of the I2C instruction to make Master, Slaver0 and Slaver1 output MIPI data in a time-sharing manner.

[0062] S4: Start the main camera to output the first frame: Start the Master sensor to output the first frame of data;

[0063] S5: Trigger the kernel processing thread: The Frame done kernel processing thread switches the three cameras to output in turn during the time-sharing period by judging the remainder value of Cmun%3, and increments Cmun by 1; the control of the time-sharing output includes:

[0064] Based on the value of Cmun%3, the switching operations of Master, Slaver0, and Slaver1 are controlled sequentially: when Cmun%3 == 0, Slaver0 and Slaver1 are turned off, and Master is turned on;

[0065] When Cmun%3 == 1, shut down Master and Slaver1, and turn on Slaver0;

[0066] When Cmun%3 == 2, shut down Master and Slaver0, and turn on Slaver1;

[0067] After each operation, the value of Cmun is incremented by 1;

[0068] S6: Cyclic Switching: Repeat step S5 to cycle through the sensor output based on the value of Cmun%3, and repeat this cycle until the application exits and terminates the cyclic switching operation.

[0069] The values ​​of I2C instruction control registers 0x74 and 0x6c are used to control the switching current operation of the sensor, specifically:

[0070] When the value of 0x74 is 0x03 and the value of 0x6c is 0xc0, the data stream of the corresponding sensor is turned off; when the value of 0x74 is 0x02 and the value of 0x6c is 0x40, the data stream of the corresponding sensor is turned on.

[0071] Therefore, the multi-camera method and system based on high-impedance switches proposed in this scheme have the following innovative advantages compared with traditional multi-camera schemes:

[0072] (1) Improved hardware reliability: By eliminating the dependence on the hardware logic of the MIPI Switch chip, the system hardware complexity is significantly reduced, thereby reducing the hardware failure rate.

[0073] (2) Cost structure optimization: Innovatively adopting instruction code timing control logic to replace the dedicated MIPI switch chip, the hardware cost of the multi-camera system is reduced, while maintaining the MIPI data chip selection performance equivalent to the hardware solution;

[0074] (3) Enhanced system compatibility: The design architecture based on the native register control of CMOS sensor can be adapted to camera modules from different manufacturers, improving the system's expansion flexibility. Attached Figure Description

[0075] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0076] Figure 1 This is a schematic diagram of the system architecture of this application.

[0077] Figure 2 This is a schematic diagram of MIPI bus data timing control in this application.

[0078] Figure 3 This is a flowchart illustrating the method used in this application.

[0079] Figure 4 This is a schematic diagram of the main pseudocode implementation of the method in this application. Detailed Implementation

[0080] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0081] To effectively reduce the hardware cost of MIPI switch chips, this invention innovatively proposes a multi-camera method and system based on high-impedance switches. This solution abandons the traditional hardware logic of MIPI switch chips and instead employs I2C instruction code timing control logic. By precisely controlling the high-impedance switch register and outgoing current switch register of the CMOS camera module, it achieves a data chip select function equivalent to that of a MIPI switch chip, thereby significantly reducing hardware costs while ensuring system performance.

[0082] This method and system can be widely applied to Linux-embedded electronic devices in the fields of multi-view photography and videography. Its core lies in controlling the high-impedance switch of the CMOS sensor via I2C command code through the main control platform, thereby replacing the hardware logic of the traditional MIPI switch chip and achieving an equivalent MIPI data chip select function.

[0083] This invention uses a triple-camera system as an example to illustrate a multi-CMOS camera method based on a high-impedance switch, and the system architecture. Figure 1 This demonstrates well the composition of a SOC main control platform with three camera systems and the relationships between the modules, including:

[0084] (1) SOC main control platform: used to control and manage the data streams of multiple image sensors;

[0085] This includes an external clock signal source (Extclk), a MIPI bus, and I2C instruction control pins. Furthermore: First, the main control platform uses I2C instructions to control the high-impedance switch register and the outgoing current switch register of the CMOS sensor, enabling the data from the three cameras to be output in a time-division multiplexing manner.

[0086] Secondly, the main control platform receives image data of the current scene in real time from the CMOS sensor camera and performs preprocessing on the image in a timely manner. The preprocessing includes optimization operations such as black level correction, demosaic, automatic white balance, automatic exposure, distortion correction, noise reduction, color space conversion, and sharpening. Then, the CMOS sensor camera is controlled in response to the image processing effect until the image achieves the expected effect.

[0087] Finally, after image processing is completed, the image data is encoded and compressed or displayed locally on the screen. (2) Multiple image sensors, i.e., CMOS sensors: Multiple image sensors are connected to the SOC main control platform through a MIPI shared bus;

[0088] This includes the main camera (Master), secondary camera 0 (Slaver0), and secondary camera 1 (Slaver1). Further:

[0089] The CMOS sensor is primarily responsible for acquiring image data of the current scene in real time. This image data is transmitted in RAW format via the MIPI data transmission protocol to the image processing unit (ISP) for preprocessing. (System architecture) Figure 1 Three CMOS sensors, or three camera modules, were displayed, labeled Master, Slaver0, and Slaver1 for easy differentiation.

[0090] (3) MIPI bus: used to transmit data streams from image sensors;

[0091] MIPI is responsible for image data transmission between the CMOS sensor and the SOC main controller. System architecture Figure 1 The demonstration showed that three cameras and the SOC main control platform shared a single MIPI bus.

[0092] (4) Synchronization mechanism module, i.e., control synchronization signal: Synchronization of multiple image sensors is achieved through the synchronization signal VSYNC and the clock signal Extclk; ensuring that multiple image sensors maintain synchronization when outputting data in a time-sharing manner. Further:

[0093] The synchronization signal is initiated by the master camera and simultaneously transmitted to the other secondary cameras. The synchronization signal lines of all three CMOS sensors are connected to the master camera's synchronization signal (VSYNC) pin. Using the synchronization signal as a reference and leveraging the controllable nature of sensor frame delay, the MIPI data from the three CMOS sensors are output asynchronously and in turn.

[0094] The clock signals for the three CMOS sensors are all provided by the main control platform, meaning they are all connected to the clock pin of the main control unit, thus avoiding errors caused by clock signal differences.

[0095] (5) I2C instruction control pin module, i.e., I2C control bus: used to control the switching operation of the image sensor via the I2C protocol; including I2C addresses 0x80, 0x88, and 0x8c, corresponding to the main camera, secondary camera 0, and secondary camera 1, respectively. Further:

[0096] The configuration registers, high impedance registers, and outflow registers of the three CMOS sensors are all controlled by the master controller via the I2C bus. At the same time, the three CMOS sensors need to have different I2C addresses (Master address is 0x80, Slaver0 address is 0x88, and Slaver1 address is 0x8c) in order to achieve precise control requirements.

[0097] The MIPI bus uses the synchronization signal VSYNC and the clock signal Extclk to synchronize the operation of multiple image sensors, ensuring that only one image sensor outputs data at a time.

[0098] The MIPI bus performs data timing control, such as Figure 2 As shown, the relationship between the data waveforms of the MIPI bus composed of three camera systems and the I2C instruction control logic is well illustrated, and it consists of the following three parts:

[0099] (1) Frame synchronization signal timing: The frame synchronization signal is issued by the Master. When Slaver0 and Slaver1 receive the falling edge of this signal, the sensor will expose a frame. By utilizing the controllable delay of the sensor's frame output, the MIPI data of the three CMOS sensors can be output asynchronously in turn. In order to control the three cameras to output 10 frames of data asynchronously in a time-division manner within one cycle, the cycle of the synchronization signal needs to be controlled at 33.3ms to achieve the goal of each of the three cameras outputting 10 frames of data.

[0100] (2) MIPI Bus Data Waveform: Three cameras share a single MIPI bus. To ensure that only one sensor MIPI pin is connected to the main controller's MIPI pin at any given time, the CMOS sensor must function as a high-impedance switch for MIPI data, effectively closing and opening the connection between the MIPI line and the main controller. The CMOS sensor used in this invention is model JXH63P. This sensor has a register that controls the high-impedance switch. Below are the relevant control parameters for the high-impedance switch of the JXH63P model:

[0101]

[0102] (3) I2C control timing: In the initial stage, the three sensors need to be initialized in sequence. After that, when the master controller detects the frame done signal of the MIPI bus, it will trigger the hardware interrupt handling thread of the frame done. This thread will control the values ​​of bits 0x74[1] and 0x6c[7] so that Master, Slaver0 and Slaver1 will output MIPI data in turn in a time-sharing manner.

[0103] The SOC main control platform controls the switching operation of the image sensor via I2C commands, specifically:

[0104] When the I2C address is 0x80, the switching flow of the main camera (Master) is controlled by the following instructions:

[0105] Master off flow: i2c_write(0x80,0x74,0x03); i2c_write(0x80,0x6c,0xc0);

[0106] Master opens the stream: i2c_write(0x80,0x74,0x02); i2c_write(0x80,0x6c,0x40);

[0107] When the I2C address is 0x88, the corresponding instruction for controlling the switching of secondary camera 0 (Slaver0) is:

[0108] Slaver0 closes the flow: i2c_write(0x88,0x74,0x03); i2c_write(0x88,0x6c,0xc0);

[0109] Slaver0 open stream: i2c_write(0x88,0x74,0x02); i2c_write(0x88,0x6c,0x40);

[0110] When the I2C address is 0x8c, the corresponding instruction for controlling the switching of secondary camera 1 (Slaver1) is:

[0111] Slaver1 closes the flow: i2c_write(0x8c,0x74,0x03); i2c_write(0x8c,0x6c,0xc0);

[0112] Slaver1 open stream: i2c_write(0x8c,0x74,0x02); i2c_write(0x8c,0x6c,0x40); where i2c_write is the I2C function to write to the sensor register. The first parameter is the sensor device address, the second parameter is the sensor register address, and the third parameter is the value to be written to the register.

[0113] The system uses a high-impedance switch to enable time-sharing output from multiple image sensors, ensuring that the data streams of the MIPI bus do not conflict.

[0114] The system also includes a Framed hardware interrupt service routine and a Framed kernel processing thread, which are used to implement time-sharing output control of multiple image sensors through the synchronization mechanism semaphore fd_comp.

[0115] like Figure 3 The diagram illustrates the detailed steps of a multi-camera method based on a high-impedance switch. This method is applicable to the aforementioned system, and its specific implementation steps are as follows:

[0116] Step S1: Initialize configuration:

[0117] Multiple CMOS sensors are initialized and configured via the I2C bus to enable them to function properly, and the variable Cmun is initialized to 0. This includes downloading the initialization configuration for three CMOS sensors.

[0118] Step S2: Create a synchronization mechanism semaphore. Create a semaphore fd_comp of type completion structure so that the Frame done hardware interrupt service routine of the main control platform can trigger the semaphore fd_comp, thereby starting the Frame done kernel processing thread.

[0119] Step S3: Create the Frame-done kernel processing thread. Create and run the Frame-done kernel processing thread. When the master controller detects the frame-done signal of the MIPI bus, it wakes up the thread through the completion synchronization mechanism. Then, the thread controls the values ​​of bits 0x74[1] and 0x6c[7] through I2C to make Master, Slaver0, and Slaver1 output MIPI data in a time-sharing manner. The pseudocode of the thread is as follows: Figure 4 As shown.

[0120] Step S4: Start the main camera to output the first frame. Start the Master sensor to output the first frame of data. Step S5: Trigger the kernel processing thread. After the frame is done, the kernel processing thread switches the three cameras to output in turn according to the remainder value of Cmun%3, and increments Cmun by 1.

[0121] The time-sharing output of Master, Slaver0, and Slaver1 is achieved by using the value of Cmun%3, ensuring that only one sensor outputs data at a time. The control of the time-sharing output includes:

[0122] Based on the value of Cmun%3, the switching operations of Master, Slaver0, and Slaver1 are controlled sequentially: when Cmun%3 == 0, Slaver0 and Slaver1 are turned off, and Master is turned on. The relevant commands are as follows:

[0123] Slaver0 closes the flow: i2c_write(0x88,0x74,0x03); i2c_write(0x88,0x6c,0xc0);

[0124] Slaver1 closes the flow: i2c_write(0x8c,0x74,0x03); i2c_write(0x8c,0x6c,0xc0);

[0125] Master opens the stream: i2c_write(0x80,0x74,0x02); i2c_write(0x80,0x6c,0x40);

[0126] When Cmun%3 == 1, shut down Master and Slaver1, and start Slaver0. The relevant commands are as follows:

[0127] Master off flow: i2c_write(0x80,0x74,0x03); i2c_write(0x80,0x6c,0xc0);

[0128] Slaver1 closes the flow: i2c_write(0x8c,0x74,0x03); i2c_write(0x8c,0x6c,0xc0);

[0129] Slaver0 open stream: i2c_write(0x88,0x74,0x02); i2c_write(0x88,0x6c,0x40);

[0130] When Cmun%3 == 2, shut down Master and Slaver0, and start Slaver1. The relevant commands are as follows:

[0131] Master off flow: i2c_write(0x80,0x74,0x03); i2c_write(0x80,0x6c,0xc0);

[0132] Slaver0 closes the flow: i2c_write(0x88,0x74,0x03); i2c_write(0x88,0x6c,0xc0);

[0133] Slaver1 open stream: i2c_write(0x8c,0x74,0x02); i2c_write(0x8c,0x6c,0x40);

[0134] After each operation, the value of Cmun is incremented by 1.

[0135] Step S6: Cyclic Switching: Repeat step S5 to cycle through the sensor output based on the value of Cmun%3, and repeat this cycle until the application exits and terminates the cyclic switching operation.

[0136] The values ​​of I2C control registers 0x74 and 0x6c are used to control the switching current operation of the sensor, specifically:

[0137] When the value of 0x74 is 0x03 and the value of 0x6c is 0xc0, the data stream of the corresponding sensor is turned off; when the value of 0x74 is 0x02 and the value of 0x6c is 0x40, the data stream of the corresponding sensor is turned on.

[0138] In summary, the present invention can at least achieve the following:

[0139] (1) Reduce hardware failure rate: This method eliminates the hardware logic of the MIPI Switch chip, which naturally reduces the complexity of the hardware logic and thus reduces the risk of hardware failure.

[0140] (2) Reduce hardware costs: This method replaces the hardware logic of the MIPI Switch chip with the timing control logic of the instruction code, thereby reducing the hardware cost compared with the traditional multi-camera method in terms of multi-camera.

[0141] This technical solution can be applied to all series of chips from Ingenic Semiconductor and similar chips.

[0142] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-camera system based on a high-impedance switch, characterized in that, The system includes: SOC main control platform: used to control and manage the data streams of multiple image sensors; including external clock signal source Extclk, MIPI bus, and I2C instruction control pins; Multiple image sensors: Multiple image sensors are connected to the SOC main control platform via a shared MIPI bus; including the main camera denoted as Master, secondary camera 0 denoted as Slaver0, and secondary camera 1 denoted as Slaver1; MIPI bus: Used to transmit data streams from image sensors; I2C instruction control pin module: used to control the switching operation of the image sensor via the I2C protocol; including I2C addresses 0x80, 0x88 and 0x8c, corresponding to the main camera, secondary camera 0 and secondary camera 1 respectively; Synchronization mechanism module: It realizes the synchronous operation of multiple image sensors through the synchronization signal VSYNC and the clock signal Extclk; and ensures that multiple image sensors maintain synchronization when outputting data in turn in a time-sharing manner.

2. The multi-camera system based on a high-impedance switch according to claim 1, characterized in that, The SOC main control platform further includes: First, the main control platform uses I2C commands to control the high-impedance switch register and the outgoing current switch register of the CMOS sensor, so that the data from the three cameras are output in a time-sharing manner. Secondly, the main control platform receives image data of the current scene in real time from the CMOS sensor camera and performs preprocessing on the image in a timely manner. The preprocessing includes optimization operations such as black level correction, demosaic, automatic white balance, automatic exposure, distortion correction, noise reduction, color space conversion, and sharpening. Then, the CMOS sensor camera is controlled in response to the image processing effect until the image achieves the expected effect. Finally, after image processing is completed, the image data is encoded and compressed or displayed locally on the screen; the plurality of image sensors, i.e., CMOS sensors, further include: The CMOS sensor is mainly responsible for acquiring image data of the current scene in real time. Its image data is transmitted to the image processing unit (ISP) in RAW format via the MIPI data transmission protocol for preprocessing. Assuming there are three CMOS sensors, i.e. three camera modules, they are labeled as Master, Slaver0, and Slaver1 respectively for easy differentiation. The MIPI bus further includes: MIPI is responsible for image data transmission between the CMOS sensor and the SOC main control platform. The three cameras and the SOC main control platform share a MIPI bus. The synchronization mechanism module, i.e., the control synchronization signal, further includes: The synchronization signal is a signal initiated by the main camera and simultaneously transmitted to the other secondary cameras. The synchronization signal lines of the three CMOS sensors are all connected to the synchronization signal VSYNC pin of the main camera. Using the synchronization signal as a reference and taking advantage of the controllability of sensor delay frame output, the MIPI data of the three CMOS sensors are output asynchronously in turn. The clock signal is provided by the main control platform for the clock signal sources of the three CMOS sensors, that is, they are all connected to the clock pin of the main control, thus avoiding errors caused by clock signal differences; The I2C command control pin module, i.e., the I2C control bus, further includes: The configuration registers, high-impedance registers, and outflow registers of the three CMOS sensors are all controlled by the master controller via the I2C bus. At the same time, the three CMOS sensors need to have different I2C addresses. For example, the Master address is 0x80, the Slaver0 address is 0x88, and the Slaver1 address is 0x8c, in order to achieve the requirements of precise control.

3. The multi-camera system based on a high-impedance switch according to claim 1, characterized in that, The SOC main control platform controls the switching operation of the image sensor through I2C instructions. Specifically, when the I2C address is 0x80, it controls the switching operation of the main camera, i.e., the Master. When the I2C address is 0x88, the switching flow of the secondary camera 0, i.e., Slaver0, is controlled. When the I2C address is 0x8c, the switching flow of the secondary camera 1 (Slaver1) is controlled.

4. A multi-camera system based on a high-impedance switch according to claim 1, characterized in that, The occupancy status of the MIPI bus is controlled by the synchronization signal VSYNC and the clock signal Extclk to synchronize the operation of multiple image sensors, ensuring that only one image sensor outputs data at a time.

5. A multi-camera system based on a high-impedance switch according to claim 1, characterized in that, The MIPI bus performs data timing control, including the relationship between the data waveform of the MIPI bus composed of three camera systems and the I2C control logic, which consists of the following three parts: (1) Frame synchronization signal timing: The frame synchronization signal is issued by the Master. When Slaver0 and Slaver1 receive the falling edge of this signal, the sensor will be triggered to expose the frame. By utilizing the controllable delay of the sensor's frame output, the MIPI data of the three CMOS sensors can be output asynchronously in turn. In order to control the three cameras to output 10 frames of data asynchronously in one cycle, the cycle of the synchronization signal needs to be controlled within 33.3ms to achieve the goal of each of the three cameras outputting 10 frames of data. (2) MIPI bus data waveform: Three cameras share one MIPI bus. In order to ensure that only one sensor MIPI pin is connected to the main control MIPI pin at the same time, the CMOS sensor must have the function of high impedance switching of MIPI data, that is, to play the function of closing and opening the connection between the MIPI line and the main control. (3) I2C control timing: In the initial stage, the three sensors need to be initialized in sequence. After that, when the master controller detects the Frame done signal of the MIPI bus, it will trigger the hardware interrupt processing thread of Frame done. This thread will control the values ​​of bits 0x74[1] and 0x6c[7] through I2C instructions so that Master, Slaver0 and Slaver1 can output MIPI data in turn in a time-sharing manner.

6. A multi-camera system based on a high-impedance switch according to claim 1, characterized in that, The selected CMOS sensor is model JXH63P. This sensor has a register for controlling the high-impedance switch. Below are the relevant control parameters for the high-impedance switch of model JXH63P:

7. A multi-camera system based on a high-impedance switch according to claim 1, characterized in that, The system uses a high-impedance switch to enable time-sharing output from multiple image sensors, ensuring that the data streams of the MIPI bus do not conflict.

8. A multi-camera system based on a high-impedance switch according to claim 1, characterized in that, The system also includes a Framed hardware interrupt service routine and a Framed kernel processing thread, which are used to implement time-sharing output control of multiple image sensors through the synchronization mechanism semaphore fd_comp.

9. A multi-camera method based on a high-impedance switch, characterized in that, The method is applicable to the system described in any one of claims 1-8 above, and the method includes the following steps: S1: Initialization Configuration: Multiple CMOS sensors are initialized and configured via the I2C bus to enable them to function properly, and the variable Cmun is initialized to 0. S2: Create a synchronization mechanism semaphore: Create a semaphore fd_comp of type completion structure so that the main control platform's Frame done hardware interrupt service routine can trigger the semaphore fd_com, thereby starting the Frame done kernel processing thread; S3: Create a Frame done kernel processing thread: Create and run the Frame done kernel processing thread. When the master detects the Frame done signal of the MIPI bus, it wakes up the thread through the completion synchronization mechanism. Then the thread will use the values ​​of bits 0x74[1] and 0x6c[7] of the I2C control register to make Master, Slaver0 and Slaver1 output MIPI data in a time-sharing manner. S4: Start the main camera to output the first frame: Start the Master sensor to output the first frame of data; S5: Trigger the kernel processing thread: The Frame done kernel processing thread switches the three cameras to output in turn during the time-sharing period by judging the remainder value of Cmun%3, and increments Cmun by 1; the control of the time-sharing output includes: Based on the value of Cmun%3, the switching operations of Master, Slaver0, and Slaver1 are controlled sequentially via I2C instructions: When Cmun%3 == 0, Slaver0 and Slaver1 are turned off, and Master is turned on. When Cmun%3 == 1, shut down Master and Slaver1, and turn on Slaver0; When Cmun%3 == 2, shut down Master and Slaver0, and turn on Slaver1; After each operation, the value of Cmun is incremented by 1; S6: Cyclic Switching: Repeat step S5 to cycle through the sensor output based on the value of Cmun%3, and repeat this cycle until the application exits and terminates the cyclic switching operation.

10. A multi-camera method based on a high-impedance switch according to claim 9, characterized in that, The values ​​of I2C instruction control registers 0x74 and 0x6c are used to control the switching current operation of the sensor, specifically: When the value of 0x74 is 0x03 and the value of 0x6c is 0xc0, the data stream of the corresponding sensor is turned off; When the value of 0x74 is 0x02 and the value of 0x6c is 0x40, the data stream of the corresponding sensor is opened.

Images