Communication system and communication method

Abstract

The present disclosure relates to a communication system and a communication method that enable easier achievement of time synchronization. This communication system comprises a first communication device that collectively stores image data and audio data in one CSI-2 packet based on the CSI-2 standard, and transmits the CSI-2 packet in accordance with the CSI-2 standard, and a second communication device that receives the CSI-2 packet and separates the image data and the audio data. The communication system utilizes the I2C standard to time-synchronize the transmission side for transmitting the CSI-2 packet and the reception side for receiving the CSI-2 packet. The present technology can be applied to, for example, a communication system that transmits image data and audio data.

Description

Communication system and communication method

[0001] The present disclosure relates to a communication system and a communication method, and particularly to a communication system and a communication method that can more easily achieve time synchronization.

[0002] Conventionally, CSI (Camera Serial Interface)-2 by the MIPI (Mobile Industry Processor Interface) Alliance has been used as a standard for transmitting image data from an imaging device such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0003] The current MIPI CSI-2 standard (see, for example, Non-Patent Document 1) is not assumed to transmit data other than image data such as audio data together with the image data, and a function for time synchronization between the transmission side and the reception side is not required.

[0004] Specification for Camera Serial Interface 2 (CSI-2) Version 4.0

[0005] Conventionally, when transmitting audio data over a long distance together with image data, for example, it was necessary to transmit the image data using the MIPI CSI-2 standard while transmitting the audio data using another communication standard. In this way, when transmitting image data and audio data through separate transmission paths, the image data and audio data are transmitted without time synchronization on the transmission side, and time synchronization cannot be achieved between the transmission side and the reception side with the MIPI CSI-2 standard, so it was difficult to time-synchronize the image data and audio data on the reception side.

[0006] The present disclosure has been made in view of such a situation and enables easier time synchronization.

[0007] A communication system in one aspect of this disclosure includes a first communication device that stores image data and data other than the image data together in a single packet based on a first communication standard and transmits the packet in accordance with the first communication standard, and a second communication device that receives the packet and separates the image data and data other than the image data, and synchronizes the time between the transmitting side that transmits the packet and the receiving side that receives the packet using a second communication standard other than the first communication standard.

[0008] One aspect of the present disclosure is a communication method which includes a communication system storing image data and data other than the image data together in a single packet based on a first communication standard, transmitting the packet in accordance with the first communication standard, and receiving the packet and separating the image data and data other than the image data, and using a second communication standard other than the first communication standard to synchronize the time between the transmitting side that transmits the packet and the receiving side that receives the packet.

[0009] In one aspect of this disclosure, image data and non-image data are bundled together and stored in a single packet based on a first communication standard, the packet is transmitted according to the first communication standard, and upon receiving the packet, the image data and non-image data are separated. Then, using a second communication standard other than the first communication standard, the transmitting side that sends the packet and the receiving side that receives the packet are synchronized in time.

[0010] This is a block diagram showing an example configuration of a first embodiment of a communication system to which this technology is applied. This is a block diagram showing an example configuration of a CMOS image sensor, microphone, and aggregator. This is a block diagram showing an example configuration of a disaggregator and ECU. This is a flowchart explaining the time synchronization command generation process. This is a diagram explaining time synchronization using the time synchronization command. This is a block diagram showing an example configuration of the timer on the aggregator side. This is a diagram explaining the synchronization of time Timer#1 with time Timer#0 depending on whether or not a PI or PID control circuit (PI / PID control circuit) is implemented. This is a diagram explaining the operation of the MCK divider, PMC, and PMC_result register. This is a block diagram showing an example configuration of an MCK generator. This is a diagram showing an example of voice transmission setting information. This is a diagram explaining the restoration of the clock signal TxREFCK, clock signal MCK_Div, master clock signal MCK, and bit clock signal BCLK. This is a diagram showing a detailed example of setting information related to I2S. This is a diagram showing an example of the format of a CSI-2 packet. This is a diagram explaining the data flow in the aggregator. This is a block diagram showing an example configuration of a CSI-2 multi-encoder. This is a diagram showing an example of the data flow in a CSI-2 multi-encoder. This is a flowchart explaining the operation of the VC0 line buffer of the aggregator. This is a flowchart illustrating the operation of the aggregator's VC1 line buffer. This is a diagram illustrating the data flow in the disaggregator. This is a block diagram showing an example configuration of a CSI-2 multidecoder. This is a diagram showing an example of the data flow in a CSI-2 multidecoder. This is a flowchart illustrating the operation of the disaggregator's VC0 line buffer. This is a flowchart illustrating the operation of the disaggregator's VC1 line buffer. This is a diagram showing an example of a packet transmitted using D-PHY. This is a diagram showing an example of the data format of a long packet. This is a diagram illustrating the format of I2S data for each SDBW. This is a block diagram showing an example configuration of a CMOS image sensor, microphone, and aggregator in a second embodiment.This is a block diagram illustrating an example configuration of a disaggregator and ECU in the second embodiment. This diagram illustrates the operation of the BCLK divider, PMC, and PMC_result register. This is a block diagram illustrating an example configuration of a BCLK generator. This diagram illustrates the restoration of the clock signal TxREFCK, the clock signal MCK_Div, and the bit clock signal BCLK. This is a block diagram illustrating an example configuration of a third embodiment of a communication system to which this technology is applied. This is a block diagram illustrating an example configuration of an aggregator and ECU. This is a block diagram illustrating an example configuration of a disaggregator, display, and speaker. This is a block diagram illustrating an example configuration of a fourth embodiment of a communication system to which this technology is applied. This is a block diagram illustrating an example configuration of an aggregator. This is a block diagram illustrating an example configuration of a disaggregator. This is a diagram showing another example of the data format of a CSI-2 packet. This is a diagram showing an example of the data format of a long packet transmitted using D-PHY.

[0011] The following describes in detail a specific embodiment of this technology, with reference to the drawings.

[0012] <First Configuration Example of a Communication System> Referring to Figures 1 to 26, a configuration example of a first embodiment of a communication system to which this technology is applied will be described.

[0013] Figure 1 is a block diagram showing an example configuration of the communication system 11.

[0014] As shown in Figure 1, the communication system 11 is comprised of a CMOS image sensor 21, a microphone 22, an aggregator 23, a disaggregator 24, an ECU (Electronic Control Unit) 25, a display 26, and a speaker 27.

[0015] The CMOS image sensor 21 supplies image data obtained by capturing images to the aggregator 23.

[0016] The microphone 22 supplies the audio data obtained by capturing sound to the aggregator 23.

[0017] The aggregator 23 uses, for example, a C-PHY or D-PHY as the physical layer to combine image data supplied from the CMOS image sensor 21 and audio data supplied from the microphone 22 into a single CSI-2 packet, which is then transmitted to the disaggregator 24. The aggregator 23 also communicates with the disaggregator 24 according to the I2C (Inter-Integrated Circuit) standard, allowing it to receive I2C data containing various information necessary for time synchronization between the aggregator 23 and the disaggregator 24 (for example, PMC_result and Time_info).

[0018] The disaggregator 24, for example, uses a D-PHY or C-PHY as its physical layer to receive CSI-2 packets transmitted from the aggregator 23, separates the image data and audio data stored together in the CSI-2 packets, and supplies them to the ECU 25. The disaggregator 24 can also communicate with the aggregator 23 according to the I2C standard and transmit I2C data containing various information (for example, PMC_result and Time_info) necessary for time synchronization between the aggregator 23 and the disaggregator 24.

[0019] The ECU 25 supplies image data from the disaggregator 24 to the display 26 and audio data from the disaggregator 24 to the speaker 27.

[0020] The display 26 displays an image captured by the CMOS image sensor 21 based on image data supplied from the ECU 25.

[0021] The speaker 27 outputs the sound picked up by the microphone 22 based on the audio data supplied from the ECU 25.

[0022] In the communication system 11 configured in this way, it is possible to transmit image data and non-image data in CSI-2 and synchronize their time. For example, in the communication system 11, the aggregator 23 and the disaggregator 24 synchronize their time using the I2C standard, thereby synchronizing the timing of sending and receiving voice data between the aggregator 23 and the disaggregator 24. This allows the image data and voice data to be stored in a single CSI-2 packet and transmitted from the aggregator 23 to the disaggregator 24 using the MIPI CSI-2 standard. Therefore, the communication system 11 can transmit image data and voice data over long distances using the MIPI CSI-2 standard with a single transmission cable. Of course, this technology can also be applied to communication systems that transmit multiple types of data other than image data (up to three types in the current CSI-2 standard) in synchronization with image data using the MIPI CSI-2 standard, instead of voice data.

[0023] Figure 2 is a block diagram showing an example configuration of a CMOS image sensor 21, a microphone 22, and an aggregator 23.

[0024] As shown in Figure 2, the CMOS image sensor 21 is configured to include an I2C slave 31, an AD (Analog to Digital) converter 32, a CSI-2 encoder 33, and a C-PHY 34. For example, the CMOS image sensor 21 receives light focused by a lens (not shown) on a sensor surface where multiple pixels are arranged in an array, and acquires an analog pixel signal by performing photoelectric conversion at each pixel.

[0025] The I2C slave 31 communicates with the I2C master 51 of the aggregator 23 in accordance with the I2C standard. For example, the I2C slave 31 can perform various settings related to the imaging of the CMOS image sensor 21 according to the setting information transmitted through communication with the I2C master 51.

[0026] The AD converter 32 performs A / D conversion on the analog pixel signals acquired by the CMOS image sensor 21 according to the setting information set via the I2C slave 31, for example, and supplies the image data obtained by the A / D conversion to the CSI-2 encoder 33.

[0027] The CSI-2 encoder 33 encodes the image data supplied from the AD converter 32 into a data format compliant with the CSI-2 standard, and transmits the image data in CSI-2 data format to the aggregator 23 via the C-PHY 34.

[0028] C-PHY 34 is a physical layer used to transmit image data in CSI-2 data format from the CMOS image sensor 21 to the aggregator 23.

[0029] The microphone 22 is configured to include an I2C slave 41, an AD converter 42, and an I2S (Inter-IC Sound) slave 43. For example, the microphone 22 acquires an analog audio signal by converting sound into an electrical signal.

[0030] The I2C slave 41 communicates with the I2C master 51 of the aggregator 23 in accordance with the I2C standard. For example, the I2C slave 41 can make various settings related to sound pickup of the microphone 22 according to the setting information transmitted through communication with the I2C master 51.

[0031] The AD converter 42 performs AD conversion on the analog audio signal acquired by the microphone 22 according to the configuration information set via the I2C slave 41, for example, and supplies the resulting audio data to the I2S slave 43.

[0032] The I2S slave 43 communicates with the I2S master 71 of the aggregator 23 in accordance with the I2S standard. For example, the I2S slave 43 transmits audio data (SDATA) supplied from the AD converter 42 to the I2S master 71 in accordance with the bit clock signal BCLK and LR clock signal LRCLK supplied from the I2S master 71 of the aggregator 23.

[0033] The aggregator 23 is comprised of an I2C master 51, a C-PHY 52, a CSI-2 decoder 53, a CSI-2 decode FIFO 54, an oscillator 55, I2C slaves 56-58, a configuration information register 59, a controller 60, a timer 61, a PMC_result register 62, an audio encoder 63, an I2S encode FIFO 64, a CSI-2 multi-encoder 65, and a D-PHY 66.

[0034] The I2C master 51 can read the setting information for the CMOS image sensor 21 from the setting information register 59 and transmit it to the I2C slave 31 in accordance with the I2C standard to set the CMOS image sensor 21. Similarly, the I2C master 51 can read the setting information for the microphone 22 from the setting information register 59 and transmit it to the I2C slave 41 in accordance with the I2C standard to set the microphone 22.

[0035] C-PHY 52 is a physical layer used to receive image data in CSI-2 data format transmitted from the CMOS image sensor 21 to the aggregator 23.

[0036] The CSI-2 decoder 53 receives image data in CSI-2 data format transmitted by the CSI-2 encoder 33 via the C-PHY 34 and C-PHY 52. The CSI-2 decoder 53 then decodes the image data in CSI-2 data format to obtain image data restored from the data format compliant with the CSI-2 standard back to the original data format, and writes it to the CSI-2 decode FIFO 54.

[0037] The CSI-2 decode FIFO 54 stores the image data (CSI_WDATA) to be written by the CSI-2 decoder 53 if the writable signal CSI_WEN supplied from the CSI-2 decoder 53 indicates that image data can be written. Then, the CSI-2 decode FIFO 54 outputs the image data in the order in which it was input if the readable signal CSI_REN supplied from the CSI-2 multi-encoder 65 indicates that image data can be read. As a result, the image data (CSI_RDATA) is read from the CSI-2 decode FIFO 54 to the CSI-2 multi-encoder 65.

[0038] The oscillator 55 synchronizes the time of the timer 61 with the time of the timer 94 (Figure 3) of the disaggregator 24, generates a clock signal RxREFCK which is referenced when synchronizing the time between the aggregator 23 and the disaggregator 24, and supplies it to the timer 61 and the MCK generator 72 of the audio encoder 63.

[0039] The I2C slave 56 acquires the configuration information stored in the I2C data transmitted from the I2C master 88 (Figure 3) of the disaggregator 24 in accordance with the I2C standard, and writes it to the configuration information register 59.

[0040] The I2C slave 57 acquires the time Timer#0 (time information indicating the time of timer 94 in Figure 3) stored in the I2C data transmitted from the I2C master 88 (Figure 3) of the disaggregator 24 in accordance with the I2C standard, and supplies it to timer 61.

[0041] The I2C slave 58 retrieves the PMC_result value stored in the I2C data transmitted from the I2C master 88 (Figure 3) of the disaggregator 24 in accordance with the I2C standard, and writes it to the PMC_result register 62.

[0042] The configuration information register 59 stores the configuration information that is written via the I2C slave 56.

[0043] The controller 60 reads the setting information necessary for the audio encoder 63 to encode the audio data from the setting information register 59, and sets it in the MCK generator 72 and the I2S encoder 73 of the audio encoder 63.

[0044] The timer 61 outputs a time Timer#1 indicating the timing at which the I2S encoder 73 of the audio encoder 63 outputs I2S - format audio data. At this time, the timer 61 can output a time Timer#1 synchronized with the time Timer#0 transmitted from the I2C master 88 (FIG. 3) of the disaggregator 24 according to the I2C standard and supplied via the I2C slave 57. That is, the timer 61 has a function of synchronizing the time with the timer 94 in FIG. 3 using communication according to the I2C standard. The detailed configuration of the timer 61 (for example, the PI / PID control circuit 122, etc.) will be described later with reference to FIGS. 6 and 7.

[0045] The PMC_result register 62 stores the PMC_result value written via the I2C slave 58, and outputs the PMC_result value to the MCK generator 72 of the audio encoder 63.

[0046] The audio encoder 63 is configured to include an I2S master 71, an MCK generator 72, and an I2S encoder 73.

[0047] The I2S master 71 communicates with the I2S slave 43 of the microphone 22 according to the I2S standard. For example, the I2S master 71 receives the audio data (SDATA) transmitted from the I2S slave 43 in response to the bit - clock signal BCLK and the LR - clock signal LRCLK transmitted to the I2S slave 43, and supplies it to the I2S encoder 73.

[0048] The MCK generator 72 generates a bit clock signal BCLK and an LR clock signal LRCLK according to the clock signal RxREFCK supplied from the oscillator 55, the setting information supplied from the controller 60, and the PMC_result value supplied from the PMC_result register 62, and supplies them to the I2S master 71 and the I2S encoder 73, respectively. The detailed configuration of the MCK generator 72 will be described later with reference to FIGS. 9 to 11.

[0049] The I2S encoder 73 encodes the audio data supplied from the I2S master 71 into a data format according to the I2S standard at a timing according to the bit clock signal BCLK and the LR clock signal LRCLK supplied from the MCK generator 72 based on the setting information (see FIG. 10) supplied from the controller 60. Then, the I2S encoder 73 writes the audio data in the I2S format into the I2S encoding FIFO 64 according to the time timer#1 supplied from the timer 61.

[0050] The I2S encoding FIFO 64 stores the audio data in the I2S format (I2S_WDATA) written by the I2S encoder 73 when the writable signal I2S_WEN supplied from the I2S encoder 73 indicates that writing of the audio data is possible. Then, the I2S encoding FIFO 64 outputs the audio data in the order in which the audio data was input when the readable signal I2S_REN supplied from the CSI-2 multi-encoder 65 indicates that reading of the audio data is possible. Thereby, the audio data in the I2S format (I2S_RDATA) is read from the I2S encoding FIFO 64 to the CSI-2 multi-encoder 65.

[0051] The CSI-2 multi-encoder 65 reads image data from the CSI-2 decode FIFO 54 and reads I2S format audio data from the I2S encoding FIFO 64. The CSI-2 multi-encoder 65 generates a CSI-2 packet in which the image data is stored in the data area of ​​virtual channel VC0 and the I2S format audio data is stored in the data area of ​​virtual channel VC1, according to the CSI-2 standard data format. At this time, the CSI-2 multi-encoder 65 stores the audio data in the CSI-2 packet in synchronization with the image data at the frame rate of the image data, and can assign an Audio Type to the Data Type, as will be described later with reference to Figure 24.

[0052] The CSI-2 multi-encoder 65 then transmits the CSI-2 packets containing the image data and audio data to the disaggregator 24 via the D-PHY 66. For example, the CSI-2 multi-encoder 65 is connected to the D-PHY 66 via PPI (Phy Protocol I / F). The detailed configuration of the CSI-2 multi-encoder 65 will be described later with reference to Figures 16 and 17.

[0053] The D-PHY 66 is a physical layer used to transmit CSI-2 packets containing image data and audio data from the aggregator 23 to the disaggregator 24. The CSI-2 packets containing image data and audio data transmitted via the D-PHY 66 are treated as D-PHY Long Packets as shown in Figure 24, which will be described later. The Long Packet structure when audio data is included is shown in Figure 25, which will be described later.

[0054] Figure 3 is a block diagram showing an example configuration of the disaggregator 24 and the ECU 25.

[0055] The disaggregator 24 is configured to include a D-PHY 81, a CSI-2 multi-decoder 82, a CSI-2 encoding FIFO 83, a CSI-2 encoder 84, a C-PHY 85, an I2S decoding FIFO 86, an audio decoder 87, an I2C master 88, I2C slaves 89-91, a configuration information register 92, a controller 93, a timer 94, and a PMC_result register 95.

[0056] The D-PHY 81 is a physical layer used to receive CSI-2 packets transmitted from the aggregator 23 to the disaggregator 24. For example, the D-PHY 81 is connected to the CSI-2 multidecoder 82 via PPI.

[0057] The CSI-2 multi-decoder 82 receives CSI-2 packets transmitted from the CSI-2 multi-encoder 65 via D-PHY 66 and D-PHY 81. The CSI-2 multi-decoder 82 obtains image data and I2S format audio data from the CSI-2 packets according to the data format of the CSI-2 standard. For example, the CSI-2 multi-decoder 82 refers to the packet header of the CSI-2 packet, separates it according to the data type of the virtual channel (VC), obtains the image data stored in virtual channel VC0 of the CSI-2 packet, and obtains the I2S format audio data stored in virtual channel VC1 of the CSI-2 packet. Then, the CSI-2 multi-decoder 82 writes the image data obtained from virtual channel VC0 of the CSI-2 packet to the CSI-2 encoded FIFO 83 and writes the I2S format audio data obtained from virtual channel VC1 of the CSI-2 packet to the I2S decoded FIFO 86. The detailed configuration of the CSI-2 multi-decoder 82 will be described later with reference to Figures 20 and 21.

[0058] The CSI-2 encoded FIFO 83 stores the image data (CSI_WDATA) to be written by the CSI-2 multidecoder 82 if the writable signal CSI_WEN supplied from the CSI-2 multidecoder 82 indicates that image data can be written. Then, the CSI-2 encoded FIFO 83 outputs the image data in the order in which it was input if the readable signal CSI_REN supplied from the CSI-2 encoder 84 indicates that image data can be read. As a result, the image data (CSI_RDATA) is read from the CSI-2 encoded FIFO 83 to the CSI-2 encoder 84.

[0059] The CSI-2 encoder 84 encodes the image data read from the CSI-2 encoding FIFO 83 into a data format conforming to the CSI-2 standard. Then, according to the time Timer#0 supplied by the timer 94, the CSI-2 encoder 84 transmits the CSI-2 formatted image data to the ECU 25 via the C-PHY 85.

[0060] C-PHY 85 is a physical layer used to transmit CSI-2 format image data from the disaggregator 24 to the ECU 25.

[0061] The I2S decoding FIFO 86 stores the I2S format audio data (I2S_WDATA) to be written by the CSI-2 multi-decoder 82 if the writable signal I2S_WEN supplied from the CSI-2 multi-decoder 82 indicates that audio data can be written. Then, the I2S decoding FIFO 86 outputs the audio data in the order in which it was input if the readable signal I2S_REN supplied from the I2S decoder 101 of the audio decoder 87 indicates that audio data can be read. As a result, the I2S format audio data (I2S_RDATA) is read from the I2S decoding FIFO 86 to the I2S decoder 101 of the audio decoder 87.

[0062] The audio decoder 87 is configured to include an I2S decoder 101, an I2S slave 102, an MCK divider 103, and a PMC 104.

[0063] The I2S decoder 101 decodes the I2S-formatted audio data read from the I2S decode FIFO 86 at timings according to the bit clock signal BCLK and LR clock signal LRCLK, based on the parameters (SDBW, num of Ch) supplied from the controller 93. The I2S decoder 101 then supplies the audio data, restored from the I2S standard-compliant data format to the original data format, to the I2S slave 102 according to the time Timer#0 supplied from the timer 94.

[0064] The I2S slave 102 communicates with the I2S master 113 of the ECU 25 in accordance with the I2S standard. For example, the I2S slave 102 transmits audio data (SDATA) supplied from the I2S decoder 101 to the I2S master 113 of the ECU 25 in accordance with the bit clock signal BCLK and LR clock signals LRCLK supplied from the I2S master 113 of the ECU 25.

[0065] The MCK divider 103 divides the master clock signal MCK supplied from the ECU 25 according to the N value supplied from the controller 93, and supplies the resulting clock signal MCK_Div with frequency (=MCK / N) to the PMC 104. For example, as shown in Figure 3, if the master clock signal MCK is 24.576MHz and the N value is 1536, the MCK divider 103 supplies a clock signal MCK_Div of 16KHz (=24.576MHz / 1536) to the PMC 104.

[0066] The PMC 104 counts the period corresponding to the clock signal MCK_Div supplied from the MCK divider 103 according to the clock signal TxREFCK supplied from the oscillator 114 of the ECU 25, and writes the resulting PMC_result value to the PMC_result register 95. For example, if the clock signal MCK_Div supplied from the MCK divider 103 is 16KHz and the clock signal TxREFCK supplied from the oscillator 114 of the ECU 25 is 250MHz, the PMC 104 writes 15625 (= 16KHz / 250MHz), which is the PMC_result value, to the PMC_result register 95.

[0067] The I2C master 88 writes the configuration information for the disaggregator 24, supplied from the ECU 25, to the configuration information register 92 via the I2C slave 89. The I2C master 88 also stores the configuration information for the aggregator 23, supplied from the ECU 25, into I2C data and sends it to the aggregator 23, which then writes it to the configuration information register 59 via the I2C slave 56 shown in Figure 2.

[0068] The I2C master 88 obtains the time Timer#0 of the timer 94 via the I2C slave 90. The I2C master 88 then stores the time Timer#0 obtained from the timer 94 in I2C data and sends it to the aggregator 23, which supplies it to the timer 61 via the I2C slave 57 in Figure 2. For example, the I2C master 88 uses a time synchronization command (n), as described later with reference to Figures 4 and 5, to send the I2C data containing the time Timer#0 to the aggregator 23.

[0069] The I2C master 88 reads the PMC_result value stored in the PMC_result register 95 via the I2C slave 91. Then, the I2C master 88 stores the PMC_result value read from the PMC_result register 95 into I2C data and sends it to the aggregator 23, and writes it to the PMC_result register 62 via the I2C slave 58.

[0070] The I2C slave 89 supplies the configuration information provided by the I2C master 88 to the configuration information register 92.

[0071] The I2C slave 90 supplies the time Timer#0, which it obtained from the timer 94, to the I2C master 88.

[0072] The I2C slave 91 supplies the PMC_result value obtained from the PMC_result register 95 to the I2C master 88.

[0073] The configuration information register 92 stores configuration information written by the I2C master 88 via the I2C slave 89.

[0074] The controller 93 reads the configuration information necessary for the audio decoder 87 to decode the audio signal from the configuration information register 92 and sets it in the I2S decoder 101 and MCK divider 103 of the audio decoder 87.

[0075] Timer 94 outputs Timer#0, which indicates the timing when the CSI-2 encoder 84 transmits CSI-2 format image data to the ECU 25, and Timer#0, which indicates the timing when the I2S decoder 101 of the audio decoder 87 supplies audio data to the I2S slave 102. In addition, Timer#0 of Timer 94 is supplied to the I2C master 88 via the I2C slave 90 and transmitted to Timer 61 in Figure 2 in order to synchronize the time with Timer#1 of Timer 61 in Figure 2.

[0076] The PMC_result register 95 stores the PMC_result value written by the PMC 104. The PMC_result value stored in the PMC_result register 95 is read by the I2C master 88 via the I2C slave 91 and written to the PMC_result register 62 in Figure 2.

[0077] The ECU 25 is comprised of a C-PHY 111, a CSI-2 decoder 112, an I2S master 113, and an oscillator 114.

[0078] C-PHY 111 is a physical layer used to transmit image data in CSI-2 data format from the disaggregator 24 to the ECU 25.

[0079] The CSI-2 decoder 112 receives CSI-2 format image data transmitted from the CSI-2 encoder 84 via the C-PHY 85 and C-PHY 111. The CSI-2 decoder 112 then decodes the CSI-2 data format image data to obtain image data restored from the CSI-2 standard data format back to the original data format, and supplies it to the display 26 in Figure 1.

[0080] The I2S master 113 communicates with the I2S slave 102 of the disaggregator 24 in accordance with the I2S standard. For example, the I2S master 113 receives audio data (SDATA) transmitted from the I2S slave 102 in accordance with the bit clock signal BCLK and LR clock signal LRCLK transmitted to the I2S slave 102, and supplies it to the speaker 27 in Figure 1.

[0081] The oscillator 114 generates a clock signal TxREFCK, which is referenced when synchronizing the time between the aggregator 23 and the disaggregator 24, and supplies it to the timer 94 and PMC 104 of the disaggregator 24.

[0082] Furthermore, the ECU 25 can write the setting information of the disaggregator 24 to the setting information register 92 via the I2C master 88. Similarly, the ECU 25 can write the setting information of the CMOS image sensor 21, microphone 22, and audio encoder 63 to the setting information register 59 in Figure 2 via the I2C master 88.

[0083] Figure 4 is a flowchart illustrating the time synchronization command generation process in which the I2C master 88 generates a time synchronization command (n) to synchronize the time of Timer #1 of Timer 61 with the time of Timer #0 of Timer 94.

[0084] The I2C master 88 performs initialization in step S11 and determines in step S12 whether or not the initialization is complete. The I2C master 88 waits until it determines in step S12 that the initialization is complete, and if it determines that the initialization is complete, the process proceeds to step S13.

[0085] In step S13, the I2C master 88 sets an initial value (for example, n=1) for the parameter n that specifies the time synchronization command (n).

[0086] In step S14, the I2C master 88 issues a time synchronization command (n) specified by the current parameter n. For example, if parameter n is 1, a time synchronization command (1) as shown in Figure 5 is issued; if parameter n is 2, a time synchronization command (2) as shown in Figure 5 is issued; and if parameter n is 3, a time synchronization command (3) as shown in Figure 5 is issued.

[0087] In step S15, the I2C master 88 determines whether the time synchronization command (n) issued in step S14 has finished, and waits until it determines that the time synchronization command (n) has finished. If the I2C master 88 determines that the time synchronization command (n) has finished, the process proceeds to step S16.

[0088] In step S16, the I2C master 88 determines whether or not to terminate the time synchronization between timer 61 and timer 94.

[0089] If, in step S16, the I2C master 88 determines that it will not terminate the time synchronization between timer 61 and timer 94, the process proceeds to step S17.

[0090] In step S17, the I2C master 88 determines whether the current parameter n is 3 or greater.

[0091] In step S17, if the I2C master 88 determines that the current parameter n is 3 or greater, the process returns to step S13, and the same process is repeated thereafter.

[0092] On the other hand, if in step S17 the I2C master 88 determines that the current parameter n is not 3 or greater (i.e., the current parameter n is 1 or 2), the process proceeds to step S18.

[0093] In step S18, the I2C master 88 increments the current parameter n (n = ++), and then the process returns to step S14, and the same process is repeated thereafter.

[0094] On the other hand, if in step S16 the I2C master 88 determines that it has finished the time synchronization between timer 61 and timer 94, the time synchronization command generation process is terminated.

[0095] By executing the time synchronization command generation process described above, time synchronization command (1), time synchronization command (2), and time synchronization command (3) are repeatedly issued, and the time information Timer#1 of timer 61 can be periodically synchronized with the time information Timer#0 of timer 94.

[0096] Referring to Figure 5, a time synchronization method using the time synchronization command (n) will be explained.

[0097] As shown in the upper part of Figure 5, the I2C master 88 issues a time synchronization command (1) and sends a CSI-2 packet containing the time information Timer#0(t0) of the timer 94 at timing t0. Furthermore, the I2C master 88 issues a time synchronization command (3) and sends a CSI-2 packet containing the time information Timer#0(t3) of the timer 94 at timing t3, and a CSI-2 packet containing the time information Timer#0(t4) of the timer 94 at timing t4. The I2C slave 57 then obtains the time information Timer#0(t0), Timer#0(t3), and Timer#0(t4) from these CSI-2 packets and supplies them to the timer 61.

[0098] Meanwhile, Timer 61 acquires the time information Timer#1(t1) at timing t1 according to the time synchronization command (1), and acquires the time information Timer#1(t2) at timing t2 according to the time synchronization command (2). Then, Timer 61 calculates a one-way communication delay time (d) (= {Time difference on Timer 94 side (a) - Time difference on Timer 61 side (b)} / 2) based on the time difference on Timer 94 side (a) from time information Timer#0(t0) to time information Timer#0(t3) and the time difference on Timer 61 side (b) from time information Timer#1(t1) to time information Timer#1(t2).

[0099] For example, as shown in Figure 5, if the time information Timer#0(t0) is 14 and the time information Timer#0(t3) is 32, the time difference (a) on the timer 94 side will be 18 (= 32 - 14), and if the time information Timer#1(t1) is 20 and the time information Timer#1(t2) is 34, the time difference (b) on the timer 61 side will be 14 (= 34 - 20). Therefore, timer 61 calculates 2 (= (18 - 14) / 2) as the one-way communication delay time (d).

[0100] Subsequently, at timing t5, Timer 61 calculates the synchronized time information Timer#1(t5) (= Timer#0(t4) + unidirectional communication delay time (d)) synchronized with Timer 94, and updates the pre-synchronization time information Timer#1(t5) with the synchronized time information Timer#1(t5). For example, as shown in Figure 5, if the time information Timer#0(t4) is 48 and the unidirectional communication delay time (d) is 2, Timer 61 calculates 50 (= 48 + 2) as the updated time information Timer#1(t5).

[0101] Therefore, the timer 61 can synchronize with the timer 94's time information Timer#0 at timing t5 by updating the pre-synchronization time information Timer#1(t5) (54 in the illustrated example) at timing t5 with the post-synchronization time information Timer#1(t5) calculated as described above.

[0102] Figure 6 is a block diagram showing an example configuration of the timer 61 in Figure 2.

[0103] As shown in Figure 6, the timer 61 is configured to include a synchronization time calculation unit 121, a PI / PID control circuit 122, and a counter 123.

[0104] As explained with reference to Figure 5 above, the synchronous time calculation unit 121 is supplied with time information Timer#0(t0), Timer#0(t3), and Timer#0(t4) via the I2C slave 57. The synchronous time calculation unit 121 also acquires time information Timer#1(t1) and Timer#1(t2) output from the counter 123. The synchronous time calculation unit 121 then calculates the one-way communication delay time (d), calculates the synchronized time information Timer#1(t5) synchronized with the timer 94 at timing t5, and updates the pre-synchronization time information Timer#1(t5) with the synchronized time information Timer#1(t5). The synchronous time calculation unit 121 supplies the synchronized time information Timer#1(t5) to the PI / PID control circuit 122.

[0105] Here, the cyclical period during which the synchronization time calculation unit 121 performs time synchronization according to the time synchronization command, for example, for each time information Timer#1(t5), is called the time synchronization period.

[0106] The PI / PID control circuit 122 corrects the time information Timer#1 output from the counter 123 using PI (Integral-Differential) control or PID (Proportional-Integral-Differential) control so that the error of Timer#1 with respect to Timer#0 of Timer94 is reduced. Then, at correction periods shorter than the time synchronization period, the PI / PID control circuit 122 supplies the corrected time information Timer#1, which has been corrected from the time information Timer#1 output from the counter 123, to the counter 123.

[0107] The counter 123 updates the time information Timer#1 with the corrected time information Timer#1 at the time the corrected time information Timer#1 is supplied from the PI / PID control circuit 122, and counts the time.

[0108] The timer 61 configured in this way can reduce the error between the time information Timer#1 and the time information Timer#0, even when performing time synchronization using the slow communication speed I2C, with a correction period shorter than the time synchronization period.

[0109] Referring to Figure 7, the error in time Timer#1 relative to time Timer#0, depending on whether or not the PI / PID control circuit 122 is implemented, will be explained.

[0110] Figure 7A shows an example of Timer#1 controlled to synchronize with Timer#0 when the PI / PID control circuit 122 is implemented. Figure 7B shows an example of Timer#1 controlled to synchronize with Timer#0 when the PI / PID control circuit 122 is not implemented.

[0111] As shown in Figure 7B, if the PI / PID control circuit 122 is not implemented, the time information Timer#1 is updated to synchronize with time Timer#0 at each time synchronization period according to timing t5.

[0112] In contrast, as shown in Figure 7A, when the PI / PID control circuit 122 is implemented, the time information Timer#1 is corrected to synchronize with time Timer#0 at correction periods shorter than the time synchronization period according to timing t5 (in the illustrated example, a correction period of 1 / 3 of the time synchronization period). By correcting the time information Timer#1 to synchronize with time Timer#0 at correction periods shorter than the time synchronization period in this way, the error of time information Timer#1 relative to time information Timer#0 can be reduced.

[0113] Referring to Figure 8, the operation of the MCK divider 103, PMC 104, and PMC_result register 95 in Figure 3 will be explained.

[0114] First, the controller 93 reads the necessary setting information for each block of the audio decoder 87 from the setting information register 92 and writes it to each block in advance. For example, the controller 93 can write the N value that the MCK divider 103 uses to divide the master clock signal MCK to the MCK divider 103 in advance.

[0115] The MCK divider 103 divides the master clock signal MCK supplied from the ECU 25 according to the N value written by the controller 93, and supplies the resulting clock signal MCK_Div with frequency (=MCK / N) to the PMC 104. For example, as shown in Figure 8, if the master clock signal MCK is 24.576MHz and the N value is 1536, the MCK divider 103 supplies a clock signal MCK_Div of 16KHz (=24.576MHz / 1536) to the PMC 104.

[0116] The PMC 104 counts the period corresponding to the clock signal MCK_Div supplied from the MCK divider 103 according to the clock signal TxREFCK supplied from the oscillator 114 of the ECU 25, and writes the resulting PMC_result value to the PMC_result register 95. For example, if the clock signal MCK_Div supplied from the MCK divider 103 is 16KHz and the clock signal TxREFCK supplied from the oscillator 114 of the ECU 25 is 250MHz, the PMC 104 writes the resulting PMC_result value of 15625 (= 16KHz@62500ns / 250MHz@4ns) to the PMC_result register 95.

[0117] The PMC_result value written to the PMC_result register 95 in this manner is read by the I2C master 88 via the I2C slave 91, stored in I2C data, sent to the aggregator 23, and written to the PMC_result register 62 via the I2C slave 58.

[0118] Figure 9 is a block diagram showing an example configuration of the MCK generator 72 in Figure 2.

[0119] As shown in Figure 9, the MCK generator 72 is configured to include a controller register 131, a TxREFCK generator 132, a TxREFCK divider 133, an MCK generator core 134, a BCLK generator 135, and an LRCLK generator 136.

[0120] The controller register 131 stores the audio transmission setting information (see Figure 10) that is written by the controller 60 from the setting information stored in the setting information register 59.

[0121] The TxREFCK generator 132 divides or multiplies the clock signal RxREFCK supplied from the oscillator 55 according to the audio transmission setting information TxREFCK_INFO and RxREFCK_INFO stored in the controller register 131. As a result, the TxREFCK generator 132 generates the clock signal TxREFCK (= RxREFCK / (RxREFCK_INFO / TxREFCK_INFO)) and supplies it to the TxREFCK divider 133.

[0122] For example, as shown in the audio transmission setting information in Figure 10, if TxREFCK_INFO is 250MHz, RxREFCK_INFO is 500MHz, and the clock signal RxREFCK is 500MHz as shown in Figure 11, the TxREFCK generator 132 generates a clock signal TxREFCK of 250MHz (= 500MHz / (500MHz / 250MHz)). Note that if the clock signal TxREFCK and the clock signal RxREFCK are equal, the generation of the clock signal TxREFCK by the TxREFCK generator 132 is unnecessary, and the RxREFCK INFO field and TxREFCK INFO field in the audio transmission setting information are also unnecessary.

[0123] The TxREFCK divider 133 divides the clock signal TxREFCK supplied from the TxREFCK generator 132 by the PMC_result value read from the PMC_result register 62. This allows the TxREFCK divider 133 to reconstruct the clock signal MCK_Div (=TxREFCK / PMC_result) and supply it to the MCK generator core 134. For example, as shown in Figure 11, if the clock signal TxREFCK is 250MHz and the PMC_result value is 15625, the TxREFCK divider 133 reconstructs a clock signal MCK_Div of 16KHz (=250MHz / 15625).

[0124] The MCK generator core 134 multiplies the clock signal MCK_Div supplied from the TxREFCK divider 133 by the N value of the audio transmission setting information stored in the controller register 131. This allows the MCK generator core 134 to reconstruct the master clock signal MCK (=MCK_Div / N), supply it to the BCLK generator 135, and output it externally. For example, as shown in Figure 11, if the clock signal MCK_Div is 16KHz and the N value is 1536 as shown in Figure 10, the MCK generator core 134 reconstructs the master clock signal MCK at 24.576MHz (=16KHz / 1536).

[0125] The BCLK generator 135 calculates a frequency division value (= K value / (SDBW × Number of Channels)) for dividing the master clock signal MCK based on the K value, SDBW (I2S Data Bit Width), and Ch number (Number of Channels) of the audio transmission setting information stored in the controller register 131. For example, as shown in Figure 10, if the K value is 512, the SDBW is 32, and the Ch number is 8, the BCLK generator 135 calculates a frequency division value of 2 (= 512 / (32 × 8)). The BCLK generator 135 then divides the master clock signal MCK supplied from the MCK generator core 134 by the frequency division value to calculate the bit clock signal BCLK, which is supplied to the LRCLK generator 136 and also output externally. For example, as shown in Figure 11, if the master clock signal MCK is 24.576MHz and the frequency divider value is 2, the BCLK generator 135 calculates a bit clock signal BCLK of 12.288MHz (= 24.576MHz / 2).

[0126] The LRCLK generator 136 divides the bit clock signal BCLK supplied from the BCLK generator 135 based on the SDBW (I2S Data Bit Width) and the number of channels (Ch) of the audio transmission setting information stored in the controller register 131. This allows the LRCLK generator 136 to reconstruct the LR clock signal LRCLK (= BCLK / (SDBW × number of channels)) and output it externally. For example, as shown in Figure 11, if the bit clock signal BCLK is 12.288MHz, and as shown in Figure 10, if the SDBW is 32 and the number of channels is 8, the LRCLK generator 136 reconstructs the LR clock signal LRCLK at 48KHz (= 12.288MHz / (32 × 8)).

[0127] Figure 12 shows a detailed example of the I2S configuration information written to the controller register 131. For example, only the necessary information from this configuration information may be implemented according to the implementation configuration.

[0128] Figure 13 shows an example of the format of a CSI-2 packet containing image data and I2S format audio data generated by the CSI-2 multi-encoder 65 in Figure 2.

[0129] As shown in Figure 13, in the CSI-2 packet format, the virtual channel VC0 is assigned to the image data Video1 Data, and each image data Video1 Data(n) stores one frame of image data Video1 Data. In addition, in the CSI-2 packet format, the virtual channel VC1 is assigned to the audio data Audio Data, and each audio data Audio Data(n) stores the audio data corresponding to one frame, while dummy data is stored in the area where no audio data is stored.

[0130] Thus, the CSI-2 packet format to which this technology is applied is novel compared to conventional CSI-2 packet formats in that image data (Video1 Data) and audio data (Audio Data) are stored together in a single CSI-2 packet.

[0131] Figure 14 is a diagram illustrating the data flow in the aggregator 23 shown in Figure 2.

[0132] As described above, the image data Video1 Data0 supplied from the AD converter 32 of the CMOS image sensor 21 to the CSI-2 encoder 33 is encoded in a data format compliant with the CSI-2 standard by the CSI-2 encoder 33 and input to the CSI-2 decoder 53 via the C-PHY 34 and C-PHY 52. The image data Video1 Data0, which has been decoded by the CSI-2 decoder 53 and restored from the data format compliant with the CSI-2 standard to the original data format, is then stored in the CSI-2 decode FIFO 54.

[0133] Furthermore, the audio data (SDATA) supplied from the AD converter 42 of the microphone 22 to the I2S encoder 73 via the I2S slave 43 and I2S master 71 is encoded in the I2S encoder 73 into a data format compliant with the I2S standard. The I2S format audio data, Audio Data0, is then stored in the I2S encoded FIFO 64.

[0134] As a result, the CSI-2 multi-encoder 65 can read out the image data Video1 Data0 stored in the CSI-2 decode FIFO 54 and the I2S format audio data Audio Data0 stored in the I2S encode FIFO 64.

[0135] Figure 15 is a block diagram showing an example configuration of the CSI-2 multi-encoder 65 in Figure 2, and Figure 16 shows an example of the data flow in the CSI-2 multi-encoder 65.

[0136] As shown in Figure 15, the CSI-2 multi-encoder 65 is configured to include a VC0 line buffer 141, a VC1 line buffer 142, and a Tx controller 143.

[0137] The VC0 line buffer 141 is a double buffer consisting of VC0 Line Buffer0 and VC0 Line Buffer1. The VC1 line buffer 142 is a double buffer consisting of VC1 Line Buffer0 and VC1 Line Buffer1. The detailed operation of the VC0 line buffer 141 and VC1 line buffer 142 will be explained by referring to the flowcharts shown in Figures 17 and 18 below.

[0138] The Tx controller 143 is configured to include an SP / LPH generator 151, a packet formatter 152, and a line distribution 153.

[0139] The SP / LPH generator 151 generates the necessary short packets (SP) based on the values ​​(VC0Ln, VC1Ln) of the line counter 154 of the CSI-2 packets supplied from the packet formatter 152, and supplies them to the packet formatter 152. For example, the SP / LPH generator 151 generates short packets as shown in Figure 16, namely, frame start FS, frame end FE, line start LS, and line end LE. The SP / LPH generator 151 also generates a packet header (LPH) for a long packet as shown in Figure 16 and supplies it to the packet formatter 152. In addition, the SP / LPH generator 151 generates Embedded Data as needed and supplies it to the packet formatter 152.

[0140] The packet formatter 152 supplies the short packets (frame start FS, frame end FE, line start LS, and line end LE) supplied from the SP / LPH generator 151 to the line distribution 153 as packet data for CSI-2 packets. It also supplies the packet header of the long packet supplied from the SP / LPH generator 151, and the long packet containing the image data Video1 Data0 read from the VC0 line buffer 141, to the line distribution 153 as packet data for CSI-2 packets. Similarly, it supplies the packet header of the long packet supplied from the SP / LPH generator 151, the I2S formatted audio data Audio Data0 read from the VC1 line buffer 142, and the long packet containing dummy data to the line distribution 153 as packet data for CSI-2 packets.

[0141] For example, the packet formatter 152 updates the WC section of the packet header with WC values ​​(VC0_WC, VC1WC) from the VC0 line buffer 141 and VC1 line buffer 142, as shown in the long packet in Figure 16. The image data (VC0__RDATA) read from the VC0 line buffer 141, the image data (VC1__RDATA) read from the VC1 line buffer 142, and I2S_WC are stored in the Packet Data section of the Long Packet. The storage location for I2S_WC is the Audio Header shown in Figure 24, which will be described later.

[0142] The line distribution 153 converts the packet data of the CSI-2 packets supplied from the packet formatter 152 into a PPI format corresponding to the number of lines implemented in the D-PHY 66 and outputs it to the D-PHY 66. For example, a detailed example of one line of data from a CSI-2 packet transmitted from the CSI-2 multi-encoder 65 to the D-PHY 66 is shown at the bottom of Figure 16.

[0143] The operation of the VC0 line buffer 141 will be explained with reference to the flowchart shown in Figure 17.

[0144] In step S21, the VC0 line buffer 141 sets an initial value (for example, n=0) to parameter n, which specifies one of VC0 Line Buffer0 and VC0 Line Buffer1 as the line buffer to which image data will be written. For example, if parameter n is set to 0, VC0 Line Buffer0 will be the target to which image data will be written, and if parameter n is set to 1, VC0 Line Buffer1 will be the target to which image data will be written.

[0145] In step S22, the VC0 line buffer 141 determines whether the CSI-2 decode FIFO 54 is empty or not according to the empty_csi2 signal output from the CSI-2 decode FIFO 54. If the CSI-2 decode FIFO 54 is empty, the VC0 line buffer 141 waits for processing to begin. If the VC0 line buffer 141 determines that the CSI-2 decode FIFO 54 is not empty, the process proceeds to step S23. In this case, the CSI-2 decode FIFO 54 stores the image data (CSI_WDATA) written from the CSI-2 decoder 53.

[0146] In step S23, the VC0 line buffer 141 determines whether the line buffer specified by parameter n (Line Buffer(n)) is in a state where image data can be written, and waits until it determines that the line buffer specified by parameter n is in a state where image data can be written. If the VC0 line buffer 141 determines that the line buffer specified by parameter n is in a state where image data can be written, the process proceeds to step S24.

[0147] In step S24, the VC0 line buffer 141 outputs a read-enabled signal CSI_REN (CSI_REN = enable) to the CSI-2 decode FIFO 54 for one clock cycle, indicating that image data can be read. Then, the VC0 line buffer 141 acquires the image data (CSI_RDATA) output from the CSI-2 decode FIFO 54 in response to the read-enabled signal CSI_REN and writes it to the line buffer (Line Buffer(n)) specified by parameter n.

[0148] In step S25, the VC0 line buffer 141 determines whether the line buffer (Line Buffer(n)) specified by parameter n has become full with image data.

[0149] In step S25, if the VC0 line buffer 141 determines that the line buffer specified by parameter n is not full with image data, the process returns to step S22, and the same process is repeated thereafter. On the other hand, in step S25, if the VC0 line buffer 141 determines that the line buffer specified by parameter n is full with image data, the process proceeds to step S26.

[0150] In step S26, the VC0 line buffer 141 outputs a VC0LB_full signal (VC0LB_full == full) to the packet formatter 152 for one clock cycle, indicating that it has reached a full state with image data, and also outputs a word count value VC0_WC, which indicates the existing setting value, to the packet formatter 152.

[0151] In step S27, the VC0 line buffer 141 determines whether the packet formatter 152 is in a state where it can read image data, according to the read-enabled signal VC0_REN output from the packet formatter 152, and waits to process until it determines that the packet formatter 152 is in a state where it can read image data. If the VC0 line buffer 141 determines that the packet formatter 152 is in a state where it can read image data (VC0_REN == enable), the process proceeds to step S28.

[0152] In step S28, the VC0 line buffer 141 supplies the image data (VC0_RDATA) stored in the line buffer specified by parameter n to the packet formatter 152. The VC0 line buffer 141 then inverts parameter n, which specifies the line buffer to which the image data will be written (for example, if parameter n is set to 0, it is set to 1, and if parameter n is set to 1, it is set to 0).

[0153] In step S29, the VC0 line buffer 141 determines whether to terminate reading image data from the CSI-2 decode FIFO 54. If it determines that it will not terminate reading image data from the CSI-2 decode FIFO 54, the process returns to step S22, and the same process is repeated thereafter. On the other hand, in step S29, if the VC0 line buffer 141 determines that it will terminate reading image data from the CSI-2 decode FIFO 54, the process is terminated.

[0154] The operation of the VC1 line buffer 142 will be explained with reference to the flowchart shown in Figure 18.

[0155] In step S31, the VC1 line buffer 142 sets an initial value (for example, n=0) for parameter n, which specifies one of VC1 Line Buffer0 and VC1 Line Buffer1 as the line buffer to which audio data will be written. For example, if parameter n is set to 0, VC1 Line Buffer0 will be the target to which audio data will be written, and if parameter n is set to 1, VC1 Line Buffer1 will be the target to which audio data will be written.

[0156] In step S32, the VC1 line buffer 142 determines whether the I2S encoding FIFO 64 is empty according to the empty_i2s signal output from the I2S encoding FIFO 64. If the I2S encoding FIFO 64 is empty, the VC1 line buffer 142 waits for processing to begin. If the VC1 line buffer 142 determines that the I2S encoding FIFO 64 is not empty, the process proceeds to step S33. In this case, the I2S encoding FIFO 64 stores the audio data (I2S_WDATA) written from the I2S encoder 73.

[0157] In step S33, the VC1 line buffer 142 determines whether the line buffer specified by parameter n (Line Buffer(n)) is in a state where audio data can be written, and waits until it determines that the line buffer specified by parameter n is in a state where audio data can be written. If the VC1 line buffer 142 determines that the line buffer specified by parameter n is in a state where audio data can be written, the process proceeds to step S34.

[0158] In step S34, the VC1 line buffer 142 determines whether the packet formatter 152 is in a state where it can read image data, according to the readable signal VC0_REN output from the packet formatter 152. If the VC1 line buffer 142 determines in step S34 that the packet formatter 152 is not in a state where it can read image data, the process proceeds to step S35.

[0159] In step S35, the VC1 line buffer 142 outputs a read-enabled signal I2S_REN (I2S_REN = enable) to the I2S encoding FIFO 64 for one clock cycle, indicating that audio data is readable. Then, the VC1 line buffer 142 acquires the audio data (I2S_RDATA) output from the I2S encoding FIFO 64 in response to the read-enabled signal I2S_REN and writes it to the line buffer (Line Buffer(n)) specified by parameter n.

[0160] In step S36, the VC1 line buffer 142 determines whether the line buffer (Line Buffer(n)) specified by parameter n has become full with audio data.

[0161] In step S36, if the VC1 line buffer 142 determines that the line buffer specified by parameter n is not full with audio data, the process returns to step S32, and the same process is repeated thereafter. On the other hand, in step S36, if the VC1 line buffer 142 determines that the line buffer specified by parameter n is full with audio data, the process proceeds to step S37.

[0162] In step S37, the VC1 line buffer 142 outputs a VC1LB_full signal (VC1LB_full == full) to the packet formatter 152 for one clock cycle, indicating that it has reached a full state with image data. It also outputs a word count value VC1_WC, indicating the existing setting value, to the packet formatter 152, after which the process proceeds to step S39.

[0163] On the other hand, if in step S34 the VC1 line buffer 142 determines that the packet formatter 152 is in a state where it can read image data, the process proceeds to step S38.

[0164] In step S38, the VC1 line buffer 142 writes dummy data to all unwritten areas in the line buffer (Line Buffer(n)) specified by parameter n where no audio data has been written. Then, the VC1 line buffer 142 outputs a VC1LB_full signal (VC1LB_full == full) to the packet formatter 152 for one clock cycle, indicating that the image data is full, and also outputs a word count value VC1_WC, which indicates the existing setting value, to the packet formatter 152, after which the process proceeds to step S39.

[0165] In step S39, the VC1 line buffer 142 determines whether the packet formatter 152 is in a state where it can read audio data from the line buffer (Line Buffer(n)) specified by parameter n, according to the read-able signal VC1_REN output from the packet formatter 152, and waits until the packet formatter 152 determines that it is in a state where it can read audio data. If the VC1 line buffer 142 determines that the packet formatter 152 is in a state where it can read audio data (VC1_REN == enable), the process proceeds to step S40.

[0166] In step S40, the VC1 line buffer 142 supplies the audio data (I2S_RDATA) stored in the line buffer specified by parameter n to the packet formatter 152, and outputs a word count value I2S_WC, which indicates the amount of data excluding dummy data, to the packet formatter 152. Then, the VC1 line buffer 142 inverts parameter n, which specifies the line buffer to which the audio data will be written (for example, if parameter n is set to 0, it is set to 1, and if parameter n is set to 1, it is set to 0).

[0167] In step S41, the VC1 line buffer 142 determines whether to finish reading audio data from the I2S encoded FIFO 64. If it determines that it will not finish reading audio data from the I2S encoded FIFO 64, the process returns to step S32, and the same process is repeated thereafter. On the other hand, in step S41, if the VC1 line buffer 142 determines that it will finish reading audio data from the I2S encoded FIFO 64, the process is terminated.

[0168] As described above, the operation of the VC0 line buffer 141 and the VC1 line buffer 142 ensures that the line data on the VC1 line buffer 142 side, which transmits audio data, is read at a timing synchronized with the reading of line data on the VC0 line buffer 141 side, which transmits image data. Therefore, in the flowchart shown in Figure 17, for example, when the VC0 Line Buffer0 of the VC0 line buffer 141 has finished writing all the image data, only some of the audio data has been written to the VC1 Line Buffer0 of the VC1 line buffer 142. However, the remaining unwritten area is forcibly filled with dummy data, and the audio data on the VC1 line buffer 142 side is read to the packet formatter 152 at a timing synchronized with the VC0 line buffer 141 side.

[0169] Therefore, the CSI-2 multi-encoder 65 can store audio data in CSI-2 packets in synchronization with image data at the frame rate of the image data.

[0170] Figure 19 is a diagram illustrating the data flow in the disaggregator 24 shown in Figure 3.

[0171] As described above, CSI-2 packets are supplied from the CSI-2 multi-encoder 65 of the aggregator 23 in Figure 2 to the CSI-2 multi-decoder 82 in Figure 3 via the D-PHY 66 and D-PHY 81. The CSI-2 multi-decoder 82 then stores the image data Video1 Data0 stored in the virtual channel VC0 of the CSI-2 packet into the CSI-2 encoded FIFO 83, and the I2S format audio data Audio Data0 stored in the virtual channel VC1 of the CSI-2 packet into the I2S decoded FIFO 86.

[0172] The image data Video1 Data0, read from the CSI-2 encoded FIFO 83 to the CSI-2 encoder 84, is encoded by the CSI-2 encoder 84 into a data format compliant with the CSI-2 standard. The CSI-2 formatted image data Video1 Data0 is then supplied to the CSI-2 decoder 112 of the ECU 25 via the C-PHY 85 and C-PHY 111, where it is decoded by the CSI-2 decoder 112 and restored from the CSI-2 standard data format back to the original data format.

[0173] The I2S-formatted audio data, Audio Data0, read from the I2S decode FIFO 86 to the I2S decoder 101 is decoded by the I2S decoder 101 and restored from the data format according to the I2S standard back to the original data format. The audio data, Audio Data0, is then supplied to the ECU 25 via the I2S slave 102 and the I2S master 113.

[0174] Figure 20 is a block diagram showing an example configuration of the CSI-2 multidecoder 82, and Figure 21 shows an example of the data flow in the CSI-2 multidecoder 82.

[0175] As shown in Figure 20, the CSI-2 multi-decoder 82 is configured to include an Rx controller 161, a line adjustment unit 162, a packet decoder 163, a VC0 line buffer 164, and a VC1 line buffer 165. CSI-2 packets transmitted from the aggregator 23 are input to the CSI-2 multi-decoder 82 in PPI format corresponding to the number of lines implemented in the D-PHY 81. For example, a detailed example of one line of data from a CSI-2 packet transmitted from the D-PHY 81 to the CSI-2 multi-decoder 82 is shown at the bottom of Figure 21.

[0176] The Rx controller 161 controls the line adjustment unit 162 and the packet decoder 163.

[0177] The line adjustment unit 162, in accordance with the control of the Rx controller 161, adjusts the disks between each line for PPI format data input via multiple lines, integrates them to obtain the original CSI-2 packet, and supplies it to the packet decoder 163.

[0178] The packet decoder 163, in accordance with the control of the Rx controller 161, decodes the CSI-2 packets supplied from the line adjustment unit 162 into short packets (frame start FS, frame end FE, line start LS, and line end LE) as shown in Figure 21, long packets containing image data Video1 Data0, and long packets containing audio data Audio Data0 in I2S format.

[0179] The packet decoder 163 then refers to the packet header of the long packet containing the image data Video1 Data0. If packet data for the virtual channel VC0 follows, it supplies a VC0LB_DEC signal (VC0LB_DEC == finish) to the VC0 line buffer 164, indicating that the decoding of the image data is complete. If the VC0 line buffer 164 is writable, it supplies a read-enabled signal VC0_REN (VC0_REN = enable) to the packet decoder 163, indicating that the image data is readable. In that case, the packet decoder 163 supplies the image data Video1 Data to the VC0 line buffer 164.

[0180] On the other hand, the packet decoder 163 refers to the packet header of the long packet containing the I2S formatted audio data Audio Data0, and if packet data for virtual channel VC1 follows, it supplies a VC1LB_DEC signal (VC1LB_DEC == finish) to the VC1 line buffer 165 indicating that the decoding of the audio data is complete. Also, if the VC1 line buffer 165 is writable, a read-enabled signal VC1_REN (VC1_REN = enable) is supplied to the packet decoder 163 indicating that the audio data is readable. In that case, the packet decoder 163 supplies the audio data Audio Data to the VC1 line buffer 165 along with Data_Valid, which indicates a valid audio data area (i.e., an area that does not contain dummy data).

[0181] In the example shown in Figure 21, the Audio Data written to the VC1 line buffer 165 has a Data_Valid of 0 = 0 (Disable) because one of the two blocks, the latter half, is dummy data. For example, the packet decoder 163 can determine how much dummy data is included based on the word count value VC1_WC contained in the long packet header of the decoded CSI-2 packet and the word count value I2S_WC contained in the header of the audio data.

[0182] The VC0 line buffer 164 is a double buffer consisting of VC0 Line Buffer0 and VC0 Line Buffer1. The VC1 line buffer 165 is a double buffer consisting of VC1 Line Buffer0 and VC1 Line Buffer1. The detailed operation of the VC0 line buffer 164 and VC1 line buffer 165 will be explained by referring to the flowcharts shown in Figures 22 and 23 below.

[0183] The operation of the VC0 line buffer 164 will be explained with reference to the flowchart shown in Figure 22.

[0184] In step S51, the VC0 line buffer 164 sets an initial value (for example, n=0) for parameter n, which specifies one of VC0 Line Buffer0 and VC0 Line Buffer1 as the line buffer to which image data will be written. For example, if parameter n is set to 0, VC0 Line Buffer0 will be the target to which image data will be written, and if parameter n is set to 1, VC0 Line Buffer1 will be the target to which image data will be written.

[0185] In step S52, the VC0 line buffer 164 determines whether the packet decoder 163 has finished decoding one line of image data according to the VC0LB_DEC signal output from the packet decoder 163, and waits for processing until it determines that the packet decoder 163 has finished decoding one line of image data. If the VC0 line buffer 164 determines that the packet decoder 163 has finished decoding one line of image data (VC0LB_DEC == finish), the process proceeds to step S53.

[0186] In step S53, the VC0 line buffer 164 outputs a read-enabled signal VC0_REN (VC0_REN = enable) to the packet decoder 163 for one clock cycle, indicating that the image data is readable. Then, the VC0 line buffer 164 acquires the image data (VC0_Data) output from the packet decoder 163 in response to the read-enabled signal VC0_REN and writes it to the line buffer (Line Buffer(n)) specified by parameter n.

[0187] In step S54, the VC0 line buffer 164 determines whether the CSI-2 encoding FIFO 83 is full of image data according to the full_csi2 signal supplied from the CSI-2 encoding FIFO 83. If the CSI-2 encoding FIFO 83 is full of image data, the VC0 line buffer 164 waits for processing to begin. If the VC0 line buffer 164 determines that the CSI-2 encoding FIFO 83 is not full of image data, the process proceeds to step S55.

[0188] In step S55, the VC0 line buffer 164 outputs a writable signal CSI_WEN (CSI_WEN = enable) to the CSI-2 encoded FIFO 83 for one clock cycle, indicating that image data is writable, and outputs image data (CSI_WDATA) from the line buffer specified by parameter n (Line Buffer(n)) and writes it to the CSI-2 encoded FIFO 83. Then, the VC0 line buffer 164 inverts the parameter n that specifies the line buffer to which the image data is to be written (for example, if parameter n is set to 0, it is set to 1, and if parameter n is set to 1, it is set to 0).

[0189] In step S56, the VC0 line buffer 164 determines whether or not to finish writing the image data to the CSI-2 encoded FIFO 83. If it determines that it will not finish writing the image data to the CSI-2 encoded FIFO 83, the process returns to step S52, and the same process is repeated thereafter. On the other hand, in step S56, if the VC0 line buffer 164 determines that it will finish writing the image data to the CSI-2 encoded FIFO 83, the process is terminated.

[0190] The operation of the VC1 line buffer 165 will be explained with reference to the flowchart shown in Figure 23.

[0191] In step S61, the VC1 line buffer 165 sets an initial value (for example, n=0) for parameter n, which specifies one of VC1 Line Buffer0 and VC1 Line Buffer1 as the line buffer to which audio data will be written. For example, if parameter n is set to 0, VC1 Line Buffer0 will be the target to which audio data will be written, and if parameter n is set to 1, VC1 Line Buffer1 will be the target to which audio data will be written.

[0192] In step S62, the VC1 line buffer 165 determines whether the packet decoder 163 has finished decoding one line of audio data according to the VC1LB_DEC signal output from the packet decoder 163, and waits for processing until it determines that the packet decoder 163 has finished decoding one line of audio data. If the VC1 line buffer 165 determines that the packet decoder 163 has finished decoding one line of audio data (VC1LB_DEC == finish), the process proceeds to step S63.

[0193] In step S63, the VC1 line buffer 165 outputs a read-enabled signal VC1_REN (VC1_REN = enable) to the packet decoder 163 for one clock cycle, indicating that audio data is readable. Then, the VC1 line buffer 165 retrieves the audio data according to Data_Valid, which identifies the write area other than dummy data from the data (VC1_Data) output from the packet decoder 163 in response to the read-enabled signal VC1_REN, and writes it to the line buffer specified by parameter n.

[0194] In step S64, the VC1 line buffer 165 determines whether the I2S decode FIFO 86 is full of audio data according to the full_i2s signal supplied from the I2S decode FIFO 86. If the I2S decode FIFO 86 is full of audio data, the VC1 line buffer 165 waits for processing to begin. If the VC1 line buffer 165 determines that the I2S decode FIFO 86 is not full of audio data, the process proceeds to step S65.

[0195] In step S65, the VC1 line buffer 165 outputs a writable signal I2S_WEN (I2S_WEN = enable) to the I2S decode FIFO 86 for one clock cycle, indicating that audio data can be written, and outputs audio data (I2S_WDATA) from the line buffer specified by parameter n (Line Buffer(n)) and writes it to the I2S decode FIFO 86. Then, the VC1 line buffer 165 inverts parameter n, which specifies the line buffer to which the audio data will be written (for example, if parameter n is set to 0, it is set to 1, and if parameter n is set to 1, it is set to 0).

[0196] In step S66, the VC1 line buffer 165 determines whether to finish writing audio data to the I2S decode FIFO 86. If it determines that it will not finish writing audio data to the I2S decode FIFO 86, the process returns to step S62, and the same process is repeated thereafter. On the other hand, in step S66, if the VC1 line buffer 165 determines that it will finish writing audio data to the I2S decode FIFO 86, the process is terminated.

[0197] Figure 24 shows an example of a long packet transmitted using D-PHY. Note that similar long packets can be transmitted using C-PHY instead of D-PHY.

[0198] Furthermore, in this long packet, the data type for voice data is 0x3A in the example shown in Figure 24A, and 0x30 in the example shown in Figure 24B. However, other data types (for example, 0x3B~0x3D, 0x31~0x37) may also be assigned to image data and data other than voice data.

[0199] Figure 25 shows an example of the data format of a long packet in which voice data is included in the data area of ​​the virtual channel VC1 within the CSI-2 packet.

[0200] Figure 26 shows an example of I2S data (Audio Data) for each SDBW (I2S Data Bit Width). As shown in Figure 26, the SDBW can be set to 8 bits, 12 bits, 16 bits, 20 bits, 24 bits, and 32 bits.

[0201] The communication system 11 configured as described above can synchronize time between the aggregator 23 and the disaggregator 24 using I2C, and can transmit image data and voice data from the aggregator 23 to the disaggregator 24 via D-PHY in accordance with the MIPI CSI-2 standard.

[0202] Furthermore, the communication system 11 can assign an Audio Type other than Video (for example, I2S Data) to the Data Type of CSI-2. The aggregator 23 and disaggregator 24 perform time synchronization using I2C, and the timer 61 can synchronize time Timer#1 with time Timer#0 using the synchronization time calculation unit 121. In addition, by including a PI / PID control circuit 122, the timer 61 can reduce the error between time information Timer#1 and time information Timer#0.

[0203] <Second Configuration Example of the Communication System> Referring to Figures 27 to 31, a configuration example of a second embodiment of a communication system to which this technology is applied will be described. In the communication system 11A described with reference to Figures 27 to 31, components common to the communication system 11 in Figure 1 are denoted by the same reference numerals, and their detailed explanations are omitted.

[0204] For example, as shown in Figures 27 and 28, the communication system 11A of the second embodiment is configured similarly to the communication system 11 in Figure 1, with a CMOS image sensor 21 and a microphone 22 connected to an aggregator 23A, and a display 26 and a speaker 27 connected to a disaggregator 24A via an ECU 25A. Between the aggregator 23A and the disaggregator 24A, CSI-2 packets are transmitted, in which image data and audio data are combined and stored in a single packet using a D-PHY as the physical layer, and time synchronization is performed using I2C.

[0205] Figure 27 is a block diagram showing an example configuration of a CMOS image sensor 21, a microphone 22, and an aggregator 23A.

[0206] As shown in Figure 27, the aggregator 23A is configured similarly to the aggregator 23 in Figure 2, comprising an I2C master 51, a C-PHY 52, a CSI-2 decoder 53, a CSI-2 decode FIFO 54, an oscillator 55, I2C slaves 56-58, a configuration information register 59, a controller 60, a timer 61, a PMC_result register 62, an I2S encode FIFO 64, a CSI-2 multi-encoder 65, and a D-PHY 66.

[0207] Furthermore, the aggregator 23A has a different configuration from the aggregator 23 in Figure 2, in that it includes an audio encoder 63A. The audio encoder 63A is configured similarly to the audio encoder 63 in Figure 2, in that it has an I2S master 71 and an I2S encoder 73, but it differs from the audio encoder 63 in Figure 2 in that it includes a BCLK generator 72A.

[0208] The BCLK generator 72A generates the bit clock signal BCLK and the LR clock signal LRCLK according to the clock signal RxREFCK supplied from the oscillator 55, the setting information supplied from the controller 60, and the PMC_result value supplied from the PMC_result register 62, and supplies them to the I2S master 71 and the I2S encoder 73, respectively. The configuration of the BCLK generator 72A will be explained later with reference to Figure 30.

[0209] Figure 28 is a block diagram showing an example configuration of the disaggregator 24A and the ECU 25A.

[0210] As shown in Figure 28, the disaggregator 24A is configured similarly to the disaggregator 24 in Figure 3, comprising a D-PHY 81, a CSI-2 multi-decoder 82, a CSI-2 encoding FIFO 83, a CSI-2 encoder 84, a C-PHY 85, an I2S decoding FIFO 86, an I2C master 88, I2C slaves 89-91, a configuration information register 92, a controller 93, a timer 94, and a PMC_result register 95.

[0211] Furthermore, the disaggregator 24A differs from the disaggregator 24 in Figure 3 in that it includes an audio decoder 87A. The audio decoder 87A is configured similarly to the audio decoder 87 in Figure 3 in that it has an I2S decoder 101, an I2S slave 102, and a PMC 104, but differs from the audio decoder 87 in Figure 3 in that it includes a BCLK divider 103A.

[0212] The BCLK divider 103A is supplied with the bit clock signal BCLK from the I2S master 113A of the ECU 25A. The BCLK divider 103A then supplies the PMC 104 with a clock signal BCLK_Div, which has a frequency (=BCLK / N) obtained by dividing the bit clock signal BCLK supplied from the ECU 25 according to the N value supplied from the controller 93.

[0213] ECU 25A is configured similarly to ECU 25 in Figure 3, in that it includes a C-PHY 111, a CSI-2 decoder 112, and an oscillator 114. However, ECU 25A differs from ECU 25 in that it includes an I2S master 113A.

[0214] The I2S master 113A is configured to also supply the bit clock signal BCLK to the BCLK divider 103A.

[0215] As described above, the PMC_result value was determined between the aggregator 23 in Figure 2 and the disaggregator 24 in Figure 3 based on the master clock signal MCK, whereas the PMC_result value is determined between the aggregator 23A and the disaggregator 24A based on the bit clock signal BCLK.

[0216] Referring to Figure 29, the operation of the BCLK divider 103A, PMC 104, and PMC_result register 95 will be described.

[0217] First, the controller 93 reads the necessary setting information for each block of the audio decoder 87A from the setting information register 92 and writes it to each block in advance. For example, the controller 93 can write the N value that the BCLK divider 103A uses to divide the bit clock signal BCLK to the BCLK divider 103A in advance.

[0218] The BCLK divider 103A supplies the PMC 104 with a clock signal BCLK_Div whose frequency is obtained by dividing the bit clock signal BCLK supplied from the ECU 25 according to the N value supplied from the controller 93 (= BCLK / N). For example, as shown in Figure 29, if the bit clock signal BCLK is 12.288MHz and the N value is 768, the BCLK divider 103A supplies the PMC 104 with a clock signal BCLK_Div of 16KHz (= 12.288MHz / 768).

[0219] The PMC 104 counts the period corresponding to the clock signal BCLK_Div supplied from the BCLK divider 103A according to the clock signal TxREFCK supplied from the oscillator 114 of the ECU 25, and writes the resulting PMC_result value to the PMC_result register 95. For example, if the clock signal MCK_Div supplied from the MCK divider 103 is 16KHz and the clock signal TxREFCK supplied from the oscillator 114 of the ECU 25 is 250MHz, the PMC 104 writes the resulting PMC_result value of 15625 (= 16KHz@62500ns / 250MHz@4ns) to the PMC_result register 95.

[0220] The PMC_result value written to the PMC_result register 95 in this manner is read by the I2C master 88 via the I2C slave 91, stored in I2C data, sent to the aggregator 23A, and written to the PMC_result register 62 via the I2C slave 58.

[0221] Figure 30 is a block diagram showing an example configuration of the BCLK generator 72A.

[0222] As shown in Figure 30, the BCLK generator 72A is configured to include a controller register 131, a TxREFCK generator 132, a TxREFCK divider 133A, an LRCLK generator 136, and a BCLK generator core 137.

[0223] The controller register 131 stores the audio transmission setting information (see Figure 10 above) that is written by the controller 60 from the setting information stored in the setting information register 59.

[0224] The TxREFCK generator 132 generates the clock signal TxREFCK (= RxREFCK / (RxREFCK_INFO / TxREFCK_INFO)) by dividing or multiplying the clock signal RxREFCK supplied from the oscillator 55 according to the audio transmission setting information TxREFCK_INFO and RxREFCK_INFO stored in the controller register 131, and supplies it to the TxREFCK divider 133A.

[0225] For example, as shown in the audio transmission setting information in Figure 10 above, if TxREFCK_INFO is 250MHz, RxREFCK_INFO is 500MHz, and the clock signal RxREFCK is 500MHz as shown in Figure 31, the TxREFCK generator 132 generates a clock signal TxREFCK of 250MHz (= 500MHz / (500MHz / 250MHz)). Note that if the clock signal TxREFCK and the clock signal RxREFCK are equal, the generation of the clock signal TxREFCK by the TxREFCK generator 132 is unnecessary, and the RxREFCK INFO field and TxREFCK INFO field in the audio transmission setting information are also unnecessary.

[0226] The TxREFCK divider 133A divides the clock signal TxREFCK supplied from the TxREFCK generator 132 by the PMC_result value read from the PMC_result register 62. This allows the TxREFCK divider 133A to reconstruct the clock signal BCLK_Div (=TxREFCK / PMC_result) and supply it to the BCLK generator core 137. For example, as shown in Figure 31, if the clock signal TxREFCK is 250MHz and the PMC_result value is 15625, the TxREFCK divider 133A reconstructs a clock signal BCLK_Div of 16KHz (=250MHz / 15625).

[0227] The BCLK generator core 137 multiplies the clock signal BCLK_Div supplied from the TxREFCK divider 133A by the N value of the audio transmission setting information stored in the controller register 131. This allows the BCLK generator core 137 to reconstruct the bit clock signal BCLK (= BCLK_Div / N), supply it to the LRCLK generator 136, and output it externally. For example, as shown in Figure 31, if the clock signal BCLK_Div is 16KHz and the N value is 768, the BCLK generator core 137 reconstructs a bit clock signal BCLK of 12.288MHz (= 16KHz / 768).

[0228] The LRCLK generator 136 divides the bit clock signal BCLK supplied from the BCLK generator core 137 based on the SDBW (I2S Data Bit Width) and the number of channels (Ch) of the audio transmission setting information stored in the controller register 131. This allows the LRCLK generator 136 to reconstruct the LR clock signal LRCLK (= BCLK / (SDBW × number of channels)) and output it externally. For example, as shown in Figure 31, if the bit clock signal BCLK is 12.288 MHz, and as shown in Figure 10 above, if the SDBW is 32 and the number of channels is 8, the LRCLK generator 136 reconstructs the LR clock signal LRCLK at 48 kHz (= 12.288 MHz / (32 × 8)).

[0229] As described above, the communication system 11A is configured to determine the PMC_result value based on the bit clock signal BCLK. Furthermore, similar to the communication system 11 in Figure 1, the communication system 11A can synchronize time between the aggregator 23A and the disaggregator 24A using I2C, and can transmit image data and audio data from the aggregator 23A to the disaggregator 24A via D-PHY in accordance with the MIPI CSI-2 standard.

[0230] <Third Configuration Example of the Communication System> Referring to Figures 32 to 34, a configuration example of a third embodiment of a communication system to which this technology is applied will be described. In the communication system 11B described with reference to Figures 32 to 34, components common to the communication system 11 in Figure 1 are denoted by the same reference numerals, and their detailed explanations are omitted.

[0231] Figure 32 is a block diagram showing an example configuration of the communication system 11B.

[0232] As shown in Figure 32, the communication system 11B is configured to include an aggregator 23B, a disaggregator 24B, an ECU 25B, a display 26, and a speaker 27.

[0233] In the communication system 11B, image data and audio data are transmitted from the ECU 25B to the display 26 and speaker 27. Therefore, the communication system 11B is configured such that the ECU 25B is connected to the aggregator 23B, and the display 26 and speaker 27 are connected to the disaggregator 24B. Between the aggregator 23B and the disaggregator 24B, CSI-2 packets are transmitted, in which image data and audio data are combined and stored in a single packet using D-PHY as the physical layer, and time synchronization is performed using I2C.

[0234] Figure 33 is a block diagram showing an example configuration of the aggregator 23B and the ECU 25B.

[0235] ECU25B shares the same configuration as ECU25 in Figure 3, in that it includes an oscillator 114. However, ECU25B differs from ECU25 in Figure 3 in that it includes a C-PHY 111B, a CSI-2 encoder 112B, and an I2S master 113B.

[0236] The CSI-2 encoder 112B encodes image data supplied from a server (not shown) into a data format compliant with the CSI-2 standard, and transmits the image data in CSI-2 data format to the aggregator 23B via the C-PHY 111B.

[0237] C-PHY111B is a physical layer used to transmit image data in CSI-2 data format from ECU25B to aggregator23B.

[0238] The I2S master 113B is set to Tx mode in I2S master mode (see Figure 12), and transmits, for example, voice data (SDATA) supplied from a server (not shown) to the aggregator 23B.

[0239] Aggregator 23B has the same configuration as disaggregator 24 in Figure 3, in that it includes an I2C master 88, I2C slaves 89-91, a setting information register 92, a controller 93, a timer 94, and a PMC_result register 95. However, aggregator 23B has a different configuration from disaggregator 24 in Figure 3, in that it includes a D-PHY 81B, a CSI-2 multi-encoder 82B, a CSI-2 decode FIFO 83B, a CSI-2 decoder 84B, a C-PHY 85B, an I2S encoding FIFO 86B, and an audio encoder 87B.

[0240] C-PHY85B is the physical layer used to transmit CSI-2 format image data from ECU25B to aggregator23B.

[0241] The CSI-2 decoder 84B receives image data in CSI-2 data format transmitted by the ECU 25B via the C-PHY 111B and C-PHY 85B. The CSI-2 decoder 84B then decodes the image data in CSI-2 data format, obtains the image data restored from the data format compliant with the CSI-2 standard back to the original data format, and writes it to the CSI-2 decode FIFO 83B.

[0242] The CSI-2 decode FIFO 83B stores the image data (CSI_WDATA) to be written by the CSI-2 decoder 84B if the writable signal CSI_WEN supplied from the CSI-2 decoder 84B indicates that image data can be written. Then, the CSI-2 decode FIFO 83B outputs the image data in the order in which it was input if the readable signal CSI_REN supplied from the CSI-2 multi-encoder 82B indicates that image data can be read. As a result, the image data (CSI_RDATA) is read from the CSI-2 decode FIFO 83B to the CSI-2 multi-encoder 82B.

[0243] The audio encoder 87B has the same configuration as the audio decoder 87 in Figure 3, in that it has an MCK divider 103 and a PMC 104. However, the audio encoder 87B has a different configuration from the audio decoder 87 in Figure 3, in that it has an I2S encoder 101B and an I2S slave 102B.

[0244] The I2S slave 102B receives audio data (SDATA) transmitted from the I2S master 113B in accordance with the bit clock signal BCLK and LR clock signal LRCLK supplied from the I2S master 113B of the ECU 25B, and supplies it to the I2S encoder 101B.

[0245] Based on parameters supplied by the controller 93, the I2S encoder 101B encodes the audio data supplied from the I2S slave 102B into a data format conforming to the I2S standard at timings according to the bit clock signal BCLK and the LR clock signal LRCLK. Then, according to the time Timer#0 supplied by the timer 94, the I2S encoder 101B writes the I2S formatted audio data to the I2S encoded FIFO 86B.

[0246] If the I2S encoding FIFO 86B receives a writable signal I2S_WEN from the I2S encoder 101B indicating that audio data can be written, it stores the I2S format audio data (I2S_WDATA) to be written by the I2S encoder 101B. Then, if the readable signal I2S_REN from the CSI-2 multi-encoder 82B indicates that audio data can be read, the I2S encoding FIFO 86B outputs the audio data in the order in which it was input. As a result, the I2S format audio data (I2S_RDATA) is read from the I2S encoding FIFO 86B to the CSI-2 multi-encoder 82B.

[0247] The CSI-2 multi-encoder 82B reads image data from the CSI-2 decode FIFO 83B and I2S formatted audio data from the I2S encoding FIFO 86B. The CSI-2 multi-encoder 82B generates a CSI-2 packet in which the image data is stored in the data area of ​​virtual channel VC0 and the I2S formatted audio data is stored in the data area of ​​virtual channel VC1, according to the CSI-2 standard data format. The CSI-2 multi-encoder 82B then transmits the CSI-2 packet containing the image data and audio data to the disaggregator 24B via the D-PHY 81B.

[0248] D-PHY 81B is a physical layer used to transmit CSI-2 packets containing image data and audio data from the aggregator 23B to the disaggregator 24B.

[0249] Figure 34 is a block diagram showing an example configuration of the disaggregator 24B, display 26, and speaker 27.

[0250] The disaggregator 24B has the same configuration as the aggregator 23 in Figure 2, in that it includes an I2C master 51, oscillator 55, I2C slaves 56-58, configuration information register 59, controller 60, timer 61, and PMC_result register 62. However, the disaggregator 24B has a different configuration from the aggregator 23 in Figure 2, in that it includes a C-PHY 52B, CSI-2 encoder 53B, CSI-2 encoding FIFO 54B, audio decoder 63B, I2S decoding FIFO 64B, CSI-2 multi-decoder 65B, and D-PHY 66B.

[0251] D-PHY66B is a physical layer used to receive CSI-2 packets transmitted from aggregator 23B to disaggregator 24B.

[0252] The CSI-2 multidecoder 65B receives CSI-2 packets transmitted from the CSI-2 multiencoder 82B via the D-PHY 66B and D-PHY 81B. The CSI-2 multidecoder 65B obtains image data and I2S format audio data from the CSI-2 packets according to the data format of the CSI-2 standard. For example, the CSI-2 multidecoder 65B refers to the packet header of the CSI-2 packet to separate it by data type of virtual channel (VC), obtains the image data stored in virtual channel VC0 of the CSI-2 packet, and obtains the I2S format audio data stored in virtual channel VC1 of the CSI-2 packet. Then, the CSI-2 multidecoder 65B writes the image data obtained from virtual channel VC0 of the CSI-2 packet to the CSI-2 encoded FIFO 54B and writes the I2S format audio data obtained from virtual channel VC1 of the CSI-2 packet to the I2S decoded FIFO 64B.

[0253] The CSI-2 encoding FIFO 54B stores the image data (CSI_WDATA) to be written by the CSI-2 multidecoder 65B if the writable signal CSI_WEN supplied from the CSI-2 multidecoder 65B indicates that image data can be written. Then, the CSI-2 encoding FIFO 54B outputs the image data in the order in which it was input if the readable signal CSI_REN supplied from the CSI-2 encoder 53B indicates that image data can be read. As a result, the image data (CSI_RDATA) is read from the CSI-2 encoding FIFO 54B to the CSI-2 encoder 53B.

[0254] The CSI-2 encoder 53B encodes the image data read from the CSI-2 encoding FIFO 54B into a data format conforming to the CSI-2 standard. Then, according to the time Timer#1 supplied by the timer 61, the CSI-2 encoder 53B transmits the CSI-2 formatted image data to the display 26 via the C-PHY 52B.

[0255] C-PHY 52B is a physical layer used to transmit CSI-2 format image data from the disaggregator 24B to the display 26.

[0256] If the writable signal I2S_WEN supplied from the CSI-2 multi-decoder 65B indicates that audio data can be written, the I2S-formatted audio data (I2S_WDATA) to be written by the CSI-2 multi-decoder 65B is stored in the I2S-formatted audio FIFO 64B. Then, if the readable signal I2S_REN supplied from the I2S decoder 73B of the audio decoder 63B indicates that audio data can be read, the I2S-formatted audio data (I2S_RDATA) is read from the I2S-formatted audio FIFO 64B to the I2S decoder 73B of the audio decoder 63B.

[0257] The audio decoder 63B has the same configuration as the audio encoder 63 in Figure 2, in that it has an MCK generator 72. However, the audio decoder 63B has a different configuration from the audio encoder 63 in Figure 2, in that it has an I2S master 71B and an I2S decoder 73B.

[0258] The I2S decoder 73B decodes the I2S-formatted audio data read from the I2S decode FIFO 64B at timings according to the bit clock signal BCLK and the LR clock signal LRCLK, based on parameters supplied from the controller 60. The I2S decoder 73B then supplies the audio data in the I2S standard format to the I2S master 71B according to the time Timer#1 supplied from the timer 61.

[0259] The I2S master 71B is set to Tx mode in I2S master mode (see Figure 12) and transmits the audio data (SDATA) supplied from the I2S decoder 73B to the speaker 27.

[0260] The display 26 is configured to include an I2C slave 181, a C-PHY 182, and a CSI-2 decoder 183.

[0261] The I2C slave 181 communicates with the I2C master 51 of the disaggregator 24B in accordance with the I2C standard. For example, the I2C slave 181 can perform various settings related to the display of the display 26 according to the setting information transmitted through communication with the I2C master 51.

[0262] C-PHY 182 is a physical layer used to transmit image data in CSI-2 data format from the disaggregator 24B to the display 26.

[0263] The CSI-2 decoder 183 receives CSI-2 format image data transmitted from the CSI-2 encoder 53B via the C-PHY 52B and C-PHY 182. The CSI-2 decoder 183 then decodes the CSI-2 data format image data to obtain the image data restored from the data format compliant with the CSI-2 standard back to the original data format, and displays it on the display 26.

[0264] The speaker 27 is configured with an I2C slave 191, an I2S slave 192, and a DA (Digital-to-Analog) converter 193.

[0265] The I2C slave 191 communicates with the I2C master 51 of the disaggregator 24B in accordance with the I2C standard. For example, the I2C slave 191 can make various settings related to the display of the speaker 27 according to the setting information transmitted through communication with the I2C master 51.

[0266] The I2S slave 192 communicates with the I2S master 71B of the disaggregator 24B in accordance with the I2S standard. For example, the I2S slave 192 receives voice data (SDATA) transmitted from the I2S master 71B in accordance with the bit clock signal BCLK and LR clock signal LRCLK supplied from the I2S master 71B of the disaggregator 24B, and supplies it to the DA converter 193.

[0267] The DA converter 193 converts the audio data supplied from the I2S slave 192 into a digital format and outputs the audio from the speaker 27.

[0268] As described above, the communication system 11B is configured to transmit image data and audio data from the ECU 25B to the display 26 and speaker 27. Furthermore, similar to the communication system 11 in Figure 1, the communication system 11B can synchronize time between the aggregator 23B and the disaggregator 24B using I2C, and can transmit image data and audio data from the aggregator 23B to the disaggregator 24B via D-PHY in accordance with the MIPI CSI-2 standard.

[0269] <Fourth Configuration Example of the Communication System> Referring to Figures 35 to 37, a configuration example of a fourth embodiment of a communication system to which this technology is applied will be described. In the communication system 11C described with reference to Figures 35 to 37, components common to the communication system 11 in Figure 1 are denoted by the same reference numerals, and their detailed explanations are omitted.

[0270] Figure 35 is a block diagram showing an example configuration of the communication system 11C.

[0271] As shown in Figure 35, the communication system 11C is configured to include a CMOS image sensor 21C, a microphone 22, an aggregator 23C, a disaggregator 24C, an ECU 25C, a display 26, and a speaker 27. The microphone 22, display 26, and speaker 27 are configured in the same way as the communication system 11 in Figure 1.

[0272] The CMOS image sensor 21C differs from the CMOS image sensor 21 in Figure 1 in that it uses a D-PHY as the physical layer to transmit image data to the aggregator 23C. Specifically, the CMOS image sensor 21C is equipped with a D-PHY instead of the C-PHY 34 found in the CMOS image sensor 21 in Figure 2. In other respects, the CMOS image sensor 21C is configured similarly to the CMOS image sensor 21 in Figure 1, and a detailed explanation of these aspects is omitted.

[0273] ECU 25C differs from ECU 25 in Figure 1 in that it uses a D-PHY as the physical layer to transmit image data from the disaggregator 24C. Specifically, ECU 25C is equipped with a D-PHY instead of the C-PHY 111 found in ECU 25 in Figure 3. In other respects, ECU 25C is configured similarly to ECU 25 in Figure 1, and a detailed explanation of these aspects is omitted.

[0274] In the communication system 11C, data is transmitted and received between the aggregator 23C and the disaggregator 24C using the A-PHY as the physical layer. For example, an A-Packet containing a CSI-2 packet, which stores image data and voice data together in a single packet, is transmitted from the aggregator 23C to the disaggregator 24C. In addition, an A-Packet containing I2C data, which includes various information necessary for time synchronization between the aggregator 23C and the disaggregator 24C (for example, PMC_result and Time_info), is transmitted from the disaggregator 24C to the aggregator 23C.

[0275] Figure 36 is a block diagram showing an example configuration of the aggregator 23C.

[0276] As shown in Figure 36, the aggregator 23C has the same configuration as the aggregator 23 in Figure 2, in that it includes an I2C master 51, a CSI-2 decoder 53, a CSI-2 decode FIFO 54, an oscillator 55, I2C slaves 56-58, a configuration information register 59, a controller 60, a timer 61, a PMC_result register 62, an audio encoder 63, an I2S encode FIFO 64, and a CSI-2 multi-encoder 65.

[0277] Furthermore, the aggregator 23C has a different configuration from the aggregator 23 in Figure 2, in that it includes a D-PHY 67 and an A-PHY serializer 201.

[0278] D-PHY 67 is a physical layer used to receive image data in CSI-2 data format transmitted from the CMOS image sensor 21C to the aggregator 23C.

[0279] The A-PHY serializer 201 comprises a PAL CIS-2 communication unit 211, a PAL I2C communication unit 212, an A-PHY data link layer 213, an A-PHY downlink 214, and an A-PHY uplink 215. The A-PHY data link layer 213 has an RTBC (Running Time-Base Counter) 216 for synchronization with the A-PHY data link layer 223 shown in Figure 37.

[0280] For example, the CSI-2 multi-encoder 65 converts the CSI-2 packets containing image data and audio data into PPI format and supplies them to the PAL CIS-2 communication unit 211. The CSI-2 packets containing the image data and audio data are then transmitted to the disaggregator 24C via the A-PHY data link layer 213 and the A-PHY downlink 214.

[0281] Meanwhile, the A-Packet containing the I2C data transmitted from the disaggregator 24C is supplied to the PAL I2C communication unit 212 via the A-PHY uplink 215 and the A-PHY data link layer 213. The PAL I2C communication unit 212 then writes the configuration information obtained from the I2C data to the configuration information register 59 via the I2C slave 56. The PAL I2C communication unit 212 also supplies the time Timer#0 obtained from the I2C data to the timer 61 via the I2C slave 57. Furthermore, the PAL I2C communication unit 212 writes the PMC_result value obtained from the I2C data to the PMC_result register 62 via the I2C slave 58.

[0282] Figure 37 is a block diagram showing an example configuration of the disaggregator 24C.

[0283] As shown in Figure 37, the disaggregator 24C has the same configuration as the disaggregator 24 in Figure 3, in that it includes a CSI-2 multi-decoder 82, a CSI-2 encoding FIFO 83, a CSI-2 encoder 84, an I2S decoding FIFO 86, an audio decoder 87, an I2C master 88, I2C slaves 89-91, a configuration information register 92, a controller 93, a timer 94, and a PMC_result register 95.

[0284] Furthermore, the disaggregator 24C has a different configuration from the disaggregator 24 in Figure 3, in that it includes a D-PHY 96 and an A-PHY deserializer 202.

[0285] D-PHY96 is a physical layer used to transmit image data in CSI-2 data format from the disaggregator 24C to the ECU 25C.

[0286] The A-PHY deserializer 202 comprises an A-PHY downlink 221, an A-PHY uplink 222, an A-PHY data link layer 223, a PAL CIS-2 communication unit 224, and a PAL I2C communication unit 225. The A-PHY data link layer 223 has an RTBC 226 for synchronization with the A-PHY data link layer 213 shown in Figure 36.

[0287] For example, CSI-2 packets containing image data and audio data transmitted from aggregator 23C are supplied to PAL CIS-2 communication unit 224 via A-PHY downlink 221 and A-PHY data link layer 223. The PAL CIS-2 communication unit 224 then converts the CSI-2 packets containing image data and audio data into PPI format and supplies them to CSI-2 multidecoder 82.

[0288] Meanwhile, an A-Packet containing I2C data to be transmitted from the disaggregator 24C is supplied to the PAL I2C communication unit 225 and transmitted to the aggregator 23C via the A-PHY data link layer 223 and the A-PHY uplink 222. For example, the I2C communication unit 225 stores the setting information of the aggregator 23C supplied from the I2C master 88, the time of the timer 94 (Timer#0), and the PMC_result value in the I2C data.

[0289] As described above, the communication system 11C is configured to use A-PHY as the physical layer between the aggregator 23C and the disaggregator 24C. Similar to the communication system 11 in Figure 1, the communication system 11C can synchronize time between the aggregator 23C and the disaggregator 24C using I2C, and can transmit image data and voice data from the aggregator 23C to the disaggregator 24C via A-PHY in accordance with the MIPI CSI-2 standard.

[0290] <Other examples of data formats>

[0291] In the embodiment described above, the communication system 11 used CSI-2 packets in a data format in which image data was assigned to virtual channel VC0 and voice data was assigned to virtual channel VC1, as explained with reference to Figure 13, for example. In contrast, as shown in Figure 38, the communication system 11 can use CSI-2 packets in a data format in which both image data and voice data are assigned to virtual channel VC0. Figure 39 shows an example of a data format for a long packet that transmits CSI-2 packets like those shown in Figure 38 using D-PHY. For example, such a CSI-2 packet is generated by a CSI-2 multi-encoder 65 and restored by a CSI-2 multi-decoder 82.

[0292] In a CSI-2 packet with the data format shown in Figure 38, two lines of embedded data (L(0) and L(1)) are provided above the image data (Video1 Data), and two lines of embedded data (L(n+3) and L(end)) are provided below the image data (Video1 Data). The I2S format audio data (Audio Data) is stored in these four lines of embedded data.

[0293] Therefore, in Figure 15, the packet formatter 152 reads I2S formatted audio data from the VC1 line buffer 142 when the value of the line counter 154 is L(0), L(1), L(n+3), L(end), and reads image data (Video1 data) from the VC0 line buffer 141 when the value of the line counter 154 is anything other than L(0), L(1), L(n+3), L(end).

[0294] On the other hand, the packet decoder 163 in Figure 20 writes audio data to the VC1 line buffer 165 when the value of the line counter 166 is L(0), L(1), L(n+3), L(end), and writes image data (Video1 data) to the VC0 line buffer 164 when the value of the line counter 154 is anything other than L(0), L(1), L(n+3), L(end).

[0295] By using CSI-2 packets in this data format, the communication system 11 can assign image data (Video1 Data) and audio data (Audio Data) to VC0 and transmit the image data (Video1 Data) and audio data (Audio Data).

[0296] In the communication systems 11 of each embodiment described above, the aggregator 23 and the disaggregator 24 synchronized their time using I2C. However, instead of using I2C, for example, I3C (Improved Inter Integrated Circuits) could be used to synchronize the time.

[0297] <Examples of Configuration Combinations> The technology can also take the following configurations: (1) A communication system comprising: a first communication device that stores image data and data other than the image data together in a single packet based on a first communication standard and transmits the packet in accordance with the first communication standard; and a second communication device that receives the packet and separates the image data and data other than the image data, wherein the transmitting side that transmits the packet and the receiving side that receives the packet are synchronized in time using a second communication standard other than the first communication standard. (2) The communication system according to (1) above, wherein the data other than the image data is voice data, and the first communication device stores the voice data in the packet in synchronization with the image data in units of the frame rate of the image data. (3) The communication system according to (2) above, wherein CSI (Camera Serial Interface)-2 is used as the first communication standard and I2C (Inter-Integrated Circuit) is used as the second communication standard. (4) The communication system according to (3) above, wherein the first communication device assigns an Audio Type to the Data Type included in the packet header of the packet data containing the audio data in a single CSI-2 packet in which the image data and the audio data are stored together. (5) The communication system according to (4) above, wherein the CSI-2 packet is transmitted from the first communication device to the second communication device using D-PHY or C-PHY as the physical layer. (6) The communication system according to any one of (2) to (5) above, wherein the first timer of the first communication device has a synchronization time calculation unit that calculates a communication delay time between the first communication device and the second communication device based on a time synchronization command transmitted from the second communication device, and calculates a time synchronized with the time of the second timer of the second communication device. (7) The communication system according to (6) above, wherein the first timer further has a control unit that corrects the time output by the first timer itself at correction periods shorter than the time synchronization period based on the time synchronization command.(8) The communication system according to any one of (4) to (7) above, wherein the first communication device further comprises a multi-encoder that stores the image data and the audio data together in a single CSI-2 packet. (9) The communication system according to (8) above, wherein the multi-encoder generates the CSI-2 packet in a data format that assigns the image data to a first virtual channel and assigns the image data to a second virtual channel. (10) The communication system according to (8) above, wherein the multi-encoder assigns the image data and the audio data to the same virtual channel and generates the CSI-2 packet in a data format that stores the audio data in embedded data line areas provided above and below the image data. (11) The communication system according to any one of (8) to (10) above, wherein the first communication device further comprises a clock signal generation unit that generates a clock signal synchronized with the time of a first timer of the first communication device and supplies it to an I2S encoder that encodes the audio data into a data format compliant with the I2S standard. (12) A communication system according to any one of (4) to (11) above, wherein an A-Packet containing the CSI-2 packet is transmitted from the first communication device to the second communication device using the A-PHY as the physical layer. (13) A communication method comprising: a communication system storing image data and data other than the image data together in a single packet based on a first communication standard, transmitting the packet in accordance with the first communication standard, and receiving the packet and separating the image data and data other than the image data, wherein a second communication standard other than the first communication standard is used to synchronize the time between the transmitting side that transmits the packet and the receiving side that receives the packet.

[0298] It should be noted that this embodiment is not limited to the embodiment described above, and various modifications are possible without departing from the spirit of this disclosure. Furthermore, the effects described herein are merely illustrative and not limiting, and other effects may also exist.

[0299] 11 Communication system, 21 CMOS image sensor, 22 Microphone, 23 Aggregator, 24 Disaggregator, 25 ECU, 26 Display, 27 Speaker

Claims

1. A communication system comprising: a first communication device that stores image data and data other than the image data together in a single packet based on a first communication standard and transmits the packet in accordance with the first communication standard; and a second communication device that receives the packet and separates the image data and data other than the image data, wherein the transmitting side that transmits the packet and the receiving side that receives the packet are synchronized in time using a second communication standard other than the first communication standard.

2. The communication system according to claim 1, wherein data other than the image data is audio data, and the first communication device stores the audio data in the packet in synchronization with the image data in units of the frame rate of the image data.

3. The communication system according to claim 2, wherein CSI (Camera Serial Interface)-2 is used as the first communication standard and I2C (Inter-Integrated Circuit) is used as the second communication standard.

4. The communication system according to claim 3, wherein the first communication device assigns an Audio Type to the Data Type included in the packet header of the packet data containing the audio data in a single CSI-2 packet in which the image data and the audio data are stored together.

5. The communication system according to claim 4, wherein the CSI-2 packet is transmitted from the first communication device to the second communication device using D-PHY or C-PHY as the physical layer.

6. The communication system according to claim 2, wherein the first timer of the first communication device has a synchronization time calculation unit that calculates a communication delay time between the first communication device and the second communication device based on a time synchronization command transmitted from the second communication device, and calculates a time synchronized with the time of the second timer of the second communication device.

7. The communication system according to claim 6, wherein the first timer further comprises a control unit that corrects the time output by the first timer itself at correction periods shorter than the time synchronization period based on the time synchronization command.

8. The communication system according to claim 4, wherein the first communication device further comprises a multi-encoder that stores the image data and the voice data together in a single CSI-2 packet.

9. The communication system according to claim 8, wherein the multi-encoder generates the CSI-2 packets in a data format that assigns the image data to a first virtual channel and assigns the image data to a second virtual channel.

10. The communication system according to claim 8, wherein the multi-encoder assigns the image data and the audio data to the same virtual channel and generates the CSI-2 packet in a data format in which the audio data is stored in the line areas of embedded data provided above and below the image data.

11. The communication system according to claim 8, further comprising a clock signal generation unit that generates a clock signal synchronized with the time of a first timer of the first communication device and supplies it to an I2S encoder that encodes the audio data into a data format conforming to the I2S standard.

12. The communication system according to claim 4, wherein the A-Packet containing the CSI-2 packet is transmitted from the first communication device to the second communication device using the A-PHY as the physical layer.

13. A communication method comprising: a communication system storing image data and data other than the image data together in a single packet based on a first communication standard, transmitting the packet in accordance with the first communication standard, and receiving the packet and separating the image data and data other than the image data, wherein a second communication standard other than the first communication standard is used to synchronize the time of a transmitting side that transmits the packet and a receiving side that receives the packet.

Images