Information processing device, mobile device, and communication system

Abstract

The present disclosure relates to an information processing device, a mobile device, and a communication system which make it possible to update session keys. A first communication, which includes the transmission or reception of a command for controlling a second communication of higher speed than the first communication or a response to said command, is used to derive a first secret from a key schedule, a first session key related to said first secret is derived, said first session key is used in encryption or message authentication of the first communication, the first communication is used to receive, transmit, or derive a second session key, said second session key is used in encryption or message authentication of the second communication, the first communication is used to receive, transmit, or derive a third session key, and use of said third session key in place of the second session key is initiated. The present technology is applicable to, for example, communication systems compliant with MIPI standards.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a mobile device, and a communication system, and more particularly to an information processing device, a mobile device, and a communication system that are capable of updating a session key. [Background technology]

[0002] CSI (Camera Serial Interface)-2 ver. 4.0, which is currently undergoing standardization, defines two types of packet structures: one that uses C-PHY in the physical layer, and one that uses D-PHY in the physical layer.

[0003] In recent years, the CSI-2 standard has been used not only in mobile devices but also in a wide variety of applications, such as in-vehicle and IoT (Internet of Things), and as a result, it is expected that the existing packet structure will not be able to support these applications. Therefore, the MIPI (Mobile Industry Processor Interface) Alliance is currently studying extended packets that extend the packet structure, including the existing packet header and packet footer, to support a wider range of applications.

[0004] Incidentally, in the SPDM (Security Protocol and Data Model) standard described in Non-Patent Document 1, a key schedule is made public. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] "Security Protocol and Data Model (SPDM) Specification", DSP0274, Version: 1.1.0, DMTF, 2020-07-15 Summary of the Invention [Problem to be solved by the invention]

[0006] However, when applying this SPDM to the MIPI CSI-2 standard or DSI (Display Serial Interface)-2 standard, the export master secret does not support a key update function, and therefore, when applying a session key derived from the export master secret, it is not possible to update the session key. On the other hand, when applying a session key transmitted or received via communication protected by SPDM, for example, it is necessary to define session key update.

[0007] The present disclosure has been made in light of such circumstances, and makes it possible to update a session key. [Means for solving the problem]

[0008] An information processing device according to one aspect of the present technology includes a first communication Second Communication and a protective part that protects the The aforementioned The protector derives a first secret from a key schedule using the first communication, derives a first session key associated with the first secret, and transmits the first session key to the first communication. is used for protected operations, The protection unit receives a second session key using the first communication protected by the first session key. Or, deriving a second secret from the key schedule to derive a second session key associated with the second secret; an initialization vector including a source ID value, a virtual channel or extended virtual channel value, a frame counter value or an additional frame number value, and the received or derived second session key for use in an operation to protect the second communication; It is an information processing device.

[0009] According to one aspect of the present technology, a mobile device Second Communication and a protective part that protects the The aforementioned The protector derives a first secret from a key schedule using the first communication, derives a first session key associated with the first secret, and transmits the first session key to the first communication. is used for protected operations,The protection unit receives a second session key using the first communication protected by the first session key. Or, deriving a second secret from the key schedule to derive a second session key associated with the second secret; an initialization vector including a source ID value, a virtual channel or extended virtual channel value, a frame counter value or an additional frame number value, and the received or derived second session key for use in an operation to protect the second communication; It is a mobile device.

[0010] A communication system according to one aspect of the present technology includes a first communication Second Communication and a protective part that protects the The aforementioned The protector derives a first secret from a key schedule using the first communication, derives a first session key associated with the first secret, and transmits the first session key to the first communication. is used for protected operations, The protection unit receives a second session key using the first communication protected by the first session key. Or, deriving a second secret from the key schedule to derive a second session key associated with the second secret; an initialization vector including a source ID value, a virtual channel or extended virtual channel value, a frame counter value or an additional frame number value, and the received or derived second session key for use in an operation to protect the second communication; It is a communication system.

[0011] In one aspect of the present technology, First Communication A first secret is derived from the key schedule using , the first communication is used for the protected operation; The second session key is received using the first communication protected by the first session key. or deriving a second secret from the key schedule and deriving a second session key associated with the second secret; An initialization vector including the source ID value, the virtual channel or extended virtual channel value, the frame counter value or the additional frame number value, and the received or derived second session key are used in the operation by which the second communication is protected. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a configuration example of a first embodiment of a communication system to which the present technology is applied. [Figure 2] FIG. 10 is a block diagram showing a configuration example of a second embodiment of a communication system to which the present technology is applied. [Figure 3] FIG. 10 is a diagram illustrating a first example of the overall packet structure of an extended packet for D-PHY. [Figure 4] FIG. 10 is a diagram illustrating a first example of the packet structure of an extended short packet for D-PHY. [Figure 5] FIG. 10 is a diagram illustrating a first example of the packet structure of an extended long packet for D-PHY. [Figure 6] FIG. 10 is a diagram illustrating a first example of the overall packet structure of an extended packet for C-PHY. [Figure 7] FIG. 10 is a diagram illustrating a first example of the packet structure of an extended short packet for C-PHY. [Figure 8] FIG. 10 is a diagram illustrating a first example of the packet structure of an extended long packet for C-PHY. [Figure 9] FIG. 2 is a block diagram illustrating a configuration example of an image sensor. [Figure 10] FIG. 2 is a block diagram illustrating a configuration example of an application processor. [Figure 11] 10 is a flowchart illustrating a process in which the image sensor transmits a packet. [Figure 12] 10 is a flowchart illustrating an extended mode transmission process. [Figure 13] 10 is a flowchart illustrating a process in which an application processor receives a packet. [Figure 14] 10 is a flowchart illustrating an extended mode reception process. [Figure 15] FIG. 10 is a diagram illustrating a second example of the overall packet structure of an extended packet for D-PHY. [Figure 16] FIG. 10 is a diagram illustrating a second example of the packet structure of an extended long packet for D-PHY. [Figure 17] FIG. 10 is a diagram illustrating a second example of the packet structure of an extended short packet for C-PHY. [Figure 18] FIG. 10 is a diagram illustrating a second example of the packet structure of an extended long packet for C-PHY. [Figure 19] FIG. 10 is a block diagram illustrating a modified example of the configuration for switching between D-PHY and C-PHY. [Figure 20]FIG. 10 is a block diagram showing a configuration example of a third embodiment of a communication system to which the present technology is applied. [Figure 21] FIG. 10 is a diagram illustrating an example of the structure of an extended packet for D-PHY that complies with the packet modification prohibition provision. [Figure 22] FIG. 10 is a diagram illustrating an example of the structure of an extended packet for C-PHY that complies with the packet modification prohibition provision. [Figure 23] FIG. 10 is a diagram illustrating an example of the structure of an extended packet for A-PHY that complies with the packet modification prohibition provision. [Figure 24] 10 is a flowchart illustrating a packet transmission / reception process that complies with the packet modification prohibition rule. [Figure 25] FIG. 1 is a block diagram showing an example of the configuration of an image sensor that complies with the packet modification prohibition provision. [Figure 26] FIG. 1 is a block diagram showing an example of the configuration of an application processor that complies with the packet modification prohibition provision. [Figure 27] FIG. 1 is a block diagram illustrating an example of the configuration of a communication system in which an image sensor and an application processor are directly connected to each other. [Figure 28] FIG. 10 is a diagram illustrating an example of a packet configuration of a read command generated on the application processor side. [Figure 29] FIG. 10 is a diagram illustrating an example of a packet configuration of a read command transferred by A-PHY. [Figure 30] 10A and 10B are diagrams illustrating an example of a packet configuration of a read command and read data on the image sensor side. [Figure 31] FIG. 10 is a diagram illustrating an example of a packet configuration of read data transferred by A-PHY. [Figure 32] FIG. 10 is a diagram illustrating an example of a packet configuration of read data acquired on the application processor side. [Figure 33] FIG. 10 is a diagram illustrating an example of a packet configuration of write data generated on the application processor side. [Figure 34] 10 is a diagram illustrating an example of a packet configuration of write data transferred by A-PHY. FIG. [Figure 35] 10 is a diagram illustrating an example of a packet configuration of write data acquired on the image sensor side. FIG. [Figure 36] FIG. 10 is a diagram illustrating an overview of an extended packet header ePH and an extended packet footer ePF. [Figure 37] 10 is a flowchart illustrating an initial setting and confirmation operation of communication processing using CCI-FS. [Figure 38] 10 is a flowchart illustrating a write operation using CCI-FS. [Figure 39] 10 is a flowchart illustrating a read operation using CCI-FS. [Figure 40] FIG. 1 is a block diagram illustrating an example of the configuration of a communication system in which an image sensor and an application processor are connected via SerDes. [Figure 41] FIG. 10 is a diagram illustrating an example of a packet configuration of a read command generated on the application processor side. [Figure 42] FIG. 10 is a diagram illustrating an example of a packet configuration of a read command output via I2C / I3C. [Figure 43] FIG. 10 is a diagram illustrating an example of a packet configuration of a read command transferred by A-PHY. [Figure 44] 10 is a diagram illustrating an example of a packet configuration of read data generated by a slave-side SerDes device. FIG. [Figure 45] 10A and 10B are diagrams illustrating an example of a packet configuration of a read command and read data on the image sensor side. [Figure 46] FIG. 10 is a diagram illustrating an example of a packet configuration of read data output via I2C / I3C. [Figure 47] FIG. 10 is a diagram illustrating an example of a packet configuration of read data transferred by A-PHY. [Figure 48] FIG. 10 is a diagram illustrating an example of a packet configuration of read data output via I2C / I3C. [Figure 49] FIG. 10 is a diagram illustrating an example of a packet configuration of read data acquired on the application processor side. [Figure 50]10 is a flowchart illustrating an initial setting and confirmation operation of communication processing using CCI-FS. [Figure 51] 10 is a flowchart illustrating a write operation using CCI-FS. [Figure 52] 10 is a flowchart illustrating a read operation using CCI-FS. [Figure 53] 10 is a flowchart illustrating Sequence A_Write (AP time) processing. [Figure 54] 10 is a flowchart illustrating Sequence A_Read_CMD (AP) processing. [Figure 55] 10 is a flowchart illustrating Sequence C (AP) processing. [Figure 56] 10 is a flowchart illustrating Sequence B (SerDes (Slave)) processing. [Figure 57] 10 is a flowchart illustrating Sequence A_Read_Data (AP time) processing. [Figure 58] 10 is a diagram showing details of an extension packet header ePH0, an extension packet header ePH1, and an extension packet header ePH2. [Figure 59] FIG. 10 is a diagram showing details of an extended packet header ePH3. [Figure 60] FIG. 10 is a diagram showing details of an extended DT of an extended packet header ePH. [Figure 61] FIG. 1 is a block diagram showing an example of a conventional I2C hardware configuration. [Figure 62] FIG. 10 is a diagram showing an example of a waveform during data transfer on an I2C bus. [Figure 63] FIG. 1 is a block diagram illustrating an example of a CCI-related configuration in a communication system having an A-PHY direct connection configuration. [Figure 64] FIG. 1 is a diagram illustrating an example of a network connection configuration. [Figure 65] FIG. 2 is a block diagram showing an example of a circuit configuration of a CCI-FS processing unit. [Figure 66] FIG. 2 is a diagram illustrating an example of a register configuration. [Figure 67] FIG. 10 is a diagram illustrating an example of a register configuration in a bridge configuration. [Figure 68] FIG. 10 is a diagram illustrating an example of the register configuration of an error-related register. [Figure 69] FIG. 10 is a diagram showing a modified example of an extended packet header ePH in the packet configuration of write data generated on the application processor side. [Figure 70] FIG. 10 is a diagram illustrating a modified example of an extended packet header ePH in the packet configuration of a read command generated on the application processor side. [Figure 71] FIG. 10 is a diagram illustrating a flow between an application processor and an image sensor in an A-PHY direct connection configuration. [Figure 72] FIG. 10 is a diagram illustrating a flow using the Clock Stretch method. [Figure 73] FIG. 2 is a block diagram showing a detailed configuration example of an image sensor including a CCI-FS processing unit. [Figure 74] FIG. 2 is a block diagram illustrating a detailed configuration example of an application processor including a CCI-FS processing unit. [Figure 75] FIG. 10 is a block diagram showing a configuration example of a fourth embodiment of a communication system to which the present technology is applied. [Figure 76] FIG. 2 is a block diagram showing a detailed configuration example of an image sensor. [Figure 77] FIG. 2 is a block diagram illustrating a detailed configuration example of an application processor. [Figure 78] 10 is a flowchart illustrating a first processing example of a communication process. [Figure 79] 10 is a flowchart illustrating a first processing example of a communication process. [Figure 80] 10 is a flowchart illustrating a first processing example of a communication process. [Figure 81] FIG. 10 is a diagram illustrating a verification packet and a packet to be verified. [Figure 82] FIG. 10 is a diagram illustrating a verification packet and a packet to be verified. [Figure 83] 10 is a flowchart illustrating a data verification process. [Figure 84] 10 is a flowchart illustrating a message count value transmission process. [Figure 85] FIG. 10 is a diagram illustrating embedded data. [Figure 86] FIG. 2 is a diagram illustrating an example of a data structure of image data. [Figure 87] 10 is a flowchart illustrating an image data transmission process. [Figure 88] 10 is a flowchart illustrating an integrity calculation value transmission process. [Figure 89] FIG. 10 is a diagram showing a first modified example of the data structure of image data. [Figure 90] FIG. 10 is a diagram showing a second modified example of the data structure of image data. [Figure 91] FIG. 10 is a diagram showing a third modified example of the data structure of image data. [Figure 92] 10 is a flowchart illustrating a first example of an integrity operation value process. [Figure 93] 10 is a flowchart illustrating a second example of the integrity operation value processing. [Figure 94] 10 is a flowchart illustrating a third example of the integrity operation value processing. [Figure 95] 10 is a flowchart illustrating a fourth example of the integrity operation value processing. [Figure 96] FIG. 10 is a diagram illustrating an example of an initial counter block in which an initialization vector is stored. [Figure 97] FIG. 1 is a diagram illustrating a GHASH function. [Figure 98] FIG. 1 is a diagram illustrating a GCTR function. [Figure 99] FIG. 1 is a diagram showing a GCM-AE function. [Figure 100] FIG. 1 is a diagram illustrating a GCM-AD function. [Figure 101] FIG. 10 is a diagram illustrating an example of the data structure of image data for transmitting an integrity calculation value MAC for each line. [Figure 102]FIG. 10 is a diagram illustrating an example of an initialization vector. [Figure 103] FIG. 10 is a diagram illustrating an example of transmitting an initialization vector from a transmitting side to a receiving side. [Figure 104] FIG. 1 is a diagram illustrating an example of an extended format of CSI-2 or CCI. [Figure 105] 10 is a flowchart illustrating a transmission process in the line MAC method. [Figure 106] FIG. 10 is a diagram showing an example of the data structure of image data in which an integrity calculation value MAC is allocated for each frame. [Figure 107] FIG. 10 is a diagram illustrating an example of an initialization vector. [Figure 108] FIG. 10 is a diagram illustrating an example of transmitting an initialization vector from a transmitting side to a receiving side. [Figure 109] 10 is a flowchart illustrating a transmission process in the frame MAC method. [Figure 110] 10 is a flowchart illustrating a selection process. [Figure 111] FIG. 10 is a diagram illustrating an example of security MAC information. [Figure 112] FIG. 10 is a diagram illustrating an example of a rollover period of a message count value and a frame count value. [Figure 113] FIG. 10 is a diagram illustrating the configuration of an initialization vector. [Figure 114] 10 is a flowchart illustrating a data verification process. [Figure 115] FIG. 10 is a diagram illustrating a reflection process. [Figure 116] FIG. 1 illustrates an example of a security protocol. [Figure 117] FIG. 10 is a diagram showing an example of a Source ID or a Final Destination ID. [Figure 118] FIG. 1 is a block diagram showing a detailed configuration example of an image sensor that diagnoses whether or not there is an abnormality in the image sensor itself. [Figure 119] 10 is a flowchart illustrating interference detection processing (part 1) by an interference detection unit. [Figure 120]10A and 10B are diagrams illustrating a method for storing a light emission pattern (light reception pattern) as a storage pattern when a ToF distance measuring sensor is realized by an image sensor. [Figure 121] 10A and 10B are diagrams illustrating a method for storing a light emission pattern (light reception pattern) as a storage pattern when a ToF distance measuring sensor is realized by an image sensor. [Figure 122] 10 is a flowchart illustrating a second interference detection process performed by an interference detection unit. [Figure 123] 10 is a flowchart illustrating a failure detection process performed by a failure detection unit. [Figure 124] 10 is a flowchart illustrating an abnormality detection process of the security unit by the intrusion detection unit. [Figure 125] 10 is a flowchart illustrating an abnormality detection process performed by a temperature detection unit. [Figure 126] FIG. 2 is a block diagram showing a detailed configuration example of an application processor that detects whether or not there is an abnormality in the image sensor. [Figure 127] 10 is a flowchart illustrating processing of the image sensor when the application processor performs processing to detect whether or not there is an abnormality in the image sensor. [Figure 128] 10 is a flowchart illustrating processing by the application processor when the application processor performs processing to detect whether or not there is an abnormality in the image sensor. [Figure 129] 10 is a diagram showing an example of the data structure of image data for explaining a location where a peculiar message is stored when high-speed data transmission of the peculiar message is realized without impeding high-speed data transmission of the image data. FIG. [Figure 130] 10 is a flowchart illustrating a process in which high-speed data transmission of a peculiar message is executed without interfering with high-speed data transmission of image data. [Figure 131] 10 is a flowchart illustrating an imaging transmission process (part 1). [Figure 132] 10 is a flowchart illustrating an application example of the imaging transmission process (part 1). [Figure 133] 10 is a flowchart illustrating an imaging transmission process (part 2). [Figure 134] 10 is a flowchart illustrating an image capturing and transmission process (part 3) by the image sensor. [Figure 135] 10 is a flowchart illustrating an imaging transmission process (part 3) by the application processor. [Figure 136] 10 is a flowchart illustrating an image capturing and transmission process (part 4) by the image sensor. [Figure 137] 10 is a flowchart illustrating an imaging transmission process (part 4) by the application processor. [Figure 138] 10 is a flowchart illustrating an imaging and transmission process (part 5) by the image sensor. [Figure 139] 10 is a flowchart illustrating an imaging transmission process (part 5) by the application processor. [Figure 140] 10 is a flowchart illustrating an image capturing and transmission process (part 6) by the image sensor. [Figure 141] 10 is a flowchart illustrating an imaging transmission process (part 6) by the application processor. [Figure 142] 10 is a flowchart illustrating an image capturing and transmission process (part 7) by the image sensor. [Figure 143] 10 is a flowchart illustrating an imaging transmission process (part 7) by the application processor. [Figure 144] 10 is a flowchart illustrating an eighth example of an imaging and transmission process by an image sensor. [Figure 145] 10 is a flowchart illustrating an imaging transmission process (part 8) by the application processor. [Figure 146] 10 is a flowchart illustrating an imaging transmission process (No. 9). [Figure 147] 10 is a flowchart illustrating an imaging transmission process (part 10). [Figure 148] 11 is a flowchart illustrating an imaging transmission process (part 11). [Figure 149] FIG. 10 is a diagram illustrating message count values ​​using two types of count values ​​with different Hamming distances. [Figure 150] FIG. 10 is a diagram illustrating a method for detecting whether or not there is a defect or tampering in a message count value using two types of count values. [Figure 151] FIG. 10 is a diagram illustrating a method for detecting whether or not there is a defect or tampering in a message count value using two types of count values. [Figure 152] 10 is a flowchart illustrating a message counting process. [Figure 153] FIG. 10 is a diagram illustrating an example of the configuration of an extension packet header ePH2 when a Warning Descriptor is set in a reserved area (Reserved) in the extension packet header ePH2. [Fig. 154] 10 is a diagram illustrating an example of description of identification information using each bit of a Warning Descriptor (specific message). FIG. [Figure 155] FIG. 10 is a diagram illustrating a configuration example when an alert alert (for example, physical attack detection) is set as a first peculiar message in an extended packet header. [Figure 156] 10 is a flowchart illustrating a transmission process of the image sensor when a peculiar message is separated and transmitted. [Figure 157] 10 is a flowchart illustrating a transmission process of the application processor when separating and transmitting a peculiar message. [Figure 158] 10 is a flowchart illustrating a transmission process for separating and transmitting a peculiar message when a command to read out warning details is transmitted after a warning bulletin has been transmitted. [Figure 159] 12 is a diagram illustrating an example of the configuration of a Security Descriptor in which a specific message is set, such as whether or not there is an abnormality inside or outside the image sensor 1211, or whether or not there is interference or attack on the image sensor 1211. FIG. [Figure 160]FIG. 1 is a block diagram showing an example configuration of a propulsion device equipped with an image sensor and an application processor. [Figure 161] FIG. 161 is a diagram for explaining a propulsion control process (part 1) for controlling the propulsion of the propulsion device of FIG. 160. [Figure 162] FIG. 161 is a diagram for explaining a propulsion control process (part 2) for controlling the propulsion of the propulsion device of FIG. 160. [Figure 163] FIG. 161 is a diagram for explaining the propulsion control process (part 3) by the microcomputer that controls the propulsion of the propulsion device of FIG. 160. [Fig. 164] 161 is a diagram illustrating a propulsion control process (part 3) by an imaging unit that controls the propulsion of the propulsion device of FIG. 160. FIG. [Figure 165] FIG. 10 is a diagram illustrating a configuration example of Responder flag fields definitions for setting whether the HEARTBEAT function is enabled (HBEAT_CAP=1) or disabled (HBEAT_CAP=0). [Figure 166] FIG. 10 is a diagram illustrating an example of the configuration of a HEARTBEAT request message. [Figure 167] FIG. 10 is a diagram illustrating an example of the configuration of a HEARTBEAT_ACK response message. [Figure 168] FIG. 10 is a diagram illustrating an example of the configuration of a HEARTBEAT_NAK response message. [Figure 169] FIG. 10 is a diagram illustrating an example of the configuration of an END_SESSION request message. [Figure 170] 10 is a flowchart illustrating a first HEARTBEAT process. [Figure 171] FIG. 10 is a diagram illustrating an example of the configuration of an END_SESSION_NAK response message. [Fig. 172] 10 is a flowchart illustrating a second HEARTBEAT process of a CCI host (requester). [Figure 173] 10 is a flowchart illustrating a second HEARTBEAT process of a CCI device (responder). [Fig. 174]10 is a flowchart illustrating a HEARTBEAT process (part 3) of a CCI host (requester). [Figure 175] 10 is a flowchart illustrating a HEARTBEAT process (part 3) of a CCI device (responder). [Figure 176] FIG. 10 is a diagram illustrating an example of the configuration of an ERROR response message. [Figure 177] FIG. 10 is a diagram illustrating an example of setting an error code and error data. [Figure 178] FIG. 10 is a diagram illustrating an example of setting ExtendedErrorData. [Figure 179] 10 is a diagram illustrating an example of setting a Registry or standards body ID when a pseudo HEARTBEAT function is used. FIG. [Figure 180] FIG. 10 is a diagram illustrating an example of settings in a VENDOR_DEFINED_REQUEST request message. [Figure 181] FIG. 10 is a diagram illustrating an example of settings in a VENDOR_DEFINED_RESPONSE response message. [Figure 182] FIG. 1 is a diagram illustrating a key schedule of SPDM. [Figure 183] FIG. 10 is a diagram illustrating an example of KEY_UPDATA_operations. [Figure 184] 10 is a flowchart illustrating an example of a processing flow regarding key update. [Figure 185] FIG. 1 is a diagram illustrating an example of ePH2. [Figure 186] 10 is a flowchart illustrating an example of a flow of updating a session key. [Figure 187] 10 is a flowchart illustrating an example of the flow of processor processing. [Figure 188] 10 is a flowchart illustrating an example of the flow of a sensor process. [Figure 189] 10 is a flowchart illustrating an example of a flow of updating a session key. [Figure 190] 10 is a flowchart illustrating an example of the flow of processor processing. [Figure 191] 10 is a flowchart illustrating an example of the flow of a sensor process. [Figure 192] 10 is a flowchart illustrating an example of the flow of processor processing. [Figure 193] 10 is a flowchart illustrating an example of the flow of a sensor process. [Figure 194] 10 is a flowchart illustrating an example of the flow of a sensor process. [Figure 195] FIG. 10 is a diagram showing an example of KeyUpdataReq and KeySwitchTiming. [Figure 196] 10 is a flowchart illustrating an example of a flow of updating a session key. [Figure 197] 10 is a flowchart illustrating an example of the flow of processor processing. [Figure 198] 10 is a flowchart illustrating an example of the flow of a sensor process. [Figure 199] 10 is a flowchart illustrating an example of the flow of processor processing. [Figure 200] 10 is a flowchart illustrating an example of the flow of a sensor process. [Figure 201] 10 is a flowchart illustrating an example of the flow of processor processing. [Figure 202] 10 is a flowchart illustrating an example of the flow of a sensor process. [Figure 203] 10 is a flowchart illustrating an example of the flow of processor processing. [Figure 204] 10 is a flowchart illustrating an example of the flow of a sensor process. [Figure 205] FIG. 10 is a diagram showing an example of EvenOddkey. [Figure 206] FIG. 10 is a diagram illustrating an example of derivation of a session key. [Figure 207] FIG. 10 is a diagram illustrating an example of derivation of a session key. [Figure 208] 1 is a block diagram illustrating an example of the configuration of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

[0014] <Example of communication system configuration> FIG. 1 is a block diagram showing a configuration example of a first embodiment of a communication system to which the present technology is applied.

[0015] 1, the communication system 11 is configured by connecting an image sensor 21 and an application processor 22 via a bus 23. For example, the communication system 11 is used for CSI-2 connection inside an existing mobile device such as a smartphone.

[0016] The image sensor 21 is configured by incorporating, for example, a lens, an imaging element (neither of which are shown), and an extended mode-compatible CSI-2 transmission circuit 31. For example, the image sensor 21 transmits image data of an image captured by the imaging element to the application processor 22 via the extended mode-compatible CSI-2 transmission circuit 31.

[0017] The application processor 22 is configured by incorporating an extended mode-compatible CSI-2 receiver circuit 32 together with an LSI (Large Scale Integration) that performs processing according to various applications executed in a mobile device equipped with the communication system 11. For example, the application processor 22 receives image data transmitted from the image sensor 21 using the extended mode-compatible CSI-2 receiver circuit 32, and can process the image data according to the application using the LSI.

[0018] The bus 23 is a communication path that transmits signals in accordance with the CSI-2 standard, and the possible transmission distance for signals is, for example, approximately 30 cm. The bus 23 also connects the image sensor 21 and the application processor 22 via multiple signal lines (I2C, CLKP / N, D0P / N, D1P / N, D2P / N, D3P / N) as shown in the figure.

[0019] The extended-mode-compatible CSI-2 transmitter circuit 31 and the extended-mode-compatible CSI-2 receiver circuit 32 support communication in extended mode, an extension of the CSI-2 standard, and can transmit and receive signals to and from each other. Note that the detailed configurations of the extended-mode-compatible CSI-2 transmitter circuit 31 and the extended-mode-compatible CSI-2 receiver circuit 32 will be described later with reference to FIGS. 9 and 10.

[0020] FIG. 2 is a block diagram showing a configuration example of a second embodiment of a communication system to which the present technology is applied.

[0021] 2, communication system 11A is configured such that image sensor 21 and SerDes device 25 are connected via bus 24-1, application processor 22 and SerDes device 26 are connected via bus 24-2, and SerDes device 25 and SerDes device 26 are connected via bus 27. For example, communication system 11A is used for connection in an existing vehicle-mounted camera.

[0022] Here, the image sensor 21 and the application processor 22 are configured in the same manner as the image sensor 21 and the application processor 22 in FIG. 1, and a detailed description thereof will be omitted.

[0023] Buses 24-1 and 24-2, like bus 23 in Figure 1, are communication paths that transmit signals in accordance with the CSI-2 standard, and are configured with multiple signal lines (HS-GPIO, I2C / I3C, CLKP / N, D0P / N, D1P / N, D2P / N, D3P / N) as shown in the figure.

[0024] The SerDes device 25 is configured to include a CSI-2 receiving circuit 33 and a SerDes (Serializer Deserializer) transmitting circuit 34. For example, the CSI-2 receiving circuit 33 communicates with the extended-mode-compatible CSI-2 transmitting circuit 31 in accordance with the normal CSI-2 standard, thereby acquiring a bit-parallel signal transmitted from the image sensor 21. The SerDes device 25 then converts the acquired signal into a bit-serial signal, and transmits the signal to the SerDes device 26 by the SerDes transmitting circuit 34 communicating with the SerDes receiving circuit 35 over one lane.

[0025] The SerDes device 26 is configured to include a SerDes receiver circuit 35 and a CSI-2 transmitter circuit 36. For example, the SerDes device 26 acquires a bit-serial signal transmitted by the SerDes receiver circuit 35 performing one-lane communication with the SerDes transmitter circuit 34. The SerDes device 26 then converts the acquired signal into a bit-parallel signal, and transmits it to the application processor 22 by the CSI-2 transmitter circuit 36 ​​performing communication with the extended-mode-compatible CSI-2 receiver circuit 32 in accordance with the normal CSI-2 standard.

[0026] Bus 27 is a communication path that transmits signals in accordance with standards such as A-PHY and FPD (Flat Panel Display)-LINK III, and the transmission distance over which signals can be transmitted is long, for example, about 15 m.

[0027] These long-distance physical layer interfaces enable the automotive industry to enable advanced driver assistance systems (ADAS), autonomous driving systems (ADS), and other surround sensor applications, including cameras and in-vehicle infotainment (IVI) displays. MIPI A-PHY has an asymmetric data link layer (asymmetric upper layer) in a point-to-point topology, allowing the same physical wiring to be shared for high-speed data transmission, control data, and power. It serves as the foundation for end-to-end systems designed to simplify the integration of cameras, sensors, and displays, while also incorporating functional safety and security.

[0028] The communication systems 11 and 11A configured in this manner can transmit and receive data in packets with an extended packet structure, as described below, using the extended mode-compatible CSI-2 transmitter circuit 31 and the extended mode-compatible CSI-2 receiver circuit 32. This allows for support for a wider variety of applications, such as RAW24, SmartROI (Region of Interest), and GLD (Graceful Link Degradation), as described below.

[0029] <First example of packet structure> A first example of the packet structure of a packet used in communication between the enhanced mode-compatible CSI-2 transmitter circuit 31 and the enhanced mode-compatible CSI-2 receiver circuit 32 will be described with reference to FIGS.

[0030] FIG. 3 shows the overall packet structure of a packet used in the extended mode of CSI-2 when the physical layer is D-PHY (hereinafter referred to as an extended packet for D-PHY).

[0031] As shown in Figure 3, the packet header and footer of the D-PHY extended packet have the same packet structure as the existing CSI-2 standard. For example, the packet header stores VC (Virtual Channel), which indicates the number of virtual channel lines, DataType, which indicates the data type, WC (Word Count), which indicates the data length of the payload, and VCX / ECC. The packet footer also stores CRC (Cyclic Redundancy Check).

[0032] In the existing CSI-2 standard, data types 0x38 to 0x3F are defined as reserved for data types transmitted in packet headers. Therefore, in the D-PHY extended packet, new setting information is defined to identify the extended mode on the receiving side using the existing reserved data types.

[0033] For example, if DataType[5:3]=3'b111 as the data type, the extension mode, DataType[2]=Reserve (RES: reserved for future extension), and DataType[1:0]=extension mode type (four extension modes are available) are defined.

[0034] That is, among the data types 0x38 to 0x3F defined as reserved in the existing CSI-2 standard, for example, DataType[5:3] is defined as extended mode setting information, and DataType[1:0] is defined as extended type setting information. The extended mode setting information indicates whether or not the mode is extended. For example, when DataType[5:3] is 3'b111, it indicates extended mode. Furthermore, when four types of extended mode are provided, namely extended mode 0, extended mode 1, extended mode 2, and extended mode 3, the extended type setting information indicates which of these types it is. For example, when DataType[1:0] is 2'b00, it indicates that the type of extended mode is extended mode 0.

[0035] In extended mode 0 (DataType[1:0]=2'b00), for example, a packet structure in which the payload is separated into four parts is defined. That is, the payload in extended mode 0 is separated into an extended packet header (ePH), an optional extended packet header (OePH), a legacy payload, and an optional extended packet footer (OePF), as shown in FIG. 3. The extended packet header may be transmitted repeatedly.

[0036] The extended packet header is placed at the beginning of the payload in the existing CSI-2 standard, and must be transmitted in extended mode. For example, as shown in the figure, the extended packet header consists of configuration information such as an SROI identification flag, an extended VC (Virtual Channel), an extended DataType, an OePH selection flag, and an OePF selection flag. Here, the extended VC extends the VC, which was 4 bits in the existing CSI-2 standard, to 8 bits, and the extended DataType extends the DataType, which was 4 bits in the existing CSI-2 standard, to 8 bits.

[0037] For example, in a D-PHY packet, the existing packet header already has 4 bits for VC, and by defining the extended packet header's extended VC as 4 bits, a total of 8 bits can be achieved. Specifically, OePH[7:0] = {5'h00,RSID,XY_POS,MC} and OePF[3:0] = {3'h0,pCRC} can be defined, allowing for on / off control of packet transmission as required for each application.

[0038] The optional extended packet header and optional extended packet footer are selectively transmitted depending on the application.

[0039] The legacy payload corresponds to the same payload as the existing CSI-2 standard.

[0040] In this way, by setting the extended packet header, optional extended packet header, and optional extended packet footer as needed, it becomes possible to transmit data suitable for various applications. Furthermore, data transmitted in the extended packet header, optional extended packet header, and optional extended packet footer uses a 26-bit + 6-bit ECC (Error Correction Code). This allows the existing packet header circuit to be reused, suppressing an increase in circuit size and improving error tolerance.

[0041] As a specific application example of such an extended packet for D-PHY, Fig. 4 shows the packet structure of a short packet (hereinafter referred to as an extended short packet for D-PHY) used in the extended mode of CSI-2 when the physical layer is D-PHY. Similarly, Fig. 5 shows the packet structure of a long packet (hereinafter referred to as an extended long packet for D-PHY) used in the extended mode of CSI-2 when the physical layer is D-PHY.

[0042] 4, in the D-PHY extended short packet, the data type extended type setting information stored in the packet header indicates that the type of extended mode is extended mode 0 (DT[5:0]=0x1C(5'b111_0_0)). Also, the data type short packet setting information stored in the extended packet header indicates that it is a short packet (DT[7:0]=0x00 (Frame Start Code(Short Packet))).

[0043] In this way, when the extended mode is selected and the data type stored in the extended packet header is DT[7:0]=0x00-0x0F, the packet is treated as an extended short packet, and the optional extended packet header always contains data including the Short Packet Data Field of the extended short packet. This Short Packet Data Field is the same as that defined in the existing CSI-2 standard.

[0044] When transmitting an extended short packet, the MC (MessageCount for GLD) and RSID (Vehicle Line Number and Source ID) of the optional extended packet header may be transmitted, but the legacy payload and pCRC are not required and therefore must not be transmitted. If they are transmitted by mistake, they will be ignored by the receiving side.

[0045] 4, the extended short packet can expand the bit width of the data type and virtual channel compared to the extended short packet conforming to the existing CSI-2 standard, and can support various applications defined by the optional extension packet header. Furthermore, if these functions are not required, the extended short packet conforming to the existing CSI-2 standard may be transmitted together with the extended long packet.

[0046] In an extended long packet for D-PHY as shown in Figure 5, the data type extended type setting information stored in the packet header indicates that the type of extended mode is extended mode 0 (DT[5:0] = 0x1C (5'b111_0_0)). Also, the data type short packet setting information stored in the extended packet header indicates that it is a type other than a short packet (DT[7:0] is other than 0x00 to 0x0F (= extended long packet)). Therefore, data including a short packet data field is not transmitted in an extended long packet.

[0047] Furthermore, in accordance with the settings of the extension packet header, the optional extension packet header, legacy payload, and optional extension packet footer are stored in a payload conforming to the existing CSI-2 standard and transmitted. Because they are stored in the existing payload and transmitted in this way, the existing SerDes transmitter circuit 34 and SerDes receiver circuit 35 (FIG. 2) recognize them as the same as image data transmitted in the existing payload, and transmit them as is to the subsequent stages.

[0048] The application processor 22 at the final stage can determine the extended mode based on the data type DT[5:0] in the packet header. Therefore, the application processor 22 can interpret the contents of the payload in order, starting from the extended packet header, and extract the desired extended mode data.

[0049] Fig. 6 shows the overall packet structure of a packet used in the extended mode of CSI-2 when the physical layer is C-PHY (hereinafter referred to as an extended packet for C-PHY). Note that in the extended packet for C-PHY shown in Fig. 6, a description of the configuration common to the extended packet for D-PHY in Fig. 3 will be omitted, and only different configurations will be described.

[0050] For example, in an extended packet for C-PHY, the extended mode is identified by the data type, as in the extended packet for D-PHY in Figure 3, and all data corresponding to each application executed by the application processor 22 is embedded in the payload and transmitted.

[0051] As shown in Figure 6, the extended packet for C-PHY transmits the packet header twice, just like a packet for C-PHY that complies with the existing CSI-2 standard, and arranges data in 16-bit units because C-PHY converts 16 bits to 7 symbols. An extended packet header is placed at the beginning of the payload, but with regard to virtual channels, the beginning of the existing packet header for C-PHY was reserved for this purpose, so virtual channels are not stored in the extended packet header. Of course, virtual channels may be stored in the extended packet header, just like an extended packet for D-PHY.

[0052] Furthermore, because the optional extended packet header and optional extended packet footer have a large number of bits, a flag called OePHF is prepared, and when this flag is 1, the OePH / OePF information is transmitted next. After the ePH information and OePH information, a CRC is transmitted as an extended packet header, and a similarly configured packet header is transmitted twice. In this way, by maintaining the same structure as the existing mechanism for transmitting packet headers twice, it is possible to achieve both circuit reuse and error tolerance.

[0053] As a specific application example of such an extended packet for C-PHY, Fig. 7 shows the packet structure of a short packet (hereinafter referred to as an extended short packet for C-PHY) used in the extended mode of CSI-2 when the physical layer is C-PHY. Similarly, Fig. 8 shows the packet structure of a long packet (hereinafter referred to as an extended long packet for C-PHY) used in the extended mode of CSI-2 when the physical layer is C-PHY.

[0054] The extended short packet for C-PHY shown in FIG. 7 has a packet structure that is not significantly different from that of the extended short packet for D-PHY shown in FIG. 4, and the extended long packet for C-PHY shown in FIG. 8 has a packet structure that is not significantly different from that of the extended long packet for D-PHY shown in FIG. 5.

[0055] <Configuration example of image sensor and application processor> (Image sensor configuration example) FIG. 9 is a block diagram showing an example of the configuration of the image sensor 21 including the extended mode-compatible CSI-2 transmission circuit 31.

[0056] 9 , in addition to the extended mode-compatible CSI-2 transmission circuit 31, the image sensor 21 is configured to include pixels 41, an AD converter 42, an image processing unit 43, a pixel CRC calculation unit 44, a physical layer processing unit 45, an I2C / I3C slave 46, and a register 47. The extended mode-compatible CSI-2 transmission circuit 31 is also configured to include a packing unit 51, a packet header generation unit 52, an extended packet header generation unit 53, an extended packet footer generation unit 54, selection units 55 and 56, a CRC calculation unit 57, a lane distribution unit 58, a CCI slave 59, and a controller 60.

[0057] The pixels 41 output analog pixel signals corresponding to the amount of light received, and an analog-to-digital converter (ADC) 42 converts the pixel signals output from the pixels 41 into digital signals and supplies them to an image processing unit 43. The image processing unit (ISP) 43 supplies image data obtained by performing various image processes on an image based on the pixel signals to a pixel CRC calculation unit 44 and a packing unit 51. The image processing unit 43 also supplies a data enable signal data_en indicating whether the image data is valid to the packing unit 51 and the controller 60.

[0058] The pixel CRC calculation unit 44 calculates and obtains a CRC for each pixel in the image data supplied from the image processing unit 43, and supplies the CRC to an extended packet footer generation unit 54.

[0059] The physical layer processing unit 45 can perform physical layer processing for both C-PHY and D-PHY. For example, the physical layer processing unit 45 performs physical layer processing for C-PHY when a C-layer enable signal cphy_en supplied from the controller 60 is enabled, and performs physical layer processing for D-PHY when the C-layer enable signal cphy_en is disabled. The physical layer processing unit 45 then transmits the packets divided into four lanes by the lane distribution unit 58 to the application processor 22.

[0060] The I2C / I3C slave 46 performs communication under the initiative of an I2C / I3C master 72 (FIG. 10) of the application processor 22, based on the I2C (Inter-Integrated Circuit) or I3C (Improved Inter Integrated Circuits) standard.

[0061] Various settings transmitted from the application processor 22 are written to the register 47 via the I2C / I3C slave 46 and the CCI slave 59. Here, the settings written to the register 47 include, for example, communication settings in accordance with the CSI-2 standard, extended mode settings indicating whether or not extended mode is used, and fixed communication settings required for communication in extended mode.

[0062] The packing unit 51 performs packing processing to store the image data supplied from the image processing unit 43 in the payload of a packet, and supplies the payload to the selection unit 55 and the lane distribution unit 58 .

[0063] When the packet header generator 52 is instructed to generate a packet header in accordance with a packet header generation instruction signal ph_go supplied from the controller 60 , the packet header generator 52 generates a packet header and supplies it to the selector 55 and lane distributor 58 .

[0064] That is, the packet header generator 52 generates a packet header that stores setting information indicating conditions set for data transmitted in a packet, for example, a data type indicating the type of data, in accordance with the existing CSI-2 standard. Furthermore, the packet header generator 52 stores extended mode setting information indicating whether the extended mode uses an extension header in an unused area of ​​the data type, which is setting information indicating the type of data transmitted in a packet and is defined as unused in the existing CSI-2 standard. Furthermore, the packet header generator 52 stores extended type setting information indicating which of multiple types of extended modes provided is used in the unused area.

[0065] The extension packet header generator 53 generates an extension packet header and an optional extension packet header in accordance with an extension packet header generation instruction signal eph_go and an extension packet header enable signal ePH_en supplied from the controller 60, and supplies them to the selector 56 and the lane distributor 58. In addition, the extension packet header generator 53 is supplied with an in-vehicle row number, a source ID (identification), and the like depending on the application of the image sensor 21, and stores these in the extension packet header or the optional extension packet header as necessary.

[0066] That is, the extension packet header generator 53 generates an extension packet header that stores setting information such as that shown in Fig. 3, separately from the packet header generated by the packet header generator 52. Furthermore, when transmitting an optional extension packet header, the extension packet header generator 53 stores optional extension packet header setting information indicating that an optional extension packet header is to be transmitted in the extension packet header as optional extension packet header setting information (OePH[7:0]) indicating whether or not an optional extension packet header is to be transmitted, and generates the optional extension packet header following the extension packet header.

[0067] The extended packet footer generator 54 generates an optional extended packet footer in accordance with the extended packet footer generation instruction signal epf_go and the extended packet header enable signal ePF_en supplied from the controller 60 , and supplies the generated optional extended packet footer to the selector 56 and the lane distributor 58 .

[0068] That is, when a packet transmitted in extended mode is an extended long packet that stores data transmitted as a payload in the existing CSI-2 standard, the extended packet footer generation unit 54 generates an optional extended packet footer that is placed following the legacy payload in which the data is stored.

[0069] Furthermore, a C-layer enable signal cphy_en is supplied from the controller 60 to the packet header generator 52, the extended packet header generator 53, and the extended packet footer generator 54. When the C-layer enable signal cphy_en indicates valid, the packet header generator 52 generates a packet header for C-PHY, the extended packet header generator 53 generates an extended packet header and optional extended packet header for C-PHY, and the extended packet footer generator 54 generates an optional extended packet footer for C-PHY. On the other hand, when the C-layer enable signal cphy_en indicates invalid, the packet header generator 52 generates a packet header for D-PHY, the extended packet header generator 53 generates an extended packet header and optional extended packet header for D-PHY, and the extended packet footer generator 54 generates an optional extended packet footer for D-PHY.

[0070] In accordance with the C-layer enable signal cphy_en supplied from the controller 60, if the C-layer enable signal cphy_en is valid, the selector 55 selects the packet header supplied from the packet header generator 52 and supplies it to the selector 56. On the other hand, if the C-layer enable signal cphy_en is invalid, the selector 55 selects the payload supplied from the packing unit 51 and supplies it to the selector 56.

[0071] In accordance with a data selection signal data_sel supplied from the controller 60, the selector 56 selects one of the packet header or payload selectively supplied via the selector 55, the extended packet header and optional extended packet header supplied from the extended packet header generator 53, and the optional extended packet footer supplied from the extended packet footer generator 54, and supplies the selected one to the CRC calculator 57.

[0072] The CRC calculation unit 57 calculates the CRC of the packet header, payload, extended packet header, optional extended packet header, or optional extended packet footer selectively supplied via the selection unit 56, and supplies the CRC to the lane distribution unit 58.

[0073] Under the control of the controller 60, the lane distribution unit 58 distributes the payload supplied from the packing unit 51, the packet header supplied from the packet header generation unit 52, the extended packet header and optional extended packet header supplied from the extended packet header generation unit 53, the optional extended packet footer supplied from the extended packet footer generation unit 54, and the CRC supplied from the CRC calculation unit 57 to four lanes in accordance with the CSI-2 standard, and supplies them to the physical layer processing unit 45.

[0074] A CCI (Camera Control Interface) slave 59 performs communication under the initiative of a CCI master 88 (FIG. 10) of the application processor 22 based on the CSI-2 standard.

[0075] The controller 60 reads out various settings stored in the register 47 and, according to those settings, controls each block constituting the extended mode-compatible CSI-2 transmission circuit 31. For example, the controller 60 controls switching between transmitting packets with a packet structure conforming to the existing CSI-2 standard and transmitting packets with a packet structure in extended mode, depending on the content of the data to be transmitted.

[0076] The image sensor 21 is configured in this way, and can generate an extended packet having the packet structure as described with reference to FIGS.

[0077] (Example of application processor configuration) FIG. 10 is a block diagram showing an example of the configuration of an application processor 22 including an extended mode-compatible CSI-2 receiver circuit 32.

[0078] 10 , in addition to the extended mode-compatible CSI-2 receiver circuit 32, the application processor 22 is configured to include a physical layer processing unit 71, an I2C / I3C master 72, a register 73, and a controller 74. The extended mode-compatible CSI-2 receiver circuit 32 is configured to include a packet header detection unit 81, a lane merging unit 82, an interpretation unit 83, selection units 84 and 85, a CRC calculation unit 86, an unpacking unit 87, and a CCI master 88.

[0079] The physical layer processing unit 71 can perform physical layer processing for both C-PHY and D-PHY. As described above, the physical layer processing unit 45 of the image sensor 21 performs physical layer processing for either C-PHY or D-PHY, and the physical layer processing unit 71 performs the same physical layer processing as that performed by the physical layer processing unit 45.

[0080] The I2C / I3C master 72 takes the lead in communicating with the I2C / I3C slave 46 (FIG. 9) of the image sensor 21 based on the I2C or I3C standard.

[0081] In the register 73, various settings to be written by the controller 74 to the register 47 of the image sensor 21 are recorded.

[0082] The controller 74 controls each block that constitutes the application processor 22 .

[0083] The packet header detection unit 81 detects a packet header from the packet supplied from the physical layer processing unit 71 and checks the data type stored in the packet header. If the extended mode setting information in the data type of the packet header indicates the extended mode (DataType[5:3]=3'b111), the packet header detection unit 81 supplies an extended mode detection flag indicating the extended mode to the interpretation unit 83, the selection unit 84, and the selection unit 85. The packet header detection unit 81 also supplies a merging enable signal mrg_en indicating whether or not to enable merging of the four divided lanes to the lane merging unit 82 based on the packet header.

[0084] That is, the packet header detection unit 81 detects a packet header that stores setting information (such as a data type) indicating conditions set for data transmitted in a packet in accordance with the existing CSI-2 standard. In this case, the packet header detection unit 81 outputs an extended mode detection flag in accordance with extended mode setting information indicating whether an extended mode using an extended header is in effect, and the extended mode setting information is stored in an unused area defined as unused in the existing CSI-2 standard in the data type, which is setting information indicating the type of data transmitted in the packet. This allows the unit to switch between receiving packets with a packet structure in accordance with the existing CSI-2 standard and receiving packets with a packet structure in the extended mode. Furthermore, the packet header detection unit 81 recognizes which type of extended mode is in effect among multiple types of extended modes provided as extended modes, in accordance with extended mode type information stored in an unused area of ​​a data type defined as unused in the existing CSI-2 standard.

[0085] When the merge enable signal mrg_en supplied from the packet header detection unit 81 is valid, the lane merging unit 82 merges the packets divided into four lanes supplied from the physical layer processing unit 71. Then, the lane merging unit 82 supplies the packet of one lane to the interpretation unit 83, the selection unit 84, and the selection unit 85.

[0086] If the extended mode detection flag supplied from the packet header detection unit 81 indicates the extended mode, the interpretation unit 83 reads the extended packet header, optional extension packet header, and optional extension packet footer from the packet supplied from the lane merging unit 82 based on the packet structure of the extended mode. Then, the interpretation unit 83 interprets the setting information stored in the extended packet header, optional extension packet header, and optional extension packet footer.

[0087] That is, the interpretation unit 83 receives an extension packet header placed at the beginning of a payload conforming to the existing CSI-2 standard as an extension header and interprets the setting information stored in the extension packet header. Furthermore, when the optional extension packet header setting information stored in the extension packet header indicates that an optional extension packet header that is selectively transmitted depending on the application is to be transmitted, the interpretation unit 83 receives an optional extension packet header following the extension packet header and interprets the setting information stored in the optional extension packet header. Furthermore, when a packet transmitted in extended mode is an extended long packet that stores data transmitted as a payload in the existing CSI-2 standard, the interpretation unit 83 receives an optional extension packet footer placed following the legacy payload in which the data is stored and interprets the optional extension packet footer.

[0088] Then, the interpretation unit 83 reads, for example, the vehicle row number and source ID stored in the optional extension packet header, and outputs them to a subsequent LSI (not shown).

[0089] If the extended mode detection flag supplied from the packet header detection unit 81 does not indicate the extended mode, that is, if a packet with an existing packet structure is supplied, the interpretation unit 83 stops without performing the above-mentioned processing.

[0090] The selector 84 selectively supplies data to an unpacking unit 87 based on the packet structure of the existing packet or the packet structure of the extended packet in accordance with the extended mode detection flag supplied from the packet header detector 81 .

[0091] The selector 85 selectively supplies data to a CRC calculator 86 based on the packet structure of the existing packet or the packet structure of the extended packet in accordance with the extended mode detection flag supplied from the packet header detector 81 .

[0092] The CRC calculation unit 86 calculates the CRC of the packet header, payload, extension packet header, optional extension packet header, or optional extension packet footer selectively supplied via the selection unit 85. If a CRC error is detected, the CRC calculation unit 86 outputs a crcCRC error detection signal indicating this to a downstream LSI (not shown).

[0093] The unpacking unit 87 performs an unpacking process to extract image data stored in the payload selectively supplied via the selection unit 84, and outputs the acquired image data to a downstream LSI (not shown).

[0094] The CCI master 88 takes the lead in communicating with the CCI slave 59 (FIG. 9) of the image sensor 21 based on the CSI-2 standard.

[0095] The application processor 22 is configured in this way, and can receive an extension packet sent from the image sensor 21, interpret the setting information stored in the extension packet header, optional extension packet header, and optional extension packet footer, and acquire image data.

[0096] <Communication processing> The communication processing performed by the image sensor 21 and the application processor 22 will be described with reference to FIGS.

[0097] FIG. 11 is a flowchart illustrating a process in which the image sensor 21 transmits a packet.

[0098] For example, processing begins when the image sensor 21 is connected to the application processor 22 via the bus 23. In step S11, the controller 60 determines whether or not to use the extended mode when starting communication with the application processor 22. For example, the controller 60 checks the extended mode setting stored in the register 47, and determines that the extended mode will be used if the extended mode setting indicating the use of the extended mode has been written by the application processor 22.

[0099] If the controller 60 determines in step S11 that the extended mode will not be used, the process proceeds to step S12.

[0100] In step S12, the I2C / I3C slave 46 receives a command to start transmitting image data transmitted from the application processor 22 (in step S54 of FIG. 13, which will be described later). Furthermore, the I2C / I3C slave 46 receives communication settings conforming to the CSI-2 standard transmitted together with the transmission start command, and writes the settings to the register 47 via the CSI slave 59.

[0101] In step S13, the image sensor 21 performs a conventional packet transmission process, transmitting a packet having a packet structure conforming to the existing CSI-2 standard to the application processor 22 based on the communication settings stored in the register 47.

[0102] On the other hand, if the controller 60 determines in step S11 that the extended mode is to be used, the process proceeds to step S14.

[0103] In step S14, the I2C / I3C slave 46 receives fixed communication settings required for communication in extended mode (for example, a copy of each lane of PH / PF in GLD mode) and writes them to the register 47 via the CCI slave 59.

[0104] In step S15, the I2C / I3C slave 46 receives a command to start transmitting image data transmitted from the application processor 22 (in step S57 of FIG. 13, which will be described later). Furthermore, the I2C / I3C slave 46 receives communication settings conforming to the CSI-2 standard transmitted together with the transmission start command, and writes them into the register 47 via the CSI slave 59.

[0105] In step S16, the controller 60 determines whether or not to start transmitting packets, and waits until it determines to start transmitting packets.

[0106] If it is determined in step S16 that packet transmission should be started, the process proceeds to step S17, where the controller 60 determines whether the data should be transmitted in the extended mode. Here, the controller 60 determines that the data should be transmitted in the extended mode depending on the content of the data to be transmitted, for example, if the data is data that would be transmitted in a use case of an application example described below.

[0107] If the controller 60 determines in step S17 that the data should be transmitted in the extended mode, the process proceeds to step S18, where an extended mode transmission process (see FIG. 12) is performed to transmit an extended packet corresponding to the extended mode.

[0108] On the other hand, if the controller 60 determines in step S17 that the data is not to be transmitted in the extended mode, the process proceeds to step S19.

[0109] In step S19, the controller 60 determines whether or not to transmit a short packet. For example, the controller 60 determines to transmit a short packet at the start and end of a frame.

[0110] If the controller 60 determines in step S19 to transmit a short packet, the process proceeds to step S20. In step S20, the packet header generator 52 generates a packet header and transmits a short packet with a conventional packet structure to the application processor 22.

[0111] On the other hand, if the controller 60 determines in step S19 not to transmit a short packet (i.e., to transmit a long packet), the process proceeds to step S21. In step S21, the packing unit 51 stores the image data in the payload, and the CRC calculation unit 57 calculates the CRC to generate a long packet with a conventional packet structure, which is then transmitted to the application processor 22.

[0112] After the processing of step S18, step S20, or step S21, the process proceeds to step S22, where the controller 60 ends the packet transmission process. Thereafter, the process returns to step S16, and the process of transmitting the packet is similarly repeated for the next packet.

[0113] FIG. 12 is a flowchart illustrating the extended mode transmission process performed in the process of step S18 in FIG.

[0114] In step S31, the packet header generation unit 52 generates a packet header that stores the VC, data type, WC, etc., and transmits it to the application processor 22. At this time, the packet header generation unit 52 writes, to the data type of the packet header, extended mode setting information (DataType[5:3]=3'b111) indicating the extended mode, and extended type setting information (DataType[1:0]=2'b00) identifying that the mode setting of the extended mode is extended mode 0.

[0115] In step S32, the application processor 22 determines whether to transmit an extended short packet. For example, the controller 60 determines to transmit an extended short packet at the start and end of a frame.

[0116] If the application processor 22 determines in step S32 that an extended short packet is to be transmitted, the process proceeds to step S33.

[0117] In step S33, the extension packet header generator 53 transmits an extension packet header in which the data type (DataType[7:0]) is set to short packet in the first byte of the payload. At this time, the extension packet header generator 53 performs various settings (e.g., OePH[7:0], OePF[3:0], etc.) to be stored in the extension packet header.

[0118] In step S34, the extended packet header generator 53 stores a frame number (FN: Frame Number) in the second byte of the payload and transmits it.

[0119] In step S35, the extension packet header generator 53 generates and transmits an optional extension packet header as shown in FIG. 4 in accordance with the setting (OePH[7:0]) made in step S33.

[0120] In step S36, the CRC calculation unit 57 calculates the CRC and transmits it as a packet footer.

[0121] On the other hand, if the application processor 22 determines in step S32 not to transmit an extended short packet (that is, to transmit a long packet), the process proceeds to step S37.

[0122] In step S37, the extension packet header generator 53 transmits an extension packet header in which the data type (DataType[7:0]) is set to other than a short packet in the first byte of the payload. At this time, the extension packet header generator 53 performs various settings (e.g., OePH[7:0], OePF[3:0], etc.) to be stored in the extension packet header.

[0123] In step S38, the extension packet header generator 53 generates and transmits an optional extension packet header as shown in FIG. 5 in accordance with the setting (OePH[7:0]) made in step S37.

[0124] In step S39, the packing unit 51 packs the image data supplied from the image processing unit 43, generates a legacy payload, and transmits it.

[0125] In step S40, the extended packet footer generating unit 54 generates and transmits an optional extended packet footer as shown in FIG. 4 in accordance with the setting (OePF[3:0]) made in step S37.

[0126] In step S41, the CRC calculation unit 57 calculates a CRC and transmits it as a packet footer.

[0127] After the processing of step S36 or S41, the extended mode transmission processing ends.

[0128] As described above, the image sensor 21 can generate and transmit an extended short packet or an extended long packet.

[0129] FIG. 13 is a flowchart illustrating the process in which the application processor 22 receives a packet.

[0130] For example, the process starts when the image sensor 21 is connected to the application processor 22 via the bus 23. In step S51, the controller 74 writes the initial settings of the image sensor 21 (for example, whether C-PHY or D-PHY is to be used as the physical layer) to the register 73 and transmits the settings to the image sensor 21 via the I2C / I3C master 72 and the CCI master 88. As a result, the initial settings are written to the register 47 of the image sensor 21.

[0131] In step S52, the controller 74 determines whether or not the image sensor 21 supports the extended mode. For example, the controller 74 can determine whether or not the image sensor 21 supports the extended mode by acquiring a setting value (e.g., extended PH / PF capability) stored in the register 47 of the image sensor 21 via the I2C / I3C master 72. Alternatively, the controller 74 can determine in advance whether or not the image sensor 21 supports the extended mode based on, for example, a manual input.

[0132] In step S53, the controller 74 determines whether the image sensor 21 supports the extended mode and whether the application executed by the application processor 22 requests the use of the extended mode.

[0133] If the controller 74 determines in step S53 that the image sensor 21 does not support the extended mode or that use of the extended mode is not desired, the process proceeds to step S54.

[0134] In step S54, the controller 74 transmits an image data transmission start command to the image sensor 21 via the I2C / I3C master 72. At this time, the controller 74 also transmits communication settings in accordance with the CSI-2 standard.

[0135] In step S55, the application processor 22 performs conventional packet reception processing to receive packets with a packet structure conforming to the existing CSI-2 standard, based on the communication settings transmitted in step S54.

[0136] On the other hand, if the controller 74 determines in step S53 that the image sensor 21 supports the extended mode and that the application executed by the application processor 22 requests the use of the extended mode, the process proceeds to step S56.

[0137] In step S56, before communication in the extended mode is started, the I2C / I3C master 72 transmits fixed communication settings required for communication in the extended mode, which are then written to the register 47 of the image sensor 21 (step S14 in FIG. 11).

[0138] In step S57, the controller 74 transmits an image data transmission start command to the image sensor 21 via the I2C / I3C master 72. At this time, the controller 74 also transmits communication settings in accordance with the CSI-2 standard.

[0139] In step S58, the packet header detection unit 81 determines whether packet reception has started by checking the data supplied from the physical layer processing unit 71, and waits until it determines that packet reception has started. For example, when the packet header detection unit 81 detects a packet header from the data supplied from the physical layer processing unit 71, it determines that packet reception has started.

[0140] If the packet header detection unit 81 determines in step S58 that packet reception has started, the process proceeds to step S59.

[0141] In step S59, the packet header detection unit 81 checks the data type of the packet header detected in step S58 and determines whether the packet that has started to be received is an extended packet that supports the extended mode. For example, if the extended mode setting information in the data type of the packet header indicates the extended mode (DataType[5:3]=3'b111), the packet header detection unit 81 determines that the packet that has started to be received is an extended packet.

[0142] If the packet header detection unit 81 determines in step S59 that the packet that has started to be received is an extended packet, the process proceeds to step S60, where extended mode reception processing (see FIG. 14) is performed to receive the extended packet.

[0143] On the other hand, if the packet header detection unit 81 determines in step S59 that the packet that has started to be received is not an extended packet, the process proceeds to step S61.

[0144] In step S61, the packet header detection unit 81 checks the data type (DataType[5:0]) of the packet header detected in step S58, and determines whether the packet that has started to be received is a short packet.

[0145] In step S61, if the packet header detection unit 81 determines that the packet that has started to be received is a short packet, the process proceeds to step S62. In step S62, the packet header detection unit 81 receives a short packet with a conventional packet structure transmitted from the image sensor 21.

[0146] On the other hand, if the packet header detection unit 81 determines in step S61 that the packet that has started to be received is not a short packet (i.e., reception of a long packet has started), the process proceeds to step S63. In step S63, the unpacking unit 87 receives the payload of the long packet with the conventional packet structure transmitted from the image sensor 21 and extracts the image data, and the CRC calculation unit 86 receives the WC+1-th byte transmitted following the packet header as a CRC.

[0147] After the processing of step S60, step S62, or step S63, the process proceeds to step S64, where the controller 74 ends the packet reception process. Thereafter, the process returns to step S58, and the packet reception process is repeated in the same manner for the next packet.

[0148] FIG. 14 is a flowchart illustrating the extended mode reception process performed in step S60 of FIG.

[0149] In step S71, the packet header detection unit 81 determines whether the mode setting of the extended mode is extended mode 0. For example, if the extended type setting information in the data type of the packet header indicates extended mode 0 (DataType[1:0] = 2'b00), the packet header detection unit 81 determines that the mode setting of the extended mode is extended mode 0.

[0150] In step S71, if the packet header detection unit 81 determines that the mode setting of the extended mode is extended mode 0, the process proceeds to step S72. In step S72, the interpretation unit 83 receives the first byte of the payload as an extended packet header.

[0151] In step S73, the interpretation unit 83 checks the data type (DataType[7:0]) of the extension packet header received in step S72 to determine whether the packet that has started to be received is an extension short packet.

[0152] If the interpretation unit 83 determines in step S73 that the packet is an extended short packet, the process proceeds to step S74. In step S74, the interpretation unit 83 receives an optional extension packet header in accordance with the setting (OePH[7:0]) stored in the extension packet header received in step S72.

[0153] In step S75, the CRC calculation unit 86 receives the WC+1 byte transmitted following the optional extension packet header as a CRC.

[0154] On the other hand, if the interpretation unit 83 determines in step S73 that the packet is not an extended short packet (i.e., reception of an extended long packet has started), the process proceeds to step S76. In step S76, the interpretation unit 83 receives an optional extension packet header in accordance with the setting (OePH[7:0]) stored in the extension packet header received in step S72.

[0155] In step S77, the unpacking unit 87 receives the legacy payload of the extended long packet transmitted from the image sensor 21 and extracts the image data.

[0156] In step S78, the interpretation unit 83 receives the optional extension packet footer in accordance with the setting (OePF[3:0]) stored in the extension packet header received in step S72.

[0157] In step S79, the CRC calculation unit 86 receives the WC+1 byte transmitted following the optional extension packet footer as a CRC.

[0158] If it is determined in step S71 that the mode setting of the extended mode is not extended mode 0, the extended mode reception process is terminated after the process of step S75 or the process of step S79.

[0159] As described above, the application processor 22 can receive an extended short packet or an extended long packet and obtain data.

[0160] <Second example of packet structure> A second example of the packet structure of a packet used in communication between the enhanced mode-compatible CSI-2 transmitter circuit 31 and the enhanced mode-compatible CSI-2 receiver circuit 32 will be described with reference to FIGS.

[0161] 3 to 8, emphasis is placed on maintaining compatibility with the existing CSI-2 standard, and the packet header and packet footer have the same packet structure as the existing CSI-2 standard, with the packet structure being extended by an extended packet header, optional extended packet header, and optional extended packet footer.In contrast, in the second structure example described below, the packet header and packet footer are different from those of the existing CSI-2 standard, and the packet structure is extended by an extended packet header and extended packet footer.

[0162] FIG. 15 shows the packet structure of a short packet used in the extended mode of CSI-2 when the physical layer is D-PHY (hereinafter referred to as an extended short packet for D-PHY).

[0163] In the extended short packet for D-PHY shown in Figure 15, like the extended short packet for D-PHY of the first structure example shown in Figure 4, the extended mode is identified by the data type stored in the packet header, which is the same as that of the existing CSI-2 standard.

[0164] On the other hand, in the extended short packet for D-PHY shown in Fig. 15, the frame number is stored in the short packet data field in the 16 bits following the data type of the packet header, just like a short packet conforming to the existing CSI-2 standard. Then, following the packet header, an extended packet header configured in the same way as the extended packet header shown in Fig. 4 is transmitted.

[0165] Therefore, the receiving application processor 22 can interpret the data type stored in the extended packet header and determine that, if it is an extended short packet, the frame number is stored in the data field of the packet header.

[0166] The optional extension packet header in the D-PHY extended short packet shown in Fig. 15 has the same structure as the optional extension packet header in the D-PHY extended short packet of the first structure example shown in Fig. 4. However, since the optional extension packet header has a packet structure that is not embedded in the payload, there is no need to add a CRC to the end.

[0167] FIG. 16 shows the packet structure of a long packet (hereinafter referred to as an extended long packet for D-PHY) used in the extended mode of CSI-2 when the physical layer is D-PHY.

[0168] In the D-PHY extended long packet shown in Figure 16, the extended data is not embedded in the payload but is transmitted as part of the packet header or packet footer. Therefore, the WC in the first packet header simply indicates the byte length of the payload, as in the existing standard.

[0169] FIG. 17 shows the packet structure of a short packet used in the extended mode of CSI-2 when the physical layer is C-PHY (hereinafter referred to as an extended short packet for C-PHY).

[0170] The extended portion of the C-PHY extended short packet shown in Figure 17 is transmitted as an extension of the packet header in accordance with the existing CSI-2 standard, so the extended portion, such as the extended packet header, is inserted after the frame number. As with the existing CSI-2 standard, the packet header ends with a CRC. Furthermore, the packet structure, which is transmitted twice with a SYNC in between, is the same as that of a short packet in accordance with the existing CSI-2 standard.

[0171] FIG. 18 shows the packet structure of a long packet used in the extended mode of CSI-2 when the physical layer is C-PHY (hereinafter referred to as an extended long packet for C-PHY).

[0172] As described above, the extended long packet for C-PHY shown in FIG. 18 differs from the extended long packet for C-PHY of the first structure example shown in FIG. 8 in that the WC in the first packet header indicates the byte length of the payload, as in existing standards.

[0173] As described above, the packet structure of the extended packet of the second structure example shown in Figures 15 to 18 can accommodate a wider variety of applications than conventional ones, similar to the packet structure of the extended packet of the first structure example (Figures 3 to 8).

[0174] However, the extended packet of the second structure example has a packet structure in which the existing packet header and footer are extended without embedding the extended data in the existing payload. Therefore, when the packet structure of the extended packet of the second structure example is adopted, the impact of requiring changes to the conventional communication system cannot be minimized compared to when the packet structure of the extended packet of the first structure example is adopted. That is, for example, the existing SerDes transmitter circuit 34 would need to be changed to the SerDes receiver circuit 35 (FIG. 2).

[0175] As described above, by adopting the extended packet of the first structural example, it is possible to accommodate a variety of applications, including in-vehicle applications, and to construct an in-vehicle system while minimizing the impact of requiring changes to conventional communication systems.

[0176] Furthermore, by adopting the extended packet of the second structural example, although changes are required from the conventional communication system, it is possible to accommodate a variety of applications, such as in-vehicle applications.

[0177] <Modifications of Image Sensor and Application Processor> (Image sensor modification) A modification of the image sensor and application processor will be described with reference to FIG.

[0178] The blocks constituting the image sensor 21 in Fig. 9 and the application processor 22 in Fig. 10 described above are configured to be able to process both D-PHY and C-PHY packets. However, for example, it is also possible to provide both a block that processes D-PHY packets exclusively and a block that processes C-PHY packets exclusively, and switch between the processes performed by each block.

[0179] The image sensor 21A shown in A of FIG. 19 is configured to include a D-layer processing block section 101, a C-layer processing block section 102, a switching section 103, and a controller 60.

[0180] The D-layer processing block unit 101 includes a block that processes packets for D-PHY exclusively, among the blocks that make up the image sensor 21 in Fig. 9. The C-layer processing block unit 102 includes a block that processes packets for C-PHY exclusively, among the blocks that make up the image sensor 21 in Fig. 9. Under the control of the controller 60, the switching unit 103 switches so as to output packets for D-PHY generated in the D-layer processing block unit 101 when D-PHY is used for the physical layer, and to output packets for C-PHY generated in the C-layer processing block unit 102 when C-PHY is used for the physical layer.

[0181] (Application Processor Variation) The application processor 22A shown in FIG. 19B is configured to include a switching unit 111, a D-layer processing block unit 112, a C-layer processing block unit 113, and a controller 74.

[0182] The switching unit 111 switches, under the control of the controller 74, so as to supply packets transmitted from the image sensor 21A to either the D-layer processing block unit 112 or the C-layer processing block unit 113. The D-layer processing block unit 112 has a block that is dedicated to processing packets for D-PHY, among the blocks that make up the application processor 22 in Fig. 10. The C-layer processing block unit 113 has a block that is dedicated to processing packets for C-PHY, among the blocks that make up the application processor 22 in Fig. 10.

[0183] In the image sensor 21A and application processor 22A configured as described above, before starting communication, the physical layer to be used can be set between the controller 60 and the controller 74. For example, when D-PHY is used for the physical layer, packets for D-PHY generated in the D-layer processing block unit 101 are transmitted via the switching unit 103 and supplied to the D-layer processing block unit 112 via the switching unit 111 for processing. For example, when C-PHY is used for the physical layer, packets for C-PHY generated in the C-layer processing block unit 102 are transmitted via the switching unit 103 and supplied to the C-layer processing block unit 113 via the switching unit 111 for processing.

[0184] <Example of application of extended packets> The application of the above-mentioned extended packets to the following use cases is being considered, for example.

[0185] For example, the extended packet is being considered for use in a use case such as transmitting higher resolution images (RAW24).

[0186] For example, when transmitting image data in RAW format, the existing CSI-2 standard defines RAW6, RAW7, RAW8, RAW10, RAW12, RAW14, RAW16, and RAW20 as data types to be stored in the packet header. However, in recent years, there has been a demand for the transmission of higher-resolution images to support autonomous driving using in-vehicle cameras. Therefore, by applying extended packets to extend the number of bits in the data type, it is possible to define, for example, the higher-resolution RAW24 as the data type in the extended packet header.

[0187] In addition, the extended packet is being considered for application to SmartROI, a technology that transmits only the image area of ​​interest on the screen.

[0188] For example, currently, many cameras are installed in stadiums, airports, and other locations. If the entire image captured by these cameras is transmitted from the camera to a cloud server via a network such as the Internet, it is expected that the Internet bandwidth will be insufficient and the amount of calculation or data on the cloud side will increase. Therefore, by extracting only the image region of interest at the edge (camera side) and transmitting that image region, it is expected that the Internet bandwidth will be insufficient and the amount of calculation or data on the cloud side will be reduced.

[0189] When transmitting such an SROI, the coordinates of the upper left corner of the rectangular region of interest (ROI) must also be transmitted to inform the receiving side of where on the entire screen the image area of ​​interest corresponds. Furthermore, data for the entire captured image must be sent at a specific timing in response to a command from the receiving side. Therefore, for example, the SROI image and data for the entire image (existing packet header) will be mixed on a frame-by-frame basis.

[0190] Therefore, by applying an extended packet, it becomes possible to transmit coordinate data of 16 bits or more for each of the X and Y coordinates, for example.

[0191] Furthermore, the use case of the extended packet is being considered for application to GLD, which reduces the bandwidth and number of lanes to continue communication even when the channel is degraded. GLD is a proposal under consideration for CSI-2 ver. 3.0.

[0192] For example, in the case of autonomous driving, even if a part of the cable connecting the camera during a collision is disconnected, it is required to continue communication using the non-disconnected cable and automatically stop the vehicle after evacuating to the safety belt. Therefore, an in-vehicle camera interface must at least have a disconnection detection function, and information such as a line number (16 bits) indicating which line of information on the screen, a Source ID (8 bits) indicating which camera sent the information, and a message counter (16 bits) indicating the transmission number is required. Furthermore, when used in combination with the SROI as described above, it is conceivable that this information is transmitted in frame units.

[0193] Therefore, by applying an extended packet, it becomes possible to transmit this information.

[0194] <First configuration example adapted to E2E protection> Referring to FIGS. 20 to 26, a configuration example adapted to a regulation prohibiting packet modification or the like on a transmission path will be described.

[0195] For example, in the communication system 11A having the configuration described with reference to FIG. 2 above, when the interfaces between the image sensor 21 and the application processor 22 are different, it is necessary to convert the packets on the transmission path. That is, in the case of a configuration where the physical layer of the image sensor 21 is D-PHY and the physical layer of the application processor 22 is C-PHY, for example, it is necessary to convert the packets from D-PHY to C-PHY in the SerDes device 26.

[0196] In a configuration where packet conversion is performed in the SerDes device 26 in this way, for example, it violates the regulations defined in ISO26262 (Functional Safety), that is, the regulations prohibiting packet modification or the like on the transmission path (hereinafter referred to as E2E (End-to-End) protection).

[0197] FIG. 20 is a block diagram showing an example of the configuration of a communication system 201 adapted to E2E protection as a third embodiment of a communication system to which the present technology is applied.

[0198] As shown in Fig. 20, communication system 201 is configured by connecting image sensor 211, SerDes device 212, SerDes device 213, and application processor 214. Note that Fig. 20 illustrates an example in which SERDES is A-PHY, but also includes cases in which connection is made using other SERDES standards such as FPD-LINK3. In addition, in the SERDES standard, communication may be performed based on the SERDES standard while maintaining the CIS-2 format (at least the application specific payload). In addition, in the SERDES, physical layer processing units 237 and 247 may include multiple physical layer processing units of other SERDES standards in addition to A-PHY, and the physical layer processing units can be switched depending on the application.

[0199] The image sensor 211 has at least an extended mode compatible CSI-2 transmitter circuit 221, a physical layer processing unit (hereinafter referred to as a C / D-PHY physical layer processing unit) 222 compatible with C-PHY or D-PHY, or both, a slave (hereinafter referred to as an I2C / I3C slave) 223 compatible with I2C or I3C, or both, and a CCI slave 224.

[0200] The SerDes device 212 has at least a CSI-2 receiving circuit 231, a C / D-PHY physical layer processing unit 232, an I2C / I3C master 233, a CCI master 234, a CSI-2 A-PHY packet generating unit 235, a CCI A-PHY packet transmitting / receiving unit 236, and an A-PHY-compatible physical layer processing unit 237. For example, the SerDes device 212 converts packets for C-PHY or D-PHY into packets for A-PHY, and this conversion is determined based on register settings, etc.

[0201] The SerDes device 213 has at least a CSI-2 transmission circuit 241, a C / D-PHY physical layer processing unit 242, an I2C / I3C slave 243, a CCI slave 244, a CSI-2 A-PHY packet receiving unit 245, a CCI A-PHY packet transmitting / receiving unit 246, and an A-PHY-compatible physical layer processing unit 247. For example, the SerDes device 213 converts A-PHY packets into C-PHY or D-PHY packets, and this conversion is determined based on register settings, etc.

[0202] The application processor 214 includes at least an extended mode compatible CSI-2 receiving circuit 251 , a C / D-PHY physical layer processing unit 252 , an I2C / I3C master 253 , and a CCI master 254 .

[0203] The communication system 201 is configured in this manner, and an extended packet having the above-described structure is transmitted from the image sensor 211 and received by the application processor 214. Even if the communication system 201 is configured so that the physical layer processing unit 222 of the image sensor 211 supports D-PHY and the physical layer processing unit 252 of the application processor 22 supports C-PHY, it is necessary to ensure that E2E protection is not violated.

[0204] Therefore, in order to be able to apply E2E protection, the communication system 201 limits the protection scope of E2E protection to an application specific payload (hereinafter referred to as AS payload), which is a payload specific to an application. That is, the AS payload is prohibited from being changed when converting an A-PHY packet to a C-PHY or D-PHY packet, or when converting a C-PHY or D-PHY packet to an A-PHY packet.

[0205] FIG. 21 shows an example of the structure of an extended packet for D-PHY that is extended to support E2E protection.

[0206] As shown in the figure, the AS payload of an extended packet for D-PHY, which is made up of an extended packet header (ePH), packet data, and an extended packet footer (ePF), is limited as the protection scope of E2E protection.

[0207] The extended packet header contains predetermined information that is required when the scope of E2E protection is limited to the AS payload. For example, a packet count PC (Packet Count) indicating the data length of the data stored in the AS payload is added as predetermined information to be written in the extended packet header so that the data length of the packet data can be identified. That is, the packet data has the number of bytes determined by the packet count PC. Also, as predetermined information to be written in the extended packet header, a virtual channel VC (Virtual Channel) indicating the number of lines of the virtual channel is copied to the existing packet header.

[0208] FIG. 22 shows an example of the structure of an extended packet for C-PHY that is extended to support E2E protection.

[0209] As shown in the figure, in the case of an extended packet for C-PHY, the protection scope of E2E protection is limited to the AS payload consisting of an extended packet header (ePH), packet data, and extended packet footer (ePF), just like in the case of an extended packet for D-PHY. Similarly to the case of an extended packet for D-PHY, the extended packet header contains a packet count PC and a virtual channel VC as predetermined information required when the protection scope of E2E protection is limited to the AS payload.

[0210] FIG. 23 shows an example of the structure of an extended packet for A-PHY that is extended to support E2E protection.

[0211] As shown in the figure, in the extended packet for A-PHY, the AS payload consisting of the extended packet header (ePH), packet data, and extended packet footer (ePF) is limited as the protection range of E2E protection.

[0212] Here, as described with reference to FIG. 20, in the communication system 201, an extended packet for A-PHY is generated from an extended packet for D-PHY or C-PHY transmitted from the image sensor 211 to the SerDes device 212. Therefore, the packet count PC and the virtual channel VC are already described in the extended packet header of the extended packet for A-PHY.

[0213] By adopting such a packet structure, the communication system 201 can avoid the AS payload being modified on the transmission path and comply with E2E protection. Note that the packet structures shown in FIGS. 21 to 23 can be partially replaced with the corresponding packets of the packet structures shown in FIGS. 3 to 8 and FIGS. 15 to 18, and a part of the packet generation is replaced.

[0214] <Packet transmission / reception processing adapted to E2E protection> FIG. 24 is a flowchart for explaining packet transmission / reception processing adapted to E2E protection.

[0215] For example, processing begins when data to be stored in packet data (e.g., image data) is supplied to the extended mode-compatible CSI-2 transmitter circuit 221. Then, in step S101, the extended mode-compatible CSI-2 transmitter circuit 221 in the image sensor 211 stores the supplied data in packet data. Furthermore, the extended mode-compatible CSI-2 transmitter circuit 221 generates an extended packet header that describes a virtual channel VC and a packet count PC, as shown in FIG. 21 or 22 above. Then, the extended mode-compatible CSI-2 transmitter circuit 221 generates an AS payload by adding an extended packet header and an extended packet footer to the packet data.

[0216] In step S102, the extended mode-compatible CSI-2 transmitter circuit 221 generates an extended packet for C-PHY or D-PHY by adding a packet header for C-PHY or D-PHY and a packet footer for C-PHY or D-PHY to the AS payload generated in step S101. The extended mode-compatible CSI-2 transmitter circuit 221 then transmits the extended packet for C-PHY or D-PHY to the SerDes device 212 via the C / D-PHY physical layer processor 222.

[0217] In step S103, in the SerDes device 212, the CSI-2 receiving circuit 231 receives the C-PHY or D-PHY extended packet transmitted from the image sensor 211 in step S102 via the C / D-PHY physical layer processing unit 232. Then, the CSI-2 receiving circuit 231 obtains the AS payload, excluding the packet header and packet footer, from the received extended packet, and supplies the AS payload to the CSI-2 A-PHY packet generating unit 235 as is.

[0218] In step S104, in the SerDes device 212, the CSI-2 A-PHY packet generator 235 generates an extended packet for A-PHY by adding an A-PHY packet header and an A-PHY packet footer to the AS payload supplied from the CSI-2 receiving circuit 231. Then, the CSI-2 A-PHY packet generator 235 transmits the extended packet for A-PHY to the SerDes device 213 via the physical layer processor 237 that supports A-PHY.

[0219] In step S105, in the SerDes device 213, the CSI-2 A-PHY packet receiver 245 receives the A-PHY extended packet transmitted from the SerDes device 212 in step S104, via the A-PHY-compatible physical layer processor 247. The CSI-2 A-PHY packet receiver 245 then acquires the AS payload, excluding the packet header and packet footer, from the received extended packet, and supplies the AS payload to the CSI-2 transmitter circuit 241 as is.

[0220] In step S106, the CSI-2 transmission circuit 241 generates an extended packet for C-PHY or D-PHY by adding a packet header for C-PHY or D-PHY and a packet footer for C-PHY or D-PHY to the AS payload supplied in step S105 from the CSI-2 A-PHY packet reception unit 245. Then, the CSI-2 transmission circuit 241 transmits the extended packet for C-PHY or D-PHY to the application processor 214 via the C / D-PHY physical layer processing unit 242.

[0221] In step S107, in the application processor 214, the extended mode-compatible CSI-2 receiver circuit 251 receives the C-PHY or D-PHY extended packet transmitted from the SerDes device 213 in step S106 via the C / D-PHY physical layer processor 252. The extended mode-compatible CSI-2 receiver circuit 251 then acquires the AS payload, excluding the packet header and packet footer, from the received extended packet, and outputs various data stored in the packet data of the AS payload to a downstream LSI (not shown). Thereafter, the packet transmission / reception process adapted to E2E protection ends, and the same process is repeated for the next extended packet.

[0222] As described above, the communication system 201 can transmit and receive extended packets without modifying the AS payload on the transmission path by executing packet transmission and reception processing that conforms to E2E protection. In this case, even if the physical layer of the image sensor 211 is D-PHY and the physical layer of the application processor 214 is C-PHY, that is, even if the interfaces are different, E2E protection can be observed.

[0223] <Detailed configuration example of image sensor 211> Fig. 25 is a block diagram showing a detailed configuration example of the image sensor 211. In the image sensor 211 shown in Fig. 25, the same components as those in the image sensor 21 in Fig. 9 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0224] 9, the image sensor 211 is configured to include pixels 41, an AD converter 42, an image processing unit 43, a register 47, and a controller 60. The I2C / I3C slave 223 and the CCI slave 224 included in the image sensor 211 correspond to the I2C / I3C slave 46 and the CCI slave 59 in FIG.

[0225] The image sensor 211 includes an extended mode compatible CSI-2 transmission circuit 221 and a physical layer processing unit 222, and the physical layer processing unit 222 is compatible with A-PHY, C-PHY, and D-PHY.

[0226] The extended mode compatible CSI-2 transmitter circuit 221 is configured to include, in addition to the controller 60 and the CCI slave 224, an AS payload generator 301, a selector 302, an A-PHY packet generator 303, a C-PHY packet generator 304, a D-PHY packet generator 305, and a selector 306.

[0227] The AS payload generator 301 generates an AS payload that is limited as the protection range of E2E protection, and outputs the generated AS payload to the selector 302. For example, the AS payload generator 301 includes a packing unit 311, an extended packet header generator 312, and an extended packet footer generator 313.

[0228] The packing unit 311 packs image data supplied from the image processing unit 43 as data to be transmitted, and generates packet data of the number of bytes determined by the packet count PC. For example, the controller 60 can control the number of bytes of packet data generated by the packing unit 311 in accordance with a setting value (e.g., image size) stored in the register 47.

[0229] The extended packet header generator 312 generates an extended packet header describing a packet count PC and a virtual channel VC, and adds it to the packet data, as described with reference to Figures 21 to 23. The extended packet footer generator 313 generates an extended packet footer and adds it to the packet data.

[0230] Under the control of the controller 60, the selector 302 selects one of the A-PHY packet generator 303, the C-PHY packet generator 304, and the D-PHY packet generator 305, which are arranged in parallel, as the output destination for the AS payload supplied from the AS payload generator 301.

[0231] The A-PHY packet generator 303 generates an extended packet for A-PHY from the AS payload supplied via the selector 302, and outputs the packet to the selector 306. For example, the A-PHY packet generator 303 includes an AAL generator 321, an A-PHY packet header generator 322, and an A-PHY packet footer generator 323.

[0232] For example, the AAL (A-PHY Adaptation Layer) generation unit 321 divides the AS payload generated by the AS payload generation unit 301 into 380-byte pieces at a layer called an Adaptation Layer. Then, the A-PHY packet header generation unit 322 adds an A-PHY packet header to the divided AS payloads, and the A-PHY packet footer generation unit 323 adds an A-PHY packet footer.

[0233] The C-PHY packet generator 304 generates an extended packet for C-PHY from the AS payload supplied via the selector 302, and outputs the packet to the selector 306. For example, the C-PHY packet generator 304 includes a C-PHY packet header generator 331, a C-PHY packet footer generator 332, and a C-PHY lane distributor 333.

[0234] For example, a C-PHY packet header generator 331 adds a C-PHY packet header, and a C-PHY packet footer generator 332 adds a C-PHY packet footer to the AS payload generated by the AS payload generator 301. Then, a C-PHY lane distributor 333 distributes the C-PHY extended packet to three lanes in accordance with the CSI-2 standard.

[0235] The D-PHY packet generator 305 generates an extended packet for D-PHY from the AS payload supplied via the selector 302, and outputs the packet to the selector 306. For example, the D-PHY packet generator 305 has a D-PHY packet header generator 341, a D-PHY packet footer generator 342, and a D-PHY lane distributor 343.

[0236] For example, a D-PHY packet header generator 341 adds a D-PHY packet header, and a D-PHY packet footer generator 342 adds a D-PHY packet footer to the AS payload generated by the AS payload generator 301. Then, a D-PHY lane distributor 343 distributes the D-PHY extended packets to four lanes in accordance with the CSI-2 standard.

[0237] The selector 306, under the control of the controller 60, selects one of the A-PHY packet generator 303, the C-PHY packet generator 304, and the D-PHY packet generator 305, which are arranged in parallel, as the output source of the extended packet to be supplied to the physical layer processing unit 222.

[0238] When the physical layer processing unit 222 receives an A-PHY extension packet from the A-PHY packet generation unit 303, it transmits the A-PHY extension packet on lane 1. When the physical layer processing unit 222 receives a C-PHY extension packet from the C-PHY packet generation unit 304, it transmits the C-PHY extension packet on lane 3. When the physical layer processing unit 222 receives a D-PHY extension packet from the D-PHY packet generation unit 305, it transmits the D-PHY extension packet on lane 4.

[0239] In the image sensor 211 configured as described above, the extended mode-compatible CSI-2 transmission circuit 221 is configured so that the AS payload generator 301 is connected to the A-PHY packet generator 303, the C-PHY packet generator 304, and the D-PHY packet generator 305 via the selector 302. This allows the image sensor 211 to generate an AS payload common to the A-PHY extended packet, the C-PHY extended packet, and the D-PHY extended packet using a single AS payload generator 301. That is, the A-PHY packet generator 303, the C-PHY packet generator 304, and the D-PHY packet generator 305 can share the AS payload generator 301, thereby reducing the circuit scale. This allows the image sensor 211 to be made smaller.

[0240] <Detailed configuration example of application processor 214> Fig. 26 is a block diagram showing a detailed configuration example of the application processor 214. In the application processor 214 shown in Fig. 26, components common to the application processor 22 in Fig. 10 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0241] 10, the application processor 214 is configured to include a register 73 and a controller 74. The controller 74 may be implemented by software. The I2C / I3C master 253 and the CCI master 254 included in the application processor 214 correspond to the I2C / I3C master 72 and the CCI master 88 in FIG. 10, respectively.

[0242] The application processor 214 includes an extended mode compatible CSI-2 receiving circuit 251 and a physical layer processing unit 252, and the physical layer processing unit 252 supports A-PHY, C-PHY, and D-PHY.

[0243] The extended mode compatible CSI-2 receiver circuit 251 is configured to include a CCI master 254, a selector 401, an A-PHY packet receiver 402, a C-PHY packet receiver 403, a D-PHY packet receiver 404, a selector 405, and an AS payload receiver 406.

[0244] The selector 401 selects one of an A-PHY packet receiver 402, a C-PHY packet receiver 403, and a D-PHY packet receiver 404 arranged in parallel as the output destination of the extended packet supplied from the physical layer processor 252.

[0245] The A-PHY packet receiver 402 receives an A-PHY extended packet supplied via the selector 401 and outputs it to the selector 405. For example, the A-PHY packet receiver 402 includes an A-PHY packet header interpreter 411, an A-PHY packet footer verifier 412, and an AAL processor 413.

[0246] For example, the A-PHY packet header interpretation unit 411 interprets the contents written in the A-PHY packet header and performs processing required to receive the A-PHY extended packet, and the A-PHY packet footer verification unit 412 verifies the presence or absence of an error using the A-PHY packet footer.Then, the AAL processing unit 413 performs processing to combine the adaptation layers divided in the AAL generation unit 321 of FIG.

[0247] The C-PHY packet receiver 403 receives the C-PHY extended packet supplied via the selector 401 and outputs it to the selector 405. For example, the C-PHY packet receiver 403 includes a C-PHY lane merging unit 421, a C-PHY packet header interpretation unit 422, and a C-PHY packet footer verification unit 423.

[0248] For example, the C-PHY lane merging unit 421 merges C-PHY extended packets that are distributed to three lanes in accordance with the CSI-2 standard and supplied via the physical layer processing unit 252. Then, the C-PHY packet header interpretation unit 422 interprets the contents written in the C-PHY packet header and performs processing required to receive the C-PHY extended packets, and the C-PHY packet footer verification unit 423 verifies the presence or absence of errors using the C-PHY packet footer.

[0249] The D-PHY packet receiving unit 404 receives the D-PHY extended packet supplied via the selector 401 and outputs it to the selector 405. For example, the D-PHY packet receiving unit 404 has a D-PHY lane merging unit 431, a D-PHY packet header interpretation unit 432, and a D-PHY packet footer verification unit 433.

[0250] For example, the D-PHY lane merging unit 431 merges D-PHY extension packets that are distributed to four lanes in accordance with the CSI-2 standard and supplied via the physical layer processing unit 252. Then, the D-PHY packet header interpretation unit 432 interprets the contents written in the D-PHY packet header and performs processing required to receive the D-PHY extension packets, and the D-PHY packet footer verification unit 433 verifies the presence or absence of errors using the D-PHY packet footer.

[0251] The selector 405 selects one of the A-PHY packet receiver 402, C-PHY packet receiver 403, and D-PHY packet receiver 404, which are arranged in parallel, as the output source of the extended packet to be supplied to the AS payload receiver 406.

[0252] The AS payload receiver 406 has an unpacking unit 441, an extended packet header interpretation unit 442, and an extended packet footer verification unit 443 corresponding to the AS payload generation unit 301 in FIG. 25. The unpacking unit 441 unpacks the image data packed by the packing unit 311. The extended packet header interpretation unit 442 interprets the extended packet header generated by the extended packet header generation unit 312, for example, reads out the packet count PC and the virtual channel VC. The extended packet footer verification unit 443 verifies the presence or absence of an error using the extended packet footer added by the extended packet footer generation unit 313. Then, the AS payload receiver 406 outputs various data stored in the packet data supplied via the selector 405, such as image data, in-vehicle line numbers, SourceID, etc., including CRC errors, to a subsequent LSI (not shown).

[0253] The application processor 214 configured as described above has the extended mode-compatible CSI-2 receiving circuit 251 configured such that the AS payload receiver 406 is connected to the A-PHY packet receiver 402, the C-PHY packet receiver 403, and the D-PHY packet receiver 404 via the selector 405. Thereby, the application processor 214 can receive the AS payload common to the extended packets for A-PHY, the extended packets for C-PHY, and the extended packets for D-PHY with one AS payload receiver 406. That is, the A-PHY packet receiver 402, the C-PHY packet receiver 403, and the D-PHY packet receiver 404 can share the AS payload receiver 406, thereby reducing the circuit scale. Therefore, miniaturization of the application processor 214 can be achieved.

[0254] <Second Configuration Example Adapted to E2E Protection> Referring to FIGS. 27 to 74, a second configuration example adapted to E2E Protection will be described.

[0255] <Configuration Example of A-PHY Direct Connection> In the communication system 501 shown in FIG. 27, the image sensor 511 and the application processor 512 are directly connected by A-PHY (without passing through a SerDes device as described in FIG. 40 to be described later).

[0256] The image sensor 511 includes an A-PHY processing unit 521, a CSIA processing unit 522, a CSI2 processing unit 523, a CSI2-FS processing unit 524, a CCI processing unit 525, a CCI-FS processing unit 526, and a register 527.

[0257] In the A-PHY processing unit 521, the CCI processing unit 525 is implemented as an upper layer, and it transmits and receives data including an extended packet header ePH and an extended packet footer ePF by connecting to the A-PHY processing unit 531 of the application processor 512 through MIPI A-PHY.

[0258] The CCI-FS processing unit 526, for example, compares the Desination ID included in the extended packet header ePH with the ID (Source ID) of the image sensor 511 to determine whether it is an access to the image sensor 511.

[0259] The application processor 512 includes an A-PHY processing unit 531, a CSIA processing unit 532, a CSI2 processing unit 533, a CSI2-FS processing unit 534, a CCI processing unit 535, a CCI-FS processing unit 536, a register 537, and a CCI-FS switch 538.

[0260] In the A-PHY processing unit 531, the CCI processing unit 535 is implemented as an upper layer, and it transmits and receives data including an extended packet header ePH and an extended packet footer ePF by connecting to the A-PHY processing unit 521 of the image sensor 511 through MIPI A-PHY.

[0261] The CCI-FS processing unit 536 compares, for example, the destination ID included in the extended packet header ePH with the ID (source ID) of the application processor 512, and determines whether or not the access is to the application processor 512.

[0262] The CCI-FS switch 538 switches so that data is sent and received via the CCI-FS processing unit 536 when the CCI-FS processing unit 536 is enabled, and so that data is sent and received without going through the CCI-FS processing unit 536 when the CCI-FS processing unit 536 is disabled.

[0263] 28 to 32, the transfer of read commands and read data in the communication system 501 will be described.

[0264] FIG. 28 shows an example of the packet configuration of a read command generated by the CCI-FS processing unit 536 of the application processor 512 during a read access.

[0265] As shown in FIG. 28, the read command is made up of an extended packet header ePH* (*=n), an AP (CCI) payload, an extended packet footer ePF1, and an extended packet footer ePF0.

[0266] The extended packet header ePH* (*=n) consists of extended packet headers ePH0 to ePH3 as shown in the figure.

[0267] The extended packet header ePH0 stores an extended VC, an extended DT, an extended PFEN, and an extended PHEN. For example, the extended DT is information indicating the CCI protocol (I2C), and routing processing is performed using the extended DT.

[0268] The extended packet header ePH1 stores Source ID[7:1] and Packet Length. For example, Source ID is information indicating the sender of the CCI protocol (I2C), and response processing is performed based on the Source ID. Packet Length is information indicating the data length.

[0269] The extended packet header ePH2 contains a security descriptor and a message counter. The security descriptor indicates whether security is used or not, and indicates "8'h0" when security is not used. The message counter is information indicating the bucket order, and indicates the count value of messages, and indicates "16'h5" when the message is the fifth.

[0270] The extended packet header ePH3 stores Destination ID[7:1], Read / Write, and Destination Address. Destination ID[7:1] indicates the slave address of the CCI processing unit 525 of the image sensor 511, and is set to "7'h0D" in the illustrated example. For example, the Destination ID is information indicating the destination of the CCI protocol (I2C), and routing is performed based on the Destination ID, and the communication path is referenced. Read / Write indicates reading or writing of data, and indicates "1'b1" in the case of a read. Destination Address indicates the address of the register 527 of the image sensor 511, which is the final destination, and is set to "0x0200" in the illustrated example.

[0271] The AP (CCI) payload stores, for example, various data (Data0[7:0]). The AP (CCI) payload is not transmitted when security is off, but may store dummy data and be transmitted when security is on.

[0272] The extended packet footer ePF1 is not sent when security is turned off.

[0273] The extended packet footer ePF0 stores the CRC calculation value.

[0274] In the application processor 512 , a read command with such a packet structure is generated in the CCI-FS processing unit 536 and supplied to the A-PHY processing unit 531 .

[0275] FIG. 29 shows an example of the packet configuration of a read command output from the A-PHY processing unit 531 of the application processor 512 during read access.

[0276] As shown in FIG. 29, the A-PHY processing unit 531 adds an A-PHY header and an A-PHY footer to the read command supplied from the CCI-FS processing unit 536, treating the read command as within the protection range of E2E Protection.

[0277] A read command with such a packet structure is A-PHY transferred by the A-PHY processing unit 531 of the application processor 512. Then, in the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY footer from the read command. Thereafter, the read command is supplied to the CCI-FS processing unit 526 via the CCI processing unit 525 at the slave address "7'h0D" indicated by the Destination ID.

[0278] FIG. 30 shows an example of the packet structure of a read command supplied to the CCI-FS processing unit 526 and read data generated by the CCI-FS processing unit 526 during a read access.

[0279] As shown in FIG. 30, the read command with the packet structure shown in FIG. 28 as it is, that is, the read command that is protected by E2E Protection in A-PHY transfer, is supplied to the CCI-FS processing unit 526.

[0280] As shown in the figure, the read data is composed of an extended packet header ePH*(*=n), an AP (CCI) payload, an extended packet footer ePF1, and an extended packet footer ePF0. The AP (CCI) payload stores the read data value read from address 0x0200 of register 527 indicated by the source address information (Destination Address) of the extended packet header ePH of the read command.

[0281] In the image sensor 511 , read data with such a packet structure is generated in the CCI-FS processing unit 526 and supplied to the A-PHY processing unit 521 .

[0282] FIG. 31 shows an example of a packet configuration of read data output from the A-PHY processing unit 521 of the image sensor 511 during read access.

[0283] As shown in FIG. 31, the A-PHY processing unit 521 adds an A-PHY header and an A-PHY footer to the read data supplied from the CCI-FS processing unit 526, with the data being protected by E2E Protection.

[0284] The read data having such a packet structure is A-PHY transferred by the A-PHY processing unit 521 of the image sensor 511. Then, in the application processor 512, the A-PHY processing unit 531 removes the A-PHY header and the A-PHY footer from the read data, and the read data is supplied to the CCI-FS processing unit 536.

[0285] FIG. 32 shows an example of the packet structure of read data supplied to the CCI-FS processing unit 536 during read access.

[0286] As shown in FIG. 32, the read data with the packet structure shown in FIG. 30 as it is, that is, the read data that is protected by E2E Protection in the A-PHY transfer, is supplied to the CCI-FS processing unit 536.

[0287] 33 to 35, the transfer of write data in the communication system 501 will be described. Note that the description will be made assuming that access is made from a state in which the CCI-FS processing unit 526 on the image sensor 511 side is enabled.

[0288] FIG. 33 shows an example of a packet configuration of write data generated by the CCI-FS processing unit 536 of the application processor 512 during write access.

[0289] As shown in FIG. 33, the write data is made up of an extension packet header ePH* (*=n), an AP (CCI) payload (write data), an extension packet footer ePF1, and an extension packet footer ePF0.

[0290] The extended packet header ePH* (*=n) consists of extended packet headers ePH0 to ePH3 as shown in the figure.

[0291] The extended packet header ePH0 stores an extended VC, an extended DT, an extended PFEN, and an extended PHEN.

[0292] The extended packet header ePH1 stores Source ID[7:1] and Packet Length.

[0293] The extended packet header ePH2 contains a Security Descriptor and a Message Counter. The Security Descriptor indicates whether security is used or not, and indicates "8'h0" if security is not used. The Message Counter indicates the count value of messages, and indicates "16'h4" if the message is the fourth.

[0294] The extended packet header ePH3 stores Destination ID[7:1], Read / Write, and Destination Address. Destination ID[7:1] indicates the slave address of the CCI processing unit 525 of the image sensor 511, and is set to "7'h0D" in the illustrated example. Read / Write indicates reading or writing data, and indicates "1'b0" for writing. Destination Address indicates the address of the register 527 of the image sensor 511, which is the final destination, and is set to "0x1234" in the illustrated example.

[0295] The AP (CCI) payload stores data (Data0[7:0]) to be written to the image sensor 511, and the value 0xFF becomes the write data.

[0296] The extended packet footer ePF1 is not sent when security is turned off.

[0297] The extended packet footer ePF0 stores the CRC calculation value.

[0298] In the application processor 512 , write data having such a packet structure is generated in the CCI-FS processing unit 536 and supplied to the A-PHY processing unit 531 .

[0299] FIG. 34 shows an example of a packet configuration of write data output from the A-PHY processing unit 531 of the application processor 512 during write access.

[0300] As shown in FIG. 34, the A-PHY processing unit 531 adds an A-PHY header and an A-PHY footer to the write data supplied from the CCI-FS processing unit 536, with the data being protected by E2E Protection.

[0301] The write data with such a packet structure is A-PHY transferred by the A-PHY processing unit 531 of the application processor 512. Then, in the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY footer from the write data. Thereafter, the write data is supplied to the CCI-FS processing unit 526 via the CCI processing unit 525 at the slave address "7'h0D" indicated by the Destination ID.

[0302] FIG. 35 shows an example of the packet structure of write data supplied to the CCI-FS processing unit 526 during write access.

[0303] 35, the write data with the packet structure shown in Fig. 33 intact, i.e., the write data protected by E2E Protection in the A-PHY transfer, is supplied to the CCI-FS processing unit 526. Then, the CCI-FS processing unit 526 writes the data stored in the AP (CCI) payload from the address "0x1234" of the register 527 indicated by the CCI command ID information, i.e., the source address information (Destination Address) of the extended packet header ePH of the read command.

[0304] An overview of the extended packet header ePH and the extended packet footer ePF will be described with reference to FIG.

[0305] As shown in FIG. 36, a CCI-FS E2E packet is made up of an extended packet header ePH, packet data, and an extended packet footer ePF, and its packet length is expressed as Length=Byte Count×Data Byte width.

[0306] The extended packet header ePH uses fields such as an extended VC, an extended DT, a message counter, etc. The length of the extended packet header ePH can be changed by the field value (epFEN field) of the extended packet header ePH.

[0307] Packet data is composed of, for example, PL pieces of data (Data 0 to Data PL-1), and its length is Length = Packet Length (PL) × Data Byte Width. In the case of a read command, no data is stored in the packet data when security is off, and one byte of dummy data is stored when security is on. In the case of a write access, write data equivalent to the payload data is stored in the packet data. In the case of a read access, read data equivalent to the payload data is stored in the packet data. When Clock Stretch (Control Code Indicator of ePH0 = 1) is used, a one-byte data payload indicating the type of control is attached to the packet data.

[0308] The length of the extended packet footer ePF1 can be changed by the field setting value (epFEN field) of the extended packet header ePH, and security-related information can also be added.

[0309] The extended packet footer ePF0 is a field setting value of the extended packet header ePH, and can add a CRC-32 calculated from the packet data.

[0310] <Example of communication processing> 37 to 39, a description will be given of communication processing using CCI-FS performed in communication system 501 shown in FIG.

[0311] As shown in FIG. 37, in steps S211 to S222, initial settings and confirmation operations are performed.

[0312] In step S211, the application processor 512 performs two read accesses to the Capability register of the CCI-FS processing unit 526 in the image sensor 511. Note that the number of read accesses is not limited to two, and can be set arbitrarily in terms of functional safety, for example, and may be one or three or more times.

[0313] In step S212, the CSI2-FS processing unit 524 in the application processor 512 determines whether the Capability register value of the CCI-FS processing unit 526 is 1'b1 both times as a result of the read access in step S211. If it is determined in step S212 that the Capability register value of the CCI-FS processing unit 526 is not 1'b1 both times, the process proceeds to step S213.

[0314] In step S213, CSI2-FS processing unit 524 in application processor 512 determines whether the number of retransmissions is three or more. Note that the number of retransmissions is not limited to three and can be set to any number, and the same applies to the number of retransmissions described below. If it is determined in step S213 that the number of retransmissions is not three or more (one or two), the process returns to step S211, and the same processes are repeated thereafter.

[0315] On the other hand, if it is determined in step S212 that the Capability register value of the CCI-FS processing unit 526 is 1'b1 both times, the process proceeds to step S214.

[0316] In step S214, the application processor 512 performs a 1 write access to the Enable register of the CCI-FS processing unit 526 to the image sensor 511.

[0317] In step S215, in the image sensor 511, the CCI-FS processing unit 526 performs 1 write access to the Enable register of the CCI-FS processing unit 536 of the application processor 512.

[0318] In step S216, the slave address of the opposing image sensor 511 is set in the Destination SID register of the CCI-FS processing unit 536 of the application processor 512.

[0319] In step S217, the ePH register of the CCI-FS processing unit 536 of the application processor 512 is set.

[0320] In step S218, the application processor 512 sets the ePH register of the CCI-FS processing unit 526 to the image sensor 511.

[0321] In step S219, the application processor 512 makes a read access to the Enable register and the Error register of the CCI-FS processing unit 526 to the image sensor 511.

[0322] In step S220, the CCI-FS processing unit 536 of the application processor 512 determines whether the Enable register value of the CCI-FS processing unit 526 is 1'b1 and the Error register value is 0 as a result of the read access in step S219.

[0323] If it is determined in step S220 that the Enable register value of the CCI-FS processing unit 526 is not 1'b1 or the Error register value is not 0, the process proceeds to step S221.

[0324] In step S221, CSI2-FS processing unit 524 in application processor 512 determines whether the number of retransmissions is 3 or more. If it is determined in step S221 that the number of retransmissions is 3 or more, the process returns to step S211, and the same processes are repeated thereafter.

[0325] On the other hand, if it is determined in step S213 that the number of retransmissions is three or more, or if it is determined in step S221 that the number of retransmissions is not three or more (one or two), the process proceeds to step S222.

[0326] In step S222, communication is performed using CCI without using CCI-FS, and then the communication process is terminated.

[0327] On the other hand, if it is determined in step S220 that the Enable register value of the CCI-FS processing unit 526 is 1'b1 and the Error register value is 0, the process proceeds to step S223.

[0328] As shown in FIG. 38, in steps S223 to S234, a write operation using CCI-FS is performed.

[0329] In step S223, the CCI-FS processing unit 536 of the application processor 512 sets the ePH register to perform a write operation.

[0330] In step S224, the CCI-FS processing unit 536 of the application processor 512 sets the write data register.

[0331] In step S225, the CCI-FS processing unit 536 of the application processor 512 sets the command execution register to 1.

[0332] In step S226, in the application processor 512, the A-PHY processing unit 531, as shown in Figure 34 above, adds an A-PHY header and an A-PHY footer to the write data generated by the CCI-FS processing unit 536, treating the data as within the protection range of E2E Protection, and performs A-PHY transfer.

[0333] In step S227, the A-PHY processing unit 521 of the image sensor 511 removes the A-PHY header and the A-PHY footer from the write data, and supplies the protection range of E2E Protection to the CCI-FS processing unit 526.

[0334] In step S228, the CCI-FS processing unit 526 in the image sensor 511 checks the Source ID of the image sensor 511 and the Destination SID of the extended packet header ePH from the contents of the extended packet header ePH.

[0335] In step S229, the CCI-FS processing unit 526 in the image sensor 511 determines whether or not the Source ID of the image sensor 511 confirmed in step S228 matches the Destination SID in the extended packet header ePH.

[0336] If it is determined in step S229 that the Source ID of the image sensor 511 matches the Destination SID in the extension packet header ePH, the process proceeds to step S230.

[0337] In step S230, the CCI-FS processing unit 526 in the image sensor 511 checks the Message Counter from the contents of the extended packet header ePH.

[0338] In step S231, the CCI-FS processing unit 526 in the image sensor 511 determines whether the Message Counter (received) of the image sensor 511 confirmed in step S230 matches the Message Counter in the extension packet header ePH.

[0339] If it is determined in step S231 that the Message Counter (received) of the image sensor 511 matches the Message Counter in the extension packet header ePH, the process proceeds to step S232.

[0340] In step S232, the CCI-FS processing unit 526 in the image sensor 511 checks the CRC from the contents of the extended packet footer ePF.

[0341] In step S233, in the image sensor 511, the CCI-FS processing unit 526 determines whether the received value (ePF0) of the extended packet footer ePF confirmed in step S232 matches the CRC calculation result calculated by the CCI-FS processing unit 526.

[0342] If it is determined in step S233 that the received value (ePF0) of the extended packet footer ePF matches the CRC calculation result, the process proceeds to step S234.

[0343] In step S234, in the image sensor 511, the CCI-FS processing unit 526 performs a write process to write write data based on the contents of the extension packet header ePH and the extension packet footer ePF to an address in the register 527. Thereafter, the process proceeds to step S235.

[0344] As shown in FIG. 39, in steps S235 to S247, a read operation using CCI-FS is performed.

[0345] In step S235, the CCI-FS processing unit 536 in the application processor 512 sets the ePH register so that a read operation is performed.

[0346] In step S236, the CCI-FS processing unit 536 in the application processor 512 sets the command execution register to 1.

[0347] In step S237, in the application processor 512, the A-PHY processing unit 531, as shown in Figure 29 above, adds an A-PHY header and an A-PHY footer to the write data generated by the CCI-FS processing unit 536, treating the data as within the protection range of E2E Protection, and performs A-PHY transfer.

[0348] In step S238, the A-PHY processing unit 521 of the image sensor 511 removes the A-PHY header and the A-PHY footer from the write data, and supplies the protection range of E2E Protection to the CCI-FS processing unit 526.

[0349] In step S239, the CCI-FS processing unit 526 in the image sensor 511 checks the Source ID of the image sensor 511 and the Destination SID of the extension packet header ePH from the contents of the extension packet header ePH.

[0350] In step S240, the CCI-FS processing unit 526 in the image sensor 511 determines whether the Source ID of the image sensor 511 confirmed in step S239 matches the Destination SID in the extended packet header ePH.

[0351] If it is determined in step S240 that the Source ID of the image sensor 511 matches the Destination SID in the extended packet header ePH, the process proceeds to step S241.

[0352] In step S241, the CCI-FS processing unit 526 in the image sensor 511 checks the Message Counter from the contents of the extended packet header ePH.

[0353] In step S242, the CCI-FS processing unit 526 in the image sensor 511 determines whether the Message Counter (received) of the image sensor 511 confirmed in step S241 matches the Message Counter in the extension packet header ePH.

[0354] If it is determined in step S242 that the Message Counter (received) of the image sensor 511 matches the Message Counter in the extension packet header ePH, the process proceeds to step S243.

[0355] In step S243, the CCI-FS processing unit 526 in the image sensor 511 checks the CRC from the contents of the extended packet footer ePF.

[0356] In step S244, in the image sensor 511, the CCI-FS processing unit 526 determines whether the received value (ePF0) of the extended packet footer ePF confirmed in step S243 matches the CRC calculation result calculated by the CCI-FS processing unit 526.

[0357] If it is determined in step S244 that the received value (ePF0) of the extended packet footer ePF matches the CRC calculation result, the process ends.

[0358] On the other hand, if it is determined in step S229 of FIG. 38 or step S240 of FIG. 39 that the Source ID of the image sensor 511 and the Destination SID of the extension packet header ePH do not match, the process proceeds to step S245.

[0359] In step S245, the Error register (Routing) on the image sensor 511 side is set to 1, and then the process ends.

[0360] On the other hand, if it is determined in step S231 of FIG. 38 or step S242 of FIG. 39 that the Message Counter (received) of the image sensor 511 does not match the Message Counter of the extended packet header ePH, the process proceeds to step S246.

[0361] In step S246, the Error register (MC) on the image sensor 511 side is set to 1, and then the process ends.

[0362] On the other hand, if it is determined in step S233 of FIG. 38 or step S244 of FIG. 39 that the received value (ePF0) of the extended packet footer ePF does not match the CRC calculation result, the process proceeds to step S247.

[0363] In step S247, the Error register (CRC) on the image sensor 511 side is set to 1, and then the process ends.

[0364] <Configuration example of SerDes connection configuration> In the communication system 601 shown in FIG. 40, the image sensor 611 and the application processor 614 are connected in a SerDes connection configuration via the slave-side SerDes device 612 and the master-side SerDes device 613.

[0365] The image sensor 611 includes an I2C / I3C slave 621, a CCI processing unit 622, a CSI2-FS processing unit 623, and a register 624.

[0366] The slave-side SerDes device 612 includes an A-PHY processing unit 631, a CSIA processing unit 632, a CSI2-FS processing unit 633, an I2C / I3C master 634, a CCI processing unit 635, a CCI-FS processing unit 636, and a register 637.

[0367] The master side SerDes device 613 is configured to include an A-PHY processing unit 641, a CSIA processing unit 642, a CSI2-FS processing unit 643, an I2C / I3C slave 644, a CCI processing unit 645, a CCI-FS processing unit 646, and a register 647.

[0368] The application processor 614 includes an I2C / I3C master 651, a CCI processing unit 652, a CCI-FS processing unit 653, a register 654, and a CCI-FS switch 655.

[0369] In addition, in the SerDes connection configuration shown in Fig. 40, if the CCI configuration or the CCI-FS configuration is implemented as an upper layer protocol, other SerDes standards may be used. For example, by implementing a configuration of an extended packet header ePH, an extended packet footer ePF1, and an extended packet footer ePF0 as shown in Fig. 41 in the payload from the application layer or a layer below that, various SerDes-related standards such as PCIE, USB, DisplayPort, HDMI (registered trademark), LVDS, and FPD-LINK can be applied.

[0370] 41 to 49, the transfer of read commands and read data in the communication system 601 will be described.

[0371] FIG. 41 shows an example of the packet configuration of a read command generated by the CCI-FS processing unit 653 of the application processor 614 during a read access.

[0372] As shown in Fig. 41, the read command is composed of an extended packet header ePH* (*=n), an extended packet footer ePF1, and an extended packet footer ePF0. Details of these are the same as those of the read command described above with reference to Fig. 28.

[0373] In the application processor 614 , a read command with such a packet structure is generated in the CCI-FS processing unit 653 and supplied to the I2C / I3C master 651 .

[0374] FIG. 42 shows an example of the packet configuration of a read command output from the I2C / I3C master 651 of the application processor 614 during read access.

[0375] As shown in Fig. 42, the I2C / I3C master 651 transmits a start condition S followed by the address of the connected sensor, i.e., in the configuration shown in Fig. 40, the address (Slave Address+W 8-bit) of the CCI processing unit 645 of the master-side SerDes device 613. In the example shown in Fig. 42, the address of the CCI processing unit 645 is Slave Address[7:1]=7'h0F. Following this address, the register address (Register Address [15:8] and Register Address [7:0]) of the register 647 of the master-side SerDes device 613 is transmitted. The I2C / I3C master 651 transmits an extended packet header ePH* (*=n), an extended packet footer ePF1, and an extended packet footer ePF0, followed by a stop condition P.

[0376] A read command with such a packet structure is transferred by I2C / I3C from the I2C / I3C master 651 of the application processor 614. In the master-side SerDes device 613, the I2C / I3C slave 644 acquires the read command (extended packet header ePH*(*=n), extended packet footer ePF1, and extended packet footer ePF0). The read command is supplied to the CCI processing unit 645 with Slave Address[7:1]=7'h0F, and then supplied to the A-PHY processing unit 641 via the CCI-FS processing unit 646, CSI2-FS processing unit 643, and CSIA processing unit 642.

[0377] FIG. 43 shows an example of the packet configuration of a read command output from the A-PHY processing unit 641 of the SerDes device 613 on the master side during a read access.

[0378] 43, the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the read command acquired by the I2C / I3C slave 644, with the read command being protected by E2E Protection. Note that the CSI2-FS processing unit 643 adds the address of the CCI processing unit 635 of the master-side SerDes device 613, for example, Slave Address[7:1]=7'h0E, to the extended packet header ePH* (*=n).

[0379] A read command with such a packet structure is A-PHY transferred by the A-PHY processing unit 641 of the master-side SerDes device 613. In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY footer from the read command. The read command is supplied to the CCI processing unit 635 at the slave address "7'h0E" indicated by the Destination ID via the CSIA processing unit 632, CSI2-FS processing unit 633, and CCI-FS processing unit 636, and then supplied to the I2C / I3C master 634.

[0380] FIG. 44 shows an example of the packet configuration of a read command output from the I2C / I3C master 634 during read access.

[0381] As shown in Fig. 44, the I2C / I3C master 634 transmits a start condition S followed by the address of the connected sensor, i.e., the address (Slave Address+W 8-bit) of the CCI processing unit 622 of the image sensor 611 in the configuration shown in Fig. 40. In the example shown in Fig. 44, the address of the CCI processing unit 622 is Slave Address[7:1]=7'h0D. Following this address, the register address (Register Address [15:8] and Register Address [7:0]) of the register 624 of the image sensor 611 is transmitted. The I2C / I3C master 634 transmits an extended packet header ePH* (*=n), an extended packet footer ePF1, and an extended packet footer ePF0, followed by a stop condition P.

[0382] A read command with such a packet structure is transferred by I2C / I3C from the I2C / I3C master 634 of the slave-side SerDes device 612. Then, in the image sensor 611, the I2C / I3C slave 621 acquires the read command (extended packet header ePH*(*=n), extended packet footer ePF1, and extended packet footer ePF0). The read command is supplied to the CSI2-FS processing unit 623 via the CCI processing unit 622 with Slave Address[7:1]=7'h0D.

[0383] FIG. 45 shows an example of the packet structure of a read command supplied to the CSI2-FS processing unit 623 and read data generated by the CSI2-FS processing unit 623 during a read access.

[0384] As shown in FIG. 45, the read command with the packet structure shown in FIG. 41 as is, that is, the read command that is protected by E2E Protection in A-PHY transfer, is supplied to the CSI2-FS processing unit 623.

[0385] As shown in the figure, the read data is composed of an extended packet header ePH*(*=n), an AP (CCI) payload, an extended packet footer ePF1, and an extended packet footer ePF0. The AP (CCI) payload stores the read data value read from address “0x0200” of register 624 indicated by the source address information (Destination Address) of the extended packet header ePH of the read command.

[0386] In the image sensor 611 , read data with such a packet structure is generated in the CCI-FS processing unit 623 and supplied to the I2C / I3C slave 621 via the CCI processing unit 622 .

[0387] FIG. 46 shows an example of a packet configuration of read data output from the I2C / I3C slave 621 of the image sensor 611 during read access.

[0388] As shown in Fig. 46, the I2C / I3C slave 621 transmits a start condition S followed by the address of the sensor to which it is connected, i.e., in the configuration shown in Fig. 40, the address (Slave Address+W 8-bit) of the I2C / I3C master 634 of the slave-side SerDes device 612. In the example shown in Fig. 46, the address of the I2C / I3C master 634 is Slave Address[7:1]=7'h0D. Following this address, the storage address of the read data (the address of the register 624 of the image sensor 611) is transmitted, followed by the address (Slave Address+R 8-bit) of the I2C / I3C master 634 of the slave-side SerDes device 612. After the I2C / I3C slave 621 transmits the extended packet header ePH* (*=n), the AP (CCI) payload, the extended packet footer ePF1, and the extended packet footer ePF0, a stop condition P is transmitted.

[0389] A read command with such a packet structure is transferred via I2C / I3C from the I2C / I3C slave 621 of the image sensor 611. In the slave-side SerDes device 612, the I2C / I3C master 634 acquires read data (extended packet header ePH*(*=n), AP (CCI) payload, extended packet footer ePF1, and extended packet footer ePF0). The read data is supplied to a CCI processing unit 635 with Slave Address[7:1]=7'h0E, and then supplied to an A-PHY processing unit 631 via a CCI-FS processing unit 636, a CSI2-FS processing unit 633, and a CSIA processing unit 632.

[0390] FIG. 47 shows an example of a packet configuration of read data output from the A-PHY processing unit 631 of the slave-side SerDes device 612 during read access.

[0391] As shown in FIG. 47, the A-PHY processing unit 631 adds an A-PHY header and an A-PHY footer to the read data acquired by the I2C / I3C master 634, with the data being protected by E2E Protection.

[0392] The read data with such a packet structure is A-PHY transferred by the A-PHY processing unit 631 of the slave-side SerDes device 612. Then, in the master-side SerDes device 613, the A-PHY processing unit 641 removes the A-PHY header and A-PHY footer from the read data. The read data is supplied to the I2C / I3C slave 644 via the CSIA processing unit 642, CSI2-FS processing unit 643, CCI-FS processing unit 646, and CCI processing unit 635.

[0393] FIG. 48 shows an example of a packet configuration of read data output from the I2C / I3C slave 644 of the master-side SerDes device 613 during read access.

[0394] As shown in FIG. 48, the I2C / I3C slave 644 transmits a start condition S followed by the address of the sensor to which it is connected, i.e., in the configuration shown in FIG. 40, the address (Slave Address+W 8-bit) of the CCI processing unit 635 of the master-side SerDes device 613. In the example shown in FIG. 48, the address of the CCI processing unit 635 is Slave Address[7:1]=7'h0F. Following this address, the register address (Register Address [15:8] and Register Address [7:0]) of the register 647 of the master-side SerDes device 613 is transmitted, followed by the address (Slave Address+R 8-bit) of the CCI processing unit 635. Next, the I2C / I3C slave 644 transmits an extended packet header ePH*(*=n), an AP(CCI) payload, an extended packet footer ePF1, and an extended packet footer ePF0, and finally transmits a stop condition P.

[0395] The read data with such a packet structure is transferred by I2C / I3C from the I2C / I3C slave 644 of the master-side SerDes device 613. Then, in the application processor 614, the I2C / I3C master 651 obtains the read command (extended packet header ePH*(*=n), extended packet footer ePF1, and extended packet footer ePF0) and supplies it to the CCI-FS processing unit 653.

[0396] FIG. 49 shows an example of the packet structure of read data supplied to the CCI-FS processing unit 653 during read access.

[0397] As shown in FIG. 49, the read data in the packet structure shown in FIG. 45, that is, the read data that is protected by E2E Protection in the A-PHY transfer, is supplied to the CCI-FS processing unit 653.

[0398] <Example of communication processing> 50 to 57, a description will be given of communication processing using CCI-FS performed in the communication system 601 shown in FIG.

[0399] As shown in FIG. 50, in steps S301 to S317, initial settings and confirmation operations are performed.

[0400] In step S301, the slave address of the opposing image sensor 611 is set in the Destination SID register of the CCI-FS processing unit 653 of the application processor 614.

[0401] In step S302, the ePH register of the CCI-FS processing unit 653 of the application processor 614 is set.

[0402] In step S303, the Destination SID of the Bridge configuration of the CCI-FS processing unit 653 of the application processor 614 is set, and the master side SerDes device 613 is registered. Here, the Address, attribution, and Timeout_no1 registers are also set in the same way, and settings are assumed to be made similarly thereafter.

[0403] In step S304, the application processor 614 sets the ePH register of the CCI-FS processing unit 643 to the SerDes device 613 on the master side.

[0404] In step S305, the application processor 614 sets the Destination SID of the Bridge configuration of the CCI-FS processing unit 643 to the master side SerDes device 613, and registers the slave side SerDes device 612.

[0405] In step S306, the application processor 614 makes a read access to the Error register of the CCI-FS processing unit 643 to the SerDes device 613 on the master side.

[0406] In step S307, the CCI-FS processing unit 653 in the application processor 614 determines whether the register value of the Error register of the CCI-FS processing unit 643 of the master-side SerDes device 613 is 0 as a result of the read access in step S306.

[0407] If it is determined in step S307 that the register value of the Error register of the CCI-FS processing unit 643 of the master-side SerDes device 613 is not 0 (other than 0), the process proceeds to step S308.

[0408] In step S308, in the application processor 614, the CCI-FS processing unit 653 determines whether the number of retransmissions is three or more, and if it determines that the number of retransmissions is not three or more (one or two), the processing returns to step S304, and the same processing is repeated thereafter.

[0409] On the other hand, if it is determined in step S307 that the register value of the Error register of the CCI-FS processing unit 643 of the master-side SerDes device 613 is 0, the process proceeds to step S309.

[0410] In step S309, the application processor 614 sets the ePH register of the CCI-FS processing unit 636 to the slave side SerDes device 612.

[0411] In step S310, the application processor 614 sets the Destination SID of the Bridge configuration of the CCI-FS processing unit 636 to the slave side SerDes device 612, and registers the slave side SerDes device 612.

[0412] In step S311, the application processor 614 makes a read access to the Error register of the CCI-FS processing unit 636 to the slave side SerDes device 612.

[0413] In step S312, the CCI-FS processing unit 653 in the application processor 614 determines whether the register value of the Error register of the CCI-FS processing unit 636 in the slave SerDes device 612 is 0 as a result of the read access in step S311.

[0414] If it is determined in step S312 that the register value of the Error register of the CCI-FS processing unit 636 of the slave-side SerDes device 612 is not 0 (other than 0), the process proceeds to step S313.

[0415] In step S313, in the application processor 614, the CCI-FS processing unit 653 determines whether the number of retransmissions is three or more, and if it determines that the number of retransmissions is not three or more (one or two), the processing returns to step S309, and the same processing is repeated thereafter.

[0416] On the other hand, if it is determined in step S312 that the register value of the Error register of the CCI-FS processing unit 636 of the slave-side SerDes device 612 is 0, the process proceeds to step S314.

[0417] In step S314, the application processor 614 sets the ePH register of the CCI-FS processing unit 623 in the image sensor 611.

[0418] In step S315, the application processor 614 makes a read access to the Error register of the CCI-FS processing unit 623 to the image sensor 611.

[0419] In step S316, the CCI-FS processing unit 653 in the application processor 614 determines whether the register value of the Error register of the CCI-FS processing unit 623 of the image sensor 611 is 0 as a result of the read access in step S315.

[0420] If it is determined in step S316 that the register value of the Error register of the CCI-FS processing unit 623 of the image sensor 611 is not 0 (other than 0), the process proceeds to step S317.

[0421] In step S317, in the application processor 614, the CCI-FS processing unit 653 determines whether the number of retransmissions is three or more, and if it determines that the number of retransmissions is not three or more (one or two), the processing returns to step S314, and the same processing is repeated thereafter.

[0422] If it is determined in step S308, step S313, or step S317 that the number of retransmissions is three or more, the process returns to step S301, and the same processes are repeated thereafter.

[0423] On the other hand, if it is determined in step S316 that the register value of the Error register of the CCI-FS processing unit 623 of the image sensor 611 is 0, the process proceeds to step S318.

[0424] As shown in FIG. 51, in steps S318 to S327, a write operation using CCI-FS is performed.

[0425] In step S318, the CCI-FS processing unit 653 of the application processor 614 sets the ePH register to perform a write operation.

[0426] In step S319, the CCI-FS processing unit 653 of the application processor 614 sets the write data register.

[0427] In step S320, the CCI-FS processing unit 653 of the application processor 614 sets the command execution register to 1 and issues a write command.

[0428] In step S321, the application processor 614 performs Sequence A_Write (AP) processing, which will be described later with reference to FIG.

[0429] In step S322, the master-side SerDes device 613 performs Sequence B (SerDes (Master) mode) processing, which will be described later with reference to Fig. 56. Note that Fig. 56 describes the Sequence B (SerDes (Slave) mode) processing performed by the slave-side SerDes device 612, but the master-side SerDes device 613 can also perform similar processing using the corresponding blocks.

[0430] In step S323, the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the extended DT of the extended packet header ePH of the master side SerDes device 613 via the CSI2-FS processing unit 643 and the CSIA processing unit 642, and performs A-PHY transfer.

[0431] In step S324, the slave SerDes device 612 performs Sequence B (SerDes (Slave) mode) processing, which will be described later with reference to FIG.

[0432] In step S325, the slave-side SerDes device 612 performs Sequence A_Write (SerDes (Slave)) processing, which will be described later with reference to Fig. 53. Note that Fig. 53 describes the Sequence A_Write (AP) processing executed by the application processor 614, but similar processing can also be executed by the corresponding blocks in the slave-side SerDes device 612.

[0433] In step S326, the image sensor 611 performs Sequence B (Image Sensor) processing, which will be described later with reference to Fig. 56. Note that Fig. 56 describes Sequence B (SerDes (Slave)) processing executed by the slave-side SerDes device 612, but similar processing can also be executed by the corresponding blocks in the image sensor 611.

[0434] In step S327, the CCI-FS processing unit 623 of the image sensor 611 performs a write process to write write data based on the contents of the extension packet header ePH and the extension packet footer ePF to an address in the register 624. Thereafter, the process proceeds to step S328.

[0435] As shown in FIG. 52, in steps S328 to S344, a read operation using CCI-FS is performed.

[0436] In step S328, the CCI-FS processing unit 653 of the application processor 614 sets the ePH register to perform a read operation.

[0437] In step S329, the CCI-FS processing unit 653 of the application processor 614 sets the read data register.

[0438] In step S330, the CCI-FS processing unit 653 of the application processor 614 sets the command execution register to 1 and issues a read command.

[0439] In step S331, the application processor 614 performs Sequence A_Read_CMD (AP) processing, which will be described later with reference to Fig. 54. Here, in the Sequence A_Read_CMD (AP) processing, two branched processes are performed in parallel, and the processing proceeds to step S332 according to branch A, or to step S339 according to branch B.

[0440] In step S332, the master-side SerDes device 613 performs Sequence B (SerDes (Master) mode) processing, which will be described later with reference to Fig. 56. Note that Fig. 56 describes the Sequence B (SerDes (Slave) mode) processing performed by the slave-side SerDes device 612, but the master-side SerDes device 613 can also perform similar processing using the corresponding blocks.

[0441] In step S333, the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the extended DT of the extended packet header ePH of the master side SerDes device 613 via the CSI2-FS processing unit 643 and the CSIA processing unit 642, and performs A-PHY transfer.

[0442] In step S334, the slave SerDes device 612 performs Sequence B (SerDes (Slave) mode) processing, which will be described later with reference to FIG.

[0443] In step S355, the slave-side SerDes device 612 performs Sequence A_Read_CMD (SerDes (Slave)) processing, which will be described later with reference to Fig. 54. Note that Fig. 54 describes the Sequence A_Read_CMD (AP) processing executed in the application processor 614, but similar processing can also be executed by corresponding blocks in the slave-side SerDes device 612. Here, in the Sequence A_Read_CMD (SerDes (Slave)) processing, of the two branched processes, the processing does not proceed to branch A, and the processing proceeds to step S336 according to branch B.

[0444] In step S336, the slave-side SerDes device 612 performs Sequence A_Read_Data (SerDes (Slave)) processing, which will be described later with reference to Fig. 57. Note that Fig. 57 describes the Sequence A_Read_Data (AP) processing executed in the application processor 614, but similar processing can also be executed by the corresponding blocks in the slave-side SerDes device 612.

[0445] In step S337, the A-PHY processing unit 631 adds an A-PHY header and an A-PHY footer to the extended DT of the extended packet header ePH of the slave side SerDes device 612 via the CSI2-FS processing unit 633 and the CSIA processing unit 632, and performs A-PHY transfer.

[0446] In step S338, the master-side SerDes device 613 performs Sequence B (SerDes (Master) mode) processing, which will be described later with reference to Fig. 56. Note that Fig. 56 describes Sequence B (SerDes (Slave) mode) processing executed in the slave-side SerDes device 612, but similar processing can also be executed by the corresponding blocks in the master-side SerDes device 613.

[0447] In step S339, the application processor 614 performs Sequence A_Read_Data (AP) processing, which will be described later with reference to FIG.

[0448] In step S340, the application processor 614 performs Sequence B (AP time) processing, which will be described later with reference to Fig. 56. Note that Fig. 56 describes Sequence B (SerDes (Slave) time) processing executed in the slave-side SerDes device 612, but similar processing can also be executed by the corresponding blocks in the application processor 614.

[0449] In step S341, the CCI-FS processing unit 653 of the application processor 614 stores read data at an address in the register 654 based on the contents of the extension packet header ePH and the extension packet footer ePF.

[0450] In step S342, the image sensor 611, the slave SerDes device 612, the master SerDes device 613, and the application processor 614 perform the above-mentioned read processing and check the error register.

[0451] In step S343, the image sensor 611 and each device (slave side SerDes device 612, master side SerDes device 613, and application processor 614) determine whether the register value of the Error register of each CCI-FS processing unit is 0 or not.

[0452] If it is determined in step S343 that the register values ​​of all the CCI-FS processing units are not 0 (any of the register values ​​is not 0), the process proceeds to step S344.

[0453] In step S344, the error-related register value of the CCI-FS processing unit is checked if the register value is not 0, and the error register is cleared by writing 1 to perform retransmission processing.

[0454] On the other hand, if it is determined in step S343 that the register values ​​of all the CCI-FS processing units are 0, or after the processing of step S344, the processing ends.

[0455] Fig. 53 is a flowchart illustrating the Sequence A_Write (AP) processing performed in step S321 of Fig. 51. Note that Fig. 53 illustrates the processing performed by the application processor 614 as an example, but the Sequence A_Write (SerDes (Slave)) processing in step S325 of Fig. 51 is also performed in the same way.

[0456] In step S351, in the application processor 614, the I2C / I3C master 651 issues a start command and a slave address (Slave Address+W 8-bit shown in FIG. 42).

[0457] In step S352, the application processor 614 determines whether or not the I2C / I3C master 651 has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613. If it is determined in step S352 that an ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S353.

[0458] In step S353, the I2C / I3C master 651 in the application processor 614 issues a register address (Register Address [15:8] shown in FIG. 42). Here, each time the processing of step S353 is repeated, the payload at or below this register address is transmitted, as shown in FIG.

[0459] In step S354, the application processor 614 determines whether or not the I2C / I3C master 651 has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613. If it is determined in step S354 that an ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S355.

[0460] In step S355, the I2C / I3C master 651 determines whether or not the transfer of the final data has been completed in the application processor 614. If it is determined in step S355 that the transfer of the final data has not been completed, the process returns to step S353, and the same processes are repeated thereafter.

[0461] On the other hand, if it is determined in step S355 that the transfer of the final data has been completed, the process proceeds to step S356. In step S356, the I2C / I3C master 651 in the application processor 614 issues a stop command. This ends the Sequence A_Write (AP) process, and the process returns to step S322 in FIG.

[0462] On the other hand, if it is determined in step S352 or S354 that an ACK response has not been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S357. In step S357, the I2C / I3C master 651 in the application processor 614 issues a stop command. In this case, the Sequence A_Write (AP) process ends, and the communication process itself ends.

[0463] Fig. 54 is a flowchart illustrating the Sequence A_Read_CMD (AP) processing performed in step S331 of Fig. 52. Note that Fig. 54 illustrates the processing performed by the application processor 614 as an example, but the Sequence A_Read_CMD (SerDes (Slave)) processing in step S335 of Fig. 52 is performed in the same way.

[0464] In step S361, in the application processor 614, the I2C / I3C master 651 issues a start command and a slave address (Slave Address+W 8-bit shown in FIG. 42), and starts a timer.

[0465] In step S362, the application processor 614 determines whether or not the I2C / I3C master 651 has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613. If it is determined in step S362 that an ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S363.

[0466] In step S363, the I2C / I3C master 651 in the application processor 614 issues a register address (Register Address [15:8] shown in FIG. 42). Here, each time the process of step S363 is repeated, the payload at or below this register address is transmitted, as shown in FIG.

[0467] In step S364, the application processor 614 determines whether the I2C / I3C master 651 has received an ACK response from the I2C / I3C slave 644 of the SerDes device 613 on the master side.

[0468] If it is determined in step S364 that an ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S365.

[0469] In step S365, the I2C / I3C master 651 in the application processor 614 determines whether the transfer of the final data has been completed.

[0470] If it is determined in step S365 that the transfer of the final data has been completed, the process proceeds to step S366.

[0471] In step S366, the I2C / I3C master 651 issues a stop command in the application processor 614. Thereafter, the process branches into two, and following branch A, the process proceeds to step S332 in Fig. 52. On the other hand, following branch B, Sequence C (at AP) process (see Fig. 55, described later) is performed in step S367, and then the process proceeds to step S339 in Fig. 52.

[0472] On the other hand, if it is determined in step S365 that the transfer of the final data has not been completed, the process proceeds to step S368.

[0473] In step S368, the I2C / I3C master 651 in the application processor 614 determines whether the timer started in step S361 has timed out. If it is determined in step S368 that the timer has not timed out, the process returns to step S363, and the same processes are repeated thereafter.

[0474] On the other hand, if it is determined in step S368 that the timer has timed out, the process proceeds to step S369.

[0475] In step S369, the application processor 614 sets the Error register (Timeout) to 1, and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related register.

[0476] After the process of step S369, or if it is determined in step S362 or S364 that an ACK response has not been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S370.

[0477] In step S370, the I2C / I3C master 651 issues a stop command in the application processor 614. In this case, the Sequence A_Read_CMD (at AP) process ends, and the communication process itself ends.

[0478] Fig. 55 is a flowchart illustrating the Sequence C (AP) processing performed in step S367 of Fig. 54. Note that Fig. 55 illustrates the processing performed by the application processor 614 as an example, but similar processing can also be performed in the slave-side SerDes device 612.

[0479] In step S381, the I2C / I3C master 651 in the application processor 614 determines whether the timer started in step S361 of Fig. 54 has timed out, and the process waits until it is determined that the timer has timed out. If it is determined in step S381 that the timer has timed out, the process proceeds to step S382, and the I2C / I3C master 651 in the application processor 614 performs a polling operation.

[0480] In step S383, the I2C / I3C master 651 in the application processor 614 determines whether the Status register value of the read command is 1 or not.

[0481] If it is determined in step S383 that the Status register value of the read command is 1, the process proceeds to step S384. In step S384, the application processor 614 performs a read access, and then the process returns to step S339 in FIG.

[0482] On the other hand, if it is determined in step S383 that the Status register value of the read command is not 1 (other than 1), the process proceeds to step S385. In step S385, the application processor 614 sets the Error register (Timeout) to 1 and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related registers.

[0483] In step S386, the I2C / I3C master 651 issues a stop command in the application processor 614. In this case, the Sequence C (AP) processing ends, and the communication processing itself ends.

[0484] Fig. 56 is a flowchart illustrating the Sequence B (SerDes (Slave)) processing performed in steps S324 and S334 in Fig. 51. Note that Fig. 56 illustrates processing performed by the slave-side SerDes device 612 as an example, but the Sequence B (SerDes (Master)) processing in step S322 in Fig. 51, the Sequence B (Image Sensor) processing in step S326 in Fig. 51, and the Sequence B (SerDes (Master)) processing in step S332 in Fig. 52 are also performed in the same way.

[0485] In step S391, in the slave SerDes device 612, the CCI-FS processing unit 636 checks the Source ID of the slave SerDes device 612 and the Destination SID of the extended packet header ePH.

[0486] In step S392, in the slave SerDes device 612, the CCI-FS processing unit 636 determines whether or not the Source ID of the slave SerDes device 612 and the Destination SID of the extended packet header ePH do not match.

[0487] If it is determined in step S392 that the Source ID of the slave SerDes device 612 and the Destination SID in the extended packet header ePH do not match, the process proceeds to step S393.

[0488] In step S393, in the slave side SerDes device 612, the CCI-FS processing unit 636 checks the Destination SID of the slave side SerDes device 612 and the Destination SID of the extended packet header ePH.

[0489] In step S394, in the slave SerDes device 612, the CCI-FS processing unit 636 determines whether or not the Source ID of the slave SerDes device 612 matches the Destination SID in the extended packet header ePH.

[0490] If it is determined in step S394 that the Source ID of the slave SerDes device 612 matches the Destination SID in the extended packet header ePH, the process proceeds to step S395.

[0491] In step S395, in the slave SerDes device 612, the CCI-FS processing unit 636 checks the Message Counter from the contents of the extended packet header ePH.

[0492] In step S396, in the slave side SerDes device 612, the CCI-FS processing unit 636 determines whether the Message Counter in the slave side SerDes device 612 matches the received value of the Message Counter confirmed from the contents of the extended packet header ePH.

[0493] If it is determined in step S396 that the Message Counter in the slave SerDes device 612 matches the received value of the Message Counter confirmed from the contents of the extended packet header ePH, the process proceeds to step S397.

[0494] In step S397, in the slave side SerDes device 612, the CCI-FS processing unit 636 checks the CRC calculation result calculated from the extended packet header ePH in the slave side SerDes device 612 and the received value (ePF0) of the extended packet footer ePF.

[0495] In step S398, it is determined whether the received value (ePF0) of the extended packet footer ePF matches the CRC calculation result, and if it is determined that they match, the process returns to step S325 in FIG.

[0496] On the other hand, if it is determined in step S392 that the Source ID of the slave SerDes device 612 and the Destination SID of the extended packet header ePH do not mismatch (match), the process proceeds to step S399.

[0497] In steps S399 to S402, the same processing as in steps S395 to S398 is performed.

[0498] If it is determined in step S402 that the received value (ePF0) of the extended packet footer ePF matches the CRC calculation result, the process proceeds to step S403. In step S403, write access is made to the register 637 of the SerDes device 612 on the slave side.

[0499] If it is determined in step S394 that the Source ID of the slave SerDes device 612 and the Destination SID of the extended packet header ePH do not match, the process proceeds to step S404. In step S404, the CCI-FS processing unit 636 in the slave SerDes device 612 sets 1 to Error register [2] (Routing) and stores the data of the extended packet header ePH and the extended packet footer ePF in the error-related registers.

[0500] If it is determined in step S398 or S402 that the received value (ePF0) of the extended packet footer ePF does not match the CRC calculation result, the process proceeds to step S405. In step S405, in the slave SerDes device 612, the CCI-FS processing unit 636 sets the Error register (CRC) to 1 and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related register.

[0501] If it is determined in step S396 or S400 that the Message Counter in the slave-side SerDes device 612 does not match the received value of the Message Counter confirmed from the contents of the extended packet header ePH, the process proceeds to step S406. In step S406, the CCI-FS processing unit 636 in the slave-side SerDes device 612 sets the Error register (MC) to 1 and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related registers.

[0502] After the processing of steps S403 to S406, the processing of Sequence B (in SerDes (Slave) mode) ends, and the communication processing itself ends.

[0503] Regarding CRC calculation, it is possible to perform it only for E2E Protection, and it is expected that there will be a combination of error detection on each device and whether or not to discard packets.

[0504] Fig. 57 is a flowchart illustrating the Sequence A_Read_Data (AP) processing performed in step S339 of Fig. 52. Note that Fig. 57 illustrates the processing performed by the application processor 614 as an example, but the Sequence A_Read_Data (SerDes (Slave)) processing in step S336 of Fig. 52 is performed in the same way.

[0505] In step S411, the I2C / I3C master 651 in the application processor 614 issues a start command and a slave address (Slave Address+W 8-bit shown in FIG. 48).

[0506] In step S412, the application processor 614 determines whether or not the I2C / I3C master 651 has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613. If it is determined in step S412 that an ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S413.

[0507] In step S413, in the application processor 614, the I2C / I3C master 651 issues a start command and a slave address (Slave Address+R 8-bit shown in FIG. 48), and starts a timer.

[0508] In step S414, the application processor 614 determines whether or not the I2C / I3C master 651 has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613. If it is determined in step S414 that the ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S415.

[0509] In step S415, in the application processor 614, the I2C / I3C master 651 acquires read data from the opposing I2C / I3C slave 644 on the application processor 614 side.

[0510] In step S416, it is determined whether the I2C / I3C master 651 of the application processor 614 has transmitted an ACK and whether the opposing I2C / I3C slave 644 on the application processor 614 side has received the ACK.

[0511] If it is determined in step S416 that the I2C / I3C master 651 of the application processor 614 has transmitted an ACK and that the opposing I2C / I3C slave 644 on the application processor 614 side has received the ACK, the process proceeds to step S417.

[0512] In step S417, it is determined whether the I2C / I3C master 651 of the application processor 614 has transmitted a NACK signal in response to the completion of the transfer of the final data.

[0513] If it is determined in step S417 that a NACK has been transmitted, the process proceeds to step S418. In step S418, the I2C / I3C master 651 in the application processor 614 issues a stop command. This ends the Sequence A_Read_Data (AP) process, and the process returns to step S340 in FIG.

[0514] On the other hand, if it is determined in step S417 that a NACK has not been transmitted, the process proceeds to step S419.

[0515] In step S419, the I2C / I3C master 651 in the application processor 614 determines whether the timer started in step S413 has timed out. If it is determined in step S419 that the timer has not timed out, the process returns to step S415, and the same processes are repeated thereafter.

[0516] On the other hand, if it is determined in step S419 that the timer has timed out, the process proceeds to step S420.

[0517] In step S420, the application processor 614 sets the Error register (Timeout) to 1, and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related register.

[0518] After the process of step S420, or if it is determined in step S414 that an ACK response has not been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S421. Similarly, if it is determined in step S416 that the I2C / I3C master 651 of the application processor 614 has not transmitted an ACK, or that the opposing I2C / I3C slave 644 on the application processor 614 side has not received an ACK, the process proceeds to step S421.

[0519] In step S421, the I2C / I3C master 651 issues a stop command in the application processor 614. In this case, the Sequence A_Read_Data (at AP) process ends, and the communication process itself ends.

[0520] Here, there are three combinations of access timing from the I2C / I3C master 634 to the I2C / I3C slave 621 when the I2C / I3C slave 621 outputs (see Figure 46), and access timing from the I2C / I3C master 651 to the I2C / I3C slave 644 when the I2C / I3C slave 644 of the master-side SerDes device 613 outputs (see Figure 48), as described below.

[0521] The first access timing is to poll until the read data is acquired, and after preparations for reading the read data are complete, the I2C / I3C master starts the read process.

[0522] The second access timing is when the I2C / I3C master starts a read process after a certain time has elapsed.

[0523] The third access timing uses the Clock Stretch method (refer to FIG. 72 described later). After a certain period of time, the I2C / I3C master starts the read process. At this time, there are two forms: sending the read data in blocks and sending the read data individually (asserting the Clock Stretch Mode signal).

[0524] <Configuration example of extended packet header ePH> FIGS. 58 to 60 are diagrams showing a configuration example of the extended packet header ePH.

[0525] FIG. 58 shows detailed configuration examples of the extended packet headers ePH0, ePH1, and ePH2. The addition of the extended packet header ePH as shown is defined by reusing the ePH structure in C-PHY and D-PHY for the content of the extended packet header ePH for CCI-FS.

[0526] FIG. 59 shows a detailed configuration example of the extended packet header ePH3. The addition of the extended packet header ePH as shown is defined for the content of the extended packet header ePH for CCI-FS.

[0527] FIG. 60 shows a detailed configuration example of the extended DT of the extended packet header ePH. For example, in order to support CCI-FS, "0xC0:For I2C" and "0xC1:For I3C" are added to the data type of the extended packet header ePH.

[0528] <Example of I2C circuit configuration> FIG. 61 shows a configuration example of a conventional I2C hardware. For example, it shows a configuration example of I2C in the case of a bus connection configuration at the upper level during hardware implementation, and the slave side may be configured to receive AKC / NACK from the upper level. Of course, it is just an example shown, and the upper bus configuration does not necessarily match.

[0529] Figure 62 shows the waveforms during data transfer on the I2C bus. Note that the I2C bus standard and CCI (I2C) are considered equivalent.

[0530] <Example of CCI-related configuration in communication system 701> FIG. 63 is a block diagram showing an example of a CCI-related configuration in a communication system 701 having an A-PHY direct connection configuration, similar to the communication system 501 shown in FIG. 27 above.

[0531] As shown in FIG. 63, in a communication system 701, an image sensor 711 and an application processor 712 are directly connected via an A-PHY.

[0532] The image sensor 711 includes an A-PHY processing unit 721, a CSIA processing unit 722, a CSI2 processing unit 723, a CSI2-FS processing unit 724, a CCI processing unit 725, a CCI-FS processing unit 726, a register 727, and selectors 728-1 and 728-2. As shown in the figure, the selectors 728-1 and 728-2 are arranged to sandwich the CCI-FS processing unit 726, and can switch between enable and disable of the CCI-FS processing unit 726 in accordance with a CCI_FS_Enable signal of the register 727.

[0533] The application processor 712 is configured to include an A-PHY processing unit 731, a CSIA processing unit 732, a CSI2 processing unit 733, a CSI2-FS processing unit 734, a CCI processing unit 735, a CCI-FS processing unit 736, a register 737, and selectors 738-1 and 738-2. As shown in the figure, the selectors 738-1 and 738-2 are arranged to sandwich the CCI-FS processing unit 736, and can switch between enable and disable of the CCI-FS processing unit 736 in accordance with a CCI_FS_Enable signal of the register 737.

[0534] For example, when the CCI_FS_Enable signal indicates enabling CCI-FS (CCI_FS_Enable = 1), as shown by the arrow of the dashed line, data is transmitted and received via the CCI-FS processing unit 726 and the CCI-FS processing unit 736. On the other hand, when the CCI_FS_Enable signal indicates disabling CCI-FS (CCI_FS_Enable = 0), as shown by the arrow of the dotted line, data is transmitted and received without passing through the CCI-FS processing unit 726 and the CCI-FS processing unit 736.

[0535] <Network connection form> FIG. 64 shows an example of the network connection form (topology) of the A-PHY direct connection configuration and the SerDes connection configuration.

[0536] The application processor (AP) 801 is directly connected to the image sensor 802 via A-PHY, and the image sensor 802 can form a connection configuration in which it is connected to the sensor 803 via I2C / I3C.

[0537] The application processor 801 is connected to the master-side SerDes device 804 via I2C / I3C, and the master-side SerDes device 804 and the slave-side SerDes device 805 are connected via A-PHY. The slave-side SerDes device 805 can form a connection configuration in which it is connected to two sensors 806-1 and 806-2 via I2C / I3C.

[0538] <Circuit configuration of CCI-FS processing unit> FIG. 65 is a block diagram showing an example of the circuit configuration of the CCI-FS processing unit. The CCI-FS processing unit 901 and the register 902 shown in FIG. 65 have a common configuration with the CCI-FS processing unit and the register provided in each of the above-described devices.

[0539] 65, CCI-FS processing unit 901 has a CCI-FS switch, registers, etc. in its upper layer, and a CCI processing unit in its lower layer. CCI-FS processing unit 901 is configured with a CCI-FS transmitter 911 and a CCI-FS receiver 912. Various register setting value information is supplied from register 902 to CCI-FS processing unit 901, and an error notification is supplied from CCI-FS processing unit 901 to register 902.

[0540] The CCI-FS transmitter 911 includes an extended packet header ePH generator 921 , an extended packet footer ePF generator 922 , and a destination address checker 923 .

[0541] The extended packet header ePH generation unit 921 has an MC generation unit 941 that generates a Message Counter and a Packet Length calculation unit 942 that calculates the packet length. The extended packet footer ePF generation unit 922 has an extended packet footer ePF1 generation unit 943 that generates an extended packet footer ePF1 and a CRC calculation unit 944 that calculates the CRC to be stored in the extended packet footer ePF0.

[0542] The CCI-FS receiver 912 includes an extended packet header ePH checker 931 , an extended packet footer ePF checker 932 , and a destination address checker 933 .

[0543] The extended packet header ePH check unit 931 has an MC check unit 951 that checks the Message Counter and a packet length calculation and check unit 952 that calculates and checks the packet length. The extended packet footer ePF check unit 932 has an extended packet footer ePF1 check unit 953 that checks the extended packet footer ePF1 and a CRC calculation unit 954 that calculates the CRC to be stored in the extended packet footer ePF0.

[0544] The CCI-FS processing unit 901 can check the destination address of data from a higher layer using a CCI-FS transmission unit 911, generate an extended packet header ePH and an extended packet footer ePF, add them to the data, and supply them to a lower layer. The CCI-FS processing unit 901 can check the destination address of data from a lower layer using a CCI-FS reception unit 912, check the extended packet header ePH and an extended packet footer ePF, and supply them to a higher layer.

[0545] Here, the operation of the CCI-FS processing unit of each device constituting communication system 601 in the example of the SerDes connection configuration shown in FIG. 40 will be described.

[0546] The application processor 614 has a Source ID indicating its own device in the extended packet header ePH in the application processor 614. Then, the CCI-FS processing unit 653 adds the above information and information having a Destination ID indicating the device making the intended access.

[0547] The slave-side SerDes device 612 and the master-side SerDes device 613 have a Source ID that indicates their own device, either by being set in advance or as a unique value. The CCI-FS processing unit 636 and the CCI-FS processing unit 646 set in advance the above information and information having a Destination ID that indicates the connected device and the target device.

[0548] Furthermore, the CCI-FS processing unit 636 and the CCI-FS processing unit 646 compare the destination ID in the received extension packet header ePH with their own ID (Source ID) to determine whether the access is to themselves or indicates a target device (Destination ID). For example, if the destination ID in the received extension packet header ePH matches their own ID (Source ID), they access their own register as an access to an intermediate device (SerDes device). On the other hand, if the destination ID in the received extension packet header ePH does not match their own ID (Source ID), they transfer data to the connected device (Destination ID) as an access to a subsequent device.

[0549] As described above, data is transferred to an intermediate device (SerDes device) or a target device based on the Source ID and Destination ID embedded in the extended packet header ePH, a pre-set or unique Source ID, and pre-set connection destination information, and access is made to the target device.

[0550] The CSI2-FS processing unit 623 of the image sensor 611 accesses its own register as an access to the image sensor 611 when the Destination ID of the received extended packet header ePH matches its own ID (Source ID).

[0551] In this way, the Source ID of each device can be a unique value for each device, a preset value, or a combination thereof.

[0552] 66 to 68 are diagrams showing detailed configuration examples of the register 902. FIG.

[0553] Fig. 66 shows details of addresses 0x000 to 0x109 of the register 902. Fig. 67 shows details of addresses 0x110 to 0x125 of the register 902, illustrating an example of a configuration in a Bridge configuration.

[0554] Fig. 68 shows the error-related registers as details of address 0x200 of register 902. Fig. 68 shows the error-related registers (debug) as details of addresses 0x300 and 0x400 of register 902. Fig. 68 shows the error injection-related registers (debug) as details of address 0x800 of register 902.

[0555] <Modification of extended packet header ePH> With reference to FIGS. 69 and 70, a modification of the extension packet header ePH will be described.

[0556] Fig. 69 shows a modified example of the extension packet header ePH in the packet configuration of write data generated by the CCI-FS processing unit 536 of the application processor 512 during write access, as described above with reference to Fig. 33. The extension packet header ePH shown in Fig. 69 differs from the example configuration shown in Fig. 33 above in the configurations of the extension packet header ePH3 and extension packet header ePH4.

[0557] Fig. 70 shows a modified example of the extension packet header ePH in the packet configuration of write data generated by the CCI-FS processing unit 536 of the application processor 512 during read access, as described above with reference to Fig. 28. The extension packet header ePH shown in Fig. 70 differs from the example configuration shown in Fig. 28 above in the configurations of the extension packet header ePH3 and extension packet header ePH4.

[0558] For example, in the extended packet header ePH shown in Figures 69 and 70, the following combinations are assumed, depending on the implementation.

[0559] Read address information may be stored in the extended packet header ePH or in the AP (CCI) payload. Length information may be stored in the extended packet header ePH or in the AP (CCI) payload. CMD information may be stored in the CCI Command ID in the extended packet header ePH. Command start, restart, and end information is referenced based on the CCI Command ID. CCI information (e.g., Slave Address) may be stored in the AP (CCI) payload using the CCI Header Length. The CCI Header Length is information indicating the header length of the CCI protocol (I2C).

[0560] FIG. 71 is a diagram illustrating the flow between the image sensor 511 and the application processor 512 in the A-PHY direct connection configuration as shown in FIG.

[0561] In the application processor 512, the CCI-FS switch 538 issues a read command and a write command. The CCI-FS switch 538 supplies a slave address (Slave Address+W 8 bits), a register address (Register Address[15:8], Register Address[7:0]), and data (Data*(*=N)[7:0]) to the CCI processing unit 535. The CCI processing unit 535 converts them into an AP(CCI) payload and supplies it to the A-PHY processing unit 531. The A-PHY processing unit 531 adds an A-PHY header and an A-PHY footer to the AP(CCI) payload and performs A-PHY transfer to the image sensor 511.

[0562] In the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and the A-PHY footer and supplies the AP (CCI) payload to the CCI processing unit 525. The CCI processing unit 525 converts the AP (CCI) payload and, based on the contents of the AP (CCI) payload, writes data to a register 527 in accordance with a write command and reads data from the register 527 in accordance with a read command.

[0563] At this time, the initial setting of CCI-FS Enable is performed by the CCI processing unit 525, and bus conversion such as the register interface and AHB bus is performed. Then, the CCI-FS Enable setting is confirmed via the CCI processing unit 525 or the CCI-FS processing unit 526.

[0564] The CCI processing unit 525 converts the read data (Data*(*=M)[7:0]) read from the register 527 in response to the read command into an AP(CCI) payload and supplies it to the A-PHY processing unit 521. The A-PHY processing unit 521 adds an A-PHY header and an A-PHY footer to the AP(CCI) payload and performs A-PHY transfer to the application processor 512.

[0565] In the application processor 512, the A-PHY processing unit 531 removes the A-PHY header and the A-PHY footer and supplies the AP(CCI) payload to the CCI processing unit 535. The CCI processing unit 535 converts the AP(CCI) payload and supplies read data (Data*(*=M)[7:0]) to the CCI-FS switch 538.

[0566] The CCI-FS switch 538 sets the CCI-FS Enable setting and various CCI-FS related registers in the register 537. At this time, register access depends on the implementation. The CCI-FS switch 538 sets various CCI-FS related registers in the register 527 via the register 537, the CCI-FS processing unit 536, the A-PHY processing unit 531, the A-PHY processing unit 521, and the CCI-FS processing unit 526.

[0567] In the application processor 512, the CCI-FS switch 538 issues a read command. The CCI-FS switch 538 supplies a slave address (Slave Address+W 8 bits), a register address (Register Address[15:8], Register Address[7:0]), and data (Data*(*=N)[7:0]) to the register 537. The CCI-FS processing unit 536 converts them into an AP (CCI) payload, adds an extended packet header ePH*(*=n), an extended packet footer ePF1, and an extended packet footer ePF0, and supplies the payload to the A-PHY processing unit 531. The A-PHY processing unit 531 adds an A-PHY header and an A-PHY footer to the payload and performs A-PHY transfer to the image sensor 511.

[0568] In the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY footer, and supplies the extended packet header ePH* (*=n), AP (CCI) payload, extended packet footer ePF1, and extended packet footer ePF0 to the CCI-FS processing unit 526. The CCI-FS processing unit 526 converts the AP (CCI) payload and, based on its contents, reads data from a register 527 in accordance with a read command. At this time, register access depends on the implementation, and bus conversion such as a register interface, AHB bus, or CCI interface is performed.

[0569] The CCI-FS processing unit 526 converts the read data (Data*(*=M)[7:0]) read from the register 527 in response to the read command into an AP(CCI) payload, adds an extended packet header ePH*(*=n), an extended packet footer ePF1, and an extended packet footer ePF0, and supplies the payload to the A-PHY processing unit 521. The A-PHY processing unit 521 adds an A-PHY header and an A-PHY footer to the data, and performs A-PHY transfer to the application processor 512.

[0570] In the application processor 512, the A-PHY processing unit 531 removes the A-PHY header and the A-PHY footer, and supplies the extended packet header ePH*(*=n), the AP(CCI) payload, the extended packet footer ePF1, and the extended packet footer ePF0 to the CCI-FS processing unit 536. The CCI-FS processing unit 536 converts the AP(CCI) payload and supplies read data (Data*(*=M)[7:0]) to the CCI-FS switch 538.

[0571] The above flow has been explained using an example of I2C / I3C command generation in hardware, but there are other combinations such as the following:

[0572] In the case of software, I2C / I3C generation involves generating the slave address, register address, payload, ACK response reception, transmission, and various control codes (S, Sr, ACK, NACK, P) in software (for example, like GPIO control). In software I2C / I3C command generation, the CPU issues the slave address, register address, and payload in response to ACK reception in the CPU bus settings.

[0573] In the case of hardware, I2C / I3C is generated in the hardware by setting the transfer settings and data in the I2C / I3C HW IP. Various control codes are automatically responded to by the hardware. I2C / I3C command generation in the hardware involves setting the data in the I2C / I3C HW IP in the transfer settings and sending the command. Various control codes are automatically responded to by the hardware.

[0574] FIG. 72 is a diagram illustrating a flow using the Clock Stretch method in write access and read access between the image sensor 611 and the application processor 614 in the SerDes connection configuration as shown in FIG.

[0575] The CCI-FS switch 655 of the application processor 614 supplies the start command and write command (slave address+W 8 bits) to the CCI processing unit 645 of the master-side SerDes device 613 and asserts the Scl_enb signal. In the master-side SerDes device 613, the CCI processing unit 645 supplies the write command to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the write command and performs A-PHY transfer to the slave-side SerDes device 612.

[0576] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY footer and supplies the write command to the CCI processing unit 635 (slave). The CCI processing unit 635 (slave) negates the Scl_enb signal and supplies the write command to the CCI processing unit 635 (master). Here, the CCI processing unit 635 that communicates with the master-side SerDes device 613 and functions as a slave is referred to as the CCI processing unit 635 (slave), and the CCI processing unit 635 that communicates with the image sensor 611 and functions as a master is referred to as the CCI processing unit 635 (master).

[0577] The CCI processing unit 635 (Master) transmits a start command and a write command to the image sensor 611.

[0578] In the image sensor 611, the CCI processing unit 622 receives the start command and write command and supplies them to the CSI2-FS processing unit 623. The CSI2-FS processing unit 623 supplies an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 transmits the ACK response to the slave-side SerDes device 612.

[0579] In the slave-side SerDes device 612, when the CCI processing unit 635 (master) receives the ACK response and the Scl_enb signal is negated from the CCI processing unit 635 (slave), the CCI processing unit 635 (master) supplies the ACK response to the CCI-FS processing unit 636. Thereafter, the CCI processing unit 635 (slave) asserts the Scl_enb signal to the CCI processing unit 635 (master).

[0580] The CCI-FS processing unit 636 supplies the ACK response to the A-PHY processing unit 631. The A-PHY processing unit 631 adds an A-PHY header and an A-PHY footer to the ACK response, and performs A-PHY transfer to the SerDes device 613 on the master side.

[0581] In the master-side SerDes device 613, the A-PHY processing unit 641 removes the A-PHY header and the A-PHY footer and supplies the ACK response to the CCI processing unit 645. When the CCI-FS switch 655 of the application processor 614 negates the Scl_enb signal to the CCI processing unit 645, the CCI processing unit 645 transmits the ACK response to the application processor 614.

[0582] In the application processor 614 , the CCI processing unit 652 receives the ACK response and supplies it to the CCI-FS switch 655 via the CCI-FS processing unit 653 .

[0583] The CCI-FS switch 655 of the application processor 614 supplies the register address (Register Address[7:0]) to the CCI processing unit 645 of the master-side SerDes device 613 and asserts the Scl_enb signal. In the master-side SerDes device 613, the CCI processing unit 645 supplies the register address to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the register address and performs A-PHY transfer to the slave-side SerDes device 612.

[0584] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY footer and supplies the register address to the CCI processing unit 635 (slave). The CCI processing unit 635 (slave) negates the Scl_enb signal and supplies the register address to the CCI processing unit 635 (master). The CCI processing unit 635 (master) transmits the register address to the image sensor 611. Thereafter, the CCI processing unit 635 (slave) asserts the Scl_enb signal to the CCI processing unit 635 (master).

[0585] In the image sensor 611, the CCI processing unit 622 receives the register address and supplies it to the CSI2-FS processing unit 623. The CSI2-FS processing unit 623 supplies an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 transmits the ACK response to the slave-side SerDes device 612.

[0586] Thereafter, the ACK response is supplied to the CCI-FS switch 655 in the same manner as described above.

[0587] In the application processor 614 , the CCI-FS processing unit 653 transmits the extended packet header ePH* (*=n) to the master side SerDes device 613 under the control of the CCI-FS switch 655 .

[0588] In the master-side SerDes device 613, the CCI processing unit 645 receives the extended packet header ePH*(*=n), and when the Scl_enb signal is asserted from the CCI-FS switch 655, supplies the extended packet header ePH*(*=n) to the A-PHY processing unit 641. The CCI-FS switch 655 then negates the Scl_enb signal to the CCI processing unit 645. The A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the extended packet header ePH*(*=n), and performs A-PHY transfer to the slave-side SerDes device 612.

[0589] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY footer and supplies the extended packet header ePH*(*=n) to the CCI-FS processing unit 636. The CCI-FS processing unit 636 negates the Scl_enb signal and supplies the extended packet header ePH*(*=n) to the CCI processing unit 635 (master). The CCI processing unit 635 (master) transmits the extended packet header ePH*(*=n) to the image sensor 611. Thereafter, the CCI processing unit 635 (slave) asserts the Scl_enb signal to the CCI processing unit 635 (master).

[0590] In the image sensor 611, the CSI2-FS processing unit 623 receives the extended packet header ePH* (*=n). The CSI2-FS processing unit 623 supplies an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 transmits the ACK response to the slave-side SerDes device 612.

[0591] Thereafter, the ACK response is supplied to the CCI-FS switch 655 in the same manner as described above.

[0592] The CCI-FS switch 655 of the application processor 614 supplies the write data (Dara0[7:0]) to the CCI processing unit 645 of the master-side SerDes device 613 and asserts the Scl_enb signal. In the master-side SerDes device 613, the CCI processing unit 645 supplies the write data to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the write data and performs A-PHY transfer to the slave-side SerDes device 612.

[0593] In the master-side SerDes device 613, the CCI processing unit 645 receives the write data, and when the Scl_enb signal is asserted from the CCI-FS switch 655, supplies the write data to the A-PHY processing unit 641. Thereafter, the CSI2-FS processing unit 653 negates the Scl_enb signal to the CCI processing unit 645 under the control of the CCI-FS switch 655. The A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the write data and performs A-PHY transfer to the slave-side SerDes device 612.

[0594] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY footer and supplies the write data to the CCI processing unit 635. The CCI processing unit 635 negates the Scl_enb signal and supplies the write data to the CCI processing unit 635 (master). The CCI processing unit 635 (master) transmits the write data to the image sensor 611. Thereafter, the CCI processing unit 635 (slave) asserts the Scl_enb signal to the CCI processing unit 635 (master).

[0595] In the image sensor 611, the CCI processing unit 622 receives the write data and supplies it to the CSI2-FS processing unit 623, which then writes the write data to the register 624. The CSI2-FS processing unit 623 supplies an ACK response indicating that the write data has been successfully written to the CCI processing unit 622, and the CCI processing unit 622 transmits the ACK response to the SerDes device 612 on the slave side.

[0596] Thereafter, the ACK response is supplied to the CCI-FS switch 655 in the same manner as described above.

[0597] In the application processor 614, the CCI-FS processing unit 653 transmits the extended packet footer ePF0 to the SerDes device 613 on the master side under the control of the CCI-FS switch 655.

[0598] In the master-side SerDes device 613, the CCI processing unit 645 receives the extended packet footer ePF0, and when the Scl_enb signal is asserted from the CCI-FS switch 655, supplies the extended packet footer ePF0 to the A-PHY processing unit 641. The CCI-FS switch 655 then negates the Scl_enb signal to the CCI processing unit 645. The A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the extended packet footer ePF0 and performs A-PHY transfer to the slave-side SerDes device 612.

[0599] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY footer and supplies the extended packet footer ePF0 to the CCI-FS processing unit 636. The CCI-FS processing unit 636 negates the Scl_enb signal and supplies the extended packet footer ePF0 to the CCI processing unit 635 (master). The CCI processing unit 635 (master) transmits the extended packet footer ePF0 to the image sensor 611. Thereafter, the CCI processing unit 635 (slave) asserts the Scl_enb signal to the CCI processing unit 635 (master).

[0600] In the image sensor 611, the CSI2-FS processing unit 623 receives the extended packet footer ePF0. The CSI2-FS processing unit 623 supplies an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 transmits the ACK response to the slave-side SerDes device 612.

[0601] Thereafter, the ACK response is supplied to the CCI-FS switch 655 in the same manner as described above.

[0602] The CCI-FS switch 655 of the application processor 614 supplies the repeat start command and read command (Slave Address+R 8 bits) to the CCI processing unit 645 of the master-side SerDes device 613 and asserts the Scl_enb signal. In the master-side SerDes device 613, the CCI processing unit 645 supplies the read command to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the read command and performs A-PHY transfer to the slave-side SerDes device 612.

[0603] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY footer and supplies the read command to the CCI processing unit 635 (slave). The CCI processing unit 635 (slave) negates the Scl_enb signal and supplies the read command to the CCI processing unit 635 (master). The CCI processing unit 635 (master) transmits a repeat start command and a read command to the image sensor 611.

[0604] In the image sensor 611, the CCI processing unit 622 receives the repeat start command and the read command and accesses the register 624. The CCI processing unit 622 transmits an ACK response indicating successful reception to the slave side SerDes device 612.

[0605] Thereafter, the ACK response is supplied to the CCI-FS switch 655 in the same manner as described above.

[0606] In the image sensor 611, the CCI processing unit 622 reads out the read data (Data0[7:0]) from the register 624 and transmits it to the SerDes device 612 on the slave side.

[0607] In the slave-side SerDes device 612, the CCI processing unit 635 (master) receives the read data and supplies it to the CCI processing unit 635 (slave), and the CCI processing unit 635 (slave) supplies the read data to the A-PHY processing unit 631. The A-PHY processing unit 631 adds an A-PHY header and an A-PHY footer to the read data and performs A-PHY transfer to the master-side SerDes device 613.

[0608] In the master-side SerDes device 613, the A-PHY processing unit 641 removes the A-PHY header and A-PHY footer and supplies the read data to the CCI processing unit 645, which then transmits the read data to the application processor 614.

[0609] In the application processor 614 , the CCI processing unit 652 receives the read data and supplies it to the CCI-FS switch 655 via the CCI-FS processing unit 653 .

[0610] The CCI-FS switch 655 transmits the NACK response and the stop command to the CCI processing unit 645. The CCI processing unit 645 supplies the NACK response and the stop command to the A-PHY processing unit 641. The A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the NACK response and the stop command, and performs A-PHY transfer to the SerDes device 612 on the slave side.

[0611] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY footer and supplies a NACK response and a stop command to the CCI processing unit 635 (slave). The CCI processing unit 635 (slave) supplies the NACK response and the stop command to the CCI processing unit 635 (master), and the CCI processing unit 635 (master) transmits the NACK response and the stop command to the image sensor 611.

[0612] In the image sensor 611 , the CCI processing unit 622 receives the NACK response and the stop command and supplies them to the CSI2-FS processing unit 623 .

[0613] In the flow described in Figure 72, I2C control commands such as start, repeat start, ACK response, NACK response, and stop set the Control Code Indicator in the extended packet header ePH0 to 1 and indicate each code assigned to the 1-byte payload.

[0614] <Detailed configuration example of image sensor and application processor> (Detailed configuration example of an image sensor) Fig. 73 is a block diagram showing a configuration example of the image sensor 211 shown in Fig. 25 described above, which is equipped with a CCI-FS processing unit 1001. In the image sensor 211 shown in Fig. 73, components common to the image sensor 211 in Fig. 25 are denoted by the same reference numerals, and descriptions thereof will be omitted.

[0615] 73, the CCI-FS processing unit 1001 is disposed between the CCI slave 224 and the register 47, and MUX units 1002-1 and 1002-2 are disposed on either side of the CCI-FS processing unit 1001. When the CCI-FS processing unit 1001 is enabled in accordance with the cci_fs_en signal supplied from the register 47, the MUX units 1002-1 and 1002-2 transmit and receive data via the CCI-FS processing unit 1001. On the other hand, when the CCI-FS processing unit 1001 is disabled in accordance with the cci_fs_en signal supplied from the register 47, the MUX units 1002-1 and 1002-2 transmit and receive data without passing through the CCI-FS processing unit 1001.

[0616] (Example of detailed configuration of application processor) Fig. 74 is a block diagram showing an example of a configuration in which the application processor 214 shown in Fig. 26 described above is equipped with a CCI-FS processing unit 1101. In the application processor 214 shown in Fig. 74, components common to the application processor 214 in Fig. 26 are denoted by the same reference numerals, and descriptions thereof will be omitted.

[0617] 74, the CCI-FS processing unit 1101 is arranged between the CCI master 254 and the register 73, and MUX units 1102-1 and 1102-2 are arranged to sandwich the CCI-FS processing unit 1101. When the CCI-FS processing unit 1101 is enabled in accordance with the cci_fs_en signal supplied from the register 73, the MUX units 1102-1 and 1102-2 transmit and receive data via the CCI-FS processing unit 1101. On the other hand, when the CCI-FS processing unit 1101 is disabled in accordance with the cci_fs_en signal supplied from the register 73, the MUX units 1102-1 and 1102-2 transmit and receive data without going through the CCI-FS processing unit 1101.

[0618] The following configurations may be used to implement each field in the extended packet header ePH configuration. The extended VC is unused in Safe CCI. (A similar configuration is used to match the extended header related information in MIPI with the header field.) The extended DT may be embedded in the command-related information for the bus from the higher level, or may be implemented as a signal line setting from the register settings. The protocol is described using I2C, but the same can be done in I3C SDR mode.

[0619] <Example of communication system configuration> A fourth embodiment of a communication system to which the present technology is applied will be described with reference to FIGS.

[0620] Fig. 75 is a block diagram of a communication system according to the fourth embodiment. A of Fig. 75 shows a communication system 1201 which is a first variation, and B of Fig. 75 shows a communication system 1201A which is a second variation.

[0621] A communication system 1201 shown in A of FIG. 75 is configured by directly connecting an image sensor 1211 and an application processor 1212.

[0622] The image sensor 1211 has an ALL layer 1222 arranged on an A-PHY layer 1221, and a CSI-2 transmitter 1223, a CSI extender 1224, a CCI slave 1225, and a CCI extender 1226 arranged on the ALL layer 1222. By providing the CSI extender 1224 to the CSI-2 transmitter 1223 and the CCI extender 1226 to the CCI slave 1225, the image sensor 1211 can support each extended standard.

[0623] The application processor 1212 has an ALL layer 1232 arranged above an A-PHY layer 1231, and a CSI-2 receiver 1233, a CSI extender 1234, a CCI master 1235, and a CCI extender 1236 arranged above the ALL layer 1232. By providing the CSI extender 1234 to the CSI-2 receiver 1233 and the CCI extender 1236 to the CCI master 1235, the application processor 1212 can support each extended standard. Note that the CSI extensions may also be referred to as Camera Service Extensions (CSE).

[0624] A communication system 1201A shown in B of Fig. 75 is configured by connecting a display 1213 and an application processor 1212A. Note that the application processor 1212A is configured with a DSI-2 transmitter 1233A and a DSI extender 1234A instead of the CSI-2 receiver 1233 and CSI extender 1234 of the application processor 1212 in A of Fig. 75, and the other blocks have the same configuration as the application processor 1212.

[0625] The display 1213 has an ALL layer 1242 arranged on an A-PHY layer 1241, and a DSI-2 receiver 1243, a DSI extender 1244, a CCI slave 1245, and a CCI extender 1246 arranged on the ALL layer 1242. By providing the DSI extender 1244 for the DSI-2 receiver 1243 and the CCI extender 1246 for the CCI slave 1245, the display 1213 can support each extended standard. Note that the DSI extensions may also be referred to as Display Service Extensions (DSE).

[0626] The communication systems 1201 and 1201A configured in this manner can at least perform high-speed data transmission, which transmits frame data including image data in one direction, and low-speed command transmission, which transmits commands related to high-speed data transmission in the opposite direction (however, transmission of the command itself may also be referred to as command transmission, or transmission of a response to the command may also be referred to as command transmission). For example, low-speed command transmission involves at least transmitting a high-speed data transmission start command that requests the start of high-speed data transmission, but this is not necessarily the case. Furthermore, high-speed data transmission is faster than low-speed command transmission and is initiated in response to receipt of a high-speed data transmission start command, but this is not necessarily the case.

[0627] However, the directions of high-speed data transmission and low-speed command transmission are different between communication system 1201, in which application processor 1212 communicates with image sensor 1211, and communication system 1201A, in which application processor 1212A communicates with display 1213. That is, in communication system 1201, image data is transmitted from image sensor 1211 to application processor 1212, and in communication system 1201A, image data is transmitted from application processor 1212A to display 1213.

[0628] In the physical layer standard A-PHY, high-speed data transmission and low-speed command transmission are transmitted partially or entirely via a common communication path. A-PHY also supports an option that enables partial or complete transmission of power supply from the application processor 1212 to the image sensor 1211 and power supply from the application processor 1212A to the display 1213 via a common communication path.

[0629] Low-speed command transmission conforms to, for example, the CSI-2 standard CCI and is performed based on the I2C or I3C standard. Low-speed command transmission can transmit commands not only via an independent I2C or I3C physical layer, but also via part or all of the physical layers of D-PHY, C-PHY, and A-PHY. High-speed data transmission, on the other hand, transmits data via part or all of the physical layers of D-PHY, C-PHY, and A-PHY.

[0630] For example, when low-speed command transmission complies with Unified Serial Link (USL) in the CSI-2 standard, commands can be transmitted via part or all of the physical layer of either D-PHY or C-PHY. In other words, high-speed data transmission and low-speed command transmission can be transmitted via part or all of the physical layer of either D-PHY, C-PHY, A-PHY, I2C, or I3C.

[0631] 75 illustrates an example configuration including application processors 1212 and 1201A, but communication systems 1201 and 1201A may also be configured to include, for example, an electronic control unit (ECU). That is, the processor is not limited to application processor 1212 as long as it can communicate with image sensor 1211, display 1213, etc. via direct or indirect connection. Also, various types of sensors other than image sensor 1211 may be included.

[0632] The communication systems 1201 and 1201A configured in this manner employ a nonce value transmission method or an initialization vector configuration including a nonce value, as described below.

[0633] Specifically, a specific symmetric key encryption algorithm (e.g., AES-GCM / GMAC) requires an initialization vector including a nonce value. Therefore, rules for setting the initialization vector and nonce value are agreed upon in advance between the image sensor 1211 and the application processor 1212, or between the display 1213 and the application processor 1212A.

[0634] However, if erroneous recognition or tampering of the nonce value occurs within the image sensor 1211, the application processors 1212 and 1201A, and the display 1213, subsequent decryption of encrypted image data and authentication of messages will fail. Therefore, in order to avoid the problem of image data not being transmitted normally, a countermeasure technology against erroneous recognition and tampering of the nonce value was required.

[0635] Meanwhile, as a new security specification for the MIPI Camera Serial Interface (CSI) standard or the MIPI Display Serial Interface (DSI) standard, it was necessary to define an initialization vector suitable for the CSI standard or the DSI standard. Therefore, the present technology discloses a nonce value transmission method or an initialization vector configuration including a nonce value that is suitable for an imaging device that complies with the CSI standard and includes an image sensor 1211, or a display device that complies with the DSI standard and includes a display 1213.

[0636] Although the following describes processing performed between the image sensor 1211 and the application processor 1212, similar processing can also be performed between the display 1213 and the application processor 1212A.

[0637] <Detailed configuration example of the image sensor in Figure 75> FIG. 76 is a block diagram showing a detailed configuration example of the image sensor 1211.

[0638] The image sensor 1211 is configured with pixels 1301, an AD converter 1302, an image processing unit 1303, an extended mode-compatible CSI-2 transmission circuit 1304, a physical layer processing unit 1305, an I2C / I3C slave 1306, a storage unit 1307, a message counter 1308, a nonce updating unit 1309, and a security unit 1310. Note that the pixels 1301, the AD converter 1302, the image processing unit 1303, the extended mode-compatible CSI-2 transmission circuit 1304, the physical layer processing unit 1305, the I2C / I3C slave 1306, and the storage unit 1307 are configured in the same manner as the corresponding blocks in the other embodiments described above, and detailed description thereof will be omitted.

[0639] The message counter 1308 updates the message count value in the image sensor 1211 every time an extended packet that satisfies a predetermined count condition is transmitted.

[0640] The security unit 1310 derives a session key within the image sensor 1211 and uses the session key to generate first protection data (e.g., an integrity calculation value calculated to protect integrity, or encrypted data encrypted to protect confidentiality) for data to be transmitted at high speed.

[0641] The nonce update unit 1309 updates the nonce (number used once) value in the image sensor 1211 every time the security unit 1310 generates first protected data.

[0642] The image sensor 1211 configured in this manner transmits part or all of the nonce value and part or all of the message count value at high speed to the application processor 1212. For example, part or all of the nonce value may be a count value or a random number. Also, part or all of the nonce value is stored outside the extended packet header and transmitted, and the image data is stored and transmitted within the packet data.

[0643] In the image sensor 1211, the message counter 1308 and the nonce update unit 1309 may be configured separately or integrally. For example, if the message counter 1308 and the nonce update unit 1309 are configured separately, the nonce value and the message count value can be updated asynchronously. This allows for greater flexibility in the nonce value and the message count value.

[0644] On the other hand, if message counter 1308 and nonce updating unit 1309 are configured as an integrated unit, the updates of the nonce value and message count value can be synchronized. In this case, if a count value is used as the nonce value, the message count value can be shared with part or all of the nonce value, thereby saving the bit width of message counter 1308. In other words, message counter 1308 may be part or all of nonce updating unit 1309, and part or all of it can be shared with nonce updating unit 1309.

[0645] <Detailed configuration example of the application processor in Figure 75> FIG. 77 is a block diagram showing a detailed configuration example of the application processor 1212.

[0646] The application processor 1212 is configured to include a physical layer processing unit 1321, an extended mode-compatible CSI-2 receiving circuit 1322, an I2C / I3C master 1323, a memory unit 1324, a data verification unit 1325, a security unit 1326, and a controller 1327. The physical layer processing unit 1321, the extended mode-compatible CSI-2 receiving circuit 1322, the I2C / I3C master 1323, and the memory unit 1324 are configured in the same manner as the corresponding blocks in the other embodiments described above, and detailed description thereof will be omitted.

[0647] The data verification unit 1325 verifies the validity of the nonce value or message count value sent from the image sensor 1211 to the application processor 1212 .

[0648] The security unit 1326 derives a session key in the application processor 1212 that corresponds to the session key in the image sensor 1211, and verifies (verifies the integrity of) or decrypts the first protected data of the image data using the session key in the application processor 1212.

[0649] In the application processor 1212 configured in this manner, if the data to be verified is a count value, the data verification unit 1325 can verify its continuity. The data verification unit 1325 may also be configured to include a counter, and perform comparison and verification by updating the count value in the same way as the image sensor 1211. If the data to be verified is a random number, the data verification unit 1325 may verify the randomness of the data. The data verification unit 1325 may also include a nonce update unit 1309 (or a message counter), which may be used to verify or decrypt the first protected data, or which may be used to verify the data to be verified.

[0650] The image sensor 1211 and the application processor 1212 can be configured to be mounted on a desired mobile device. For example, the mobile device may be a portable mobile device such as a mobile phone, a smartphone, a digital camera, or a game console. The mobile device may be a propulsion device such as a vehicle, a robot, or a drone capable of propulsion (moving, running, walking, flying, etc.). The mobile device may be an autonomous vehicle, an autonomous robot, an autonomous drone, or the like equipped with an AI (artificial intelligence) function and capable of autonomous propulsion. The propulsion of the propulsion device may be controlled by a user of the propulsion device, and the propulsion device may notify the user of instructions or warnings as necessary. Alternatively, the propulsion device may be configured to automatically control the propulsion of the propulsion device.

[0651] The security units 1310 and 1326 may each include a security calculation unit that performs calculations to protect image data, for example. Therefore, the security units 1310 and 1326 can process any of encryption, decryption, hash value calculations, message authentication code calculations, digital signature calculations, ID (identification) authentication, firmware measurement, encryption session key establishment, key exchange, key update, etc., using the security calculation unit.

[0652] On the other hand, any of the security units 1310 and 1326, the nonce updating unit 1309, the message counter 1308, and the data verification unit 1325 may be configured to be electrically connected directly to the memory. This memory may be electrically connected directly to a register, and any of the security units 1310 and 1326, the nonce updating unit 1309, the message counter 1308, and the data verification unit 1325 may be electrically connected directly to a register. The memory may be a memory that is protected from either leakage or tampering of information in the memory. Such a memory and register are used as the storage units 1307 and 1324, respectively.

[0653] The storage units 1307 and 1324 may store any of key information (e.g., a pre-shared key, a private key, a public key, or a session key), a certificate (e.g., a root certificate, an intermediate certificate, or a leaf certificate), cryptographic algorithm information, etc. The storage units 1307 and 1324 may store any of function information of the image sensor 1211 or the application processor 1212, ID information of the image sensor 1211 or the application processor 1212 (e.g., a source ID, a destination ID, a final destination ID, etc.), firmware information of the image sensor 1211 or the application processor 1212, etc. The storage units 1307 and 1324 may store any of session information (e.g., a session ID), a calculation value of a security calculation unit (e.g., an initial value, an intermediate value, or a final value), an initialization vector, a nonce value, a message count value, a frame number (frame count value), etc., which will be described later.

[0654] Any of the security units 1310 and 1326, the nonce update unit 1309, the message counter 1308, and the data verification unit 1325 can determine whether or not there is a malfunction by, for example, the image sensor 1211 or the application processor 1212 storing multiple nonce values, count values, integrity calculation values, encryption information, etc. in the storage unit 1307 or 1324, and can take appropriate action (for example, requesting retransmission of data at the defective location or sending an abnormality message). Furthermore, if any of the nonce values, count values, integrity calculation values, encryption information, etc. is periodically stored in the protected storage unit 1307 or 1324, there is also the effect that, if an accident occurs to the mobile device, the cause of the accident can be more easily identified by analyzing the protected storage unit 1307 or 1324.

[0655] <Session> The requester and the responder, that is, the application processor 1212 and the image sensor 1211, can have one or more communication channels via a session. Below, the session will be described using an example in which the application processor 1212 is the requester and the image sensor 1211 is the responder. Of course, the application processor 1212 may be the responder and the image sensor 1211 may be the requester.

[0656] The requester and responder can also establish a secure communication channel using temporally fixed cryptographic information. Specifically, a session provides either encryption or message authentication, or both. A session may include, for example, three phases: a session handshake phase, an application phase, and a session termination phase.

[0657] The session handshake phase begins, for example, with a key exchange request (either PSK_EXCHANGE or KEY_EXCHANGE) from the requester, deriving session keys, such as a session secret and an encryption key, and using the session keys to secure communications. The purpose of this phase may be, for example, to first establish trust between the responder and requester before either side sends application data (e.g., image data). Additionally, it may ensure some integrity of the handshake and synchronization of the derived handshake secret.

[0658] The session may terminate immediately and proceed to session termination if an error occurs during this phase. If the handshake is successful, for example, by a finish response (FINISH_RSP or PSK_FINISH_RSP) from the responder, the application phase begins. The session reaches the application phase where either the responder or requester may send application data once the handshake is complete and all validations have passed.

[0659] The application phase ends when, for example, an end request (END_SESSION) is issued from the requester or an error occurs. The next phase is the session termination phase.

[0660] The session termination phase, for example, is purely an internal phase, with no explicit messages sent or received. Both the requestor and responder discard or clean up session keys, such as all derived session secrets and encryption keys, when the session ends. The requestor and responder may have other internal data associated with this session that they may also want to clean up.

[0661] The session secret is used, for example, to derive encryption keys and salts used in the Authenticated Encryption with Additional Data (AEAD) function. The derivation of the encryption keys may frequently use HMAC as defined in RFC 2104 and HKDF-Expand as described in RFC 5869. The session secret may consist of a single secret or multiple secrets. The session key may consist of a single key or multiple keys.

[0662] <Example of high-speed data transmission and low-speed command transmission processing> 78 to 80, a communication process in which high-speed data transmission and low-speed command transmission are performed between the image sensor 1211 and the application processor 1212 will be described.

[0663] FIG. 78 is a flowchart illustrating a first processing example of the communication processing.

[0664] Here, the extended mode-compatible CSI-2 receiver circuit 1322 of the application processor 1212 functions as a CCI host (requester) and a CSI-2 host. The extended mode-compatible CSI-2 transmitter circuit 1304 of the image sensor 1211 functions as a CCI device (responder) and a CSI-2 device. The CCI host transmits a request message to the CCI device, and in response to receiving the request message, the CCI device transmits a response message to the CCI host.

[0665] In step S501, a GET_VERSION request and a VERSION response are exchanged between the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. As a result, the extended mode-compatible CSI-2 receiver circuit 1322 acquires the SPDM (Security Protocol and Data Model) version of the endpoint.

[0666] In step S502, a GET_CAPABILITIES request and a CAPABILITIES response are exchanged between the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. As a result, the extended mode-compatible CSI-2 receiver circuit 1322 acquires the SPDM capabilities of the endpoint.

[0667] In step S503, a NEGOTIATE_ALGORITHMS request and an ALGORITHMS response are sent between the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. As a result, the extended mode-compatible CSI-2 receiver circuit 1322 negotiates an encryption algorithm with the extended mode-compatible CSI-2 transmitter circuit 1304.

[0668] In step S504, a PSK_EXCHANGE request and a PSK_EXCHANGE_RSP response are sent between the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. As a result, the extended mode-compatible CSI-2 receiver circuit 1322 and the extended mode-compatible CSI-2 transmitter circuit 1304 derive session keys for the CCI, such as a session secret and an encryption key.

[0669] In step S505, a PSK_FINISH request and a PSK_FINISH_RSP response are sent between the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. This proves to the responder that the extended mode-compatible CSI-2 receiver circuit 1322 knows the PSK (PSK: Pre-shared key) and that the session key for CCI derived in step S504 is correct.

[0670] In step S506, a PSK_EXCHANGE request and a PSK_EXCHANGE_RSP response are exchanged between the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. As a result, the extended mode-compatible CSI-2 receiver circuit 1322 and the extended mode-compatible CSI-2 transmitter circuit 1304 derive session keys for CSI-2, such as a session secret and an encryption key.

[0671] In step S507, a PSK_FINISH request and a PSK_FINISH_RSP response are sent between the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. This proves to the responder that the extended mode-compatible CSI-2 receiver circuit 1322 knows the PSK (PSK: Pre-shared key) and that the session key for CSI-2 derived in step S506 is correct.

[0672] Here, the session key authentication in steps S505 and S507 is achieved by a MAC value calculated using the requester's finished_key and the messages for this session, and the session key derived in steps S504 and S506 is used to protect subsequent CCI and CSI-2 communications.

[0673] In step S508, in the extended mode-compatible CSI-2 receiving circuit 1322, the CSI host supplies the CSI-2 session secret, session key, algorithm, other parameters, and the like to the CSI-2 host.

[0674] In step S509, in the extended mode compatible CSI-2 transmission circuit 1304, the CSI-2 session secret, session key, algorithm, other parameters, etc. are supplied from the CCI device to the CSI-2 device.

[0675] In step S510, the CSI-2 device in the extended mode-compatible CSI-2 transmitter circuit 1304 transmits image data by high-speed data communication to the CSI-2 host in the extended mode-compatible CSI-2 receiver circuit 1322. For example, the high-speed data communication continues until it is time to update the CSI-2 session key.

[0676] In step S511, a trigger for updating the CSI-2 session key is supplied from the CSI-2 host to the CCI host in the extended mode-compatible CSI-2 receiving circuit 1322. However, a trigger may be supplied from the CSI-2 device or CCI device to the CCI host, or a self-trigger may be supplied from the CCI host to the CCI host.

[0677] In step S512, a KEY_UPDATE request and a KEY_UPDATE_ACK response are exchanged between the CCI host of the extended-mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended-mode-compatible CSI-2 transmitter circuit 1304. This updates the session key and discards part of the old session key. If the session key is composed of multiple types of keys (such as a request direction key and a response direction key), some or all of them may be updated. The KEY_UPDATE request may also be issued from the responder using the GET_ENCAPSULATED_REQUEST mechanism described below.

[0678] In step S513, the same process as in step S512 is performed, and a KEY_UPDATE request and a KEY_UPDATE_ACK response are sent twice, thereby discarding the rest (all) of the old session keys that were not discarded by the process in step S512 alone.

[0679] In step S514, in the extended mode-compatible CSI-2 receiving circuit 1322, the CSI host supplies the CSI-2 session secret, session key (updated), algorithm, other parameters, and the like to the CSI-2 host.

[0680] In step S515, in the extended mode compatible CSI-2 transmission circuit 1304, the CSI device supplies the CSI-2 session secret, session key (updated), algorithm, other parameters, and the like to the CSI-2 device.

[0681] In step S516, similarly to step S510, transmission of image data by high-speed data communication is started, and thereafter, the same processes as in steps S510 to S515 are repeatedly performed.

[0682] In the first processing example of the communication processing, the session key for CCI and the session key for CSI-2 are different, the session IDs for CCI and CSI-2 are different, and the session secrets for CCI and CSI-2 are different. However, this is not limited to this, and as in the second processing example of the communication processing, the session key for CCI and the session key for CSI-2 may be the same, or the session IDs for CCI and CSI-2 may be the same, and the session secrets for CCI and CSI-2 may be the same.

[0683] FIG. 79 is a flowchart illustrating a second processing example of the communication processing.

[0684] In steps S521 to S523, the same processes as in steps S501 to S503 in FIG. 78 are performed.

[0685] In step S524, a PSK_EXCHANGE request and a PSK_EXCHANGE_RSP response are exchanged between the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. In this second processing example of the communication process, the same session secret is derived for the CCI and CSI-2.

[0686] That is, a session key for CCI and a session key for CSI-2 can be derived from the same session secret. Alternatively, a session key for uplink and a session key for downlink (the opposite direction of uplink) may be derived from the same session secret. Alternatively, a common session key for CCI and CSI-2 may be derived from the same session secret. Note that even if the session for CCI and CSI-2 is the same, the session secret and session keys for CCI and CSI-2 may be different.

[0687] Thereafter, in steps S525 to S534, the same processes as in steps S507 to S516 in FIG. 78 are performed.

[0688] Here, the pre-shared key (PSK) key exchange scheme provides an option for the requester and responder to perform mutual authentication and session key establishment using symmetric key cryptography. This option is particularly useful for endpoints that do not support asymmetric key cryptography or certificate processing. Even if asymmetric key cryptography is supported, this option can still be leveraged to speed up session key establishment. This option requires that the requester and responder know a common PSK in advance before the handshake.

[0689] Essentially, the PSK serves as the basis for mutual authentication credentials and session key establishment. As such, only the two endpoints and potentially a trusted third party that provisions the PSK to the two endpoints may know the value of the PSK. A requestor may pair with multiple responders. Similarly, a responder may pair with multiple requestors. A requestor-responder pair may be provisioned with one or more PSKs.

[0690] An endpoint may simultaneously act as a requestor to one device and a responder to another device. The transport layer must identify the peers and establish communication between the two endpoints before the PSK-based session key exchange can begin.

[0691] A PSK may be provisioned in a trusted environment, for example, during a secure manufacturing process. In an untrusted environment, a PSK may be agreed upon between two endpoints using a secure protocol. The size of the provisioned PSK depends on the security strength requirements of the application, but should be at least 128 bits, preferably 256 bits or more. During PSK provisioning, an endpoint may communicate its capabilities and supported algorithms to its peer. Therefore, the SPDM commands GET_CAPABILITIES and NEGOTIATE_ALGORITHMS are not required during session key establishment using the PSK option.

[0692] This option defines two message pairs: PSK_EXCHANGE / PSK_EXCHANGE_RSP and PSK_FINISH / PSK_FINISH_RSP. The PSK_EXCHANGE message has three roles: it prompts the responder to obtain a specific PSK, it exchanges context between the requester and the responder, and it proves to the requester that the responder knows the correct PSK and has derived the correct session key.

[0693] FIG. 80 is a flowchart illustrating a third processing example of the communication processing.

[0694] In steps S541 to S543, the same processes as in steps S501 to S503 in FIG. 78 are performed.

[0695] In step S544, a GET_DIGESTS request and a DIGESTS response are exchanged between the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. As a result, the extended mode-compatible CSI-2 receiver circuit 1322 obtains the certificate chain digest from the extended mode-compatible CSI-2 transmitter circuit 1304.

[0696] In step S545, a GET_CERTIFICATE request and a CERTIFICATE response are exchanged between the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. As a result, the extended mode-compatible CSI-2 receiver circuit 1322 obtains a certificate chain from the extended mode-compatible CSI-2 transmitter circuit 1304. Note that obtaining the certificate chain may be performed multiple times.

[0697] In step S546, a CHALLENGE request and CHALLENGE_AUTH response are exchanged between the CCI host of the extended mode-enabled CSI-2 receiver circuit 1322 and the CCI device of the extended mode-enabled CSI-2 transmitter circuit 1304. This allows the extended mode-enabled CSI-2 receiver circuit 1322 to authenticate the extended mode-enabled CSI-2 transmitter circuit 1304 through a challenge-response protocol.

[0698] In step S547, a KEY_EXCHANGE request (channel=CCI, sessionID=D) and a KEY_EXCHANGE_RSP response are exchanged between the CCI host in the extended-mode-enabled CSI-2 receiver circuit 1322 and the CCI device in the extended-mode-enabled CSI-2 transmitter circuit 1304. This initiates a handshake between the requester and responder for the purpose of authenticating the responder (or, optionally, both parties). Encryption parameters are then negotiated in addition to those negotiated in the final NEGOTIATE_ALGORITHMS / ALGORITHMS exchange, and shared key information is established.

[0699] In step S548, the CCI host of the extended mode-capable CSI-2 receiver circuit 1322 sends a GET_ENCAPSULATED_REQUEST to the CCI device of the extended mode-capable CSI-2 transmitter circuit 1304.

[0700] In step S549, the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304 transmits an ENCAPSULATED_REQUEST (GET_DIGESTS request) to the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322.

[0701] In step S550, the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 sends a DELIVER_ENCAPSULATED_RESPONSE (DIGESTS response) to the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. As a result, the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304 obtains the certificate chain digest from the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322.

[0702] In step S551, the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304 transmits an ENCAPSULATED_RESPONSE_ACK (GET_CERTIFICATE request) to the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322.

[0703] In step S552, the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322 sends a DELIVER_ENCAPSULATED_RESPONSE (CERTIFICATE response) to the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304. This allows the CCI device (responder) to obtain a certificate chain from the CCI host (requester). This process may be performed multiple times.

[0704] In step S553, the CCI device of the extended mode-compatible CSI-2 transmitter circuit 1304 transmits an ENCAPSULATED_RESPONSE_ACK to the CCI host of the extended mode-compatible CSI-2 receiver circuit 1322.

[0705] In step S554, a FINISH request and a FINISH_RSP response are sent between the CCI host of the extended mode-capable CSI-2 receiver circuit 1322 and the CCI device of the extended mode-capable CSI-2 transmitter circuit 1304. This completes the handshake between the CCI host of the extended mode-capable CSI-2 receiver circuit 1322 and the CCI device of the extended mode-capable CSI-2 transmitter circuit 1304, which was started by the KEY_EXCHANGE request in step S547.

[0706] In step S555, a GET_MEASUREMENTS request and a MEASUREMENTS response are exchanged between the CCI host of the extended mode-enabled CSI-2 receiver circuit 1322 and the CCI device of the extended mode-enabled CSI-2 transmitter circuit 1304. As a result, the CCI host of the extended mode-enabled CSI-2 receiver circuit 1322 acquires measurement data from the CCI device of the extended mode-enabled CSI-2 transmitter circuit 1304. Note that the GET_MEASUREMENTS request may be issued by the responder using the GET_ENCAPSULATED_REQUEST mechanism described above. Similarly, other requests may also be issued by the responder using the GET_ENCAPSULATED_REQUEST mechanism described above.

[0707] Thereafter, in step S556, a KEY_EXCHANGE request (channel=CSI-2, sessionID=E) and a KEY_EXCHANGE_RSP response are made in the same manner as in step S547, and in step S557, a FINISH request and a FINISH_RSP response are made in the same manner as in step S554. Then, in steps S558 to S566, the same processes as in steps S508 to S516 in Fig. 78 are performed.

[0708] <Data verification process> The data verification process using a verification packet and a packet to be verified will be described with reference to FIGS.

[0709] As shown in Figures 81 and 82, an extended packet consists of a packet header PH, an extended packet header ePH, packet data, an extended packet footer ePF, and a packet footer PF. An extended packet with this configuration can contain a frame start, embedded data, image data, user-defined data, a frame end, a write command (CCI Write), a read command (CCI Read), and a read response (CCI Read return value). Note that the packet header PH, extended packet header ePH, packet data, extended packet footer ePF, and packet footer PF may be omitted in part or in whole. In other words, a packet configuration including at least the extended packet header ePH and packet data is defined as an extended packet.

[0710] Incidentally, there is a possibility that any of the extended packet header ePH, packet data, and extended packet footer ePF may not be received correctly (the message may be lost) due to noise, interference, or attacks. Therefore, it is desirable that a verification packet for verifying the integrity of the extended packet header ePH, packet data, and the remaining part of the extended packet footer ePF1 be stored in the end of the extended packet footer ePF0. For example, a cyclic redundancy check (CRC32), which is a type of error detection code, is used to verify the integrity. The generating polynomial of CRC32 is, for example, X 32 +X 26 +X 23 +X 22 +X 16 +X 12 +X 11 +X 10 +X 8 +X 7 +X 5 +X 4 +X 2 +X+1 is used.

[0711] The packet to be verified may include packet data. Alternatively, the packet to be verified may include an extended packet header and packet data. Alternatively, the packet to be verified may include packet data and the remaining extended packet footer (ePF1). Alternatively, the packet to be verified may include an extended packet header, packet data, and the remaining extended packet footer (ePF1). At least the packet data is protected by such a packet to be verified.

[0712] That is, the image sensor 1211 includes a second protection unit (e.g., a CRC calculation unit) that generates second protection data (e.g., a CRC calculation value) for packet data without using a session key. The second protection data is stored, for example, in the extended packet footer ePF for high-speed data transmission. That is, the second protection data is stored in any of the frame start, embedded data, image data, user-defined data, frame end, write command (CCI Write), read command (CCI Read), and read response (CCI Read return value).

[0713] The extended packet footers ePF1 and ePF0 may define a security feature. That is, the image sensor 1211 may include a security calculation unit (e.g., an encryption calculation unit, a decryption calculation unit, a hash value calculation unit, a message authentication code calculation unit, and a digital signature calculation unit). The results of the security calculation (e.g., a hash value, a message authentication code, and a digital signature) may be stored in the extended packet footer ePF.

[0714] The result of the security calculation may be stored only in the extended packet footer ePF1 and not in the extended packet footer ePF0, or may be stored outside the extended packet footer and not in the extended packet footer (for example, in the embedded data or the read response). The security calculation unit provided in the image sensor 1211 is included in the security unit 1310.

[0715] The message authentication code (MAC) may be any of Galois MAC (GMAC), Cipher-based MAC (CMAC), Hash-based MAC (HMAC), etc. For example, any of AES-GMAC, AES-CMAC, SHA2-HMAC, SHA3-HMAC, etc., to which AES (Advanced Encryption Standard) or SHA (Secure Hash Algorithm) is applied, may be used. The block length of AES is 128 bits, and the key length of AES is selected from 128 bits, 192 bits, and 256 bits.

[0716] The extended packet footer may store any security information such as a hash (particularly a cryptographic hash) value, a message authentication code, or a digital signature, with the packet data as the packet to be verified, or with the extended packet header and packet data as the packet to be verified. In this case, it is possible to provide further resistance to malicious tampering by an attacker. Note that the extended packet footer "ePF1" or "ePF1 and ePF0" may store a CRC of a cyclic redundancy check, which is a type of error detection code.

[0717] That is, the image sensor 1211 may include an integrity calculation unit (e.g., the first protection unit=security calculation unit, the second protection unit=CRC calculation unit), and an integrity calculation value (e.g., first protected data, second protected data) resulting from the integrity calculation may be stored in the extended packet footer. Note that the CRC can be used for functional safety, and the integrity can be used to prevent hardware failures from going undetected. On the other hand, the integrity of the security function can be used to detect intentional interference or attacks. That is, the security calculation unit calculates an integrity calculation value based on cryptography, and the CRC calculation unit calculates an integrity calculation value that is not based on cryptography.

[0718] The application processor 1212 can verify the integrity of the packet to be verified, for example, by using a verification packet. If an abnormality is determined, any of the following processes may be executed: sending a request message requesting retransmission of a packet including the packet to be verified and the verification packet; sending a request message inquiring of the image sensor 1211 about whether there is an abnormality in the image sensor 1211; sending a request message requesting the image sensor 1211 to stop some or all of the functions of the image sensor 1211; stopping the propulsion of the propulsion device; changing the propulsion control of the propulsion device; or changing priority data used for propulsion control.

[0719] The integrity calculation value may be stored, for example, in the embedded data, image data (packet data), user-defined data, write command, read command, read response, etc. In this case, the integrity calculation value may not be stored in the extended packet footer. For example, the integrity calculation value may be stored in units of image frames rather than units of image lines, in which case the integrity is calculated efficiently. In this case, the integrity calculation value is stored, for example, in the embedded data or read response after the image data has been transmitted.

[0720] The extended packet shown in A of Figure 81 is an example of a configuration in which the packet to be verified is an extended packet header ePH, packet data, and the remaining extended packet footer ePF1, and the verification packet is an extended packet footer end ePF0 that stores a calculated value obtained by security calculation using the packet to be verified.

[0721] The extended packet shown in B of Figure 81 is an example of a configuration in which the packet data and the remaining ePF1 of the extended packet footer are used as the packet to be verified, and the end ePF0 of the extended packet footer, which stores the calculated value obtained by security calculation using the packet to be verified, is used as the verification packet.

[0722] The extended packet shown in C of Figure 81 is an example of a configuration in which the extended packet header ePH and packet data are used as the packet to be verified, and the verification packet is an extended packet footer end ePF0 that stores a calculated value obtained by security calculation using the packet to be verified.

[0723] The extended packet shown in D of Figure 81 is an example of a configuration in which packet data is used as the packet to be verified, and the verification packet is the end ePF0 of the extended packet footer, which stores the calculated value obtained by security calculation using the packet to be verified.

[0724] The extended packet shown in A of Figure 82 is an example of a configuration in which the extended packet header ePH and packet data are used as the packet to be verified, and the extended packet footer remainder ePF1, which stores a calculated value obtained by security calculation using the packet to be verified, is used as the verification packet.

[0725] The extended packet shown in B of Figure 82 is an example of a configuration in which the extended packet header ePH and packet data are used as the packet to be verified, and the extended packet footer remainder ePF1 and extended packet footer end ePF0, which store the calculated value obtained by security calculation using the packet to be verified, are used as the verification packet.

[0726] The extended packet shown in C of Figure 82 is an example of a configuration in which packet data is used as the packet to be verified, and the verification packet is the extended packet footer remaining ePF1, which stores a calculated value obtained by security calculation using the packet to be verified.

[0727] The extended packet shown in D of Figure 82 is an example of a configuration in which packet data is used as the packet to be verified, and the verification packet is the extended packet footer remainder ePF1 and extended packet footer end ePF0, which store the calculated value obtained by security calculation using the packet to be verified.

[0728] FIG. 83 is a flowchart illustrating the data verification process performed by the application processor 1212.

[0729] In step S601, when the extended mode-compatible CSI-2 receiving circuit 1322 receives an extended packet transmitted from the image sensor 1211, the security unit 1326 receives the packet to be verified of that extended packet. When the security unit 1326 completes reception of the packet to be verified, the process proceeds to step S602. Note that even if reception of the entire packet to be verified has not been completed, as long as reception of at least a portion (e.g., 128 bits) sufficient to start security calculations has been completed, the process may proceed to step S602. In this case, the remainder of the packet to be verified continues to be received until reception of the entire packet to be verified has been completed.

[0730] In step S602, the security unit 1326 starts calculating a value obtained by security calculation using at least a part of the packet to be verified received in step S601.

[0731] In step S603, the security unit 1326 receives the verification packet transmitted from the image sensor 1211 via the extended mode-compatible CSI-2 receiving circuit 1322. Then, when the security unit 1326 completes reception of the verification packet and acquires the received value stored in the verification packet (the value calculated by the image sensor 1211), the process proceeds to step S604.

[0732] In step S604, when the security unit 1326 completes the calculation of the calculated value obtained by the security calculation using the packet to be verified that started in step S602 (i.e., the entire packet to be verified has been received and the calculation using all of it has been completed), the processing proceeds to step S605.

[0733] In step S605, the security unit 1326 determines whether the value received in step S603 matches the value calculated in step S604.

[0734] If the security unit 1326 determines in step S605 that the received value and the calculated value match, the process proceeds to step S606. In this case, in step S606, the security unit 1326 determines that the extended packet received by the extended mode-compatible CSI-2 receiving circuit 1322 is normal, and the process ends.

[0735] On the other hand, if the security unit 1326 determines in step S605 that the received value and the calculated value do not match, the process proceeds to step S607. In this case, in step S607, the security unit 1326 determines that an abnormality has occurred in the extended packet received by the extended mode-compatible CSI-2 receiving circuit 1322, and the process ends.

[0736] <Ensuring functional safety using message count values> In order to ensure functional safety (for example, detecting a missing message and taking appropriate action), the image sensor 1211 can store the message count value counted by the message counter 1308 in an extended packet header or an extended packet footer. For example, the message counter 1308 included in the image sensor 1211 can store a message count value that is incremented or decremented each time a message is transmitted from the image sensor 1211. Note that the image sensor 1211 may be configured to have an independent message counter 1308 for each virtual channel, or may be configured to have a message counter 1308 that is common to all virtual channels.

[0737] Message counter 1308 sets the message count value to an initial value (e.g., 0 or the maximum value) in the first packet including an extended packet header of a certain virtual channel, and increments or decrements the message count value each time data including an extended packet header of a certain virtual channel is transmitted. Also, for example, when data not including an extended packet header is transmitted, message counter 1308 does not increment or decrement the message count value, and restarts counting the next time data including an extended packet header is transmitted.

[0738] The message counter 1308 may continue counting regardless of whether it is a frame start or a frame end. When the message count value reaches a specified value (for example, the maximum value or 0), the message counter 1308 resets the message count value to the initial value (for example, 0 or the maximum value) and continues counting. Note that part of the extension packet header may contain part of the nonce value.

[0739] Furthermore, the receiving side (image sensor 1211 or application processor 1212) that receives the message count value can immediately detect any missing messages. For example, a denial-of-service (DoS) attack that compromises the availability of the image sensor 1211 or application processor 1212 by intentionally including a huge number of messages can also be immediately detected on the receiving side. For this reason, it is desirable to store the message count value in an extended packet header. By being able to detect such missing messages or attacks in a shorter time, the receiving side can begin responding to them in a shorter time, which is particularly suitable for, for example, a propulsion device that is capable of high-speed movement or high-speed operation.

[0740] Note that a write command (CCI Write), a read command (CCI Read), or a read response (CCI Read return value) may also be configured to store a message count value or an integrity calculation value, and elements related to extended packets may be applied. In this case, it becomes possible to support functional safety and protect the integrity of the write command, read command, or read response.

[0741] FIG. 84 is a flowchart illustrating a message count value transmission process in which the image sensor 1211 transmits a message count value.

[0742] In step S611, the message counter 1308 initializes the message count value to zero.

[0743] In step S612, the extended mode-compatible CSI-2 transmitter circuit 1304 determines whether to transmit an extended packet header, and the process waits until it determines to transmit an extended packet header. If the extended mode-compatible CSI-2 transmitter circuit 1304 determines to transmit an extended packet header in step S612, the process proceeds to step S613.

[0744] In step S613, the extended mode compatible CSI-2 transmission circuit 1304 obtains the message count value from the message counter 1308 and stores it in the extended packet header.

[0745] In step S614, the extended mode-compatible CSI-2 transmission circuit 1304 transmits the extended packet header in which the message count value was stored in step S613.

[0746] In step S615, message counter 1308 determines whether the message count value has reached its maximum value. If message counter 1308 determines in step S615 that the message count value has not reached its maximum value, the process proceeds to step S616.

[0747] In step S616, the message counter 1308 increments the message count value, after which the process returns to step S612, and the same process is repeated thereafter.

[0748] On the other hand, if it is determined in step S615 that the message counter 1308 has counted up to the maximum message count value, the process returns to step S611, where the message count value is initialized, and the same process is then repeated.

[0749] In addition to incrementing the message count value in this way, for example, the message count value may be initialized to a maximum value and then decremented.

[0750] <About embedded data> The embedded data will be described with reference to FIGS.

[0751] Image sensor 1211 can include additional information, such as device configuration information, in the data stream by using embedded data. The embedded data consists of one or more lines and can include any of the following: configuration data for image sensor 1211, standard-compliant register values, vendor-specific register values, frame format descriptions, statistics, etc.

[0752] A in Figure 85 shows one line of embedded data, which consists of an embedded data format code followed by a desired amount of embedded data, with padding characters placed in the remainder.

[0753] The embedded data includes information related to the image data or user-defined data. Therefore, while the image data or user-defined data may be compressed, the embedded data is preferably uncompressed data (non-compressed data). Therefore, when data compression is used, a frame of high-speed data transmission will contain a mixture of compressed data (image data or user-defined data) and non-compressed data (embedded data).

[0754] The embedded data may have multiple lines (rows) of embedded data depending on the number of register values ​​added to the embedded data. The number of lines of embedded data may be specified as part of the frame format description for the first embedded data row in a frame. The line length of the embedded data may be shorter than the line length of the image data or user-defined data, but preferably does not exceed the line length of the image data or user-defined data, and is preferably the same as the line length of the image data or user-defined data. The first pixel value of the embedded data may indicate the format used for the embedded data.

[0755] Part or all of the nonce value may be transmitted stored in at least a portion of the embedded data indicating a vendor specific code or a reserved code (reserved for future use) as shown in B of Figure 85. Within a frame, the embedded data is stored either between the frame start and the first image data or user-defined data, or between the last image data or user-defined data and the frame end. However, the embedded data between the last image data or user-defined data and the frame end may be omitted.

[0756] FIG. 86 shows an example of the data structure of two frames of image data transmitted from the image sensor 1211.

[0757] As shown in FIG. 86, after a frame start (VC1 FS) of the first virtual channel is transmitted, a read command and a read response are followed by a frame start (VC2 FS) of the second virtual channel. Next, first embedded data (VC1 Emb Data) of the first virtual channel and first embedded data (VC2 Emb Data) of the second virtual channel are transmitted. Then, one frame's worth of image data (VC1 Image Data) of the first virtual channel and user-defined data (VC2 UD Data) of the second virtual channel are transmitted. After transmission of one frame is complete, second embedded data (VC1 Emb Data) of the first virtual channel and second embedded data (VC2 Emb Data) of the second virtual channel are transmitted. Then, a frame end (VC1 FE) of the first virtual channel is transmitted, followed by a read command and a read response, followed by a frame end (VC2 FE) of the second virtual channel.

[0758] 86 shows an example in which the message count value is shared between the first virtual channel and the second virtual channel. In this case, it is also possible to configure the first virtual channel and the second virtual channel to have independent message counters. Furthermore, the user-defined data may be image data or the like.

[0759] Here, a part or all of the nonce value is stored, for example, within the period from the frame start to the frame end, or within the period from the frame end to the frame start (frame blanking period). Furthermore, within the period from the frame start to the frame end, the nonce value can be stored, for example, within the embedded data, image data, non-image data, or within the line blanking period. It may also be stored in a second virtual channel.

[0760] By defining the frame start and frame end, for example, the image sensor can notify the processor of the start and end of high-speed data transmission. It also enables the image sensor to maintain a constant frame transmission cycle. The embedded data contains attributes that represent the image data and information (metadata) related to the image data.

[0761] In this embodiment, an example will be described in which high-speed data transmission of a nonce value is executed without interfering with high-speed data transmission of image data. That is, an example will be described in which high-speed data transmission of image data and high-speed data transmission of a nonce value are executed serially rather than in parallel. However, if the communication paths for high-speed data transmission of image data and transmission of a nonce value (high-speed data transmission or low-speed command transmission) are different, they may be executed in parallel.

[0762] Note that high-speed data transmission and low-speed command transmission can be frequency-separated using a filter, so as long as power consumption is not an issue, some or all of the transmissions may overlap (be executed in parallel). Some or all of the nonce values ​​may be transmitted every several frames, but it is preferable to transmit them every frame, for example, due to frame loss. For example, a Frame Start (FS) packet contains a Frame Start Code (Data Type = 0x00), and a Frame End (FE) packet contains a Frame End Code (Data Type = 0x01).

[0763] FIG. 87 is a flowchart illustrating an image data transmission process in which the image sensor 1211 transmits image data.

[0764] In step S621, the extended mode-compatible CSI-2 transmitter circuit 1304 determines whether a command to start high-speed data transmission has been received, and waits until it determines that the command to start high-speed data transmission has been received. If the extended mode-compatible CSI-2 transmitter circuit 1304 determines that the command to start high-speed data transmission has been received in step S621, the process proceeds to step S622.

[0765] In step S622, the pixel 1301 starts capturing an image, and the image data output from the pixel 1301 is supplied to the extended mode compatible CSI-2 transmission circuit 1304 via the AD converter 1302 and the image processing unit 1303.

[0766] In step S623, the extended mode-compatible CSI-2 transmission circuit 1304 transmits a frame start for the first virtual channel.

[0767] In step S624, the extended mode-compatible CSI-2 transmission circuit 1304 transmits a frame start for the second virtual channel.

[0768] In step S625, the enhanced mode-compatible CSI-2 transmission circuit 1304 transmits the first embedded data of the first virtual channel.

[0769] In step S626, the enhanced mode-compatible CSI-2 transmission circuit 1304 transmits the first embedded data of the second virtual channel.

[0770] In step S627, the extended mode-compatible CSI-2 transmission circuit 1304 transmits image data on the first virtual channel.

[0771] In step S628, the extended mode-compatible CSI-2 transmission circuit 1304 transmits the user-defined data of the second virtual channel.

[0772] In step S629, the extended mode-compatible CSI-2 transmission circuit 1304 determines whether transmission of one frame of image data has been completed.

[0773] If the extended mode-compatible CSI-2 transmission circuit 1304 determines in step S629 that the transmission of one frame of image data has not been completed, the process returns to step S627, and the same process is repeated thereafter.On the other hand, if the extended mode-compatible CSI-2 transmission circuit 1304 determines in step S629 that the transmission of one frame of image data has been completed, the process proceeds to step S630.

[0774] In step S630, the enhanced mode-compatible CSI-2 transmission circuit 1304 transmits the second embedded data of the first virtual channel.

[0775] In step S631, the extended mode-compatible CSI-2 transmission circuit 1304 transmits the second embedded data of the second virtual channel.

[0776] In step S632, the extended mode-compatible CSI-2 transmission circuit 1304 transmits a frame end signal for the first virtual channel.

[0777] In step S633, the extended mode-compatible CSI-2 transmission circuit 1304 transmits a frame end signal for the second virtual channel.

[0778] In step S634, the extended mode-compatible CSI-2 transmission circuit 1304 determines whether or not a command to end high-speed data transmission has been received.

[0779] If the extended mode-compatible CSI-2 transmitter circuit 1304 determines in step S634 that it has not received a command to end high-speed data transmission, the process returns to step S622, and the same process is repeated from step S622 onward. On the other hand, if the extended mode-compatible CSI-2 transmitter circuit 1304 determines in step S634 that it has received a command to end high-speed data transmission, the process ends.

[0780] The imaging may be continuously started until a command to end high-speed data transmission is received, or may be started every time a command to start high-speed data transmission is received.

[0781] FIG. 88 is a flowchart illustrating an integrity calculation value transmission process in which the image sensor 1211 transmits the integrity calculation value.

[0782] In step S641, the security unit 1310 derives a session key for the first virtual channel.

[0783] In step S642, security unit 1310 derives a session key for the second virtual channel.

[0784] In step S643, the message counter 1308 initializes the upper count value of the message count value to zero.

[0785] In step S644, the message counter 1308 initializes the lower count value of the message count value to zero.

[0786] In step S645, the extended mode-compatible CSI-2 transmission circuit 1304 determines whether or not to end the session, and if it is determined not to end the session, the process proceeds to step S646.

[0787] In step S646, the extended mode compatible CSI-2 transmission circuit 1304 determines whether or not to transmit an extended packet of the first virtual channel.

[0788] If the extended mode-compatible CSI-2 transmitter circuit 1304 determines in step S646 not to transmit an extended packet of the first virtual channel, the process returns to step S645, and the same process is repeated thereafter. On the other hand, if the extended mode-compatible CSI-2 transmitter circuit 1304 determines in step S646 to transmit an extended packet of the first virtual channel, the process proceeds to step S647.

[0789] In step S647, the security unit 1310 calculates an integrity operation value for the first virtual channel using the session key for the first virtual channel derived in step S641.

[0790] In step S648, the extended mode compatible CSI-2 transmission circuit 1304 places the integrity calculation value calculated in step S647 in the extended packet of the first virtual channel and transmits the extended packet of the first virtual channel.

[0791] In step S649, the extended mode-compatible CSI-2 transmitter circuit 1304 determines whether to transmit an extended packet of the second virtual channel and waits until it determines to transmit an extended packet of the second virtual channel. If the extended mode-compatible CSI-2 transmitter circuit 1304 determines in step S649 to transmit an extended packet of the second virtual channel, the process proceeds to step S650.

[0792] In step S650, security unit 1310 calculates an integrity calculation value for the second virtual channel using the session key for the second virtual channel derived in step S642.

[0793] In step S651, the extended mode compatible CSI-2 transmission circuit 1304 places the integrity calculation value calculated in step S650 in the extended packet of the second virtual channel, and transmits the extended packet of the second virtual channel.

[0794] In step S652, the message counter 1308 determines whether the lower count value of the message count value has counted up to the maximum value.

[0795] If message counter 1308 determines in step S652 that the lower count value of the message count value has not yet reached the maximum value, the process proceeds to step S653. In step S653, message counter 1308 increments the lower count value of the message count value, and then the process returns to step S645, where the same process is repeated.

[0796] On the other hand, if message counter 1308 determines in step S652 that the lower count value of the message count value has reached the maximum value, the process proceeds to step S654. In step S654, message counter 1308 increments the upper count value of the message count value, and then the process returns to step S644, and the same process is repeated thereafter.

[0797] If the extended mode-compatible CSI-2 transmission circuit 1304 determines in step S645 that the session should be ended, the process proceeds to step S655.

[0798] In step S655, the security unit 1310 discards or cleans up the session keys for the first virtual channel and the second virtual channel, after which the process ends.

[0799] <Modification of the data structure of image data> The data structure of image data will be described with reference to FIGS.

[0800] FIG. 89 shows a first modified example of the data structure of image data.

[0801] In the data structure of the image data shown in FIG. 89, a message count value common to the first virtual channel and the second virtual channel is used.

[0802] However, the session key or message counter may be shared between the first virtual channel and the second virtual channel. Furthermore, the image data or embedded data may be replaced with other data. For example, the embedded data may be replaced with image data. Meanwhile, the message counter may be shared by counting across virtual channels (VCs).

[0803] FIG. 90 shows a second modified example of the data structure of the image data.

[0804] In the data structure of the image data shown in FIG. 90, Write (CCI write command), Read1 (CCI read command), and Read2 (CCI read response) each use an independent message count value.

[0805] FIG. 91 shows a third modified example of the data structure of image data.

[0806] In the data structure of the image data shown in Fig. 91, independent message count values ​​are provided for the CCI uplink (Write and Read1) and the CCI downlink (Read2). In other words, the message count value may be common to Write (CCI write command) and Read1 (CCI read command).

[0807] <About nonce values> The nonce value is used once for the same session key, and is therefore used as part or all of an initialization vector for an encryption or decryption operation that uses the session key. Therefore, the nonce used by image sensor 1211 for an encryption operation is transmitted from image sensor 1211 and received by application processor 1212, allowing application processor 1212 to obtain the nonce value required for the decryption operation.

[0808] That is, it is desirable for the image sensor 1211 to transmit a nonce value before transmitting image data. Specifically, a part or all of the nonce value corresponding to image data in a certain frame is stored in any of the following locations: a read response, user-defined data, embedded data (immediately after the image data), frame end, frame start, embedded data (immediately before the image data), etc., from after the transmission of the last image data in the previous frame has been completed until before the transmission of the first image data in the frame starts.

[0809] For example, the application processor 1212, which is the master of the low-speed command transmission, may transmit a read command by low-speed command transmission requesting the application processor 1212 to read the nonce value in the image sensor 1211 in response to the start or completion of reception of any of the following data transmitted by high-speed data transmission from the image sensor 1211, which is the slave of the low-speed command transmission: a frame start, embedded data, image data, user-defined data, or frame end.

[0810] The image sensor 1211 receives a read command transmitted from the application processor 1212 and transmits a corresponding nonce value by high-speed data transmission. Then, the application processor 1212 receives a read response, allowing the image sensor 1211 to notify the application processor 1212 of the nonce value.

[0811] Because the nonce value notified from the image sensor 1211 is used within the application processor 1212, it is desirable that part or all of the nonce value be transmitted during the frame blanking period when image data is not transmitted between the end of a frame and the start of the next frame. However, for the first frame (Frame Number = 1), the image sensor 1211 and application processor 1212 must agree on the initial nonce value (initial value) in advance, or part or all of the initial nonce value must be received by the application processor 1212 before the start of image data transmission.

[0812] This read command corresponds to, for example, "Read" in the Read / Write section of the I2C or I3C standard. Meanwhile, the read response corresponds to a "Read return value." To adjust the timing of the read response, a timer may be provided to wait a predetermined time between when the application processor 1212 receives the high-speed data transmission and when it transmits the read command.

[0813] <I2CおよびI3Cについて> I2C bus or I 2 The Inter-Integrated Circuit Serial Bus, sometimes referred to as the I2C bus, is a serial single-ended computer bus intended for use in connecting low-speed peripherals to the application processor 1212. The I2C bus is a multi-master bus in which each device can act as both a master and a slave for various messages sent on the I2C bus.

[0814] The I2C bus can transmit data using only two bidirectional open-drain connectors, including a serial data line (SDA) and a serial clock line (SCL). These connectors typically contain signal lines terminated by pull-up resistors. The protocol that governs the operation of the I2C bus defines basic types of messages, each initiated with a START and terminated with a STOP. The I2C bus uses 7-bit addressing and defines two types of nodes:

[0815] A master node is a node that generates the clock and initiates communication with slave nodes. A slave node is a node that receives the clock and responds when addressed by a master. The I2C bus is a multi-master bus, which means that there can be any number of master nodes. Furthermore, the roles of master and slave may change between messages (i.e., after a STOP is sent). In this embodiment, which is a camera implementation, unidirectional transmission may be used to capture images from the sensor and transmit such image data to memory in the baseband processor, while control data may be exchanged between the baseband processor and the sensor as well as other peripheral devices.

[0816] In one example, a Camera Control Interface (CCI) protocol may be used for such control data between the baseband processor and the image sensor (or one or more slave nodes). In one example, the CCI protocol may be implemented over an I2C serial bus between the image sensor and the baseband processor. Traditional I2C systems, i.e., camera control interface-based camera systems, use a separate interrupt (IRQ) line for each slave device to allow the slave node to indicate to the master node that it wishes to use the bus.

[0817] On the other hand, the I3C communication standard is a standard that communicates via two signal lines: an SDA line that transmits data and an SCL line that transmits a clock signal. In this standard, devices (such as processors) are classified into devices that operate as either masters or slaves, and devices that operate only as slaves. For example, a processor can operate as either a master or slave, and a sensor can operate only as a slave.

[0818] Here, a master is a device that controls a slave, and a slave is a device that operates under the control of the master. Furthermore, I3C allows multiple slaves to be connected to one master. Furthermore, multiple masters can send signals to one slave; this type of communication is hereinafter referred to as "multi-master communication." Furthermore, slaves can communicate with each other without going through a master; this type of communication is called "peer-to-peer communication." Furthermore, a slave can interrupt communication while the SDA line is busy with another device's communication; this interrupt is called an "in-band interrupt."

[0819] In the above-mentioned multi-master communication, in-band interrupt, and peer-to-peer communication, signals sent simultaneously by multiple devices can collide on the SDA line. For example, if a master is sending a signal to a slave and another slave sends a signal to the master via an in-band interrupt, the signals from the master and slave will collide. For this reason, I3C devices have the ability to detect collisions and arbitrate between devices.

[0820] The use of the interrupt function described above makes it easy to synchronize with the application processor 1212, so that by executing an interrupt at a timing determined by the image sensor 1211, nonce-related information is transmitted according to the timing determined by the image sensor 1211. However, the image sensor 1211 may trigger a read command by an in-band interrupt and transmit a read response accordingly, or may omit the read command and transmit a read response by an in-band interrupt.

[0821] <Integrity calculation value processing> The integrity calculation value processing will be described with reference to FIGS.

[0822] FIG. 92 is a flowchart illustrating a first example of the integrity calculation value process in which the image sensor 1211 transmits the integrity calculation value.

[0823] In step S661, the security unit 1310 derives a session key.

[0824] In step S662, the message counter 1308 initializes the message count value to zero.

[0825] In step S663, the extended mode-compatible CSI-2 transmission circuit 1304 determines whether or not to end the session, and if it is determined not to end the session, the process proceeds to step S664.

[0826] In step S664, the extended mode compatible CSI-2 transmission circuit 1304 determines whether or not to transmit an extended packet.

[0827] If the extended mode-compatible CSI-2 transmission circuit 1304 determines in step S664 not to transmit an extended packet, the process returns to step S663, and the same process is repeated thereafter. On the other hand, if the extended mode-compatible CSI-2 transmission circuit 1304 determines in step S664 to transmit an extended packet, the process proceeds to step S665.

[0828] In step S665, the security unit 1310 uses the message count value to calculate an integrity calculation value.

[0829] In step S666, the extended mode compatible CSI-2 transmission circuit 1304 places the integrity calculation value calculated in step S665 in the extended packet and transmits the extended packet.

[0830] In step S667, message counter 1308 determines whether the message count value has reached its maximum value. If message counter 1308 determines in step S667 that the message count value has not reached its maximum value, the process proceeds to step S668.

[0831] In step S668, the message counter 1308 increments the message count value, after which the process returns to step S663, and the same processes are repeated thereafter.

[0832] On the other hand, if it is determined in step S667 that message counter 1308 has counted up to the maximum message count value, the process proceeds to step S669. In step S669, security unit 1310 updates the session key, and then the process returns to step S662, whereupon the same processes are repeated.

[0833] If the extended mode-compatible CSI-2 transmission circuit 1304 determines in step S663 that the session should be terminated, the process proceeds to step S670.

[0834] In step S670, the security unit 1310 discards or cleans up the session key, after which the process ends.

[0835] In this way, when calculating the MAC value for each image line and storing it in the extended packet footer before sending it, the message count value is incremented by 1 each time an extended packet is sent, so 2 16 For example, if you are sending 4K data with a pixel count of 4096 x 2160 (horizontal x vertical) at a frame rate of 60 fps, and you are sending an extended packet of 2163 lines, which is the sum of the three lines for the frame start, embedded data, and frame end, within one frame, the message count value will go around once every 2 16 The message count value goes around in 0.5 seconds.

[0836] For example, if the image sensor 1211 calculates a MAC value, such as a Galois Message Authentication Code (GMAC) value, for a message using the same session key and the same initialization vector value and then transmits the message and MAC value, an attacker can easily obtain the session key by calculating the simultaneous equations for the message and MAC value. In this case, the attacker can freely tamper with the MAC value, enabling attacks such as message spoofing, tampering, and replay. Therefore, if the message count value is used as the variable part of the initialization vector, i.e., the nonce value, the session key must be updated every time the message count value wraps around. For example, the session key can be updated by utilizing the frame blanking or line blanking period before the nonce value wraps around.

[0837] FIG. 93 is a flowchart illustrating a second example of the integrity calculation value process in which the image sensor 1211 transmits the integrity calculation value.

[0838] In step S681, the se...

Claims

1. a protection unit that protects the first communication and the second communication, The protective part is deriving a first secret from a key schedule using the first communication; deriving a first session key associated with the first secret; using the first session key for operations to protect the first communication; The protective part is receiving a second session key using the first communication protected by the first session key, or deriving a second secret from the key schedule to derive a second session key associated with the second secret; an initialization vector including a source ID value, a virtual channel or extended virtual channel value, a frame counter value or an additional frame number value, and the received or derived second session key for use in an operation to protect the second communication; 1. An information processing device comprising:

2. the second communication is transmitting or receiving a frame including an extended packet header and packet data; The second communication protected by the second session key is transmitted or received with a first MAC value calculated using the second session key on at least a part or all of the extended packet header and a part or all of the packet data stored in a first extended packet footer at a frame end in the frame. The information processing device according to claim 1 .

3. The second communication protected by the second session key is transmitted or received with a second MAC value calculated using the second session key for at least part or all of the embedded data stored in the packet data stored in a second extended packet footer of the embedded data. The information processing device according to claim 2 .

4. the second communication is transmitting or receiving a frame including a packet header and packet data; The second communication protected by the second session key is transmitted or received with a first MAC value calculated using the second session key for at least a portion of the packet data stored in first embedding data stored in the packet data. The information processing device according to claim 1 .

5. the second communication protected by the second session key is transmitted or received with a second MAC value calculated using the second session key on at least a portion of second embedding data stored in the packet data and stored in the second embedding data; the frame includes a frame start, the second embedded data, at least one of image data and user-defined data, the first embedded data, and a frame end; the second embedded data is transmitted or received between the frame start and at least one of the image data and the user-defined data; The first embedded data is transmitted or received between at least one of the image data and the user-defined data and the frame end. The information processing device according to claim 4 .

6. The frame includes an extended packet header The information processing device according to claim 4 .

7. the second communication is a transmission or reception of a line including an extended packet header and packet data; The second communication protected by the second session key is transmitted or received with a MAC value calculated using the second session key for at least a part or all of the extended packet header and a part or all of the packet data stored in an extended packet footer in the line. The information processing device according to claim 1 .

8. The second communication protected by the second session key is protected by a MAC mode in which related information about the MAC mode protecting the second communication is: transmitted or received in an extended packet header protected by the second session key; or the second session key is stored in embedded data stored in packet data protected by the second session key and transmitted or received. The information processing device according to claim 1 .

9. the second communication includes transmitting or receiving a frame including a frame start; The protection unit selects one of at least two of a first MAC mode, a second MAC mode, and a non-MAC mode as the MAC mode for protecting at least a part or all of the packet data in the frame before the transmission of the frame start is completed. The information processing device according to claim 8 .

10. the first communication protected by the first session key is transmitted or received with a first key ID indicating a key slot number corresponding to the second session key; the second communication protected by the second session key is transmitted or received with information related to the first key ID indicating that the second communication is protected by the second session key; the second communication is faster than the first communication; The related information of the first key ID has a smaller data volume than the first key ID. The information processing device according to claim 1 .

11. The second communication protected by the second session key is transmitted or received with information related to the first key ID, which indicates that the second communication is protected by the second session key, stored in embedded data protected by the second session key. The information processing device according to claim 1 .

12. The protective part is receiving a third session key using the first communication protected by the first session key, or deriving a third secret from the key schedule to derive a third session key associated with the third secret; the third session key is started to be used in place of the second session key; The second communication protected by the third session key is transmitted or received with information related to a second key ID, which indicates that the second communication is protected by the third session key, stored in embedded data protected by the third session key. The information processing device according to claim 11.

13. The first communication protected by the first session key is transmitted or received with information related to a first key ID stored in a write command or a read response protected by the first session key, the information indicating that the second session key can be used for the second communication. The information processing device according to claim 1 .

14. The protective part is receiving a third session key using the first communication protected by the first session key, or deriving a third secret from the key schedule to derive a third session key associated with the third secret; information related to a second key ID, which indicates that the first communication protected by the first session key can start using the third session key for the second communication, is stored in a write command or a read response protected by the first session key and is transmitted or received; The third session key begins to be used in place of the second session key. The information processing device according to claim 13.

15. The protective part is receiving a third session key using the first communication protected by the first session key, or deriving a third secret from the key schedule to derive a third session key associated with the third secret; the second communication protected by the second session key is transmitted or received with a specification of a timing for starting use of the third session key; The third session key begins to be used in place of the second session key. The information processing device according to claim 1 .

16. The protective part is receiving a third session key using the first communication protected by the first session key, or deriving a third secret from the key schedule to derive a third session key associated with the third secret; The protection unit starts using the third session key instead of the second session key from the calculation of a MAC value for at least a portion of the embedded data. The information processing device according to claim 1 .

17. The protective part is receiving a third session key using the first communication protected by the first session key, or deriving a third secret from the key schedule to derive a third session key associated with the third secret; The second session key and the third session key are alternately updated. The information processing device according to claim 1 .

18. a protection unit that protects the first communication and the second communication, The protective part is deriving a first secret from a key schedule using the first communication; deriving a first session key associated with the first secret; using the first session key for operations to protect the first communication; The protective part is receiving a second session key using the first communication protected by the first session key, or deriving a second secret from the key schedule to derive a second session key associated with the second secret; an initialization vector including a source ID value, a virtual channel or extended virtual channel value, a frame counter value or an additional frame number value, and the received or derived second session key for use in an operation to protect the second communication; A mobile device characterized by:

19. a protection unit that protects the first communication and the second communication, The protective part is deriving a first secret from a key schedule using the first communication; deriving a first session key associated with the first secret; using the first session key for operations to protect the first communication; The protective part is receiving a second session key using the first communication protected by the first session key, or deriving a second secret from the key schedule to derive a second session key associated with the second secret; an initialization vector including a source ID value, a virtual channel or extended virtual channel value, a frame counter value or an additional frame number value, and the received or derived second session key for use in an operation to protect the second communication; A communication system comprising:

Images