Information processing apparatus, mobile device, and communication system

Abstract

An information processing apparatus according to an aspect of the present disclosure is provided with a protection unit for protecting a first communication between a first device and a second device and a second communication between a third device and the second device. The protection unit performs the following (1) to (4). (1) Deriving or receiving a first session key; (2) Encryption or decryption and message authentication of the first communication using the first session key; (3) Receiving a second session key by using the first communication protected by the first session key; and (4) Encrypting, decrypting, or message authenticating the second communication using the second session key, wherein a total number of communications or a total amount of communication data of communications from start using the second session key to end using the second session key is different between a third communication between the first device and the third device and the first communication.

Description

Technical Field

[0001] This disclosure relates to information processors, mobile devices, and communication systems, and more particularly, to information processors, mobile devices, and communication systems that enable the use of session keys. Background Technology

[0002] Currently, in the standardized CSI (Camera Serial Interface)-2 ver4.0, two types are defined: a packet structure using C-PHY for the physical layer and a packet structure using D-PHY for the physical layer.

[0003] In recent years, the CSI-2 standard has been widely used not only in mobile devices but also in various applications such as vehicle mounts and IoT (Internet of Things). Consequently, it is assumed that existing packet structures are inadequate for these applications. Therefore, the MIPI (Mobile Industry Processor Interface) consortium has been researching extended packet structures (such as existing packet headers or trailers) to accommodate these applications.

[0004] Incidentally, the SPDM (Security Protocol and Data Model) standard described in NPTL 1 discloses a method for establishing an SPDM session. Additionally, the SPDM extension standard described in NPTL 2 discloses a method for applying an SPDM session.

[0005] Reference List

[0006] Non-patent literature

[0007] [NPTL]

[0008] NPTL 1:"Security Protocol and Data Model(SPDM)Specification",DSP0274,Version:1.1.0,DMTF,2020-07-15

[0009] NPTL 2:"Secured Messages using SPDM Specification",DSP0277,Version:1.0.0,DMTF,2020-09-18 Summary of the Invention

[0010] However, when session keys sent or received using SPDM sessions are applied to control system or imaging system communications according to the MIPI CSI-2 standard or DSI (Display Serial Interface)-2 standard, the following problems arise: the timing of session key updates cannot be detected from the sending side; the timing of the start of session key usage cannot be detected from the receiving side; replay attacks or tampering attacks can be performed on commands or data protected using the session key; and so on. Furthermore, it is necessary to ensure consistency or selection of security features with the communication counterpart in the control system or imaging system communications. Additionally, various problems exist in the implementation of message authentication, encryption, or decryption for control system or imaging system communications.

[0011] This disclosure is made in view of this situation and is intended to address issues or problems in information processors, mobile devices and communication systems that use session keys.

[0012] According to one aspect of this disclosure, the information processor includes a protection unit that protects a first communication between a first device and a second device, and a second communication between a third device and the second device. The protection unit performs the following (1) to (4):

[0013] (1) Export or receive the first session key;

[0014] (2) Use the first session key for encryption or decryption of the first communication and message authentication;

[0015] (3) Using the first communication protected by the first session key to receive the second session key; and

[0016] (4) Use the second session key for encryption, decryption or message authentication of the second communication.

[0017] Here, the total number of communications or the total amount of communication data between the start and end of the use of the second session key refers to the communication between the first and third communications, and the third communications refer to the communication between the first and third devices.

[0018] According to one aspect of this disclosure, the information processor includes a protection unit that protects a first communication between a first device and a second device, and a second communication between a third device and the second device. The protection unit performs the following (1) to (4):

[0019] (1) Export or receive the first session key;

[0020] (2) Use the first session key for encryption or decryption of the first communication and message authentication;

[0021] (3) Using the first communication protected by the first session key to receive the second session key; and

[0022] (4) Use the second session key for encryption, decryption or message authentication of the second communication.

[0023] Here, the total number of communications or the total amount of communication data between the start and end of the use of the second session key differs between the first and third communications, with the third communication occurring between the first and third devices.

[0024] According to one aspect of this disclosure, a mobile device includes a protection unit that protects a first communication between a first device and a second device, and a second communication between a third device and the second device. The protection unit performs the following (1) to (4):

[0025] (1) Export or receive the first session key;

[0026] (2) Use the first session key for encryption or decryption of the first communication and message authentication;

[0027] (3) Using the first communication protected by the first session key to receive the second session key; and

[0028] (4) Use the second session key for encryption, decryption or message authentication of the second communication.

[0029] Here, the total number of communications or the total amount of communication data between the start and end of the use of the second session key differs between the first and third communications, with the third communication occurring between the first and third devices.

[0030] According to one aspect of this disclosure, a communication system includes a protection unit that protects a first communication between a first device and a second device, and a second communication between a third device and the second device. The protection unit performs the following (1) to (4):

[0031] (1) Export or receive the first session key;

[0032] (2) Use the first session key for encryption or decryption of the first communication and message authentication;

[0033] (3) Using the first communication protected by the first session key to receive the second session key; and

[0034] (4) Use the second session key for encryption, decryption or message authentication of the second communication.

[0035] Here, the total number of communications or the total amount of communication data between the start and end of the use of the second session key differs between the first and third communications, with the third communication occurring between the first and third devices. Attached Figure Description

[0036] Figure 1 This is a block diagram illustrating a configuration example of a first embodiment of a communication system applying the present technology.

[0037] Figure 2 This is a diagram illustrating a configuration example of a second embodiment of a communication system applying the present technology.

[0038] Figure 3 This is a diagram illustrating a first structural example of the overall packet structure for extended packets oriented towards D-PHY.

[0039] Figure 4 This is a diagram illustrating a first structural example of a packet structure for extended short packets oriented towards D-PHY.

[0040] Figure 5 This is a diagram illustrating a first structural example of a packet structure for extended long packets oriented towards D-PHY.

[0041] Figure 6 This is a diagram illustrating a first structural example of the overall packet structure for extended packets oriented towards C-PHY.

[0042] Figure 7 This is a diagram illustrating a first structural example of a grouping structure for extended short packets oriented towards C-PHY.

[0043] Figure 8 This is a diagram illustrating a first structural example of a packet structure for extended long packets oriented towards C-PHY.

[0044] Figure 9 This is a block diagram illustrating an example of the configuration of an image sensor.

[0045] Figure 10 This is a block diagram illustrating a configuration example of an application processor.

[0046] Figure 11 This is a flowchart describing the process of sending packets from an image sensor.

[0047] Figure 12 This is a flowchart describing the extended mode sending process.

[0048] Figure 13 This is a flowchart describing the process of the application processor receiving packets.

[0049] Figure 14 This is a flowchart describing the extended mode receive processing.

[0050] Figure 15 This is a diagram illustrating a second structural example of the overall packet structure for extended packets oriented towards D-PHY.

[0051] Figure 16 This is a diagram illustrating a second structural example of a packet structure for extended long packets oriented towards D-PHY.

[0052] Figure 17 This is a diagram illustrating a second structural example of a grouping structure for extended short packets oriented towards C-PHY.

[0053] Figure 18 This is a diagram illustrating a second structural example of a grouping structure for extended long groups oriented towards C-PHY.

[0054] Figure 19 This is a block diagram illustrating a modified example of the configuration used for switching between D-PHY and C-PHY.

[0055] Figure 20 This is a block diagram illustrating a configuration example of a third embodiment of a communication system applying the present technology.

[0056] Figure 21 This is a diagram illustrating a structural example of an extended packet for D-PHY that corresponds to the prohibition of packet modification.

[0057] Figure 22 This is a diagram illustrating a structural example of an extended packet for C-PHY that corresponds to the prohibition of packet modification.

[0058] Figure 23 This is a diagram illustrating a structural example of an extended packet for A-PHY that corresponds to the prohibition of packet modification.

[0059] Figure 24 It is a flowchart describing the packet sending / receiving process that adapts to the rules prohibiting packet modification.

[0060] Figure 25 This is a block diagram illustrating an example of the configuration of an image sensor that adapts to the prohibition of group modification.

[0061] Figure 26 This is a block diagram illustrating a configuration example of an application processor that adapts to the restrictions on group modification.

[0062] Figure 27 This is a block diagram illustrating a communication system in which an image sensor and an application processor are configured as directly coupled to each other.

[0063] Figure 28 This is a diagram illustrating an example of the grouping of read commands generated on the application processor side.

[0064] Figure 29 This is a diagram illustrating an example of the packet structure of a read command transmitted via A-PHY.

[0065] Figure 30 This is a diagram illustrating an example of the grouping of read commands and read data on the image sensor side.

[0066] Figure 31 A diagram illustrating an example of the packets of read data transmitted via A-PHY is shown.

[0067] Figure 32 This is a diagram illustrating an example of the grouping of read data acquired on the application processor side.

[0068] Figure 33 This is a diagram illustrating an example of the grouping of write data generated on the application processor side.

[0069] Figure 34 A diagram illustrating an example of the composition of write data packets transmitted via A-PHY is shown.

[0070] Figure 35 This is a diagram illustrating an example of the grouping of write data acquired at the image sensor side.

[0071] Figure 36 This is a diagram illustrating the overview of the extended packet header (ePH) and extended packet trailer (ePF).

[0072] Figure 37 This is a flowchart describing the initial setup and confirmation procedures for communication processing using CCI-FS.

[0073] Figure 38 This is a flowchart describing the write operation using CCI-FS.

[0074] Figure 39 This is a flowchart describing the read operation using CCI-FS.

[0075] Figure 40 This is a block diagram illustrating a configuration example of an image sensor and an application processor configured as a communication system coupled via SerDes.

[0076] Figure 41 This is a diagram illustrating an example of the grouping of read commands generated on the application processor side.

[0077] Figure 42 This is a diagram illustrating an example of a group of read commands output by I2C / I3C.

[0078] Figure 43This is a diagram illustrating an example of the packet structure of a read command transmitted via A-PHY.

[0079] Figure 44 This is a diagram illustrating an example of packets of read data generated by a slave-side SerDes device.

[0080] Figure 45 This is a diagram illustrating an example of the grouping of read commands and read data on the image sensor side.

[0081] Figure 46 This is a diagram illustrating an example of a group of read data output from I2C / I3C.

[0082] Figure 47 This is a diagram illustrating an example of the packet structure of read data transmitted via A-PHY.

[0083] Figure 48 This is a diagram illustrating an example of a group of read data output from I2C / I3C.

[0084] Figure 49 This is a diagram illustrating an example of the grouping of read data acquired on the application processor side.

[0085] Figure 50 This is a flowchart describing the initial setup and confirmation procedures for communication processing using CCI-FS.

[0086] Figure 51 This is a flowchart describing the write operation using CCI-FS.

[0087] Figure 52 This is a flowchart describing the read operation using CCI-FS.

[0088] Figure 53 This is a flowchart describing the processing of sequence A_Write (at AP).

[0089] Figure 54 This is a flowchart describing the processing of sequence A_Read_CMD (at AP).

[0090] Figure 55 This is a flowchart describing the processing of sequence C (at AP).

[0091] Figure 56 This is a flowchart describing the processing of sequence B (in SerDes(Slave)).

[0092] Figure 57 This is a flowchart describing the processing of sequence A_Read_Data (at AP).

[0093] Figure 58This is a diagram showing the details of the extended packet headers ePH0, ePH1, and ePH2.

[0094] Figure 59 This is a diagram showing the details of the extended packet header ePH3.

[0095] Figure 60 This is a diagram showing the details of the extended DT of the extended packet header ePH.

[0096] Figure 61 This is a block diagram illustrating an example of the existing I2C configuration in hardware.

[0097] Figure 62 This is a diagram illustrating an example of waveforms during data transmission on the I2C bus.

[0098] Figure 63 This is a block diagram illustrating a construction example of a communication system directly coupled with a CCI-related A-PHY.

[0099] Figure 64 This is a diagram illustrating an example of a network coupling pattern.

[0100] Figure 65 This is a block diagram illustrating an example of the circuit configuration of the CCI-FS processing unit.

[0101] Figure 66 This is a diagram showing an example of register configuration.

[0102] Figure 67 This is a diagram illustrating an example of register configuration when a bridge is constructed.

[0103] Figure 68 This is a diagram illustrating an example of the register configuration of an error-related register.

[0104] Figure 69 This is a diagram illustrating a variant of the extended packet header ePH in the packet structure of write data generated on the application processor side.

[0105] Figure 70 This is a diagram illustrating a variant of the extended packet header ePH in the packet structure of a read command generated on the application processor side.

[0106] Figure 71 This is a diagram illustrating the flow between the application processor and the image sensor in a directly coupled A-PHY configuration.

[0107] Figure 72 This is a diagram illustrating the flow using the clock stretching method.

[0108] Figure 73This is a block diagram illustrating a detailed configuration example of an image sensor including a CCI-FS processing unit.

[0109] Figure 74 This is a block diagram illustrating a detailed configuration example of an application processor including a CCI-FS processing unit.

[0110] Figure 75 This is a block diagram illustrating a configuration example of a fourth embodiment of a communication system applying the present technology.

[0111] Figure 76 This is a block diagram illustrating a detailed example of the configuration of an image sensor.

[0112] Figure 77 This is a block diagram illustrating a detailed configuration example of an application processor.

[0113] Figure 78 This is a flowchart describing the first processing instance of the communication process.

[0114] Figure 79 This is a flowchart describing the first processing instance of the communication process.

[0115] Figure 80 This is a flowchart describing the first processing instance of the communication process.

[0116] Figure 81 This is a diagram illustrating the verification group and the group to be verified.

[0117] Figure 82 This is a diagram illustrating the verification group and the group to be verified.

[0118] Figure 83 It is a flowchart describing the data validation process.

[0119] Figure 84 This is a flowchart describing the message count value sending process.

[0120] Figure 85 It is a diagram describing the embedded data.

[0121] Figure 86 This is a diagram illustrating an example of the data structure for image data.

[0122] Figure 87 It is a flowchart describing the image data transmission and processing.

[0123] Figure 88 This is a flowchart describing the complete arithmetic value sending process.

[0124] Figure 89 The first variation of the data structure for image data is shown.

[0125] Figure 90A second variation of the data structure for image data is shown.

[0126] Figure 91 This illustrates a third variation of the data structure for image data.

[0127] Figure 92 This is a flowchart describing the first processing instance of the processing of complete arithmetic values.

[0128] Figure 93 This is a flowchart describing the second processing instance of the processing of complete arithmetic values.

[0129] Figure 94 This is a flowchart describing the third processing instance of the complete arithmetic value processing.

[0130] Figure 95 This is a flowchart describing the fourth processing instance of the complete arithmetic value processing.

[0131] Figure 96 This is a diagram showing an instance of an initial counter block that stores the initialization vector.

[0132] Figure 97 This is a diagram illustrating the GHASH function.

[0133] Figure 98 This is a diagram showing the GCTR function.

[0134] Figure 99 This is a diagram showing the GCM-AE function.

[0135] Figure 100 This is a graph showing the GCM-AD function.

[0136] Figure 101 A diagram illustrating an example of the data structure for image data is provided, in which a completeness arithmetic value (MAC) is sent for each row.

[0137] Figure 102 This is a diagram showing an instance of an initialization vector.

[0138] Figure 103 This is a diagram illustrating an example of sending an initialization vector from the sending side to the receiving side.

[0139] Figure 104 A diagram showing an example of an extended format of CSI-2 or CCI is provided.

[0140] Figure 105 This is a flowchart describing the send process in the line-MAC method.

[0141] Figure 106A diagram illustrating an example of the data structure for image data is provided, where a completeness arithmetic value (MAC) is set for each frame.

[0142] Figure 107 This is a diagram showing an instance of an initialization vector.

[0143] Figure 108 This is a diagram illustrating an example of sending an initialization vector from the sending side to the receiving side.

[0144] Figure 109 This is a flowchart describing the transmission process in the frame-MAC method.

[0145] Figure 110 It is a flowchart describing the selection process.

[0146] Figure 111 This is a diagram illustrating an example of secure MAC information.

[0147] Figure 112 This is a diagram illustrating an example of the cycle between message count and frame count values.

[0148] Figure 113 This is a diagram illustrating the composition of the initialization vector.

[0149] Figure 114 It is a flowchart describing the data validation process.

[0150] Figure 115 It is a diagram describing the reflection process.

[0151] Figure 116 This is a diagram illustrating an example of a security protocol.

[0152] Figure 117 This is a diagram showing an instance of the source ID or final destination ID.

[0153] Figure 118 This is a block diagram illustrating a detailed example of the configuration of an image sensor that diagnoses the presence or absence of its own anomalies.

[0154] Figure 119 This is a flowchart describing the interference detection process (part 1) performed by the interference detection unit.

[0155] Figure 120 This is a diagram illustrating the storage method when storing the light emission mode (light reception mode) as a storage mode when the ranging sensor of the ToF system is implemented through an image sensor.

[0156] Figure 121 This is a diagram illustrating the storage method when storing the light emission mode (light reception mode) as a storage mode when the ranging sensor of the ToF system is implemented through an image sensor.

[0157] Figure 122 This is a flowchart describing the interference detection process (part 2) performed by the interference detection unit.

[0158] Figure 123 This is a flowchart describing the fault detection process performed by the fault detection department.

[0159] Figure 124 This is a flowchart describing the anomaly detection and handling process within the security department of the infringement detection division.

[0160] Figure 125 This is a flowchart describing the abnormal detection and handling process of the temperature detection unit.

[0161] Figure 126 A block diagram illustrating a detailed configuration example of an application processor for detecting the presence or absence of anomalies in an image sensor.

[0162] Figure 127 It is a flowchart describing the image sensor's processing when the application processor detects the presence or absence of an anomaly in the image sensor.

[0163] Figure 128 It is a flowchart describing the application processor's processing when the application processor detects the presence or absence of an anomaly in the image sensor.

[0164] Figure 129 This is a diagram illustrating an example of a data structure for storing image data in a location that allows for high-speed data transmission of a specific message without inhibiting high-speed data transmission of image data.

[0165] Figure 130 It is a flowchart describing the process of performing high-speed data transmission of a specific message without prohibiting high-speed data transmission of image data.

[0166] Figure 131 This is a flowchart describing the imaging / transmission processing (part 1).

[0167] Figure 132 This is a flowchart describing a practical application example of imaging / transmission processing (Part 1).

[0168] Figure 133 This is a flowchart describing the imaging / transmission processing (part 2).

[0169] Figure 134 This is a flowchart describing the imaging / transmission processing via an image sensor (part 3).

[0170] Figure 135 This is a flowchart describing the imaging / transmission processing (part 3) via the application processor.

[0171] Figure 136 This is a flowchart describing the imaging / transmission processing via an image sensor (part 4).

[0172] Figure 137 This is a flowchart describing the imaging / transmission processing (part 4) via the application processor.

[0173] Figure 138 This is a flowchart describing the imaging / transmission processing via an image sensor (part 5).

[0174] Figure 139 This is a flowchart describing the imaging / transmission processing (part 5) via the application processor.

[0175] Figure 140 This is a flowchart describing the imaging / transmission processing via an image sensor (part 6).

[0176] Figure 141 This is a flowchart describing the imaging / transmission processing (part 6) via the application processor.

[0177] Figure 142 This is a flowchart describing the imaging / transmission processing via an image sensor (part 7).

[0178] Figure 143 This is a flowchart describing the imaging / transmission processing (part 7) via the application processor.

[0179] Figure 144 This is a flowchart describing the imaging / transmission processing via an image sensor (part 8).

[0180] Figure 145 This is a flowchart describing the imaging / transmission processing (part 8) via the application processor.

[0181] Figure 146 This is a flowchart describing the imaging / transmission process (part 9).

[0182] Figure 147 This is a flowchart describing the imaging / transmission process (part 10).

[0183] Figure 148 This is a flowchart describing the imaging / transmission process (part 11).

[0184] Figure 149 This is a diagram illustrating message count values ​​using two different types of count values ​​with different Hamming distances.

[0185] Figure 150 This is a diagram illustrating a method that uses two types of count values ​​to detect whether a message count value has been tampered with or failed.

[0186] Figure 151 This is a diagram illustrating a method that uses two types of count values ​​to detect whether a message count value has been tampered with or failed.

[0187] Figure 152 This is a flowchart describing message counting processing.

[0188] Figure 153 This is a diagram illustrating an example of the configuration of the extended packet header ePH2 when a Warning Descriptor is set in the Reserved area of ​​the extended packet header ePH2.

[0189] Figure 154 This is a diagram illustrating a descriptive instance of each bit of the identification information using the alarm descriptor (specific message).

[0190] Figure 155 This is a diagram illustrating a constituent instance when an alert flash report (e.g., physical attack detection) is set as the first specific message in an extended packet header.

[0191] Figure 156 It is a flowchart describing the transmission process of an image sensor when a specific message is separated for transmission.

[0192] Figure 157 It is a flowchart describing the application processor's sending process when a specific message is separated for sending.

[0193] Figure 158 This is a flowchart describing the sending process when a read command is used to send alarm details after an alarm flash report is sent, and specific messages are separated for sending.

[0194] Figure 159 This is a diagram illustrating a constituent instance of a Security Descriptor, in which one of a specific message is set, such as the presence or absence of an anomaly inside or outside the image sensor 1211, or the presence or absence of interference or attack on the image sensor 1211.

[0195] Figure 160 This is a block diagram illustrating a configuration example of a propulsion device equipped with an image sensor and an application processor.

[0196] Figure 161 It describes control Figure 160 A diagram illustrating the propulsion control process (part 1) of the propulsion equipment in the diagram.

[0197] Figure 162 It describes control Figure 160A diagram illustrating the propulsion control process (part 2) of the propulsion equipment in the diagram.

[0198] Figure 163 It describes the control performed by a microcomputer. Figure 160 A diagram illustrating the propulsion control process (part 3) of the propulsion equipment in the diagram.

[0199] Figure 164 It describes the control through the imaging unit. Figure 160 A diagram illustrating the propulsion control process (part 3) of the propulsion equipment in the diagram.

[0200] Figure 165 This is a diagram illustrating a constituent instance of the Responder flag fields definitions used to enable (HBEAT_CAP=1) or disable (HBEAT_CAP=0) the HEARTBEAT function.

[0201] Figure 166 This is a diagram illustrating a constituent instance of a HEARTBEAT request message.

[0202] Figure 167 This is a diagram illustrating a constituent instance of the HEARTBEAT_ACK response message.

[0203] Figure 168 This is a diagram illustrating a constituent instance of the HEARTBEAT_NAK response message.

[0204] Figure 169 This is a diagram illustrating the constituent instances of the END_SESSION request message.

[0205] Figure 170 This is a flowchart describing the HEARTBEAT process (part 1).

[0206] Figure 171 This is a diagram illustrating a constituent instance of the END_SESSION_NAK response message.

[0207] Figure 172 This is a flowchart describing the HEARTBEAT processing (part 2) of the CCI host (requesting party).

[0208] Figure 173 This is a flowchart describing the HEARTBEAT process (part 2) of the CCI device (responder).

[0209] Figure 174 This is a flowchart describing the HEARTBEAT processing (part 3) of the CCI host (requesting party).

[0210] Figure 175 This is a flowchart describing the HEARTBEAT process (part 3) of the CCI device (responder).

[0211] Figure 176 This is a diagram illustrating a constituent instance of an ERROR response message.

[0212] Figure 177 It is a diagram illustrating a setting instance that describes error codes and error data.

[0213] Figure 178 This is a diagram illustrating a setting instance for ExtendedErrorData.

[0214] Figure 179 This is a diagram illustrating an example of setting up a Registry or Standards Body ID when using the pseudo-HEARTBEAT function.

[0215] Figure 180 This is a diagram illustrating a configuration instance of the VENDOR_DEFINED_REQUEST request message.

[0216] Figure 181 This is a diagram illustrating a configuration instance of the VENDOR_DEFINED_RESPONSE response message.

[0217] Figure 182 This is a diagram illustrating the key scheduling of SPDM.

[0218] Figure 183 A diagram showing an instance of KEY_UPDATA_operations is provided.

[0219] Figure 184 It is a flowchart describing an example of the processing flow related to key updates.

[0220] Figure 185 This is a diagram showing an example of ePH2.

[0221] Figure 186 This is a flowchart describing an example of the session key update process.

[0222] Figure 187 It is a flowchart that describes an instance of the processor's processing flow.

[0223] Figure 188 This is a flowchart illustrating an example of the sensor processing flow.

[0224] Figure 189This is a flowchart describing an example of the session key update process.

[0225] Figure 190 It is a flowchart that describes an instance of the processor's processing flow.

[0226] Figure 191 This is a flowchart illustrating an example of the sensor processing flow.

[0227] Figure 192 It is a flowchart that describes an instance of the processor's processing flow.

[0228] Figure 193 This is a flowchart illustrating an example of the sensor processing flow.

[0229] Figure 194 This is a flowchart illustrating an example of the sensor processing flow.

[0230] Figure 195 This is a diagram showing instances of KeyUpdataReq and KeySwitchTiming.

[0231] Figure 196 This is a flowchart describing an example of the session key update process.

[0232] Figure 197 It is a flowchart that describes an instance of the processor's processing flow.

[0233] Figure 198 This is a flowchart illustrating an example of the sensor processing flow.

[0234] Figure 199 It is a flowchart that describes an instance of the processor's processing flow.

[0235] Figure 200 This is a flowchart illustrating an example of the sensor processing flow.

[0236] Figure 201 It is a flowchart that describes an instance of the processor's processing flow.

[0237] Figure 202 This is a flowchart illustrating an example of the sensor processing flow.

[0238] Figure 203 It is a flowchart that describes an instance of the processor's processing flow.

[0239] Figure 204 This is a flowchart illustrating an example of the sensor processing flow.

[0240] Figure 205 This is a diagram showing an instance of EvenOddkey.

[0241] Figure 206 This is a diagram showing an example of session key export.

[0242] Figure 207 This is a diagram showing an example of session key export.

[0243] Figure 208 This is a block diagram illustrating a configuration example of a fifth embodiment of a communication system applying this technology.

[0244] Figure 209 This is a block diagram illustrating a modified example of the configuration of an image sensor and a processor.

[0245] Figure 210 This is a block diagram illustrating a modified example of the configuration of an image sensor and a processor.

[0246] Figure 211 This is a block diagram illustrating a modified example of the configuration of an image sensor and a processor.

[0247] Figure 212 This is a block diagram illustrating a modified example of the configuration of an image sensor and a processor.

[0248] Figure 213 This is a block diagram illustrating a modified example of the configuration of an image sensor and a processor.

[0249] Figure 214 This is a block diagram illustrating a modified example of the configuration of an image sensor and a processor.

[0250] Figure 215 This is a block diagram illustrating a modified example of the configuration of an image sensor and a processor.

[0251] Figure 216 This is a block diagram illustrating a modified example of the configuration of an image sensor and a processor.

[0252] Figure 217 This is a diagram illustrating an example of countermeasures against replay attacks targeting control system communications (control plane).

[0253] Figure 218 This is a diagram showing an instance of an initialization vector.

[0254] Figure 219 This is a diagram showing an example of writing a message.

[0255] Figure 220 This is a diagram showing an example of writing a message.

[0256] Figure 221 This is a diagram illustrating an example of countermeasures against replay attacks targeting control system communications (control plane).

[0257] Figure 222This is a diagram illustrating a configuration instance of the VENDOR_DEFINED_REQUEST request message.

[0258] Figure 223 This is a diagram illustrating an example of how written data is grouped.

[0259] Figure 224 This is a diagram illustrating a configuration instance of the VENDOR_DEFINED_REQUEST request message.

[0260] Figure 225 This is a diagram illustrating an example of how read data is grouped.

[0261] Figure 226 This is a diagram illustrating an example of countermeasures against replay attacks targeting image system communications (data plane).

[0262] Figure 227 This is a diagram illustrating an example of the data structure for image data.

[0263] Figure 228 It is a description Figure 227 A diagram illustrating the terminology used in the text.

[0264] Figure 229 This is a diagram illustrating an example of countermeasures for replay attacks targeting image system communications (data plane).

[0265] Figure 230 This is a diagram illustrating an example of the data structure for image data.

[0266] Figure 231 This is a diagram illustrating an example of countermeasures for replay attacks targeting image system communications (data plane).

[0267] Figure 232 This is a diagram illustrating an example of the data structure for image data.

[0268] Figure 233 It is a description Figure 232 A diagram illustrating the terminology used in the text.

[0269] Figure 234 This is a flowchart illustrating an example of session key generation / sending processing in SSMC.

[0270] Figure 235 It describes the continuation Figure 234 A flowchart of an example of subsequent processing.

[0271] Figure 236 This is a flowchart illustrating an example of session key update processing at one end of the SoC or bridge.

[0272] Figure 237This is a flowchart illustrating an example of session key update processing at the other end of a sensor or bridge.

[0273] Figure 238 This is a flowchart illustrating an example of session key generation / sending processing in SSMC.

[0274] Figure 239 It describes the continuation Figure 238 A flowchart of an example of subsequent processing.

[0275] Figure 240 This is a flowchart illustrating an example of session key update processing at one end of the SoC or bridge.

[0276] Figure 241 This is a flowchart describing an instance of session key update processing at the other end of a sensor or bridge.

[0277] Figure 242 It is shown Figure 208 A block diagram of a modified example of the structure.

[0278] Figure 243 This is a diagram illustrating an example of how data is grouped for writing.

[0279] Figure 244 This is a diagram illustrating an example of data writing processing.

[0280] Figure 245 This is a diagram showing an example of a function register.

[0281] Figure 246 This is a diagram showing an example of a function register.

[0282] Figure 247 This is a diagram illustrating an example of how read data is grouped.

[0283] Figure 248 This is a diagram illustrating an example of data reading processing.

[0284] Figure 249 This is a diagram illustrating an example of how data is grouped for writing.

[0285] Figure 250 This is a diagram illustrating an example of data writing processing.

[0286] Figure 251 This is a diagram showing an example of the settings for an extended header.

[0287] Figure 252 This is a diagram illustrating an example of how written data is grouped.

[0288] Figure 253This is a diagram showing an example of an extended header configuration.

[0289] Figure 254 This is a diagram illustrating an example of how written data is grouped.

[0290] Figure 255 This is a diagram illustrating an example of data writing processing.

[0291] Figure 256 This is a flowchart illustrating an example of arithmetic processing of MAC or CRC in the second CCI mode.

[0292] Figure 257 This is a diagram showing an example of the settings for register definition and extended header.

[0293] Figure 258 This is a diagram illustrating an example of how written data is grouped.

[0294] Figure 259 This is a diagram illustrating an example of data writing processing.

[0295] Figure 260 This is a flowchart describing an instance of arithmetic operation processing of GCM or CCM in the second CCI mode.

[0296] Figure 261 This is a diagram showing an example of the settings for register definition and extended header.

[0297] Figure 262 This is a diagram illustrating an example of how written data is grouped.

[0298] Figure 263 This is a diagram illustrating an example of data writing processing.

[0299] Figure 264 This is a diagram illustrating an example of how written data is grouped.

[0300] Figure 265 This is a diagram illustrating an example of data writing processing.

[0301] Figure 266 This is a flowchart illustrating an example of arithmetic operation processing of GCM or CCM in the first CCI mode.

[0302] Figure 267 This is a diagram showing an example of the settings for the Capability register.

[0303] Figure 268 This is a diagram showing an example of a vendor-defined SPDM message configuration.

[0304] Figure 269 This is a diagram illustrating an example of how written data is grouped.

[0305] Figure 270 This is a diagram illustrating an example of data writing processing.

[0306] Figure 271 It is a flowchart describing an instance of switching internal processing sequences.

[0307] Figure 272 This is a diagram showing an example of the settings for the Capability register.

[0308] Figure 273 This is a diagram showing an example of a vendor-defined SPDM message configuration.

[0309] Figure 274 It is a flowchart describing an instance of switching internal processing sequences.

[0310] Figure 275 It is a flowchart describing an instance of switching internal processing sequences.

[0311] Figure 276 This is a flowchart illustrating an example of processing the notification initialization vector.

[0312] Figure 277 This is a diagram illustrating an example of how data is grouped for writing.

[0313] Figure 278 This is a diagram illustrating an example of how written data is grouped.

[0314] Figure 279 This is a diagram illustrating an example of how written data is grouped.

[0315] Figure 280 This is a diagram illustrating an example of how written data is grouped.

[0316] Figure 281 This is a diagram showing an example of the capability register settings.

[0317] Figure 282 This is a diagram showing an example of a vendor-defined SPDM message setting.

[0318] Figure 283This is a diagram showing an example of the settings for an extended header.

[0319] Figure 284 This is a flowchart describing an instance of the message authentication process.

[0320] Figure 285 This is a diagram illustrating an example of a message authentication policy.

[0321] Figure 286 This is a diagram illustrating an example of a message authentication policy.

[0322] Figure 287 This is a diagram illustrating an example of a message authentication policy.

[0323] Figure 288 This is a diagram showing an example of the capability register settings.

[0324] Figure 289 This is a diagram showing an example of a vendor-defined SPDM message setting.

[0325] Figure 290 This is a diagram showing an example of an initialization vector.

[0326] Figure 291 This is a diagram showing an example of an initialization vector.

[0327] Figure 292 This is a diagram showing an example of Mode in IV.

[0328] Figure 293 This is a block diagram illustrating a configuration example of an implementation of a computer using this technology. Detailed Implementation

[0329] The following will describe in detail, with reference to the accompanying drawings, specific implementation methods of this technology.

[0330] <Examples of Communication System Structure>

[0331] Figure 1 This is a block diagram illustrating a configuration example of a first embodiment of a communication system applying the present technology.

[0332] like Figure 1 As shown, the communication system 11 has the following configuration: an image sensor 21 and an application processor 22 are coupled to each other via a bus 23. For example, the communication system 11 is used for CSI-2 coupling within existing mobile devices (such as so-called smartphones).

[0333] The image sensor 21 is configured by combining the extended mode adaptive CSI-2 transmission circuit 31 with, for example, a lens and an imaging element (both not shown). For example, the image sensor 21 transmits image data of the image acquired by the imaging element to the application processor 22 via the extended mode adaptive CSI-2 transmission circuit 31.

[0334] The application processor 22 is configured by combining the extended mode adaptive CSI-2 receiver circuit 32 with the LSI (Large-Scale Integration). The LSI performs processing corresponding to various applications to be executed by the mobile device including the communication system 11. For example, the application processor 22 receives image data transmitted from the image sensor 21 via the extended mode adaptive CSI-2 receiver circuit 32. For example, the application processor 22 performs processing on the image data corresponding to the LSI application.

[0335] Bus 23 is a communication path for transmitting signals according to the CSI-2 standard. In bus 23, for example, the transmission distance for signals is approximately 30 cm. Furthermore, bus 23 couples the image sensor 21 and the application processor 22 to each other via multiple signal lines (I2C, CLKP / N, D0P / N, D1P / N, D2P / N, and D3P / N) as shown.

[0336] The extended mode adaptive CSI-2 transmitting circuit 31 and the extended mode adaptive CSI-2 receiving circuit 32 are adapted to communicate in the extended mode of the extended CSI-2 standard and are capable of sending and receiving signals to each other. Note that further details will be provided later. Figure 9 and Figure 10 A detailed description of the configuration of the extended mode adaptive CSI-2 transmitting circuit 31 and the extended mode adaptive CSI-2 receiving circuit 32 is given.

[0337] Figure 2 This is a block diagram illustrating a configuration example of a second embodiment of a communication system applying the present technology.

[0338] like Figure 2 As shown, in communication system 11A, image sensor 21 and SerDes device 25 are coupled to each other via bus 24-1. In communication system 11A, application processor 22 and SerDes device 26 are coupled to each other via bus 24-2. In communication system 11A, SerDes device 25 and SerDes device 26 are coupled to each other via bus 27. For example, communication system 11A is used for coupling in an existing vehicle camera.

[0339] Here, the image sensor 21 and the application processor 22 are connected to... Figure 1 The image sensor 21 and application processor 22 are constructed in the same manner, and their detailed description is omitted.

[0340] With Figure 1 In the same manner as bus 23, buses 24-1 and 24-2 are communication paths for transmitting signals conforming to the CSI-2 standard, and include multiple signal lines (HS-GPIO, I2C / I3C, CLKP / N, D0P / N, D1P / N, D2P / N and D3P / N), as shown in the figure.

[0341] SerDes device 25 includes a CSI-2 receiving circuit 33 and a SerDes (Serializer Deserializer) transmitting circuit 34. For example, SerDes device 25 acquires bit-parallel signals transmitted from image sensor 21 through the CSI-2 receiving circuit 33, which communicates with the extended mode adaptive CSI-2 transmitting circuit 31 in accordance with the normal CSI-2 standard. Then, SerDes device 25 converts the acquired signal into a bit-serial signal and transmits the converted signal to SerDes device 26 through the SerDes transmitting circuit 34, which communicates with the SerDes receiving circuit 35 in a channel.

[0342] SerDes device 26 includes a SerDes receiving circuit 35 and a CSI-2 transmitting circuit 36. For example, SerDes device 26 acquires the transmitted bit-serial signal through SerDes receiving circuit 35, which communicates with SerDes transmitting circuit 34 in a channel. SerDes device 26 then converts the acquired signal into bit-parallel data and transmits the converted signal to application processor 22 through CSI-2 transmitting circuit 36 ​​and extended mode adaptive CSI-2 receiving circuit 32 in accordance with the normal CSI-2 standard.

[0343] Bus 27 is a communication path that transmits signals according to standards such as A-PHY or FPD (Flat Panel Display)-LINK III. In bus 27, for example, signals can be transmitted over a distance of approximately 15m.

[0344] This long-range transmittable physical layer interface enables the automotive industry to leverage Advanced Driver Assistance Systems (ADAS), Autonomous Driving Systems (ADS), and other surround sensor applications including cameras and in-vehicle infotainment (IVI) displays. The MIPI A-PHY features an asymmetric data link layer (asymmetric upper layer) with a point-to-point topology, allowing for high-speed data transmission, control data sharing, and power sharing over the same physical cabling. The MIPI A-PHY serves as the foundation for end-to-end systems designed to simplify the integration of cameras, sensors, and displays, while also enabling the incorporation of functional and safety features.

[0345] In the communication systems 11 and 11A configured as described above, the extended mode adaptive CSI-2 transmitting circuit 31 and the extended mode adaptive CSI-2 receiving circuit 32 are capable of transmitting and receiving data in packets with an extended packet structure described later. This makes it adaptable to various applications, such as RAW24, intelligent ROI (region of interest), GLD (graceful link degradation), etc., as described later.

[0346] <First structural example of a grouping structure>

[0347] refer to Figures 3 to 8 A description of a first structural example of the packet structure used in communication between the Extended Mode Adaptive CSI-2 transmitting circuit 31 and the Extended Mode Adaptive CSI-2 receiving circuit 32 is given.

[0348] Figure 3 The overall packet structure for the packets used in CSI-2 extended mode when the physical layer is D-PHY is shown (hereinafter referred to as D-PHY-oriented extended packets).

[0349] like Figure 3 As shown, the extended packets for D-PHY have packet headers and packet trailers, which have the same packet structure as the existing CSI-2 standard. For example, the packet header stores VC (Virtual Channel) indicating the number of lines in the virtual channel, DataType indicating the data type, WC (Word Count) indicating the data length of the payload, and VCX / ECC. Additionally, the packet trailer stores CRC (Cyclic Redundancy Check).

[0350] Here, in the existing CSI-2 standard, data types 0x38 to x3F are defined as reserved for transmission in the packet header. Therefore, in extended packets for D-PHY, the reserved existing data types are used to define new settings for the receiver to identify the extended mode.

[0351] For example, in the case of DataType[5:3] = 3'b111, the extended mode / DataType[2] = reserved (RES: reserved for future extensions) / DataType[1:0] = extended mode type (the four prepared extended modes) is defined as the data type.

[0352] That is, in the existing CSI-2 standard, the reserved data types 0x38 to x3F are defined. For example, DataType[5:3] is defined as extended mode setting information, and DataType[1:0] is defined as extended type setting information. The extended mode setting information indicates whether the mode is an extended mode; for example, if DataType[5:3] is 3'b111, the mode is represented as an extended mode. Furthermore, when four types—Extended Mode 0, Extended Mode 1, Extended Mode 2, and Extended Mode 3—are prepared as extended mode types, the extended type setting information indicates which of these types the extended mode is. For example, if DataType[1:0] is 2'b00, the extended mode type is represented as Extended Mode 0.

[0353] In extended mode 0 (DataType[1:0] = 2'b00), for example, a grouping structure is defined where the payload is divided into four groups. That is, as shown below. Figure 3 As shown, the payload in extended mode 0 is divided into an extended packet header (ePH), an optional extended packet header (OePH), a legacy payload, and an optional extended packet footer (OePF). It should be noted that the extended packet header can be sent repeatedly.

[0354] The extended packet header is placed in the header corresponding to the payload of the existing CSI-2 standard and must be transmitted in extended mode. For example, as shown in the figure, the extended packet header is constructed by setting information such as the SROI identification flag, extended VC (Virtual Channel), extended DataType, OePH selection flag, and OePF selection flag. Here, the 4-bit VC in the existing CSI-2 standard is extended to 8 bits by the extended VC, and the 4-bit DataType in the existing CSI-2 standard is extended to 8 bits by the extended DataType.

[0355] For example, in a D-PHY-oriented packet, there is already a 4-bit VC in the existing packet header, and the extended VC of the extended packet header is defined as 4 bits, allowing a total of 8 bits. Specifically, it can be defined as follows: OePH[7:0] = {5'h00, RSID, XY_POS, MC}, OePF[3:0] = {3'h0, pCRC}, thereby enabling control over the on / off state of packet transmission required by each application.

[0356] Optional extended packet headers and optional extended packet trailers are sent selectively depending on the application.

[0357] The conventional payload corresponds to the same payload as the payload in the existing CSI-2 standard.

[0358] In this way, by configuring the extended packet header, optional extended packet header, and optional extended packet trailer as needed, data corresponding to various applications can be sent. Furthermore, the data transmitted in the extended packet header, optional extended packet header, and optional extended packet trailer is a 26-bit + 6-bit ECC (Error Correction Code). This allows for the suppression of increased circuit size and improved fault tolerance by utilizing existing packet header circuitry.

[0359] Figure 4 This illustrates a specific application example of this D-PHY-oriented extended packet, showing the packet structure of a short packet (hereinafter referred to as a D-PHY-oriented extended short packet) used in CSI-2 extended mode when the physical layer is D-PHY. Similarly, Figure 5 The packet structure of long packets (hereinafter referred to as D-PHY-oriented extended long packets) used in CSI-2 extended mode is shown when the physical layer is D-PHY.

[0360] In such Figure 4 In the extended short packet for D-PHY shown, the data type extension type setting information stored in the packet header indicates that the extension mode type is extension mode 0 (DT[5:0] = 0x1C(5′b111_0_0)). Additionally, the data type short packet setting information stored in the extended packet header indicates a short packet (DT[7:0] = 0x00(FrameStart Code(Short Packet)).

[0361] In this manner, in extended mode and when the data type stored in the extended packet header is DT[7:0] = 0x00 to 0x0F, extended short packets are used, and data including the Short Packet Data Field of the extended short packet is necessarily sent to the optional extended packet header. This Short Packet Data Field is the same as defined in the existing CSI-2 standard.

[0362] It should be noted that when sending extended short packets, the MC (Message Count for GLD) and RSID (Row Number and Source ID for Vehicle Mounting) in the optional extended packet header can be sent; however, the traditional payload and pCRC are unnecessary and therefore their transmission is prohibited. If the traditional payload and pCRC are sent incorrectly, they will be ignored by the receiving side.

[0363] Furthermore, compared to extended short groups based on the existing CSI-2 standard, such as Figure 4 The extended short packets shown in the diagram can extend the bit width and data type of the virtual channel, and thus can be adapted to various applications defined by the optional extended packet header. Additionally, when these features are not required, extended short packets according to the existing CSI-2 standard can be transmitted together with extended long packets.

[0364] In such Figure 5 In the extended long packet for D-PHY shown, the extended type setting information of the data type stored in the packet header indicates that the extended mode type is extended mode 0 (DT[5:0] = 0x1C(5'b111_0_0)). Additionally, the short packet setting information of the data type stored in the extended packet header indicates short packets other than short packets (DT[7:0] is short packets other than 0x00 to 0x0F (= extended Long Packet)). Therefore, in extended long packets, data including the Short Packet Data Field is not transmitted.

[0365] Additionally, depending on the extended packet header settings, the optional extended packet header, the conventional payload, and the optional extended packet trailer are stored in the payload of the existing CSI-2 standard for transmission. As mentioned above, they are stored in the existing payload for transmission, and therefore transmitted by the existing SerDes transmitting circuit 34 and SerDes receiving circuit 35. Figure 2 It is identified in the same way as image data sent by the existing payload and transmitted to the subsequent stage as is.

[0366] Then, the final-level application processor 22 can determine the extended mode by the data type DT[5:0] of the packet header. Thus, the application processor 22 can sequentially interpret the payload content from the extended packet header and retrieve the data of the desired extended mode.

[0367] Figure 6 The overall packet structure for the packets used in CSI-2 extended mode when the physical layer is C-PHY (hereinafter referred to as C-PHY-oriented extended packets) is shown. It should be noted that... Figure 6 In the extended grouping for C-PHY shown in the figure, for the case of... Figure 3 The same configuration of the extended grouping for D-PHY is omitted, and descriptions of different configurations are given.

[0368] For example, in the extended grouping for C-PHY, to be compatible with Figure 3In the same manner as the extended packets for D-PHY, the extended mode is identified by the data type, and all data corresponding to the various applications to be executed by the application processor 22 is embedded in the payload for transmission.

[0369] like Figure 6 As shown, C-PHY-oriented extended packets transmit the packet header twice in the same manner as C-PHY-oriented packets according to the existing CSI-2 standard, and multiple data lines are arranged in 16-bit units to facilitate C-PHY's conversion of 16 bits into 7 symbols. Furthermore, the extended packet header is placed in the payload header. However, for virtual channels, since the existing extended packet header is reserved in the C-PHY case, virtual channels are not stored in the extended packet header. Needless to say, virtual channels can be stored in the extended packet header in the same manner as D-PHY-oriented extended packets.

[0370] In addition, both the optional extended packet header and optional extended packet trailer have bits, thus a flag for OePHF is prepared; when this flag is 1, the OePH / OePF information is then sent. Then, after the ePH and OePH information, the CRC is sent as an extended packet header, and the packet header constructed in the same manner is sent twice. In this way, by employing the same structure as the mechanism of sending the existing packet header twice, circuit multiplexing and fault tolerance can be achieved.

[0371] Figure 7 The diagram illustrates a packet structure for short packets (hereinafter referred to as C-PHY-oriented extended short packets) to be used in CSI-2 extended mode when the physical layer is C-PHY, serving as a concrete application example of this C-PHY-oriented extended packet structure. Similarly, Figure 8 The packet structure of long packets (hereinafter referred to as C-PHY-oriented extended long packets) used in CSI-2 extended mode when the physical layer is C-PHY is shown.

[0372] It should be noted that, Figure 7 The extended short packet for C-PHY shown in the figure Figure 4 The extended short packets for D-PHY shown in the figure do not differ significantly in their packet structure, and... Figure 8 The extended long packet for C-PHY shown in the figure Figure 5 The extended long packets for D-PHY shown in the figure do not differ significantly in their packet structure.

[0373] <Examples of Image Sensor and Application Processor Configuration>

[0374] (Example of an image sensor configuration)

[0375] Figure 9 This is a block diagram illustrating an example configuration of an image sensor 21, including an extended mode adaptive CSI-2 transmission circuit 31.

[0376] like Figure 9 As shown, in addition to the extended mode adaptive CSI-2 transmission circuit 31, the image sensor 21 also includes pixels 41, an AD converter 42, an image processing unit 43, a pixel CRC arithmetic unit 44, a physical layer processing unit 45, an I2C / I3C slave device 46, and a register 47. Furthermore, the extended mode adaptive CSI-2 transmission circuit 31 includes a packet generation unit 51, a packet header generation unit 52, an extended packet header generation unit 53, an extended packet trailer generation unit 54, selection units 55 and 56, a CRC arithmetic unit 57, a channel allocation unit 58, a CCI slave device 59, and a controller 60.

[0377] Pixel 41 outputs an analog pixel signal corresponding to the amount of light received, and the AD converter (ADC: analog-to-digital converter) 42 performs digital conversion on the pixel signal output from pixel 41 and supplies the pixel signal to the image processing unit 43. The image processing unit (ISP: image signal processor) 43 provides image data obtained by performing various types of image processing on the image based on the pixel signal to the pixel CRC arithmetic unit 44 and the packetization unit 51. In addition, the image processing unit 43 provides a data enable signal data_en to the packetization unit 51 and the controller 60, indicating whether the image data is valid.

[0378] The pixel CRC arithmetic unit 44 obtains the CRC of each pixel in the image data provided by the image processing unit 43 through arithmetic operations, and provides the CRC to the extended packet trailer generation unit 54.

[0379] The physical layer processing unit 45 can perform processing on both the C-PHY and D-PHY physical layers. For example, the physical layer processing unit 45 performs processing on the C-PHY physical layer when the C-layer enable signal cphy_en is active, and performs processing on the D-PHY physical layer when the C-layer enable signal cphy_en is inactive. Then, the physical layer processing unit 45 sends the packets, which have been divided into four channels by the channel allocation unit 58, to the application processor 22.

[0380] I2C / I3C slave device 46 in application processor 22 I2C / I3C master device 72 ( Figure 10 Communication is performed actively based on the I2C (Inter-Integrated Circuit) or I3C (Improved Inter Integrated Circuits) standards.

[0381] Different settings sent from application processor 22 are written to register 47 via I2C / I3C slave device 46 and CCI slave device 59. Examples of settings to be written to register 47 include CSI-2 compliant communication settings, extended mode settings indicating the presence or absence of extended mode, and fixed communication settings required for communication in extended mode.

[0382] Packaging unit 51 performs packaging processing to store the image data provided from image processing unit 43 in the packet payload, and provides the payload to selection unit 55 and channel allocation unit 58.

[0383] When an instruction is received to generate a packet header according to the packet header generation instruction signal ph_go provided from the controller 60, the packet header generation unit 52 generates a packet header and provides the packet header to the selection unit 55 and the channel allocation unit 58.

[0384] That is, the packet header generation unit 52 generates setting information indicating the setting conditions of the data to be transmitted in the packet according to the existing CSI-2 standard. For example, it stores a packet header indicating the data type. Additionally, the packet header generation unit 52 stores extension mode setting information indicating whether an extension mode using an extension header is adopted in an unused area defined as unused in the existing CSI-2 standard within the data type setting information. This data type is setting information indicating the type of data to be transmitted in the packet. Furthermore, the packet header generation unit 52 stores extension type setting information indicating which type of extension mode among the various types of extension modes it is preparing as an extension mode in an unused area.

[0385] Based on the extended packet header generation command signal eph_go and the extended packet header enable signal ePH_en provided from the controller 60, the extended packet header generation unit 53 generates an extended packet header and an optional extended packet header, respectively, and provides them to the selection unit 56 and the channel allocation unit 58. Furthermore, depending on the application of the image sensor 21, the extended packet header generation unit 53 is provided with row numbers, source IDs (identifiers), etc., for vehicle installation, which are stored in the extended packet header or the optional extended packet header as needed.

[0386] That is, in addition to the packet header generated by the packet header generation unit 52, the extended packet header generation unit 53 also generates and stores, such as... Figure 3The extended packet header shows the setting information. In addition, when sending an optional extended packet header, the extended packet header generation unit 53 stores the optional extended packet header setting information that indicates the sending of the optional extended packet header in the extended packet header as optional extended packet header setting information (OePH[7:0]) indicating whether to send the optional extended packet header, and generates an optional extended packet header after the extended packet header.

[0387] Based on the extended packet trailer generation instruction signal epf_go and the extended packet header enable signal ePF_en provided from the controller 60, the extended packet trailer generation unit 54 generates an optional extended packet trailer and provides it to the selection unit 56 and the channel allocation unit 58.

[0388] That is, in the case where the packet to be sent in extended mode is an extended long packet that stores the data to be sent as the payload in the existing CSI-2 standard, the extended packet trailer generation unit 54 generates an optional extended packet trailer that will be placed after the conventional payload in which the data is stored.

[0389] Furthermore, the packet header generation unit 52, the extended packet header generation unit 53, and the extended packet trailer generation unit 54 are supplied with a C-layer enable signal cphy_en from the controller 60. Then, when the C-layer enable signal cphy_en is active, the packet header generation unit 52 generates a packet header for the C-PHY; the extended packet header generation unit 53 generates an extended packet header and an optional extended packet header for the C-PHY; and the extended packet trailer generation unit 54 generates an optional extended packet trailer for the C-PHY. Conversely, when the C-layer enable signal cphy_en is inactive, the packet header generation unit 52 generates a packet header for the D-PHY; the extended packet header generation unit 53 generates an extended packet header and an optional extended packet header for the D-PHY; and the extended packet trailer generation unit 54 generates an optional extended packet trailer for the D-PHY.

[0390] When the C-layer enable signal cphy_en is active, the selection unit 55 selects the packet header provided by the packet header generation unit 52 based on the C-layer enable signal cphy_en provided by the controller 60, and provides it to the selection unit 56. Conversely, when the C-layer enable signal cphy_en is inactive, the selection unit 55 selects the payload supplied by the packing unit 51, and supplies it to the selection unit 56.

[0391] Based on the data selection signal data_sel provided from the controller 60, the selection unit 56 selects one of the packet header or payload selectively provided via the selection unit 55, the extended packet header and optional extended packet header provided from the extended packet header generation unit 53, and the optional extended packet tail provided from the extended packet tail generation unit 54, and provides the selected one to the CRC arithmetic unit 57.

[0392] The CRC arithmetic unit 57 determines the CRC of the packet header, payload, extended packet header, optional extended packet header or optional extended packet trailer selectively provided by the selection unit 56 through arithmetic operations, and provides the CRC to the channel allocation unit 58.

[0393] Under the control of the controller 60, the channel allocation unit 58 allocates the payload supplied from the packing unit 51, the packet header supplied from the packet header generation unit 52, the packet header provided from the packet header generation unit 52, the extended packet header and optional extended packet header provided from the extended packet header generation unit 53, the optional extended packet tail provided from the extended packet tail generation unit 54, and the CRC provided to the four channels from the CRC arithmetic unit 57 according to the CSI-2 standard, and supplies them to the physical layer processing unit 45.

[0394] CCI (Camera Control Interface) slave device 59 is based on the CSI-2 standard in application processor 22 and CCI host 88 ( Figure 10 Communication is initiated by the user.

[0395] The controller 60 reads various settings stored in register 47 and controls each block of the extended mode adaptive CSI-2 transmission circuit 31 according to the settings. For example, depending on the content of the data to be transmitted, the controller 60 controls the switching between the transmission of packets according to the existing CSI-2 standard packet structure and the transmission of packets in the extended mode packet structure.

[0396] Image sensor 21 is therefore configured and is capable of generating images as shown in the figure. Figures 3 to 8 The described packet structure is extended for sending to application processor 22.

[0397] (Example of application processor configuration)

[0398] Figure 10 This is a block diagram illustrating an example configuration of an application processor 22 including an extended mode adaptive CSI-2 receiver circuit 32.

[0399] like Figure 10As shown, the application processor 22 includes, in addition to the extended mode adaptive CSI-2 receiver circuit 32, a physical layer processing unit 71, an I2C / I3C master device 72, a register 73, and a controller 74. Furthermore, the extended mode adaptive CSI-2 receiver circuit 32 includes a packet header detection unit 81, a channel merging unit 82, an interpretation unit 83, selection units 84 and 85, a CRC arithmetic unit 86, a packet decompression unit 87, and a CCI host 88.

[0400] The physical layer processing unit 71 can perform processing on both the C-PHY and D-PHY physical layers. As described above, the physical layer processing unit 45 of the image sensor 21 performs processing on one of the C-PHY and D-PHY physical layers, and the physical layer processing unit 71 performs the same processing on the physical layer as that performed in the physical layer processing unit 45.

[0401] I2C / I3C master device 72, based on I2C or I3C standard, actively interacts with image sensor 21; I2C / I3C slave device 46 ( Figure 9 ) to communicate.

[0402] The controller 74 records the different settings to be written into the register 47 of the image sensor 21 into the register 73.

[0403] Controller 74 controls each block of application processor 22.

[0404] The packet header detection unit 81 detects packet headers from the packets provided by the physical layer processing unit 71 and confirms the data type stored in the packet header. Then, if the extended mode setting information indicates that the data type of the packet header is in the extended mode (DataType[5:3] = 3'b111), the packet header detection unit 81 provides an extended mode detection flag indicating the extended mode to the interpretation unit 83, the selection unit 84, and the selection unit 85. In addition, the packet header detection unit 81 provides a merge enable signal mrg_en to the channel merging unit 82 based on the packet header. This merge enable signal mrg_en indicates whether the merging of the four divided channels is valid.

[0405] That is, according to the existing CSI-2 standard, the packet header detection unit 81 detects the packet header that stores setting information (data type, etc.), which indicates the setting conditions for the data to be transmitted in the packet. At this time, according to the extended mode setting information stored in the unused area defined as unused in the existing CSI-2 standard, the extended mode setting information indicates whether an extended mode using an extended header is used. In the data type setting information indicating the type of data to be transmitted in the packet, the packet header detection unit 81 outputs an extended mode detection flag, thereby switching between receiving packets according to the packet structure of the existing CSI-2 standard and receiving packets with the packet structure during extended mode. In addition, according to the extended mode type information stored in the unused area defined as data type unused in the existing CSI-2 standard, the packet header detection unit 81 identifies which of the various types of extended modes is intended to be an extended mode.

[0406] When the merger enable signal mrg_en provided by the packet header detection unit 81 is valid, the channel merging unit 82 merges the packets that have been divided into four channels provided by the physical layer processing unit 71. Then, the channel merging unit 82 provides the single-channel packets to the interpretation unit 83, the selection unit 84, and the selection unit 85.

[0407] When the extended mode detection flag provided by the packet header detection unit 81 indicates an extended mode, the interpretation unit 83 reads the extended packet header, optional extended packet header, and optional extended packet trailer from the packets provided by the channel merging unit 82 based on the packet structure of the extended mode. Then, the interpretation unit 83 interprets the setting information stored in the extended packet header, optional extended packet header, and optional extended packet trailer.

[0408] That is, the interpretation unit 83 receives the extended packet header arranged in the header of the payload according to the existing CSI-2 standard as an extended header, and interprets the setting information stored in the extended packet header. Additionally, if the optional extended packet header setting information stored in the extended packet header indicates the transmission of an optional extended packet header selectively sent according to the application, the interpretation unit 83 receives the optional extended packet header following the extended packet header and interprets the setting information stored in the optional extended packet header. Furthermore, if the packet to be transmitted in extended mode is an extended long packet storing data to be transmitted as payload in the existing CSI-2 standard, the interpretation unit 83 receives the optional extended packet trailer arranged after the conventional payload storing the data and interprets the optional extended packet trailer.

[0409] Then, the interpretation unit 83 reads the line number, source ID, etc. for vehicle installation stored in, for example, the optional extended packet header, and outputs them to the LSI (not shown) of the subsequent stage.

[0410] It should be noted that if the extended mode detection flag provided by the packet header detection unit 81 indicates that there is no extended mode, that is, if a packet with the existing packet structure is provided, the interpretation unit 83 does not perform the processing described above, and the processing stops.

[0411] Based on the extended mode detection flag provided by the packet header detection unit 81, the selection unit 84 selectively provides data to the unpacking unit 87 based on the packet structure of the existing packets or the packet structure of the extended packets.

[0412] Based on the extended mode detection flag provided by the packet header detection unit 81, the selection unit 85 selectively provides data to the CRC arithmetic unit 86 based on the packet structure of the existing packet or the packet structure of the extended packet.

[0413] The CRC arithmetic unit 86 performs CRC arithmetic operations on the packet header, payload, extended packet header, optional extended packet header, or optional extended packet trailer selectively provided via the selection unit 85. In the event of a CRC error, the CRC arithmetic unit 86 outputs a crcCRC error detection signal indicating this condition to the subsequent LSI (not shown).

[0414] The unpacking unit 87 performs unpacking processing to retrieve image data stored in the payload selectively supplied via the selection unit 84, and outputs the acquired image data to the subsequent LSI (not shown).

[0415] CCI host 88 is based on CSI-2 standard active and image sensor 21, CCI slave device 59 ( Figure 9 ) to communicate.

[0416] Application processor 22 is thus configured to receive extended packets sent from image sensor 21, interpret setting information stored in extended packet header, optional extended packet header and optional extended packet tail, and acquire image data.

[0417] <Communication Processing>

[0418] Reference Figures 11 to 14 This describes the communication processing performed by the image sensor 21 and the application processor 22.

[0419] Figure 11 This is a flowchart describing the process of image sensor 21 sending packets.

[0420] For example, processing begins when image sensor 21 is coupled to application processor 22 via bus 23. In step S11, when communication with application processor 22 begins, controller 60 determines whether to use extended mode. For example, controller 60 checks the extended mode setting stored in register 47, and determines to use extended mode if application processor 22 writes an extended mode setting indicating the use of extended mode.

[0421] In step S11, if the controller 60 determines that the extended mode is not used, the process proceeds to step S12.

[0422] In step S12, the I2C / I3C slave device 46 receives a transmission start command for image data sent from the application processor 22 (described later). Figure 13 (In step S54). In addition, the I2C / I3C slave device 46 receives the communication settings according to the CSI-2 standard sent along with the start command, and writes the received communication settings into register 47 via the CCI slave device 59.

[0423] In step S13, based on the communication settings stored in register 47, existing packet transmission processing is performed in image sensor 21, wherein packets according to the existing CSI-2 standard packet structure are sent to application processor 22.

[0424] Meanwhile, if the controller 60 determines in step S11 that the extended mode is used, the process proceeds to step S14.

[0425] In step S14, the I2C / I3C slave device 46 receives the fixed communication settings required for communication in extended mode (e.g., copying PH / PF channel by channel in GLD) and writes them to register 47 via the CCI slave device 59.

[0426] In step S15, the I2C / I3C slave device 46 receives a transmission start command for image data sent from the application processor 22 (described later). Figure 13 (In step S57). In addition, the I2C / I3C slave device 46 receives the communication settings according to the CSI-2 standard sent along with the start command, and writes them into register 47 via the CCI slave device 59.

[0427] In step S16, the controller 60 determines whether to start sending packets and waits for the process to complete until it is determined that packet sending should begin.

[0428] Then, if it is determined in step S16 that packet transmission will begin, the process proceeds to step S17, and the controller 60 determines whether to transmit data in extended mode. Here, the controller 60 determines whether to transmit data in extended mode based on the content of the data to be transmitted, for example, in the case of transmitting data in the application example described later.

[0429] In step S17, if the controller 60 determines that data should be sent in extended mode, the process proceeds to step S18, where extended mode transmission processing for sending extended packets corresponding to the extended mode is performed (see [link to step S18]). Figure 12 ).

[0430] Meanwhile, if the controller 60 determines in step S17 that it will not send data in extended mode, the process proceeds to step S19.

[0431] In step S19, controller 60 determines whether to send short packets. For example, controller 60 determines to send short packets at the beginning and end of a frame.

[0432] If the controller 60 determines in step S19 that a short packet needs to be sent, the process proceeds to step S20. In step S20, the packet header generation unit 52 generates a packet header and sends the short packet with the existing packet structure to the application processor 22.

[0433] Meanwhile, if the controller 60 determines in step S19 that it will not send short packets (i.e., send long packets), the process proceeds to step S21. In step S21, the packetization unit 51 stores the image data in the payload, and the CRC arithmetic unit 57 obtains the CRC, thereby generating a long packet with the existing packet structure and sending it to the application processor 22.

[0434] After the processing in step S18, step S20, or step S21, the process proceeds to step S22, and the controller 60 ends the packet transmission process. Thereafter, the process returns to step S16, and the processing for transmitting packets is repeated in the same manner for the next packet.

[0435] Figure 12 It is described in Figure 11 The flowchart shows the extended mode sending process performed in step S18.

[0436] In step S31, the packet header generation unit 52 generates a packet header storing VC, data type, WC, etc., and sends it to the application processor 22. At this time, the packet header generation unit 52 writes the extended mode setting information (DataType[5:3] = 3'b111) representing the extended mode and the extended type setting information (DataType[1:0] = 2'b00) that identifies the extended mode mode setting as extended mode 0 into the data type of the packet header.

[0437] In step S32, application processor 22 determines whether to send extended short packets. For example, controller 60 determines to send extended short packets at the beginning and end of a frame.

[0438] In step S32, if the application processor 22 determines that an extended short packet will be sent, the process proceeds to step S33.

[0439] In step S33, the extended packet header generation unit 53 sends an extended packet header with a data type (DataType[7:0]) set to short packet at the first byte of the payload. At this time, the extended packet header generation unit 53 performs various settings (e.g., OePH[7:0], OePF[3:0], etc.) to store in the extended packet header.

[0440] In step S34, the extended packet header generation unit 53 stores the frame number (FN: FrameNumber) in the second byte of the payload for transmission.

[0441] In step S35, according to the settings (OePH[7:0]) executed in step S33, the extended packet header generation unit 53 generates and sends an optional extended packet header, such as... Figure 4 As shown in the image.

[0442] In step S36, the CRC arithmetic unit 57 obtains the CRC and sends it as the packet trailer.

[0443] Meanwhile, if the application processor 22 determines in step S32 not to send extended short packets (i.e., to send long packets), the process proceeds to step S37.

[0444] In step S37, the extended packet header generation unit 53 sends an extended packet header whose data type (DataType[7:0]) is set to a type other than short packets at the first byte of the payload. At this time, the extended packet header generation unit 53 performs various settings (e.g., OePH[7:0], OePF[3:0], etc.) to store in the extended packet header.

[0445] In step S38, according to the settings (OePH[7:0]) executed in step S37, the extended packet header generation unit 53 generates and sends an optional extended packet header, such as... Figure 5 As shown in the image.

[0446] In step S39, the packaging unit 51 packages the image data provided by the image processing unit 43 and generates and sends a conventional payload.

[0447] In step S40, according to the settings (OePF[3:0]) executed in step S37, the extended packet trailer generation unit 54 generates and sends an optional extended packet trailer, such as... Figure 4 As shown in the image.

[0448] In step S41, the CRC arithmetic unit 57 obtains the CRC and sends it as the packet trailer.

[0449] Then, after the processing in step S36 or S41, the extended mode sending process ends.

[0450] As described above, the image sensor 21 is capable of generating and transmitting extended short packets or extended long packets.

[0451] Figure 13 This is a flowchart describing the process of application processor 22 receiving packets.

[0452] For example, processing begins when image sensor 21 is coupled to application processor 22 via bus 23. In step S51, controller 74 writes the initial settings of image sensor 21 (e.g., whether to use C-PHY or D-PHY as the physical layer) into register 73 and sends them to image sensor 21 via CCI host 88 through I2C / I3C master device 72. This allows the initial settings to be written into register 47 of image sensor 21.

[0453] In step S52, the controller 74 identifies whether the image sensor 21 is adapted to the extended mode. For example, the controller 74 can identify whether the image sensor 21 is adapted to the extended mode by obtaining the setting value (e.g., extended PH / PF adaptive capability) stored in the register 47 of the image sensor 21 via the I2C / I3C host device 72. Alternatively, the controller 74 can pre-identify whether the image sensor 21 is adapted to the extended mode based on, for example, manual input.

[0454] In step S53, the controller 74 determines whether the image sensor 21 is adapted to extended mode and whether the application executed by the application processor 22 needs to use extended mode.

[0455] If the controller 74 determines in step S53 that the image sensor 21 is not adapted to the extended mode or does not need to use the extended mode, the process proceeds to step S54.

[0456] In step S54, the controller 74 sends a command to start transmitting image data to the image sensor 21 via the I2C / I3C host 72. At this time, the controller 74 also sends communication settings according to the CSI-2 standard.

[0457] In step S55, existing packet reception processing is performed in application processor 22, wherein packets according to the existing CSI-2 standard packet structure are received based on the communication settings sent in step S54.

[0458] Meanwhile, if the controller 74 determines in step S53 that the image sensor 21 is adapted to the extended mode and the application executed by the application processor 22 requires the use of the extended mode, the process proceeds to step S56.

[0459] In step S56, the I2C / I3C host 72 sends the fixed communication settings required for communication in extended mode before communication begins in extended mode. This allows the fixed communication settings to be written into register 47 of the image sensor 21. Figure 11 Step S14 in the process.

[0460] In step S57, the controller 74 sends a command to start transmitting image data to the image sensor 21 via the I2C / I3C host 72. At this time, the controller 74 also sends communication settings according to the CSI-2 standard.

[0461] In step S58, the packet header detection unit 81 determines whether packet reception has started by verifying the data provided by the physical layer processing unit 71, and waits for this process until it is determined that packet reception has started. For example, if a packet header has been detected from the data provided by the physical layer processing unit 71, the packet header detection unit 81 determines that packet reception has started.

[0462] If the packet header detection unit 81 determines in step S58 that packet reception has begun, the process proceeds to step S59.

[0463] In step S59, the packet header detection unit 81 confirms the data type of the packet header detected in step S58 and determines whether the packet that has started to be received is an extended packet that is adapted to the extended mode. For example, if the extended mode setting information indicates that the data type of the packet header is an extended mode (DataType[5:3] = 3'b111), the packet header detection unit 81 determines that the packet that has started to be received is an extended packet.

[0464] If the packet header detection unit 81 determines in step S59 that the packet that has begun to be received is an extended packet, the process proceeds to step S60. In step S60, extended mode reception processing for receiving extended packets is performed (see...). Figure 14 ).

[0465] Meanwhile, if the packet header detection unit 81 determines that the packet that has been started to be received is not an extended packet, the process proceeds to step S61.

[0466] In step S61, the packet header detection unit 81 confirms the data type (DataType[5:0]) of the packet header detected in step S58, and determines whether the packet that has started to be received is a short packet.

[0467] If the packet header detection unit 81 determines in step S61 that the packet being received is a short packet, the process proceeds to step S62. In step S62, the packet header detection unit 81 receives short packets with existing packet structures sent from the image sensor 21.

[0468] Meanwhile, if the packet header detection unit 81 determines in step S61 that the packet being received is not a short packet (i.e., a long packet has been received), the process proceeds to step S63. In step S63, the unpacking unit 87 receives the payload of the long packet with the existing packet structure sent from the image sensor 21 to retrieve image data, and the CRC arithmetic unit 86 receives the WC+ first byte sent after the packet header as the CRC.

[0469] After multiple processes in steps S60, S62, or S63, the process proceeds to step S64, and the controller 74 terminates the packet reception process. Thereafter, the process returns to S58, and the packet reception process is then repeated in the same manner for the next packet.

[0470] Figure 14 It is described in Figure 13 The flowchart shows the extended mode receiving process performed in step S60.

[0471] In step S71, the packet header detection unit 81 determines whether the mode setting of the extended mode is extended mode 0. For example, if the extended type setting information in the data type of the packet header indicates extended mode 0 (DataType[1:0] = 2'b00), the packet header detection unit 81 determines that the mode setting of the extended mode is extended mode 0.

[0472] If the packet header detection unit 81 determines in step S71 that the extended mode is set to extended mode 0, the process proceeds to step S72. In step S72, the interpretation unit 83 receives the first byte of the payload as the extended packet header.

[0473] In step S73, the interpretation unit 83 confirms the data type (DataType[7:0]) of the extended packet header received in step S72, and determines whether the packet that has started to be received is an extended short packet.

[0474] If the interpretation unit 83 determines in step S73 that the packet is an extended short packet, the process proceeds to step S74. In step S74, the interpretation unit 83 receives an optional extended packet header based on the settings (OePH[7:0]) stored in the extended packet header received in step S72.

[0475] In step S75, the CRC arithmetic unit 86 receives the WC+ first byte sent after the optional extended packet header as the CRC.

[0476] Meanwhile, if the interpretation unit 83 determines in step S73 that the packet is not an extended short packet (i.e., reception of extended long packets has already begun), the process proceeds to step S76. In step S76, the interpretation unit 83 receives an optional extended packet header according to the settings (OePH[7:0]) stored in the extended packet header received in step S72.

[0477] In step S77, the unpacking unit 87 receives the conventional payload of the extended long packet sent from the image sensor 21 and retrieves the image data.

[0478] In step S78, the interpretation unit 83 receives the optional extended packet trailer according to the settings (OePF[3:0]) stored in the extended packet header received in step S72.

[0479] In step S79, the CRC arithmetic unit 86 receives the WC+ first byte sent after the optional extended packet tail as the CRC.

[0480] Then, if it is determined in step S71 that the mode setting of the extended mode is not extended mode 0, the extended mode receiving process ends after the processing in step S75 or after the processing in step S79.

[0481] As described above, the application processor 22 is capable of receiving extended short packets or extended long packets to obtain data.

[0482] <Second structural example of grouping structure>

[0483] refer to Figures 15 to 18A description of a second structural example of the packet structure used in communication between the Extended Mode Adaptive CSI-2 transmitting circuit 31 and the Extended Mode Adaptive CSI-2 receiving circuit 32 is given.

[0484] exist Figures 3 to 8 In the first structural example shown above, the packet header and packet trailer have the same packet structure as those in the existing CSI-2 standard, with the emphasis on maintaining compatibility with the existing CSI-2 standard; extended packet headers, optional extended packet headers, and optional extended packet trailers allow the packet structure to be extended. Conversely, in the second structural example described below, the packet header and packet trailer differ from those in the existing CSI-2 standard, and extended packet headers and extended packet trailers allow the packet structure to be extended.

[0485] Figure 15 The diagram shows the packet structure for short packets (hereinafter referred to as D-PHY-oriented extended short packets) to be used in CSI-2 extended mode when the physical layer is D-PHY.

[0486] for Figure 15 The extended short packet for D-PHY shown in the figure, the extended mode to be compatible with Figure 4 The first structural instance shown is identified in the same way as the extended short packet for D-PHY in the existing CSI-2 standard by the data type stored in the same packet header.

[0487] At the same time, Figure 15 In the extended short packet for D-PHY shown, the frame number is stored in the short packet data field of the next 16 bits of the data type in the packet header, in the same manner as the short packet according to the existing CSI-2 standard. Then, following the packet header, data is transmitted... Figure 4 The extended packet header shown is constructed in the same way as the extended packet header.

[0488] Therefore, the application processor 22 used as the receiving side can interpret the data type stored in the extended packet header to determine, in the case of an extended short packet, that the frame number is stored in the data field of the packet header.

[0489] It should be noted that, in order to Figure 4 The optional extended packet header in the extended short packet for D-PHY shown in the first structural instance is constructed in the same way. Figure 15 The optional extended packet header in the extended short packet for D-PHY is shown in the figure. However, the optional extended packet header has a packet structure that is not embedded in the payload, and therefore does not need to be provided with a CRC at the end.

[0490] Figure 16 The diagram shows the packet structure for long packets (hereinafter referred to as D-PHY-oriented extended long packets) to be used in CSI-2 extended mode when the physical layer is D-PHY.

[0491] exist Figure 16 In the extended long packet format for D-PHY shown, the extended data is sent as part of the packet header or packet trailer and is not embedded in the payload. Therefore, the WC (Write-Cover) of the packet header strictly indicates the byte length of the payload in the same manner as existing standards.

[0492] Figure 17 The packet structure of the short packets used in CSI-2 extended mode (hereinafter referred to as C-PHY-oriented extended short packets) is shown when the physical layer is C-PHY.

[0493] According to the existing CSI-2 standard, strictly send in Figure 17 The extension portion of the extended short packet for C-PHY shown is an extension of the packet header, and thus an extension such as the extended packet header is inserted after the frame number. Then, in the same manner as the existing CSI-2 standard, the packet header ends with a CRC. Furthermore, the packet structure, which uses a SYNC signal in between to send them twice, is similar to the short packet structure according to the existing CSI-2 standard.

[0494] Figure 18 The packet structure of the long packet (hereinafter referred to as the C-PHY-oriented extended long packet) used in the CSI-2 extended mode is shown when the physical layer is C-PHY.

[0495] As mentioned above, Figure 18 The extended long packet for C-PHY shown in the figure Figure 8 The difference between the first structural example shown and the extended long packet for C-PHY is that the WC of the header packet strictly indicates the byte length of the payload in the same manner as existing standards.

[0496] As mentioned above, compared with the existing grouping structure, Figures 15 to 18 The extended grouping structure of the second structure instance shown can be compared with the first structure instance ( Figures 3 to 8 The extended grouping structure of ) is the same as that of ) and is suitable for various applications.

[0497] However, the extended packet of the second structural instance has a packet structure in which the existing packet header or trailer is extended without embedding extended data into the existing payload. Therefore, in the case of using the extended packet structure of the second structural instance, it is impossible to minimize the impact of changes required from the already used communication system compared to the case of using the extended packet structure of the first structural instance. That is, for example, the existing SerDes transmitting circuit 34 requires the SerDes receiving circuit 35 ( Figure 2 )Change.

[0498] As described above, the extended grouping using the first structural instance makes it adaptable to various applications, such as vehicle installations, and allows for the construction of in-vehicle systems by minimizing the impact of needing to change the communication systems already in use.

[0499] Furthermore, the extended packetization using the second architecture instance makes it suitable for various applications, such as vehicle installations, even though existing communication systems may require modifications.

[0500] <Variations on Image Sensors and Application Processors>

[0501] (Example of an image sensor variation)

[0502] Reference Figure 19 Descriptions of variations of the image sensor and application processor are given.

[0503] Constituting the above Figure 9 Image sensor 21 and Figure 10 The individual blocks of the application processor 22 are configured to perform processing in an adaptive manner to both D-PHY-oriented and C-PHY-oriented packets. Alternatively, for example, blocks specifically for performing processing on D-PHY-oriented packets and blocks specifically for performing processing on C-PHY-oriented packets may be provided, with processing switched for each block.

[0504] Figure 19 The image sensor 21A shown in Figure A includes a D-layer processing block 101, a C-layer processing block 102, a switching unit 103, and a controller 60.

[0505] constitute Figure 9 Within the image sensor 21 block, the D-layer processing block 101 includes a block specifically designed for packet processing of the D-PHY. In the constitutive... Figure 9Within the image sensor 21 block, the C-layer processing block 102 includes a block specifically for processing packets directed to the C-PHY. Under the control of the controller 60, when the D-PHY is used for the physical layer, the switching unit 103 outputs the D-PHY-oriented packets generated in the D-layer processing block 101; when the C-PHY is used for the physical layer, a switching is performed to output the C-PHY-oriented packets generated in the C-layer processing block 102.

[0506] (A variation of the application processor)

[0507] Figure 19 The application processor 22A shown in B includes a switching unit 111, a D-layer processing block unit 112, a C-layer processing block unit 113, and a controller 74.

[0508] Under the control of the controller 74, the switching unit 111 switches to provide packets sent from the image sensor 21A to one of the D-layer processing block unit 112 or the C-layer processing block unit 113. The D-layer processing block unit 112 includes components constituting... Figure 10 The application processor 22 in the middle is a block specifically for packet execution processing for D-PHY. The C-layer processing block 113 is included in the structure Figure 10 The application processor 22 in the block is a block specifically for group execution processing for C-PHY.

[0509] In the image sensor 21A and application processor 22A configured in this manner, the physical layer to be used can be set between the controller 60 and the controller 74 before communication begins. Then, for example, if D-PHY is used as the physical layer, packets destined for D-PHY generated in the D-layer processing block 101 are sent via the switching unit 103 and provided to the D-layer processing block 112 for processing via the switching unit 111. Furthermore, for example, if C-PHY is used as the physical layer, packets destined for C-PHY generated in the C-layer processing block 102 are sent via the switching unit 103 and provided to the C-layer processing block 113 for processing via the switching unit 111.

[0510] <Application Examples of Extended Grouping>

[0511] For example, the above extended grouping is considered for use in the following scenarios.

[0512] For example, consider the use case where extended packets are applied to send higher resolution images (RAW24).

[0513] For example, RAW6, RAW7, RAW8, RAW10, RAW12, RAW14, RAW16, and RAW20 are defined as data types to be stored in the packet header according to the existing CSI-2 standard when transmitting image data in RAW format. Meanwhile, in recent years, to accommodate autonomous driving using in-vehicle cameras, there is a desire to transmit higher-resolution images. Therefore, for example, extending the bit count of data types by applying extended packets allows for the definition of higher-resolution RAW24 for data types in extended packet headers.

[0514] In addition, extended grouping is being considered for application to Smart ROI, a technique for sending only the region of interest on the screen.

[0515] For example, many cameras are currently installed in stadiums, airports, and other similar locations. When the entire image captured by these cameras is transmitted from the camera to a cloud server via a network such as the internet, it is assumed that insufficient internet bandwidth, increased data volume or computational demands in the cloud may occur. Therefore, by only cutting out the region of interest at the edge (camera side) and transmitting that region, it is expected to mitigate issues such as insufficient internet bandwidth and increased data volume or computational demands in the cloud.

[0516] When transmitting such a SROI, the coordinates of the upper left corner of the rectangular region (ROI) need to be transmitted together to inform the receiving side where the region of interest corresponds to within the entire image. Additionally, data for the entire image needs to be transmitted at predetermined intervals according to commands from the receiving side. Therefore, for example, the SROI image and data about the entire image (existing packet headers) exist in a mixed manner, frame by frame.

[0517] Therefore, applying extended grouping makes it possible to send 16 bits or more of coordinate data for each of the X and Y coordinates, for example.

[0518] Furthermore, for extended packet handling, consider the use case of GLD, where communication continues even in the event of channel degradation by reducing the number of channels or bandwidth. It should be noted that GLD is a recommendation considered in CSI-2 ver3.0.

[0519] For example, in autonomous driving, even if part of the cable connecting the camera breaks during a collision, communication must continue using the remaining cable, and the vehicle must automatically stop after evacuating to a safe area. Therefore, the onboard camera interface needs to have at least a disconnection detection function and must provide information such as: a line number (16 bits) indicating which line of the image the information is on; a source ID (8 bits) indicating which camera the data was sent from; and a message counter (16 bits) indicating the transmission number. Furthermore, in the case of use with SROI as described above, it is conceivable to send this information in frames.

[0520] Therefore, applying extended packets makes it possible to send this information.

[0521]

[0522] Reference Figures 20 to 26 This describes a configuration example of a rule suitable for prohibiting packet alteration on a transmission path.

[0523] For example, in the case of the above references Figure 2 In the described communication system 11A, when the interfaces of the image sensor 21 and the application processor 22 are different, it is necessary to convert packets on the transmission path. That is, when the physical layer of the image sensor 21 is D-PHY and the physical layer of the application processor 22 is C-PHY, for example, in the SerDes device 26, it is necessary to convert packets from D-PHY-oriented packets to C-PHY-oriented packets.

[0524] As stated above, the undesirable behavior of performing packet switching in SerDes device 26 violates, for example, the provisions of ISO 26262 (functional safety), which prohibits packet changes on the transmission path (hereinafter referred to as E2E (end-to-end) protection).

[0525] Figure 20 This is a block diagram illustrating a configuration example of a communication system 201 suitable for E2E protection, which is a third embodiment of a communication system applying this technology.

[0526] like Figure 20 As shown, the communication system 201 has an image sensor 211, a SerDes device 212, a SerDes device 213, and an application processor 214 coupled to each other. It should be noted that... Figure 20 The example illustrates the case of SERDES using A-PHY, but it also includes cases where another SERDES standard, such as FPD-LINK3, is used for coupling. Alternatively, in the SERDES standard, communication can be performed based on the SERDES standard while maintaining the CIS-2 format (at least with a specific payload). Furthermore, in SERDES, physical layer processing units 237 and 247 can include multiple physical layer processing units of other SERDES standards besides A-PHY, and the physical layer processing units can be switched according to the application.

[0527] The image sensor 211 includes at least an extended mode adaptive CSI-2 transmission circuit 221, a physical layer processing unit adapted to C-PHY or D-PHY or both (hereinafter referred to as C / D-PHY physical layer processing unit) 222, a slave device adapted to I2C and / or I3C or both (hereinafter referred to as I2C / I3C slave device) 223, and a CCI slave device 224.

[0528] SerDes device 212 includes at least a CSI-2 receiving circuit 231, a C / D-PHY physical layer processing unit 232, an I2C / I3C master device 233, a CCI master 234, a CSI-2-oriented A-PHY packet generation unit 235, a CCI-oriented A-PHY packet transmission / reception unit 236, and an A-PHY-adaptive physical layer processing unit 237. For example, in SerDes device 212, packets oriented to C-PHY or D-PHY are converted into packets oriented to A-PHY, and this conversion is determined based on register settings, etc.

[0529] SerDes device 213 includes at least a CSI-2 transmitting circuit 241, a C / D-PHY physical layer processing unit 242, an I2C / I3C slave device 243, a CCI slave device 244, a CSI-2-oriented A-PHY packet receiving unit 245, a CCI-oriented A-PHY packet transmitting / receiving unit 246, and an A-PHY-adaptive physical layer processing unit 247. For example, in SerDes device 213, A-PHY-oriented packets are converted into C-PHY-oriented or D-PHY-oriented packets, and this conversion is determined based on register settings, etc.

[0530] The application processor 214 includes at least an extended mode adaptive CSI-2 receiver circuit 251, a C / D-PHY physical layer processing unit 252, an I2C / I3C master device 253, and a CCI master device 254.

[0531] The communication system 201 is thus configured, and the extended packets described above are sent from the image sensor 211 and received by the application processor 214. Here, even when the communication system 201 is configured to allow the physical layer processing unit 222 of the image sensor 211 to adapt to the D-PHY and the physical layer processing unit 252 of the application processor 22 to adapt to the C-PHY, E2E protection need not be violated.

[0532] Therefore, communication system 201 limits the scope of E2E protection to application-specific payloads (hereinafter referred to as AS payloads) as application-specific payloads in order to accommodate E2E protection. That is, the AS payload is prohibited from being changed when converting A-PHY-oriented packets to C-PHY-oriented or D-PHY-oriented packets, or when converting C-PHY-oriented or D-PHY-oriented packets to A-PHY-oriented packets.

[0533] Figure 21 An example of a structure for an extended packet oriented towards D-PHY that is adapted to E2E protection is shown.

[0534] As shown in the figure, in extended packets for D-PHY, the AS payload, which includes the extended packet header (ePH), packet data, and extended packet trailer (ePF), is limited to the protection range of E2E protection.

[0535] The extended packet header describes predetermined information required when the E2E protection scope is limited to the AS payload. For example, as predetermined information described in the extended packet header, a packet count (PC) indicating the data length of the data stored in the AS payload is added to enable identification of the packet data length. That is, the packet data has a byte count determined by the packet count (PC). Furthermore, as predetermined information described in the extended packet header, the Virtual Channel (VC), indicating the number of virtual channels, is copied to the existing packet header.

[0536] Figure 22 An example of a structure for an extended packet oriented to C-PHY adapted for E2E protection is shown.

[0537] As shown in the figure, similar to the extended packets for D-PHY, the extended packets for C-PHY limit the AS payload, including the extended packet header (ePH), packet data, and extended packet trailer (ePF), to the protection range of E2E protection. Furthermore, in the same manner as the extended packets for D-PHY, the extended packet header describes the packet count (PC) and virtual channel (VC) as necessary predefined information when the protection range of E2E protection is limited to the AS payload.

[0538] Figure 23 An example of a structure for an A-PHY-oriented extended packet adapted to accommodate E2E protection is shown.

[0539] As shown in the figure, in the extended packets facing A-PHY, the AS payload, including the extended packet header (ePH), packet data, and extended packet trailer (ePF), is limited to the protection range of E2E protection.

[0540] Here, for reference Figure 20 As described, in communication system 201, A-PHY-oriented extended packets are generated based on D-PHY-oriented or C-PHY-oriented extended packets sent from image sensor 211 to SerDes device 212. Therefore, the packet count PC and virtual channel VC have already been described in the extended packet header of the A-PHY-oriented extended packets.

[0541] This grouping structure enables the communication system 201 to prevent the AS payload from being altered along the transmission path, thus ensuring compliance with E2E protection. It should be noted that... Figures 21 to 23 The grouping structure shown can be achieved by using... Figures 3 to 8 and Figures 15 to 18 The corresponding group structure shown is used to replace the group structure, thereby allowing a portion of the group generation to be replaced.

[0542] <Packet Transmission / Reception Processing Adapted to E2E Protection>

[0543] Figure 24 This is a flowchart describing packet transmission / reception processing adapted to E2E protection.

[0544] For example, processing begins when data to be stored in packet data (e.g., image data, etc.) is provided to the Extended Mode Adaptive CSI-2 Transmitter Circuit 221. Then, in step S101, in the image sensor 211, the Extended Mode Adaptive CSI-2 Transmitter Circuit 221 stores the supplied data in packet data. Furthermore, the Extended Mode Adaptive CSI-2 Transmitter Circuit 221 generates an extended packet header describing the Virtual Channel VC and the Packet Count PC, as described above. Figure 21 or Figure 22 As shown in the diagram. Then, the Extended Mode Adaptive CSI-2 Transmitter Circuit 221 generates the AS payload by adding an extended packet header and an extended packet trailer to the packet data.

[0545] In step S102, the Extended Mode Adaptive CSI-2 Transmitting Circuit 221 generates extended packets for C-PHY or D-PHY by adding a packet header or a packet trailer for C-PHY or D-PHY to the AS payload generated in step S101. Then, the Extended Mode Adaptive CSI-2 Transmitting Circuit 221 transmits the extended packets for C-PHY or D-PHY to the SerDes device 212 via the C / D-PHY Physical Layer Processing Unit 222.

[0546] In step S103, in the SerDes device 212, the CSI-2 receiving circuit 231 receives the extended packets for C-PHY or D-PHY sent from the image sensor 211 in step S102 via the C / D-PHY physical layer processing unit 232. Then, the CSI-2 receiving circuit 231 obtains the AS payload (excluding packet header and packet trailer) from the received extended packets and provides the AS payload as is to the A-PHY packet generation unit 235 for CSI-2.

[0547] In step S104, in SerDes device 212, the CSI-2-oriented A-PHY packet generation unit 235 generates an A-PHY-oriented extended packet by adding an A-PHY-oriented packet header and an A-PHY-oriented packet trailer to the AS payload supplied from the CSI-2 receiving circuit 231. Then, the CSI-2-oriented A-PHY packet generation unit 235 transmits the A-PHY-oriented extended packet to SerDes device 213 via the A-PHY-adapted physical layer processing unit 237.

[0548] In step S105, in SerDes device 213, the CSI-2-oriented A-PHY packet receiving unit 245 receives the A-PHY-oriented extended packet sent from SerDes device 212 in step S104 via the A-PHY-adapted physical layer processing unit 247. Then, the CSI-2-oriented A-PHY packet receiving unit 245 obtains the AS payload (excluding packet header and packet trailer) from the received extended packet and provides the AS payload to the CSI-2 transmitting circuit 241 as is.

[0549] In step S106, the CSI-2 transmitting circuit 241 generates extended packets for C-PHY or D-PHY by adding a packet header and a packet trailer for C-PHY or D-PHY to the AS payload provided in step S105 from the A-PHY packet receiving unit 245 for CSI-2. Then, the CSI-2 transmitting circuit 241 transmits the extended packets for C-PHY or D-PHY to the application processor 214 via the C / D-PHY physical layer processing unit 242.

[0550] In step S107, within the application processor 214, the extended mode adaptive CSI-2 receiving circuit 251 receives the extended packets for C-PHY or D-PHY transmitted from the SerDes device 213 in step S106 via the C / D-PHY physical layer processing unit 252. Then, the extended mode adaptive CSI-2 receiving circuit 251 extracts the AS payload (excluding packet headers and trailers) from the received extended packets and outputs various types of data stored in the packet data of the AS payload to the subsequent LSI (not shown). After this, the packet transmission / reception processing adapted to E2E protection ends, and similar processing is repeated for the next extended packet.

[0551] As described above, the communication system 201 can send and receive extended packets by performing packet transmission / reception processing adapted to E2E protection without changing the AS payload on the transmission path. In this case, for example, even when the physical layer of the image sensor 211 is D-PHY and the physical layer of the application processor 214 is C-PHY, i.e., when the interfaces differ, E2E protection can still be complied with.

[0552] <Detailed Configuration Example of Image Sensor 211>

[0553] Figure 25 This is a block diagram illustrating a detailed configuration example of the image sensor 211. It should be noted that... Figure 25 In the image sensor 211 shown, and in Figure 9 The configuration of the image sensor 21 is the same as that of the image sensor 21, and its detailed description is omitted.

[0554] That is, with Figure 9 Similar to the image sensor 21 in the previous example, image sensor 211 includes pixels 41, an AD converter 42, an image processing unit 43, a register 47, and a controller 60. Furthermore, image sensor 211 includes an I2C / I3C slave device 223 and a CCI slave device 224, which respectively correspond to... Figure 9 The I2C / I3C slave device 46 and the CCI slave device 59 are included.

[0555] Furthermore, the image sensor 211 includes an extended mode adaptive CSI-2 transmission circuit 221 and a physical layer processing unit 222, and the physical layer processing unit 222 is adapted to A-PHY, C-PHY and D-PHY.

[0556] The extended mode adaptive CSI-2 transmitter circuit 221 includes, in addition to the controller 60 and the CCI slave device 224, an AS payload generator 301, a selector 302, an A-PHY packet generator 303, a C-PHY packet generator 304, a D-PHY packet generator 305, and a selector 306.

[0557] AS payload generator 301 generates AS payloads limited to the protection range of E2E protection and outputs them to selector 302. For example, AS payload generator 301 includes a packing unit 311, an extended packet header generation unit 312, and an extended packet trailer generation unit 313.

[0558] Packaging unit 311 packages the image data provided by image processing unit 43 into data to be sent, and generates packet data with a byte count determined by packet counting PC. For example, controller 60 can control the byte count of the packet data generated by packaging unit 311 according to the setting value (e.g., image size) stored in register 47.

[0559] For reference Figures 21 to 23 As described, for example, the extended packet header generation unit 312 generates an extended packet header that describes the packet count PC and the virtual channel VC, and adds it to the packet data. The extended packet trailer generation unit 313 generates an extended packet trailer and adds it to the packet data.

[0560] Under the control of the controller 60, the selector 302 selects one of the parallel A-PHY packet generator 303, C-PHY packet generator 304 or D-PHY packet generator 305 as the output destination of the AS payload provided by the AS payload generator 301.

[0561] The A-PHY packet generator 303 generates extended packets for A-PHY based on the AS payload provided via selector 302 and outputs them to selector 306. For example, the A-PHY packet generator 303 includes an AAL generation unit 321, an A-PHY-oriented packet header generation unit 322, and an A-PHY-oriented packet trailer generation unit 323.

[0562] For example, the AAL (A-PHY adaptation layer) generation unit 321 divides the AS payload generated by the AS payload generator 301 into 380-byte AS payloads in layers called adaptation layers. Then, the A-PHY-oriented packet header generation unit 322 adds the A-PHY-oriented packet header to the divided AS payload, and the A-PHY-oriented packet trailer generation unit 323 adds the A-PHY-oriented packet trailer.

[0563] C-PHY packet generator 304 generates extended C-PHY packets based on the AS payload provided via selector 302 and outputs them to selector 306. For example, C-PHY packet generator 304 includes a C-PHY packet header generation unit 331, a C-PHY packet trailer generation unit 332, and a C-PHY channel allocation unit 333.

[0564] For example, the C-PHY-oriented packet header generation unit 331 adds the C-PHY-oriented packet header to the AS payload generated by the AS payload generator 301, and the C-PHY-oriented packet trailer generation unit 332 adds the C-PHY-oriented packet trailer to it. Then, the C-PHY-oriented channel allocation unit 333 allocates the C-PHY-oriented extended packets to the three channels according to the CSI-2 standard.

[0565] D-PHY packet generator 305 generates D-PHY-oriented extended packets based on the AS payload provided via selector 302 and outputs them to selector 306. For example, D-PHY packet generator 305 includes a D-PHY-oriented packet header generation unit 341, a D-PHY-oriented packet trailer generation unit 342, and a D-PHY-oriented channel allocation unit 343.

[0566] For example, the D-PHY-oriented packet header generation unit 341 adds the D-PHY-oriented packet header to the AS payload generated by the AS payload generator 301, and the D-PHY-oriented packet trailer generation unit 342 adds the D-PHY-oriented packet trailer to it. Then, the D-PHY-oriented channel allocation unit 343 allocates the D-PHY extended packets to the four channels according to the CSI-2 standard.

[0567] For example, the D-PHY-oriented packet header generation unit 341 adds the D-PHY-oriented packet header to the AS payload generated by the AS payload generator 301, and the D-PHY-oriented packet trailer generation unit 342 adds the D-PHY-oriented packet trailer to it. Then, the D-PHY-oriented channel allocation unit 343 allocates the D-PHY-oriented extended packets to the four channels according to the CSI-2 standard.

[0568] Under the control of the controller 60, the selector 306 selects one of the parallel A-PHY packet generator 303, C-PHY packet generator 304 and D-PHY packet generator 305 as the output source of the extended packets provided to the physical layer processing unit 222.

[0569] Then, when extended packets for A-PHY are provided from A-PHY packet generator 303, physical layer processing unit 222 transmits the extended packets for A-PHY in one channel. Furthermore, when extended packets for C-PHY are provided from C-PHY packet generator 304, physical layer processing unit 222 transmits the extended packets for C-PHY in three channels. Furthermore, when extended packets for D-PHY are provided from D-PHY packet generator 305, physical layer processing unit 222 transmits the extended packets for D-PHY in four channels.

[0570] In the image sensor 211 configured as described above, the extended mode adaptive CSI-2 transmission circuit 221 is configured to allow the AS payload generator 301 to be coupled to the A-PHY packet generator 303, C-PHY packet generator 304, and D-PHY packet generator 305 via a selector 302. This enables the image sensor 211 to generate an AS payload common to the A-PHY-oriented extended packets, the C-PHY-oriented extended packets, and the D-PHY-oriented extended packets within a single AS payload generator 301. That is, the A-PHY packet generator 303, C-PHY packet generator 304, and D-PHY packet generator 305 can share the AS payload generator 301, thereby enabling a reduction in circuit size. Therefore, miniaturization of the image sensor 211 can be achieved.

[0571] <Detailed Configuration Example of Application Processor 214>

[0572] Figure 26 This is a block diagram illustrating a detailed configuration example of the application processor 214. It should be noted that... Figure 26 In the application processor 214 shown, and in Figure 10 The common configuration of the application processor 22 is represented by the same reference numerals, and its detailed description is omitted.

[0573] That is, the application processor 214 is compatible with... Figure 10 The application processor 22 in the same manner includes register 73 and controller 74. It should be noted that controller 74 can be implemented in software. Furthermore, application processor 214 includes I2C / I3C master device 253 and CCI master device 254, which respectively correspond to… Figure 10The I2C / I3C master device 72 and CCI host 88 are included.

[0574] In addition, the application processor 214 includes an extended mode adaptive CSI-2 receiver circuit 251 and a physical layer processing unit 252, and the physical layer processing unit 252 is adapted to A-PHY, C-PHY and D-PHY.

[0575] The extended mode adaptive CSI-2 receiver circuit 251 includes, in addition to the CCI master device 254, a selector 401, an A-PHY packet receiver 402, a C-PHY packet receiver 403, a D-PHY packet receiver 404, a selector 405, and an AS payload receiver 406.

[0576] Selector 401 selects one of the parallel-configured A-PHY packet receiver 402, C-PHY packet receiver 403, and D-PHY packet receiver 404 as the output destination for the extended packets supplied from the physical layer processing unit 252.

[0577] A-PHY packet receiver 402 receives A-PHY-oriented extended packets supplied via selector 401 and outputs them to selector 405. For example, A-PHY packet receiver 402 includes an A-PHY-oriented packet header interpretation unit 411, an A-PHY-oriented packet trailer verification unit 412, and an AAL processing unit 413.

[0578] For example, the A-PHY-oriented packet header interpretation unit 411 interprets the content described in the A-PHY-oriented packet header and performs the processing necessary for receiving extended A-PHY-oriented packets, while the A-PHY-oriented packet trailer verification unit 412 uses the A-PHY-oriented packet trailer to verify the presence or absence of errors. Then, the AAL processing unit 413 performs processing to... Figure 25 The AAL generation unit 321 in the middle combines the divided adaptation layers.

[0579] C-PHY packet receiver 403 receives C-PHY-oriented extended packets provided via selector 401 and outputs them to selector 405. For example, C-PHY packet receiver 403 includes a C-PHY-oriented channel merging unit 421, a C-PHY-oriented packet header interpretation unit 422, and a C-PHY-oriented packet trailer verification unit 423.

[0580] For example, the C-PHY-oriented channel merging unit 421 merges the C-PHY-oriented extended packets allocated in three channels according to the CSI-2 standard and provided via the physical layer processing unit 252. Then, the C-PHY-oriented packet header interpretation unit 422 interprets the contents described in the C-PHY-oriented packet header and performs the processing required to receive the C-PHY-oriented extended packets, and the C-PHY-oriented packet trailer verification unit 423 uses the C-PHY-oriented packet trailer to verify the presence or absence of errors.

[0581] D-PHY packet receiver 404 receives D-PHY-oriented extended packets provided via selector 401 and outputs them to selector 405. For example, D-PHY packet receiver 404 includes a D-PHY-oriented channel merging unit 431, a D-PHY-oriented packet header interpretation unit 432, and a D-PHY-oriented packet trailer verification unit 433.

[0582] For example, the D-PHY-oriented channel merging unit 431 merges the D-PHY-oriented extended packets allocated in four channels according to the CSI-2 standard and provided by the physical layer processing unit 252. Then, the D-PHY-oriented packet header interpretation unit 432 interprets the contents described in the D-PHY-oriented packet header and performs the processing required to receive the D-PHY-oriented extended packets, and the D-PHY-oriented packet trailer verification unit 433 uses the D-PHY-oriented packet trailer to verify the presence or absence of errors.

[0583] Selector 405 selects one of the parallel-configured A-PHY packet receiver 402, C-PHY packet receiver 403, and D-PHY packet receiver 404 as the output source of the extended packets to be provided to AS payload receiver 406.

[0584] With Figure 25 Corresponding to the AS payload generator 301, the AS payload receiver 406 includes an unpacking unit 441, an extended packet header interpretation unit 442, and an extended packet tail verification unit 443. The unpacking unit 441 unpacks the image data packaged by the packing unit 311. For example, the extended packet header interpretation unit 442 interprets the extended packet header generated in the extended packet header generation unit 312 and reads the packet count PC and virtual channel VC. The extended packet tail verification unit 443 uses the extended packet tail added by the extended packet tail generation unit 313 to verify the presence or absence of errors. Then, the AS payload receiver 406 outputs various types of data (e.g., image data, line numbers for vehicle mounting, source IDs, CRC errors, etc.) stored in the packet data provided via the selector 405 to the subsequent stage LSI (not shown).

[0585] In the application processor 214 configured as described above, the extended mode adaptive CSI-2 reception circuit 251 is configured to allow the AS payload receiver 406 to be coupled to the A-PHY packet receiver 402, the C-PHY packet receiver 403, and the D-PHY packet receiver 404 via the selector 405. This enables the application processor 214 to receive an AS payload common to the A-PHY-oriented extended packet, the C-PHY-oriented extended packet, and the D-PHY-oriented extended packet in one AS payload receiver 406. That is, the A-PHY packet receiver 402, the C-PHY packet receiver 403, and the D-PHY packet receiver 404 can share the AS payload receiver 406, thereby enabling a reduction in circuit size. Therefore, miniaturization of the application processor 214 can be achieved.

[0586] <Second configuration example adapted for E2E protection>

[0587] Refer to Figures 27 to 74 for a description of the second configuration example adapted for E2E protection.

[0588] <Configuration example of A-PHY direct coupling configuration>

[0589] Figure 27 The communication system 501 shown in Figure 40 has a direct coupling configuration, in which the image sensor 511 and the application processor 512 are directly coupled to each other via A-PHY (without passing through the SerDes device described later with reference to

[0590] The image sensor 511 includes an A-PHY processing unit 521, a CSIA processing unit 522, a CSI2 processing unit 523, a CSI2-FS processing unit 524, a CCI processing unit 525, a CCI-FS processing unit 526, and a register 527.

[0591] The A-PHY processing unit 521 is implemented with the CCI processing unit 525 as the upper layer and is coupled to the A-PHY processing unit 531 of the application processor 512 via MIPI A-PHY to transmit and receive an extended packet header ePH and an extended packet trailer ePF.

[0592] For example, the CCI-FS processing unit 526 compares the destination ID included in the extended packet header ePH with the ID (source ID) of the image sensor 511 to determine whether to access the image sensor 511.

[0593] The application processor 512 includes an A-PHY processing unit 531, a CSIA processing unit 532, a CSI2 processing unit 533, a CSI2-FS processing unit 534, a CCI processing unit 535, a CCI-FS processing unit 536, a register 537, and a CCI-FS switch 538.

[0594] The A-PHY processing unit 531 is implemented with the CCI processing unit 535 as the upper layer, and is coupled to the A-PHY processing unit 521 of the image sensor 511 via MIPI A-PHY to send and receive extended packet header ePH and extended packet trailer ePF.

[0595] For example, the CCI-FS processing unit 536 compares the destination ID included in the extended packet header ePH with the ID (source ID) of the application processor 512 to determine whether to access the application processor 512.

[0596] For example, the CCI-FS processing unit 536 compares the destination ID included in the extended packet header ePH with the ID (source ID of the application processor 512) and determines whether to access the application processor 512.

[0597] When the CCI-FS processing unit 536 is enabled, the CCI-FS switch 538 performs a switching action to allow data to be sent and received through the CCI-FS processing unit 536; when the CCI-FS processing unit 536 is disabled, data can be sent and received without passing through the CCI-FS processing unit 536.

[0598] refer to Figures 28 to 32 Provide a description of the transmission of read commands and read data in communication system 501.

[0599] Figure 28 An example of the grouping of read commands generated in the CCI-FS processing unit 536 of the application processor 512 during a read access is shown.

[0600] like Figure 28 As shown, the read command consists of the extended packet header ePH* (*=n), the AP (CCI) payload, the extended packet tail ePF1, and the extended packet tail ePF0.

[0601] The extended packet header ePH* (*=n) is shown in the figure and includes extended packet headers ePH0 to ePH3.

[0602] The extended packet header ePH0 stores extended VC, extended DT, extended PFEN, and extended PHEN. For example, extended DT indicates information about the CCI protocol (I2C), and extended DT is used to perform routing processing.

[0603] The extended packet header ePH1 stores the Source ID[7:1] and the Packet Length. For example, the Source ID indicates the source of the CCI protocol (I2C) transmission, and response processing is performed based on the Source ID. The Packet Length indicates the length of the data.

[0604] The extended packet header ePH2 stores a Security Descriptor and a Message Counter. The Security Descriptor indicates whether security is used, and is represented as "8'h0" when security is not used. The Message Counter indicates the packet order and the count value of the messages being counted; the fifth message is represented as "16'h5".

[0605] The extended packet header ePH3 stores the Destination ID[7:1], Read / Write, and Destination Address. The Destination ID[7:1] represents the slave address of the CCI processing unit 525 of the image sensor 511, and in the illustrated example, it is “7'h0D”. For example, the Destination ID indicates the destination of the CCI protocol (I2C) transmission, and routing and communication paths are referenced based on the Destination ID. Read / Write indicates reading or writing data, and in the case of reading, it is “1'b1”. The Destination Address indicates the address of the register 527 of the image sensor 511, which is the final destination, and in the illustrated example, it is “0x0200”.

[0606] The AP(CCI) payload stores, for example, different types of data (data 0[7:0]). The AP(CCI) payload is not transmitted when securely closed, and pseudo-data can be stored and transmitted when securely open.

[0607] When securely closed, no extended packet trailer ePF1 is sent.

[0608] The extended packet tail ePF0 stores the CRC calculation value.

[0609] In the application processor 512, read commands for such a grouping structure are generated in the CCI-FS processing unit 536 and provided to the A-PHY processing unit 531.

[0610] Figure 29 An example of a grouping of read commands output from the A-PHY processing unit 531 of the application processor 512 during a read access is shown.

[0611] like Figure 29 As shown, the A-PHY processing unit 531 uses the read command provided by the CCI-FS processing unit 536 as the protection range of E2E protection to add the A-PHY header and A-PHY trailer.

[0612] The APHY processing unit 531 of the application processor 512 performs A-PHY transmission on the read command of this packet structure. Then, in the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY trailer from the read command. Thereafter, the CCI processing unit 525, via the address "7'h0D" indicated by the destination ID, provides the read command to the CCI-FS processing unit 526.

[0613] Figure 30 An example is shown of the read command to be provided to the CCI-FS processing unit 526 during a read access and the grouping structure of the read data to be generated in the CCI-FS processing unit 526.

[0614] like Figure 30 As shown, Figure 28 The read command for the shown group structure, that is, the read command set to the protection range of E2E protection in APHY transmission, is provided to the CCI-FS processing unit 526.

[0615] As shown, the read data consists of the extended packet header ePH* (*=n), the AP (CCI) payload, the extended packet trailer ePF1, and the extended packet trailer ePF0. Furthermore, the AP (CCI) payload stores the read data value read from address "0x0200" of register 527, indicated by the Destination Address information in the extended packet header ePH of the read command.

[0616] In the image sensor 511, read data with this grouping structure is generated in the CCI-FS processing unit 526 and provided to the A-PHY processing unit 521.

[0617] Figure 31 An example of the grouping of read data output from the A-PHY processing unit 521 of the image sensor 511 during read access is shown.

[0618] like Figure 31 As shown, the A-PHY processing unit 521 uses the read data provided from the CCI-FS processing unit 526 as the protection range for E2E protection to add the A-PHY header and A-PHY trailer.

[0619] The read data with this packet structure is transmitted via A-PHY by the APHY processing unit 521 of the image sensor 511. Then, in the application processor 512, the A-PHY processing unit 521 removes the A-PHY header and A-PHY trailer from the read data and provides the read data to the CCI-FS processing unit 536.

[0620] Figure 32 An example of the grouping structure of read data that will be provided to the CCI-FS processing unit 536 during read access is shown.

[0621] like Figure 32 As shown, the data read from the grouped structure is exactly as follows: Figure 30 As shown, read data set within the protection range of E2E protection in A-PHY transmission is provided to the CCI-FS processing unit 536.

[0622] refer to Figures 33 to 35 A description of write data transmission in the communication system 501 is given. It should be noted that the description is given by assuming that access is performed from the CCI-FS processing unit 526 on the image sensor 511 side.

[0623] Figure 33 An example of the composition of write data packets generated in the CCI-FS processing unit 536 of the application processor 512 during a write access is shown.

[0624] like Figure 33 As shown, the write data consists of the extended packet header ePH* (*=n), the AP (CCI) payload (write data), the extended packet trailer ePF1, and the extended packet trailer ePF0.

[0625] The extended packet header ePH* (*=n) includes extended packet headers ePH0 to ePH3, as shown in the figure.

[0626] Extended packet header ePH0 stores extended VC, extended DT, extended PFEN, and extended PHEN.

[0627] The extended packet header ePH1 stores the source ID[7:1] and packet length.

[0628] The extended packet header ePH2 stores a Security Descriptor and a Message Counter. The Security Descriptor indicates whether security is used, and is represented as "8'h0" when security is not used. The Message Counter indicates the count value of the messages being counted; the fourth message is represented as "16'h4".

[0629] The extended packet header ePH3 stores the Destination ID[7:1], Read / Write, and Destination Address. The Destination ID[7:1] represents the slave address of the CCI processing unit 525 of the image sensor 511, and in the illustrated example, it is “7'h0D”. Read / Write indicates reading or writing data, and in the case of writing, it is “1'b0”. The Destination Address indicates the address of the register 527 of the image sensor 511, which is the final destination, and in the illustrated example, it is “0x1234”.

[0630] The AP(CCI) payload stores the data to be written to the image sensor 511 (Data0[7:0]), and the value of 0xFF is the data to be written.

[0631] During secure shutdown, no extended packet trailer ePF1 is sent.

[0632] The extended packet tail ePF0 stores the CRC calculation value.

[0633] In the application processor 512, the write data of this grouping structure is generated in the CCI-FS processing unit 536 and provided to the A-PHY processing unit 531.

[0634] Figure 34 An example is shown of the composition of packets of write data output from the A-PHY processing unit 531 of the application processor 512 during a write access.

[0635] like Figure 34 As shown, the A-PHY processing unit 531 uses the write data provided from the CCI-FS processing unit 536 as the protection range for E2E protection to add the A-PHY header and A-PHY trailer.

[0636] The A-PHY processing unit 531 of the application processor 512 performs A-PHY transmission on the write data of this packet structure. Then, in the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY trailer from the write data. Thereafter, the write data is provided to the CCI-FS processing unit 526 via the CCI processing unit 525, which is located at address "7'h0D" and is represented by the destination ID.

[0637] Figure 35 An example of a packet structure for write data to be provided to the CCI-FS processing unit 526 during a write access is shown.

[0638] like Figure 35 As shown, just as Figure 33The write data of the packet structure shown (i.e., the write data set within the protection range of E2E protection in A-PHY transmission) is provided to the CCI-FS processing unit 526. Then, the CCI-FS processing unit 526 writes the data stored in the AP (CCI) payload from address "0x1234" (i.e., the source address information (destination address) of the extended packet header ePH of the read command) in register 527 indicated by the CCI command ID information.

[0639] Reference Figure 36 This section provides an overview of the extended packet header (ePH) and extended packet trailer (ePF).

[0640] like Figure 36 As shown, a CCI-FS E2E packet consists of an extended packet header (ePH), packet data, and an extended packet trailer (ePF), and its packet length is Length = Byte count × Data byte width.

[0641] Fields such as Extended VC, Extended DT, or Message Counter are used as Extended Packet Header (ePH). The length of the Extended Packet Header (ePH) can be changed using field values ​​(epFEN fields) of the Extended Packet Header (ePH).

[0642] The packet data consists of, for example, PL data bytes (data 0 to data PL-1), with a length equal to the packet length (PL) multiplied by the data byte width. In the case of a read command, no data is stored in the packet data when securely closed; when securely open, 1 byte of pseudo-data is stored in the packet data. In the case of a write access, the write data for the payload is stored in the packet data. In the case of a read access, the read data for the payload is stored in the packet data. When clock stretching is used (ePH0 control code indicator = 1), a 1-byte data payload indicating the control type is added to the packet data.

[0643] The length of the extended packet trailer ePF1 can be changed using the epFEN field setting value of the extended packet header ePH. Additionally, security-related information can be added.

[0644] The CRC-32 calculated from the packet data can be added to the extended packet trailer ePF0 using the field setting value of the extended packet header ePH.

[0645] <Processing Examples of Communication>

[0646] Reference Figures 37 to 39 The flowchart shows that in Figure 27 Description of the communication processing using CCI-FS performed in the communication system 501 shown.

[0647] like Figure 37As shown, in steps S211 to S222, initial setup and confirmation operations are performed.

[0648] In step S211, two read accesses are performed on the capability register of the CCI-FS processing unit 526 from the application processor 512 to the image sensor 511. It should be noted that the number of read accesses is not limited to two and can be optionally set according to functional safety requirements; for example, the number of read accesses may be one, or three or more times.

[0649] In step S212, in the application processor 512, the CSI2-FS processing unit 524 determines whether the capability register value of the CCI-FS processing unit 526 is 1'b1 both times, as a result of the read access in step S211. If it is determined in step S212 that the capability register value of the CCI-FS processing unit 526 is not 1'b1 both times, the process proceeds to step S213.

[0650] In step S213, the CSI2-FS processing unit 524 in the application processor 512 determines whether the number of retransmissions is more than three. It should be noted that the number of retransmissions is not limited to three and can be set to any number; this also applies to the retransmission count described below. If it is determined in step S213 that the number of retransmissions is not more than three (one or two), the processing returns to step S211, and then a similar process is repeated.

[0651] Meanwhile, if the capability register value of the CCI-FS processing unit 526 is determined to be 1'b1 twice in step S212, the processing proceeds to step 214.

[0652] In step S214, a write access is performed on the enable register of the CCI-FS processing unit 526 from the application processor 512 to the image sensor 511.

[0653] In step S215, in the image sensor 511, the CCI-FS processing unit 526 performs a write access to the enable register of the CCI-FS processing unit 536 of the application processor 512.

[0654] In step S216, the slave address of the image sensor 511 is set in the destination SID register of the CCI-FS processing unit 536 of the application processor 512.

[0655] In step S217, the ePH register of the CCI-FS processing unit 536 of the application processor 512 is set.

[0656] In step S218, the ePH register of the CCI-FS processing unit 526 is set from the application processor 512 to the image sensor 511.

[0657] In step S219, a read access is performed on the enable register and error register of the CCI-FS processing unit 526 from the application processor 512 to the image sensor 511.

[0658] In step S220, in application processor 512, CCI-FS processing unit 536 determines the result of the read access in step S219, whether the enable register value of CCI-FS processing unit 526 is 1'b1, and whether the error register value is 0.

[0659] If, in step S220, it is determined that the enable register value of the CCI-FS processing unit 526 is not 1'b1 or the error register value is not zero, the processing proceeds to step S221.

[0660] In step S221, the CSI2-FS processing unit 524 in the application processor 512 determines whether the number of retransmissions is three or more. If it is determined in step S221 that the number of retransmissions is three or more, the processing returns to step S211, and then similar processing is repeated.

[0661] Meanwhile, if it is determined in step S213 that the number of retransmissions is three or more, or if it is determined in step S221 that the number of retransmissions is not three or more (one or two), the process proceeds to step S222.

[0662] In step S222, communication is performed via CCI without using CCI-FS, and then the communication process ends.

[0663] Meanwhile, if in step S220 it is determined that the enable register value of the CCI-FS processing unit 526 is 1'b1 and the error register value is 0, the processing proceeds to step S223.

[0664] like Figure 38 As shown, in steps S223 to S234, a write operation using CCI-FS is performed.

[0665] In step S223, the CCI-FS processing unit 536 of the application processor 512 sets the ePH register to perform a write operation.

[0666] In step S224, the CCI-FS processing unit 536 of the application processor 512 sets the write data register.

[0667] In step S225, the CCI-FS processing unit 536 of the application processor 512 sets the command execution register to one.

[0668] In step S226, in the application processor 512, the A-PHY processing unit 531 uses the write data generated by the CCI-FS processing unit 536 as the protection range for E2E protection, and performs A-PHY transmission by adding an A-PHY header and an A-PHY trailer, as described above. Figure 34 As shown.

[0669] In step S227, in the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY trailer from the write data and provides the protection range of E2E protection to the CCIFS processing unit 526.

[0670] In step S228, in the image sensor 511, the CCI-FS processing unit 526 confirms the source ID of the image sensor 511 and the destination SID of the extended packet header ePH based on the content of the extended packet header ePH.

[0671] In step S229, in image sensor 511, CCI-FS processing unit 526 determines whether the source ID of image sensor 511 and the destination SID of extended packet header ePH confirmed in step S228 are consistent with each other.

[0672] If, in step S229, it is determined that the source ID of the image sensor 511 and the destination SID of the extended packet header ePH are consistent with each other, the process proceeds to step S230.

[0673] In step S230, in the image sensor 511, the CCI-FS processing unit 526 retrieves the content acknowledgment message counter from the extended packet header ePH.

[0674] In step S231, in the image sensor 511, the CCI-FS processing unit 526 determines whether the message counter (received) of the image sensor 511 and the message counter of the extended packet header ePH confirmed in step S230 are consistent with each other.

[0675] If, in step S231, it is determined that the message counter (received) of the image sensor 511 and the message counter of the extended packet header ePH are consistent with each other, the process proceeds to step S232.

[0676] In step S232, in the image sensor 511, the CCI-FS processing unit 526 confirms the CRC from the contents of the extended packet tail ePF.

[0677] In step S233, in the image sensor 511, the CCI-FS processing unit 526 determines whether the received value (ePF0) of the extended packet tail ePF confirmed in step S232 is consistent with the CRC calculation result calculated in the CCI-FS processing unit 526.

[0678] If the received value (ePF0) of the extended packet trailer ePF and the CRC calculation result are consistent in step S233, the process proceeds to step S234.

[0679] In step S234, in the image sensor 511, the CCI-FS processing unit 526 performs a write process to write data from the contents of the Extended Packet Header (ePH) and Extended Packet Trailer (ePF) to the address of register 527. Afterward, the process proceeds to step S235.

[0680] like Figure 39 As shown, in steps S235 to S247, a read operation using CCI-FS is performed.

[0681] In step S235, in application processor 512, CCI-FS processing unit 536 sets the ePH register to allow read operations.

[0682] In step S236, in the application processor 512, the CCI-FS processing unit 536 sets the command execution register to one.

[0683] In step S237, in the application processor 512, the A-PHY processing unit 531 performs A-PHY transmission by adding the A-PHY header and A-PHY trailer as the protection range of E2E protection using the write data generated by the CCI-FS processing unit 536, as described above. Figure 29 As shown.

[0684] In step S238, in the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY trailer from the write data and provides the protection range of E2E protection to the CCI-FS processing unit 526.

[0685] In step S239, in the image sensor 511, the CCI-FS processing unit 526 confirms the source ID of the image sensor 511 and the destination SID of the extended packet header ePH from the contents of the extended packet header ePH.

[0686] In step S240, in image sensor 511, CCI-FS processing unit 526 determines whether the source ID of image sensor 511 and the destination SID of extended packet header ePH confirmed in step S239 are consistent with each other.

[0687] If, in step S240, it is determined that the source ID of the image sensor 511 and the destination SID of the extended packet header ePH are consistent with each other, the process proceeds to step S241.

[0688] In step S241, in the image sensor 511, the CCI-FS processing unit 526 retrieves the content confirmation message counter from the extended packet header ePH.

[0689] In step S242, in the image sensor 511, the CCI-FS processing unit 526 determines whether the message counter (received) of the image sensor 511 confirmed in step S241 and the message counter of the extended packet header ePH are consistent with each other.

[0690] If, in step S242, it is determined that the message counter (received) of the image sensor 511 and the message counter of the extended packet header ePH are consistent with each other, the process proceeds to step S243.

[0691] In step S243, in the image sensor 511, the CCI-FS processing unit 526 confirms the CRC based on the content of the extended packet tail ePF.

[0692] In step S244, in the image sensor 511, the CCI-FS processing unit 526 determines whether the received value (ePF0) of the extended packet tail ePF confirmed in step S243 is consistent with the CRC calculation result calculated by the CCI-FS processing unit 526.

[0693] In step S244, it is determined that the received value (ePF0) of the extended packet trailer ePF and the CRC calculation result are consistent with each other, and the process ends.

[0694] At the same time, Figure 38 Step S229 or Figure 39 In step S240, if it is determined that the source ID of the image sensor 511 and the destination SID of the extended packet header ePH are inconsistent with each other, the process proceeds to step S245.

[0695] In step S245, the error register (Routing) on ​​the image sensor 511 side is set to 1, and then the process ends.

[0696] At the same time, Figure 38 Step S231 or Figure 39In step S242, if it is determined that the message counter (received) of the image sensor 511 and the message counter of the extended packet header ePH do not match each other, the process proceeds to step S246.

[0697] In step S246, the error register (MC) on the image sensor 511 side is set to 1, and then the process ends.

[0698] Meanwhile, in Figure 38 step S233 or Figure 39 if it is determined in step S244 that the received value (ePF0) of the extended packet footer ePF and the CRC calculation result do not match each other, the process proceeds to step S247.

[0699] In step S247, the error register (CRC) on the image sensor 511 side is set to 1, and then the process ends.

[0700] <Composition example of SerDes coupling>

[0701] Figure 40 The communication system 601 shown in

[0702] has a SerDes coupling configuration, in which the image sensor 611 and the application processor 614 are coupled to each other via a SerDes device 612 acting as the slave side and a SerDes device 613 acting as the master side.

[0703] The slave side SerDes device 612 includes an A-PHY processing unit 631, a CSIA processing unit 632, a CSI2-FS processing unit 633, an I2C / I3C master device 634, a CCI processing unit 635, a CCI-FS processing unit 636, and a register 637.

[0704] The master side SerDes device 613 includes an A-PHY processing unit 641, a CSIA processing unit 642, a CS12-FS processing unit 643, an I2C / I3C slave device 644, a CCI processing unit 645, a CCI-FS processing unit 646, and a register 647.

[0705] The application processor 614 includes an I2C / I3C master device 651, a CCI processing unit 652, a CCIFS processing unit 653, a register 654, and a CCI-FS switch 655.

[0706] Note that in Figure 40In the SerDes coupling configuration shown, if the CCI or CCI-FS configuration is implemented as the upper-level protocol, another SerDes standard can be used. For example, by implementing such a configuration in the payload from the application layer or the corresponding upper-level layer. Figure 41 The extended packet header ePH, extended packet trailer ePF1, and extended packet trailer ePF0 shown can be configured according to various SerDes-related standards, such as PCIe, USB, DisplayPort, HDMI (registered trademark), LVDS, and FPD-LINK.

[0707] refer to Figures 41 to 49 The transmission of read commands and read data in the communication system 601 is described.

[0708] Figure 41 An example of the grouping of read commands generated in the CCI-FS processing unit 653 of the application processor 614 during a read access is shown.

[0709] like Figure 41 As shown, the read command consists of an extended packet header ePH* (*=n), an extended packet trailer ePF1, and an extended packet trailer ePF0. It should be noted that the details are the same as those described above. Figure 28 The details of the read command are similar.

[0710] In the application processor 614, a read command for this grouping structure is generated in the CCI-FS processing unit 653 and provided to the I2C / I3C master device 651.

[0711] Figure 42 An example of a grouping of read commands output from the I2C / I3C master device 651 of the application processor 614 during a read access is shown.

[0712] like Figure 42 As shown, after start condition S, the I2C / I3C master device 651 sends the sensor address of the coupling destination, that is, Figure 40 The address (slave address + W 8 bits) of the CCI processing unit 645 of the master device side SerDes device 613 in the configuration shown. Figure 42 In the example shown, the address of the CCI processing unit 645 is from address [7:1] = 7'h0F. Following this address, the register address (register address [15:8] and register address [7:0]) of the master-side SerDes device 613 is sent to register 647. After the extended packet header ePH* (*=n), extended packet trailer ePF1, and extended packet trailer ePF0, the I2C / I3C master device 651 finally sends the stop condition P.

[0713] The I2C / I3C master device 651 of the application processor 614 sends a read command with this packet structure. In the master-side SerDes device 613, the I2C / I3C slave device 644 obtains the read command (extended packet header ePH* (*=n), extended packet trailer ePF1, and extended packet trailer ePF0). The read command is provided to the CCI processing unit 645 at slave address [7:1]=7'h0F, and then provided to the A-PHY processing unit 641 via the CCI-FS processing unit 646, the CS12-FS processing unit 643, and the CSIA processing unit 642.

[0714] Figure 43 An example is shown of a group of read commands output by the A-PHY processing unit 641 of the master-side SerDes device 613 during a read access.

[0715] like Figure 43 As shown, the A-PHY processing unit 641 uses the read command obtained from the I2C / I3C slave device 644 as the protection range for E2E protection to add the A-PHY header and A-PHY trailer. It should be noted that in the CS12-FS processing unit 643, the address of the CCI processing unit 635 of the master-side SerDes device 613 (e.g., from address [7:1] = 7'h0E) is added to the extended packet header ePH* (* = n).

[0716] The A-PHY transmission of the read command with this packet structure is performed by the A-PHY processing unit 641 of the master-side SerDes device 613. In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY trailer from the read command. The read command is supplied to the CCI processing unit 635 at the slave address "7'h0E" indicated by the destination ID via the CSIA processing unit 632, the CSI2-FS processing unit 633, and the CCI-FS processing unit 636, and then supplied to the I2C / I3C master device 634.

[0717] Figure 44 An example of the composition of a group of read commands output from the I2C / I3C master device 634 during a read access is shown.

[0718] like Figure 44 As shown, after start condition S, the I2C / I3C master device 634 sends the sensor address of the coupling destination, that is, Figure 40 The address (from address + W 8 bits) of the CCI processing unit 622 of the image sensor 611 in the configuration shown. Figure 44In the example shown, the address of the CCI processing unit 622 is from address [7:1] = 7'h0D. After this address, the register address of the image sensor 611's register 624 (register address [15:8] and register address [7:0]) is sent. After the extended packet header ePH* (*=n), extended packet trailer ePF1, and extended packet trailer ePF0, the I2C / I3C master device 634 finally sends the stop condition P.

[0719] The SerDes device 612 on the self-side transmits a read command with this packet structure according to I2C / I3C. Then, in the image sensor 611, the I2C / I3C slave device 621 obtains the read command (extended packet header ePH* (*=n), extended packet trailer ePF1, and extended packet trailer ePF0). The read command is provided to the CS12-FS processing unit 623 from the CCI processing unit 622 at address [7:1]=7'h0D.

[0720] Figure 45 An example is shown of the read command to be provided to the CS12-FS processing unit 623 during a read access and the grouping structure of the read data generated in the CS12-FS processing unit 623.

[0721] like Figure 45 As shown, just as Figure 41 The read command for the group structure shown, that is, the read command set to the protection range of E2E protection in APHY transmission, is provided to the CSI2-FS processing unit 623.

[0722] As shown, the read data consists of the extended packet header ePH* (*=n), the AP (CCI) payload, the extended packet trailer ePF1, and the extended packet trailer ePF0. Furthermore, the AP (CCI) payload stores the read data value read from address "0x0200" of register 624, indicated by the source address information (destination address) of the extended packet header ePH of the read command.

[0723] In the image sensor 611, the read data of this grouped structure is generated in the CCI-FS processing unit 623 and provided to the I2C / I3C slave device 621 via the CCI processing unit 622.

[0724] Figure 46 An example of the grouping of read data output from the I2C / I3C slave device 621 of the image sensor 611 during a read access is shown.

[0725] like Figure 46 As shown, after start condition S, the I2C / I3C slave device 621 sends the sensor address of the coupling destination, that is, Figure 40The address (slave address + W 8 bits) of the I2C / I3C master device 634 of the slave-side SerDes device 612 in the configuration shown. Figure 46 In the example shown, the address of the I2C / I3C master device 634 is the slave address [7:1] = 7'h0D. Following this address, the storage address for the read data (the address of register 624 of the image sensor 611) is sent, along with the address of the I2C / I3C master device 634 of the slave-side SerDes device 612 (slave address + R 8 bits). After the I2C / I3C slave device 621 sends the extended packet header ePH* (*=n), AP (CCI) payload, extended packet trailer ePF1, and extended packet trailer ePF0, it finally sends the stop condition P.

[0726] Read commands with this packet structure are sent from the I2C / I3C slave device 621 of the image sensor 611 according to the I2C / I3C protocol. In the slave-side SerDes device 612, the I2C / I3C master device 634 acquires the read data (extended packet header ePH* (*=n), AP (CCI) payload, extended packet trailer ePF1, and extended packet trailer ePF0). The read data is provided to the CCI processing unit 635 at slave address [7:1]=7'h0E, and then provided to the A-PHY processing unit 631 via the CCI-FS processing unit 636, the CS12-FS processing unit 633, and the CSIA processing unit 632.

[0727] Figure 47 An example of the grouping of read data output from the A-PHY processing unit 631 of the SerDes device 612 on the self-side during a read access is shown.

[0728] like Figure 47 As shown, the A-PHY processing unit 631 uses the read data obtained by the I2C / I3C master device 634 as the protection range of E2E protection to add the A-PHY header and A-PHY tail.

[0729] The A-PHY processing unit 631 of the slave-side SerDes device 612 performs A-PHY transmission on the read data of this packet structure. Then, in the master-side SerDes device 613, the A-PHY processing unit 641 removes the A-PHY header and A-PHY trailer from the read data. The read data is provided to the I2C / I3C slave device 644 through the CSIA processing unit 642, CSI2-FS processing unit 643, CCI-FS processing unit 646, and CCI processing unit 635.

[0730] Figure 48 An example of the composition of read data packets output from the master-side SerDes device 613 to the device 644 during a read access is shown.

[0731] like Figure 48 As shown, after start condition S, the I2C / I3C slave device 644 sends the sensor address of the coupling destination, i.e., Figure 40 In the configuration shown, the address of the CCI processing unit 635 of the master-side SerDes device 613 (slave address + W 8 bits). Figure 48 In the example shown, the address of the CCI processing unit 635 is slave address [7:1] = 7'h0F. Following this address, the register address (Register Address [15:8] and RegisterAddress [7:0]) of register 647 of the master-side SerDes device 613 is sent, along with the address of the CCI processing unit 635 (slave Address + R8 bits). Subsequently, the I2C / I3C slave device 644 sends the extended packet header ePH* (*=n), the AP (CCI) payload, the extended packet trailer ePF1, and the extended packet trailer ePF0, and finally, the stop condition P is sent.

[0732] Read data with this packet structure is sent from the master-side SerDes device 613 via the I2C / I3C slave device 644. Then, in the application processor 614, the I2C / I3C master device 651 obtains the read command (extended packet header ePH* (*=n), extended packet trailer ePF1, and extended packet trailer ePF0) and provides the read command to the CCI processing unit 653.

[0733] Figure 49 An example of the grouping structure of read data provided to the CCI-FS processing unit 653 during read access is shown.

[0734] like Figure 49 As shown, just as Figure 45 The read data of the packet structure shown, that is, the read data set within the protection range of E2E protection in A-PHY transmission, is provided to the CCI-FS processing unit 653.

[0735] <Processing Examples of Communication>

[0736] Reference Figures 50 to 57 The flowchart is given in Figure 40 The communication processing using CCI-FS is described in the communication system 601 shown.

[0737] like Figure 50 As shown, in steps S301 to S317, initial setup and confirmation operations are performed.

[0738] In step S301, the slave address of the image sensor 611 is set in the destination SID register of the CCI-FS processing unit 653 of the application processor 614.

[0739] In step S302, the ePH register of the CCI-FS processing unit 653 of the application processor 614 is set.

[0740] In step S303, the destination SID of the bridge configuration of the CCI-FS processing unit 653 of the application processor 614 is set, and the master-side SerDes device 613 is registered. Here, it is assumed that the address, attribute, and Timeout_no1 registers are also set in the same manner.

[0741] In step S304, the ePH register of the CCI-FS processing unit 643 is set from the application processor 614 to the main-side SerDes device 613.

[0742] In step S305, a destination SID is set from the application processor 614 to the master-side SerDes device 613, which is configured by the bridge of the CCI-FS processing unit 643, and the slave-side SerDes device 612 is registered.

[0743] In step S306, a read access is performed on the error register of the CCI-FS processing unit 643 from the application processor 614 to the host-side SerDes device 613.

[0744] In step S307, in application processor 614, as a result of the read access in step S306, CCI-FS processing unit 653 determines whether the register value of the error register of CCIFS processing unit 643 of master-side SerDes device 613 is zero.

[0745] If, in step S307, it is determined that the register value of the error register of the CCI-FS processing unit 643 of the master-side SerDes device 613 is not zero (other than zero), the process proceeds to step S308.

[0746] In step S308, in application processor 614, CCI-FS processing unit 653 determines whether the number of retransmissions is more than three; if it is determined that the number of retransmissions is not more than three (1 or 2 times), it returns to step S304 and then repeats a similar process.

[0747] Meanwhile, if the error register value of the CCI-FS processing unit 643 of the master-side SerDes device 613 is determined to be zero in step S307, the process proceeds to step S309.

[0748] In step S309, the ePH register of the CCI-FS processing unit 636 is set from the application processor 614 to the slave-side SerDes device 612.

[0749] In step S310, the destination SID is set from the application processor 614 to the slave SerDes device 612 by the bridge of the CCI-FS processing unit 636, and the slave SerDes device 612 is registered.

[0750] In step S311, a read access is performed on the error register of the CCI-FS processing unit 636 from the application processor 614 to the slave-side SerDes device 612.

[0751] In step S312, in application processor 614, as a result of the read access in step S311, CCI-FS processing unit 653 determines whether the register value of the error register of CCI-FS processing unit 636 of slave-side SerDes device 612 is zero.

[0752] If, in step S312, it is determined that the register value of the error register of the CCI-FS processing unit 636 of the slave-side SerDes device 612 is not zero (other than zero), the process proceeds to step S313.

[0753] In step S313, in application processor 614, CCI-FS processing unit 653 determines whether the number of retransmissions is more than three; if it is determined that the number of retransmissions is not more than three (once or twice), the processing returns to step S309, and then similar processing is repeated.

[0754] Meanwhile, if it is determined in step S312 that the register value of the error register of the CCI-FS processing unit 636 of the slave-side SerDes device 612 is zero, the process proceeds to step S314.

[0755] In step S314, the ePH register of the CCI-FS processing unit 623 is set from the application processor 614 to the image sensor 611.

[0756] In step S315, a read access is performed on the error register of the CCI-FS processing unit 623 from the application processor 614 to the image sensor 611.

[0757] In step S316, in application processor 614, as a result of the read access in step S315, CCI-FS processing unit 653 determines whether the register value of the error register of CCI-FS processing unit 623 of image sensor 611 is zero.

[0758] If, in step S316, it is determined that the register value of the error register of the CCI-FS processing unit 623 of the image sensor 611 is not zero (other than zero), the process proceeds to step S317.

[0759] In step S317, in application processor 614, CCI-FS processing unit 653 determines whether the number of retransmissions is more than three; if it is determined that the number of retransmissions is not more than three (1 or 2 times), it returns to step S314 and then repeats a similar process.

[0760] Here, in step S308, step S313 or step S317, if it is determined that the number of retransmissions is three or more, the process returns to step S301, and then similar processing is repeated.

[0761] Meanwhile, if the error register value of the CCI-FS processing unit 623 of the image sensor 611 is determined to be zero in step S316, the processing proceeds to step S318.

[0762] like Figure 51 As shown, in steps S318 to S327, a write operation using CCI-FS is performed.

[0763] In step S318, the CCI-FS processing unit 653 of the application processor 614 sets the ePH register to perform a write operation.

[0764] In step S319, the CCI-FS processing unit 653 of the application processor 614 sets the write data register.

[0765] In step S320, the CCI-FS processing unit 653 of the application processor 614 sets the command execution register to 1 and issues a write command.

[0766] In step S321, the application processor 614 executes the subsequent reference Figure 53 The sequence A_Write (in AP) is described and processed.

[0767] In step S322, the master-side SerDes device 613 executes a later reference. Figure 56 Sequence B, as described, is processed (in SerDes(Master)). It should be noted that... Figure 56 The process described is sequence B (in SerDes(Slave)) performed by the slave-side SerDes device 612; however, the master-side SerDes device 613 may also perform similar processing with each corresponding block.

[0768] In step S323, the A-PHY processing unit 641 performs A-PHY transmission by adding the A-PHY header and A-PHY trailer to the extended DT of the extended packet header ePH of the master-side SerDes device 613 via the CS12-FS processing unit 643 and the CSIA processing unit 642.

[0769] In step S324, the slave-side SerDes device 612 executes the later reference. Figure 56 The sequence B described is processed (in SerDes(Slave)).

[0770] In step S325, the slave-side SerDes device 612 executes the later reference. Figure 53 The sequence described is processed by A_Write (when in SerDes(Slave)). It should be noted that... Figure 53 The description of the sequence A_Write (at AP) processing performed by the application processor 614 is given; however, the SerDes device 612 on the slave side may also perform similar processing with each corresponding block.

[0771] In step S326, the image sensor 611 performs the following reference Figure 56 Sequence B (described in the image sensor) is processed. It should be noted that... Figure 56 The process described is sequence B (in SerDes (Slave)) performed by the slave-side SerDes device 612; however, the image sensor 611 may also perform similar processing using each corresponding block.

[0772] In step S327, in the image sensor 611, the CCI-FS processing unit 623 performs a write process to write data from the contents of the Extended Packet Header (ePH) and Extended Packet Trailer (ePF) to the address of register 624. Afterwards, the process proceeds to step S328.

[0773] like Figure 52 As shown, in steps S328 to S344, a read operation using CCI-FS is performed.

[0774] In step S328, the CCI-FS processing unit 653 of the application processor 614 sets the ePH register to perform a read operation.

[0775] In step S329, the CCI-FS processing unit 653 of the application processor 614 sets the read data register.

[0776] In step S330, the CCI-FS processing unit 653 of the application processor 614 sets the command execution register to 1 and issues a read command.

[0777] In step S331, application processor 614 executes the following reference. Figure 54 The sequence A_Read_CMD (in AP) is described. Here, in the processing of sequence A_Read_CMD (in AP), the processing of two branches is executed in parallel; according to branch A, the processing proceeds to step S332, and according to branch B, the processing proceeds to step S339.

[0778] In step S332, the master-side SerDes device 613 executes the later reference. Figure 56 Sequence B, as described, is processed (in SerDes(Master)). It should be noted that... Figure 56 The process described is sequence B (in SerDes(Slave)) performed by the slave-side SerDes device 612; however, the master-side SerDes device 613 may also perform similar processing with each corresponding block.

[0779] In step S333, the A-PHY processing unit 641 performs A-PHY transmission by adding the A-PHY header and A-PHY trailer to the extended DT of the extended packet header ePH of the master-side SerDes device 613 via the CS12-FS processing unit 643 and the CSIA processing unit 642.

[0780] In step S334, the slave-side SerDes device 612 executes the later reference. Figure 56 The sequence B described is processed (in SerDes(Slave)).

[0781] In step S355, the slave-side SerDes device 612 performs the following reference. Figure 54 The sequence described is processed by A_Read_CMD (during SerDes(Slave)). Note that in... Figure 54 The text describes the sequence A_Read_CMD (at AP) processing executed in the application processor 614; however, the slave-side SerDes device 612 can also perform similar processing using each corresponding block. Here, in the sequence A_Read_CMD (SerDes(Slave)) processing in the two-branch processing, the processing does not proceed to branch A, but instead proceeds to step S336 according to branch B.

[0782] In step S336, the slave-side SerDes device 612 executes the later reference. Figure 57 The sequence A_Read_Data described is processed (during SerDes(Slave)). Note that in... Figure 57The description of the sequence A_Read_Data (at AP) processing executed in the application processor 614 is given; however, the slave-side SerDes device 612 may also perform similar processing with each corresponding block.

[0783] In step S337, the A-PHY processing unit 631 adds the A-PHY header and A-PHY trailer to the extended packet header ePH of the SerDes device 612 on the self-side via the CS12-FS processing unit 633 and the CSIA processing unit 632, and performs A-PHY transmission.

[0784] In step S338, the master-side SerDes device 613 executes the later reference. Figure 56 Sequence B, as described, is processed (in SerDes(Master)). It should be noted that... Figure 56 The description of the processing of sequence B (in SerDes(Slave)) performed in the slave-side SerDes device 612 is given; however, the master-side SerDes device 613 may also perform similar processing with each corresponding block.

[0785] In step S339, application processor 614 executes the subsequent reference Figure 57 The sequence A_Read_Data (in AP) is described and processed.

[0786] In step S340, application processor 614 executes the subsequent reference Figure 56 Sequence B (described in AP) is processed. It should be noted that... Figure 56 The description of the processing of sequence B (in the case of SerDes(Slave)) executed in the slave-side SerDes device 612 is given; however, the application processor 614 may also perform similar processing with each corresponding block.

[0787] In step S341, in the application processor 614, the CCI-FS processing unit 653 stores the read data from the extended packet header ePH and extended packet trailer ePF in the address of register 654.

[0788] In step S342, an error register confirmation is performed on the read processing in the image sensor 611, the slave-side SerDes device 612, the master-side SerDes device 613, and the application processor 614.

[0789] In step S343, the image sensor 611 and the devices (the slave-side SerDes device 612, the master-side SerDes device 613, and the application processor 614) determine whether the register value of the error register of each CCI-FS processing unit is zero.

[0790] If, in step S343, it is determined that the register values ​​of all CCI-FS processing units are not zero (there is a register value other than zero among them), the processing proceeds to step S344.

[0791] In step S344, the error-related register values ​​of the CCI-FS processing unit whose register values ​​are not zero are confirmed, and the error register is cleared once for retransmission processing.

[0792] Meanwhile, the processing ends when it is determined in step S343 that the register values ​​of all CCI-FS processing units are zero, or after the processing in step S344.

[0793] Figure 53 It is described in Figure 51 The flowchart shows the sequence A_Write (at AP) processing performed in step S321. Note that in... Figure 53 The process executed by application processor 614 is described through examples; however, Figure 51 The sequence A_Write (in SerDes(Slave)) processing in step S325 is also performed in the same way.

[0794] In step S351, within application processor 614, I2C / I3C master device 651 issues a start command and slave address ( Figure 42 (The address shown is +W8 bits).

[0795] In step S352, the application processor 614 determines whether the I2C / I3C master device 651 has received an ACK response from the master-side SerDes device 613's I2C / I3C slave device 644. If it is determined in step S352 that an ACK response has been received from the master-side SerDes device 613's I2C / I3C slave device 644, the process proceeds to step S353.

[0796] In step S353, within application processor 614, I2C / I3C master device 651 publishes the register address ( Figure 42 The register address shown in [15:8]). Here, each time the processing in step S353 is repeated, a payload equal to or less than the register address is sent, such as Figure 42 As shown in the image.

[0797] In step S354, the application processor 614 determines whether the I2C / I3C master device 651 has received an ACK response from the master-side SerDes device 613's I2C / I3C slave device 644. If it is determined in step S354 that an ACK response has been received from the master-side SerDes device 613's I2C / I3C slave device 644, the process proceeds to step S355.

[0798] In step S355, within the application processor 614, the I2C / I3C master device 651 determines whether the final data transmission has been completed. If it is determined in step S355 that the final data transmission has not been completed, the process returns to step S353, and then similar processing is repeated.

[0799] Simultaneously, if it is determined in step S355 that the final data transmission has been completed, the process proceeds to step S356. In step S356, in the application processor 614, the I2C / I3C master device 651 issues a stop command. This completes the processing of sequence A_Write (at AP), and the processing returns to... Figure 51 Step S322 in the process.

[0800] Simultaneously, if it is determined in step S352 or S354 that no ACK response has been received from the I2C / I3C slave device 644 of the master-side SerDes device 613, the process proceeds to step S357. In step S357, the I2C / I3C master device 651 issues a stop command in the application processor 614. In this case, the sequence A_Write (at AP) processing is completed, and the communication processing itself is completed.

[0801] Figure 54 It is described in Figure 52 The flowchart shows the sequence A_Read_CMD (at AP) processing executed in step S331. Note that in... Figure 54 The process executed by application processor 614 is described through examples; however, Figure 52 The sequence A_Read_CMD processing in step S335 (when SerDes(Slave)) is also performed in the same way.

[0802] In step S361, within application processor 614, I2C / I3C master device 651 issues a start command and slave address ( Figure 42 The address shown is +W8 bits), and the timer is activated.

[0803] In step S362, the application processor 614 determines whether the I2C / I3C master device 651 has received an ACK response from the master-side SerDes device 613's I2C / I3C slave device 644. If it is determined in step S362 that an ACK response has been received from the master-side SerDes device 613's I2C / I3C slave device 644, the process proceeds to step S363.

[0804] In step S363, in application processor 614, I2C / I3C master device 651 publishes register address ( Figure 42 The register address shown in [15:8]). Here, each time the processing in step S363 is repeated, a payload equal to or less than the register address is sent, such as Figure 42 As shown in the image.

[0805] In step S364, the application processor 614 determines whether the I2C / I3C master device 651 has received an ACK response from the master-side SerDes device 613 to the I2C / I3C slave device 644.

[0806] If it is determined in step S364 that an ACK response from the I2C / I3C slave device 644 has been received from the master-side SerDes device 613, the process proceeds to step S365.

[0807] In step S365, within application processor 614, I2C / I3C master device 651 determines whether the final data transmission has been completed.

[0808] If it is determined in step S365 that the final data transmission has been completed, the process proceeds to step S366.

[0809] In step S366, in application processor 614, I2C / I3C master device 651 issues a stop command. Thereafter, the process branches into two, and according to branch A, the process proceeds to... Figure 52 Step S332 in the process. Simultaneously, according to branch B, sequence C (at AP) is processed in step S367 (see subsequent description). Figure 55 Then the processing continues until... Figure 52 Step S339 in the process.

[0810] Meanwhile, if it is determined in step S365 that the final data transmission has not been completed, the process proceeds to step S368.

[0811] In step S368, within application processor 614, I2C / I3C master device 651 determines whether the timer started in step S361 has timed out. If it is determined in step S368 that the timer has not timed out, the process returns to step S363, and similar processing is repeated.

[0812] Meanwhile, if it is determined in step S368 that the timer has expired, the process proceeds to step S369.

[0813] In step S369, the application processor 614 sets the error register (Timeout) to 1 and stores the data of the extended packet header ePH and extended packet trailer ePF in the error-related register.

[0814] After the processing in step S369, or if it is determined in step S362 or S364 that no ACK response has been received from the master-side SerDes device 613 I2C / I3C slave device 644, the processing proceeds to step S370.

[0815] In step S370, within application processor 614, I2C / I3C master device 651 issues a stop command. In this case, sequence A_Read_CMD (at AP) processing is completed, and the communication processing itself is finished.

[0816] Figure 55 It is described in Figure 54 The flowchart shows the sequence C (at AP) processing performed in step S367. Note that in... Figure 55 The process performed by the application processor 614 is described in the example below; however, a similar process can also be performed by the SerDes device 612 on the slave side.

[0817] In step S381, within application processor 614, I2C / I3C master device 651 determines that... Figure 54 The process checks whether the timer started in step S361 has expired and waits until it is determined that the timer has expired. If it is determined in step S381 that the timer has expired, the process proceeds to step S382; in the application processor 614, the I2C / I3C master device 651 performs a polling operation.

[0818] In step S383, in application processor 614, I2C / I3C master device 651 determines whether the status register value of the read command is 1.

[0819] If the status register value of the read command is determined to be 1 in step S383, the process proceeds to step S384. In step S384, the application processor 614 performs a read access, and then the process returns. Figure 52 Step S339 in the process.

[0820] Meanwhile, if it is determined in step S383 that the status register value of the read command is not 1 (other than 1), the process proceeds to step S385. In step S385, the application processor 614 sets the error register (Timeout) to 1 and stores the data of the extended packet header ePH and extended packet trailer ePF in the error-related register.

[0821] In step S386, within application processor 614, I2C / I3C master device 651 issues a stop command. In this case, sequence C (at AP) processing is complete, and the communication processing itself is finished.

[0822] Figure 56 It is described in Figure 51 The flowchart shows the processing of sequence B (during SerDes(Slave)) executed in steps S324 and S334. Note that in... Figure 56 The process performed by the slave-side SerDes device 612 is described in the following section, with examples illustrating the steps taken; however, Figure 51 Sequence B in step S322 (during SerDes(Master)) Figure 51 In step S326, sequence B (in the image sensor) is processed and... Figure 52 The processing of sequence B in step S332 (in SerDes(Master)) is also performed in the same way.

[0823] In step S391, in the slave-side SerDes device 612, the CCI-FS processing unit 636 confirms the source ID and destination SID of the extended packet header ePH of the slave-side SerDes device 612.

[0824] In step S392, in the slave-side SerDes device 612, the CCI-FS processing unit 636 determines whether the source ID of the slave-side SerDes device 612 and the destination SID of the extended packet header ePH are inconsistent with each other.

[0825] If, in step S392, it is determined that the source ID of the slave-side SerDes device 612 is inconsistent with the destination SID of the extended packet header ePH, the process proceeds to step S393.

[0826] In step S393, in the slave-side SerDes device 612, the CCI-FS processing unit 636 confirms the destination SID of the slave-side SerDes device 612 and the destination SID of the extended packet header ePH.

[0827] In step S394, in the slave-side SerDes device 612, the CCI-FS processing unit 636 determines whether the source ID of the slave-side SerDes device 612 and the destination SID of the extended packet header ePH are consistent with each other.

[0828] If, in step S394, it is determined that the source ID of the slave-side SerDes device 612 and the destination SID of the extended packet header ePH are consistent with each other, the process proceeds to step S395.

[0829] In step S395, in the slave-side SerDes device 612, the CCI-FS processing unit 636 confirms the message counter from the contents of the extended packet header ePH.

[0830] In step S396, in the slave-side SerDes device 612, the CCI-FS processing unit 636 determines whether the received values ​​of the message counter at the slave-side SerDes device 612 and the message counter confirmed according to the content of the extended packet header ePH are consistent with each other.

[0831] If, in step S396, it is determined that the message counter at the slave-side SerDes device 612 matches the received value of the message counter confirmed according to the content of the extended packet header ePH, the process proceeds to step S397.

[0832] In step S397, in the slave-side SerDes device 612, the CCI-FS processing unit 636 confirms the CRC calculation result calculated based on the received values ​​(ePF0) of the extended packet header ePH and extended packet trailer ePF in the slave-side SerDes device 612.

[0833] In step S398, it is determined whether the received value (ePF0) of the extended packet trailer ePF and the CRC calculation result are consistent; if they are consistent, the process returns to... Figure 51 Step S325 in the process.

[0834] Meanwhile, if it is determined in step S392 that the source ID of the slave-side SerDes device 612 and the destination SID of the extended packet header ePH are not consistent with each other, the process proceeds to step S399.

[0835] In steps S399 to S402, similar processes to those in steps S395 to S398 are performed.

[0836] In step S402, if the received value (ePF0) of the extended packet trailer ePF and the CRC calculation result are consistent, the process proceeds to step S403. In step S403, a write access is performed on register 637 of the slave-side SerDes device 612.

[0837] If, in step S394, it is determined that the source ID of the slave-side SerDes device 612 and the destination SID of the extended packet header ePH are inconsistent with each other, the process proceeds to step S404. In step S404, in the slave-side SerDes device 612, the CCI-FS processing unit 636 sets 1 to the error register [2] (Routing) and stores the data of the extended packet header ePH and the extended packet trailer ePF in the error-related register.

[0838] If, in step S398 or S402, it is determined that the received value (ePF0) of the extended packet trailer ePF and the CRC calculation result are inconsistent, the process proceeds to step S405. In step S405, in the slave-side SerDes device 612, the CCI-FS processing unit 636 sets 1 to the error register (CRC) and stores the data of the extended packet header ePH and the extended packet trailer ePF in the error-related register.

[0839] If, in step S396 or S400, it is determined that the message counter at the slave-side SerDes device 612 is inconsistent with the received value of the message counter confirmed based on the content of the extended packet header ePH, the process proceeds to step S406. In step S406, in the slave-side SerDes device 612, the CCI-FS processing unit 636 sets 1 to the error register (MC) and stores the data of the extended packet header ePH and the extended packet trailer ePF in the error-related register.

[0840] After the processing in steps S403 to S406, the processing of sequence B (SerDes(Slave)) ends, and the communication processing itself ends.

[0841] It is important to note that the following combinations are envisioned: CRC calculation can be performed only on the E2E Protection target; errors are detected in each device, and packets are discarded or not discarded.

[0842] Figure 57 It is described in Figure 52 The flowchart shows the sequence A_Read_Data processing (at AP) performed in step S339. Note that in... Figure 57 The process executed by application processor 614 is described in the following section through examples; however, it is also executed in the same manner. Figure 52 The sequence A_Read_Data in step S336 (during SerDes(Slave) processing).

[0843] In step S411, within application processor 614, I2C / I3C master device 651 issues a start command and slave address ( Figure 48 (The address shown is +W8 bits).

[0844] In step S412, the application processor 614 determines whether the I2C / I3C master device 651 has received an ACK response from the master-side SerDes device 613's I2C / I3C slave device 644. If it is determined in step S412 that an ACK response has been received from the master-side SerDes device 613's I2C / I3C slave device 644, the process proceeds to step S413.

[0845] In step S413, within application processor 614, I2C / I3C master device 651 issues a start command and slave address ( Figure 48 As shown in the diagram (address + R8 bits), and activate the timer.

[0846] In step S414, the application processor 614 determines whether the I2C / I3C master device 651 has received an ACK response from the master-side SerDes device 613's I2C / I3C slave device 644. If it is determined in step S414 that an ACK response has been received from the master-side SerDes device 613's I2C / I3C slave device 644, the process proceeds to step S415.

[0847] In step S415, in application processor 614, I2C / I3C master device 651 obtains read data from I2C / I3C slave device 644, which is opposite to application processor 614.

[0848] In step S416, it is determined whether the I2C / I3C master device 651 of the application processor 614 has performed ACK transmission and whether the I2C / I3C slave device 644 opposite to the application processor 614 has performed ACK reception.

[0849] If, in step S416, it is determined that the I2C / I3C master device 651 of the application processor 614 has performed ACK transmission and the I2C / I3C slave device 644 opposite to the application processor 614 has performed ACK reception, the process proceeds to step S417.

[0850] In step S417, it is determined whether the I2C / I3C master device 651 of the application processor 614 has performed a NACK transmission associated with the completion of the final data transmission.

[0851] If it is determined in step S417 that a NACK transmission has been performed, the process proceeds to step S418. In step S418, in application processor 614, I2C / I3C master device 651 issues a stop command. This completes the processing of sequence A_Read_Data (at AP), and processing returns to... Figure 52 Step S340 in the process.

[0852] Meanwhile, if it is determined in step S417 that NACK transmission has not yet been performed, the process proceeds to step S419.

[0853] In step S419, within the application processor 614, the I2C / I3C master device 651 determines whether the timer started in step S413 has expired. If it is determined in step S419 that the timer has not expired, the process returns to step S415, and then similar processing is repeated.

[0854] Meanwhile, if it is determined in step S419 that the timer has expired, the process proceeds to step S420.

[0855] In step S420, the application processor 614 sets 1 to the error register (Timeout) and stores the data of the extended packet header ePH and extended packet trailer ePF in the error-related register.

[0856] After the processing in step S420, or if it is determined in step S414 that no ACK response has been received from the I2C / I3C slave device 644 of the master-side SerDes device 613, the processing proceeds to step S421. Similarly, if it is determined in step S416 that the I2C / I3C master device 651 of the application processor 614 has not yet performed ACK transmission or that the I2C / I3C slave device 644 opposite to the application processor 614 has not yet performed ACK reception, the processing proceeds to step S421.

[0857] In step S421, within the application processor 614, the I2C / I3C master device 651 issues a stop command. In this case, the sequence A_Read_Data processing (at the AP) is completed, and the communication processing itself is finished.

[0858] Here, during the output of I2C / I3C slave device 621 (see...) Figure 46), the access timing from the I2C / I3C master device 634 to the I2C / I3C slave device 621, and during the output of the I2C / I3C slave device 644 of the SerDes device 613 on the master side (see Figure 48 ), the access timing from the I2C / I3C master device 651 to the I2C / I3C slave device 644 has three combinations described below.

[0859] In the first access timing, polling is performed until read data is acquired, and the I2C / I3C master device starts the read process after preparing to complete the read data acquisition.

[0860] In the second access timing, the I2C / I3C master device starts the read process after a certain period of time.

[0861] In the third access timing, the clock stretching method is used (see Figure 72 described later), and the I2C / I3C master device starts the read process after a certain period of time has passed; at this time, there are a mode of sending read data in batches and a mode of sending read data separately (asserting the Clock Stretch Mode signal).

[0862] <Example of the composition of the extended packet header ePH>

[0863] Figures 58 to 60 Each is a diagram showing an example of the composition of the extended packet header ePH.

[0864] Figure 58 Shows detailed examples of the composition of the extended packet headers ePH0, ePH1, and ePH2. For the addition of the extended packet header ePH as shown in the figure, the content of the extended packet header ePH is defined by an appropriate ePH structure for CCI-FS at C-PHY and D-PHY.

[0865] Figure 59 Shows a detailed example of the composition of the extended packet header ePH3. For the addition of the extended packet header ePH as shown in the figure, the content of the extended packet header ePH is defined for CCI-FS.

[0866] Figure 60 Shows a detailed example of the composition of the extended DT of the extended packet header ePH. For example, "0xC0: For I2C" and "0xC1: For I3C" are added to the data type of the extended packet header ePH to adapt to CCI-FS.

[0867] <Example of the circuit composition of I2C>

[0868] Figure 61An example of an existing I2C configuration in hardware is shown. For example, an example of an I2C configuration is shown in the case of upper-layer bus coupling configuration during hardware implementation; a configuration that enables AKC / NACK to be received from the upper layer can be adopted on the slave side. Needless to say, only one example is shown, and the upper-layer bus configuration does not have to be consistent with it.

[0869] Figure 62 The waveforms during data transfer on the I2C bus are shown. It should be noted that the I2C bus standard and CCI (I2C) should be equivalent.

[0870] <Component Examples Related to CCI in Communication System 701>

[0871] Figure 63 This is shown in relation to the above. Figure 27 The block diagram shows a configuration example of a communication system 701 that is directly coupled to an A-PHY, in the same manner as the communication system 501 shown.

[0872] like Figure 63 As shown, in the communication system 701, the image sensor 711 and the application processor 712 are directly coupled to each other via an A-PHY.

[0873] The image sensor 711 includes an A-PHY processing unit 721, a CSIA processing unit 722, a CSI2 processing unit 723, a CSI2-FS processing unit 724, a CCI processing unit 725, a CCI-FS processing unit 726, a register 727, and selectors 728-1 and 728-2. As shown, selectors 728-1 and 728-2 are configured to sandwich the CCI-FS processing unit 726 and can switch the CCI-FS processing unit 726 to enable / disable according to the CCI_FS_Enable signal of register 727.

[0874] Application processor 712 includes an A-PHY processing unit 731, a CSIA processing unit 732, a CSI2 processing unit 733, a CSI2-FS processing unit 734, a CCI processing unit 735, a CCI-FS processing unit 736, a register 737, and selectors 738-1 and 738-2. As shown, selectors 738-1 and 738-2 are configured to enable / disable the CCI-FS processing unit 736, and can switch the CCI-FS processing unit 736 on / off according to the CCI_FS_Enable signal of register 737.

[0875] For example, when the CCI_FS_Enable signal indicates that CCI-FS is enabled (CCI_FS_Enable = 1), data is transmitted and received via the CCI-FS processing unit 726 and the CCI-FS processing unit 736, as indicated by the arrows of alternating long and short dashed lines. At the same time, when the CCI_FS_Enable signal indicates that CCI-FS is disabled (CCI_FS_Enable = 0), data is not transmitted and received via the CCI-FS processing unit 726 and the CCI-FS processing unit 736, as indicated by the arrows of alternating long and two short dashed lines.

[0876] <Coupling mode of the network>

[0877] Figure 64 Examples of the coupling mode (topology) of the network composed of A-PHY direct coupling and SerDes coupling are shown.

[0878] A coupling mode can be configured in which the application processor 801 is directly coupled to the image sensor 802 via A-PHY, and the image sensor 802 is coupled to the sensor 803 via I2C / I3C.

[0879] The application processor 801 is coupled to the main-side SerDes device 804 via I2C / I3C, and the main-side SerDes device 804 and the slave-side SerDes device 805 are coupled to each other via A-PHY. A coupling mode can be configured in which the slave-side SerDes device 805 is coupled to two sensors 806-1 and 806-2 via I2C / I3C.

[0880] <Circuit configuration of the CCI-FS processing unit>

[0881] Figure 65 It is a block diagram showing an example of the circuit configuration of the CCI-FS processing unit. Figure 65 The CCIFS processing unit 901 and the register 902 shown in it have the same configuration as the CCI-FS processing unit and the register included in each of the above devices.

[0882] As Figure 65 shown, the CCI-FS processor 901 includes a CCI-FS switch, a register, etc. in the upper layer, and a CCI processing unit in the lower layer. The CCI-FS processor 901 includes a CCI-FS transmitter 911 and a CCI-FS receiver 912. Different types of register setting value information are provided from the register 902 to the CCI-FS processor 901, and error notifications are provided from the CCI-FS processor 901 to the register 902.

[0883] The CCI-FS transmitter 911 includes an extended packet header ePH generator 921, an extended packet trailer ePF generator 922, and a destination address acknowledgment unit 923.

[0884] The extended packet header (ePH) generator 921 includes an MC generation unit 941 for generating a message counter and a packet length calculation unit 942 for calculating the packet length. The extended packet trailer (ePF) generator 922 includes an extended packet trailer (ePF1) generation unit 943 for generating an extended packet trailer (ePF1) and a CRC calculation unit 944 for calculating the CRC stored in the extended packet trailer (ePF0).

[0885] The CCI-FS receiver 912 includes an extended packet header ePH acknowledgment 931, an extended packet trailer ePF acknowledgment 932, and a destination address acknowledgment 933.

[0886] The extended packet header ePH acknowledgment unit 931 includes an MC acknowledgment unit 951 for acknowledging the message counter and a packet length calculation / acknowledgment unit 952 for calculating and acknowledging the packet length. The extended packet trailer ePF acknowledgment unit 932 includes an extended packet trailer ePF1 acknowledgment unit 953 for acknowledging extended packet trailer ePF1 and a CRC calculation unit 954 for calculating the CRC stored in extended packet trailer ePF0.

[0887] The CCI-FS processor 901 enables the CCI-FS transmitter 911 to acknowledge the destination address of data from the upper layer, generate extended packet headers (ePH) and extended packet trailers (ePF) to add them to the data, and provide the data to the lower layer. The CCI-FS processor 901 also enables the CCI-FS receiver 912 to acknowledge the destination address of data from the lower layer, acknowledge the extended packet headers (ePH) and extended packet trailers (ePF), and provide them to the upper layer.

[0888] Now, the description constitutes the above. Figure 40 The operation of the CCI-FS processing unit of each device in the communication system 601 constituting the example of the SerDes coupling shown.

[0889] Application processor 614 contains a source ID in its extended packet header ePH indicating the device itself. Additionally, CCI-FS processing unit 653 adds the aforementioned information along with a destination ID indicating the target device to be accessed.

[0890] Both the slave-side SerDes device 612 and the master-side SerDes device 613 have a source ID, which is either preset or used as a characteristic value to represent the device itself. The CCI-FS processing unit 636 and the CCI-FS processing unit 646 perform preset settings on the above information and the destination ID indicating the coupling device and the target device.

[0891] Furthermore, CCI-FS processing units 636 and 646 each compare the destination ID of the received extended packet header ePH with its own ID (source ID) to determine whether it is an access to itself or an instruction to the target device (destination ID). For example, when the destination ID of the received extended packet header ePH matches its own ID (source ID), a self-registration access is performed as an access to the intermediate device (SerDes device). Conversely, when the destination ID of the received extended packet header ePH does not match its own ID (source ID), data is transmitted to the coupling device (destination ID) as an access to the downstream device.

[0892] As described above, based on a preset source ID or feature value and information about a preset coupled destination, data is transmitted to the source ID and destination ID, intermediate device (SerDes device) or target device embedded in the extended packet header ePH, and access is performed toward the target device.

[0893] When the destination ID of the received extended packet header ePH matches its own ID (source ID), the CS12-FS processing unit 623 of the image sensor 611 performs its own register access as an access to the image sensor 611.

[0894] In this way, the source ID of each device can use a characteristic value, a preset value, or a combination thereof for each device.

[0895] Figures 66 to 68 Each of these is a diagram illustrating a detailed configuration example of register 902.

[0896] Figure 66 Details of register 902, from address 0x000 to address 0x109, are shown. Figure 67 An example of the configuration when the bridge is constructed is shown, as detailed by register 902 from address 0x110 to address 0x125.

[0897] Figure 68 The error-related register, register 902 at address 0x200, is shown in detail. Figure 68 The error-related registers (debug registers) are shown in detail, specifically register 902 at addresses 0x300 and 0x400. Figure 68 The details of the error injection-related register (troubleshooting), register 902 at address 0x800, are shown.

[0898] <Example of an extended group header ePH>

[0899] Reference Figure 69 and Figure 70 This describes a variant of the extended group header ePH.

[0900] Figure 69 An example of modification to the extended packet header ePH in the packet structure of write data generated by the CCI-FS processing unit 536 of the application processor 512 during write access is shown above. Figure 33 As described. Figure 69 The extended packet header ePH shown above is similar to the one described above. Figure 33 The difference between the examples shown lies in the composition of the extended packet header ePH3 and extended packet header ePH4.

[0901] Figure 70 An example of modification of the extended packet header ePH in the packet structure of write data generated in the CCI-FS processing unit 536 of the application processor 512 during read access is shown, as referenced above. Figure 28 As described. Figure 70 The extended packet header ePH shown above is similar to the one described above. Figure 28 The difference between the examples shown lies in the composition of the extended packet header ePH3 and extended packet header ePH4.

[0902] For example, in Figure 69 and Figure 70 The extended packet header ePH shown in the diagram envisions the following combinations depending on the implementation.

[0903] Address information can be stored in the extended packet header ePH or the AP (CCI) payload. Length information can be stored in the extended packet header ePH or the AP (CCI) payload. CMD information can be stored in the CCI command ID of the extended packet header ePH. Based on the CCI command ID, information about the start, restart, and end of the command is referenced. The CCI header length can be used to store CCI information (e.g., slave address, etc.) in the AP (CCI) payload. The CCI header length is information indicating the header length of the CCI protocol (I2C).

[0904] Figure 71 It is a description Figure 27 The diagram illustrates the flow between the image sensor 511 and the application processor 512 in the A-PHY direct coupling configuration shown.

[0905] In the application processor 512, the CCI-FS switch 538 issues read and write commands. The CCI-FS switch 538 provides the CCI processing unit 535 with the slave address (slave address + W8 bits), register address a (register address [15:8], register address [7:0], and data (Data*(*=N)[7:0]). The CCI processing unit 535 converts these into an AP (CCI) payload and provides it to the A-PHY processing unit 531. The A-PHY processing unit 531 adds the A-PHY header and A-PHY trailer to the AP (CCI) payload and performs A-PHY transmission to the image sensor 511.

[0906] In the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY trailer, and provides the AP (CCI) payload to the CCI processing unit 525. The CCI processing unit 525 converts the AP (CCI) payload, writes data to register 527 based on the data content according to the write command, and reads data from register 527 according to the read command.

[0907] At this time, the CCI processing unit 525 performs the initial settings for CCI-FS Enable and performs bus conversions for the register interface, AHB bus, etc. Furthermore, the CCI-FS Enable settings are confirmed by either the CCI processing unit 525 or the CCI-FS processing unit 526.

[0908] In response to a read command to the AP (CCI) payload, the CCI processing unit 525 converts the read data (Data*(*=M)[7:0]) read from register 527 and provides it to the A-PHY processing unit 521. The A-PHY processing unit 521 adds the A-PHY header and A-PHY trailer to the AP (CCI) payload and performs A-PHY transmission to the application processor 512.

[0909] In the application processor 512, the A-PHY processing unit 531 removes the A-PHY header and APHY trailer, and provides the AP (CCI) payload to the CCI processing unit 535. The CCI processing unit 535 converts the AP (CCI) payload and provides the read data (Data*(*=M)[7:0]) to the CCI-FS switch 538.

[0910] CCI-FS switch 538 performs CCI-FS Enable setting and various CCI-FS related register settings on register 537. At this time, register access depends on the implementation. CCI-FS switch 538 performs various CCI-FS related register settings on register 527 through register 537, CCI-FS processing unit 536, A-PHY processing unit 531, A-PHY processing unit 521, and CCI-FS processing unit 526.

[0911] In application processor 512, CCI-FS switch 538 issues a read command. CCI-FS switch 538 provides slave address (slave address + W8 bits), register address (register address [15:8], register address [7:0]) and data (Data*(*=N)[7:0]) to register 537. CCI-FS processing unit 536 converts them into AP (CCI) payload, adds extended packet header ePH*(*=n), extended packet trailer ePF1, and extended packet trailer ePF0 to it, and provides them to A-PHY processing unit 531. A-PHY processing unit 531 adds A-PHY header and A-PHY trailer to it and performs A-PHY transmission to image sensor 511.

[0912] In the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY trailer, and provides the CCI-FS processing unit 526 with the extended packet header ePH* (*=n), AP (CCI) payload, extended packet trailer ePF1, and extended packet trailer ePF0. The CCI-FS processing unit 526 converts the AP (CCI) payload and reads data from register 527 based on its content according to a read command. At this time, register access depends on the implementation method, and bus conversion is performed between the register interface, AHB bus, CCI interface, etc.

[0913] In response to a read command to the AP (CCI) payload, the CCI-FS processing unit 526 converts the read data (Data*(*=M)[7:0]) read from register 527, adds the extended packet header ePH*(*=n), extended packet trailer ePF1, and extended packet trailer ePF0 to it, and provides them to the A-PHY processing unit 521. The A-PHY processing unit 521 adds the A-PHY header and A-PHY trailer to it and performs A-PHY transmission to the application processor 512.

[0914] In the application processor 512, the A-PHY processing unit 531 removes the A-PHY header and A-PHY trailer, and provides the CCI-FS processing unit 536 with the extended packet header ePH* (*=n), AP (CCI) payload, extended packet trailer ePF1, and extended packet trailer ePF0. The CCIFS processing unit 536 converts the AP (CCI) payload and provides the read data (Data* (*=M) [7:0]) to the CCI-FS switch 538.

[0915] It should be noted that the above process has been described using hardware examples to illustrate I2C / I3C command generation; however, in addition to those, the following combinations exist.

[0916] In the software-based I2C / I3C generation scenario, the slave address, register address, payload, ACK response reception, transmission, and various control codes (S, Sr, ACK, NACK, and P) are generated by software (e.g., GPIO-controlled graphics). For software-based I2C / I3C command generation, CPU bus settings are used to respond to ACK reception by sending the slave address, register, and payload from the CPU.

[0917] In the hardware-based configuration, for hardware-based I2C / I3C generation, setup and data configuration are performed via HWIP sent to I2C / I3C. Various control codes are executed automatically using hardware. For hardware-based I2C / I3C command generation, data is set via HWIP sent to I2C / I3C, and sent via command execution. Various control codes are executed automatically using hardware.

[0918] Figure 72 It is described in Figure 40 The diagram illustrates the flow of write and read accesses using a clock stretching method between the image sensor 611 and the application processor 614 in the SerDes coupling configuration shown.

[0919] The CCI-FS switch 655 of the application processor 614 provides the start command and write command (from address + W8 bit) to the CCI processing unit 645 of the master-side SerDes device 613, and asserts the SCI_enb signal. In the master-side SerDes device 613, the CCI processing unit 645 provides the write command to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds the A-PHY header and A-PHY trailer to the write command, and executes the A-PHY transmission to the slave-side SerDes device 612.

[0920] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY trailer, and provides a write command to the CCI processing unit 635 (Slave). The CCI processing unit 635 (Slave) negates the Sci_enb signal and provides a write command to the CCI processing unit 635 (Master). Here, the CCI processing unit 635 that communicates with the master-side SerDes device 613 as a slave device is referred to as the CCI processing unit 635 (Slave), and the CCI processing unit 635 that communicates with the image sensor 611 as a master device is referred to as the CCI processing unit 635 (Master).

[0921] The CCI processing unit 635 (Master) sends the start command and write command to the image sensor 611.

[0922] In the image sensor 611, the CCI processing unit 622 receives the start command and the write command and provides them to the CS12-FS processing unit 623. The CS12-FS processing unit 623 provides an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 sends the ACK response to the slave-side SerDes device 612.

[0923] In the slave-side SerDes device 612, the CCI processing unit 635 (Master) receives an ACK response and provides the ACK response to the CCI-FS processing unit 636 when the slave CCI processing unit 635 (Slave) negates the Scl_enb signal. Thereafter, the slave CCI processing unit 635 (Slave) asserts the Scl_enb signal.

[0924] The CCI-FS processing unit 636 provides the ACK response to the A-PHY processing unit 631. The A-PHY processing unit 631 adds the A-PHY header and A-PHY trailer to the ACK response and performs A-PHY transmission to the master-side SerDes device 613.

[0925] In the primary SerDes device 613, the A-PHY processing unit 641 removes the A-PHY header and A-PHY trailer and provides an ACK response to the CCI processing unit 645. When the CCI-FS switch 655 of the application processor 614 negates the SCI_enb signal of the CCI processing unit 645, the CCI processing unit 645 transmits an ACK response to the application processor 614.

[0926] In the application processor 614, the CCI processing unit 652 receives the ACK response and provides it to the CCI-FS switch 655 through the CCI-FS processing unit 653.

[0927] The CCI-FS switch 655 of the application processor 614 provides the register address (Register Address[7:0]) to the CCI processing unit 645 of the master-side SerDes device 613 and asserts the SCI_enb signal. In the master-side SerDes device 613, the CCI processing unit 645 provides the register address to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds the A-PHY header and A-PHY trailer to the register address and performs A-PHY transmission to the slave-side SerDes device 612.

[0928] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY trailer and provides the register address to the CCI processing unit 635 (Slave). The CCI processing unit 635 (Slave) negates the SCI_enb signal and provides the register address to the CCI processing unit 635 (Master). The CCI processing unit 635 (Master) sends the register address to the image sensor 611. Afterwards, the CCI processing unit 635 (Slave) asserts the SCI_enb signal to the CCI processing unit 635 (Master).

[0929] In the image sensor 611, the CCI processing unit 622 receives the register address and provides it to the CSI2-FS processing unit 623. The CSI2-FS processing unit 623 provides an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 sends the ACK response to the slave-side SerDes device 612.

[0930] Subsequently, the ACK response is continuously provided to the CCI-FS switch 655 in the same manner as described above.

[0931] In the application processor 614, the CCI-FS processing unit 653, under the control of the CCI-FS switch 655, sends the extended packet header ePH* (*=n) to the master-side SerDes device 613.

[0932] In the master-side SerDes device 613, the CCI processing unit 645 receives the extended packet header ePH* (*=n), and when the Scl_enb signal is asserted from the CCI-FS switch 655, it provides the extended packet header ePH* (*=n) to the A-PHY processing unit 641. Thereafter, the CCI-FS switch 655 negates the Scl_enb signal received by the CCI processing unit 645. The A-PHY processing unit 641 adds the A-PHY header and A-PHY trailer to the extended packet header ePH* (*=n), and performs A-PHY transmission to the slave-side SerDes device 612.

[0933] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY trailer, and provides the extended packet header ePH* (*=n) to the CCI-FS processing unit 636. The CCI-FS processing unit 636 negates the Scl_enb signal and provides the extended packet header ePH* (*=n) to the CCI processing unit 635 (Master). The CCI processing unit 635 (Master) sends the extended packet header ePH* (*=n) to the image sensor 611. Thereafter, the CCI processing unit 635 (Slave) asserts the Scl_enb signal to the CCI processing unit 635 (Master).

[0934] In the image sensor 611, the CS12-FS processing unit 623 receives the extended packet header ePH* (*=n). The CS12-FS processing unit 623 provides an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 sends the ACK response to the slave-side SerDes device 612.

[0935] Subsequently, in the same manner as described above, an ACK response is provided until the CCI-FS switch 655.

[0936] The CCI-FS switch 655 of the application processor 614 provides write data (Dara0[7:0]) to the CCI processing unit 645 of the master-side SerDes device 613 and asserts the SCI_enb signal. In the master-side SerDes device 613, the CCI processing unit 645 provides the write data to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds the A-PHY header and A-PHY trailer to the write data and performs A-PHY transmission to the slave-side SerDes device 612.

[0937] In the master-side SerDes device 613, when the Scl_enb signal is asserted from the CCI-FS switch 655, the CCI processing unit 645 receives write data and provides it to the A-PHY processing unit 641. Subsequently, under the control of the CCI-FS switch 655, the CS12-FS processing unit 653 denies the Scl_enb signal from the CCI processing unit 645. The A-PHY processing unit 641 adds the A-PHY header and A-PHY trailer to the write data and performs A-PHY transmission to the slave-side SerDes device 612.

[0938] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY trailer and provides write data to the CCI processing unit 635. The CCI processing unit 635 denies the SCI_enb signal and provides write data to the CCI processing unit 635 (Master). The CCI processing unit 635 (Master) transmits the write data to the image sensor 611. Afterwards, the CCI processing unit 635 (Slave) asserts the SCI_enb signal to the CCI processing unit 635 (Master).

[0939] In the image sensor 611, the CCI processing unit 622 receives write data and provides it to the CS12-FS processing unit 623, and the CS12-FS processing unit 623 writes the write data to the register 624. The CSI2-FS processing unit 623 provides an ACK response indicating successful write of the write data to the CCI processing unit 622, and the CCI processing unit 622 sends the ACK response to the slave-side SerDes device 612.

[0940] Subsequently, the ACK response is provided up to the CCI-FS switch 655 in the same manner as described above.

[0941] In the application processor 614, the CCI-FS processing unit 653, under the control of the CCI-FS switch 655, sends the extended packet trailer ePF0 to the master-side SerDes device 613.

[0942] In the primary SerDes device 613, when the Scl_enb signal is asserted from the CCI-FS switch 655, the CCI processing unit 645 receives the extended packet trailer ePF0 and provides the extended packet trailer ePF0 to the A-PHY processing unit 641. Thereafter, the CCI-FS switch 655 denies the Scl_enb signal to the CCI processing unit 645. The A-PHY processing unit 641 adds the A-PHY header and A-PHY trailer to the extended packet trailer ePF0 and performs A-PHY transmission to the secondary SerDes device 612.

[0943] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY trailer, and provides the extended packet trailer ePF0 to the CCI-FS processing unit 636. The CCI-FS processing unit 636 negates the Scl_enb signal and provides the extended packet trailer ePF0 to the CCI processing unit 635 (Master). The CCI processing unit 635 (Master) sends the extended packet trailer ePF0 to the image sensor 611. Thereafter, the CCI processing unit 635 (Slave) asserts the Scl_enb signal to the CCI processing unit 635 (Master).

[0944] In the image sensor 611, the CS12-FS processing unit 623 receives the extended packet trailer ePF0. The CS12-FS processing unit 623 provides an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 sends the ACK response to the slave-side SerDes device 612.

[0945] Subsequently, the ACK response is provided up to the CCI-FS switch 655 in the same manner as described above.

[0946] The CCI-FS switch 655 of the application processor 614 provides the repeat start command and read command (from address + R8 bits) to the CCI processing unit 645 of the master-side SerDes device 613, and asserts the SCI_enb signal. In the master-side SerDes device 613, the CCI processing unit 645 provides the read command to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds the A-PHY header and A-PHY trailer to the read command, and executes the A-PHY transmission to the slave-side SerDes device 612.

[0947] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY trailer, and provides a read command to the CCI processing unit 635 (Slave). The CCI processing unit 635 (Slave) negates the Sci_enb signal and provides a read command to the CCI processing unit 635 (Master). The CCI processing unit 635 (Master) sends a repeat start command and a read command to the image sensor 611.

[0948] In the image sensor 611, the CCI processing unit 622 receives the repeat start command and the read command, and accesses register 624. The CCI processing unit 622 sends an ACK response indicating successful reception to the slave SerDes device 612.

[0949] Subsequently, the ACK response is provided up to the CCI-FS switch 655 in the same manner as described above.

[0950] In the image sensor 611, the CCI processing unit 622 reads read data (Data0[7:0]) from the register 624 and sends it to the slave SerDes device 612.

[0951] In the slave-side SerDes device 612, the CCI processing unit 635 (Master) receives read data and provides it to the CCI processing unit 635 (Slave), and the CCI processing unit 635 (Slave) provides the read data to the A-PHY processing unit 631. The A-PHY processing unit 631 adds the A-PHY header and A-PHY trailer to the read data and performs A-PHY transmission to the master-side SerDes device 613.

[0952] In the SerDes device 613 on the main side, the A-PHY processing unit 641 removes the A-PHY header and A-PHY trailer, and supplies the read data to the CCI processing unit 645, and the CCI processing unit 645 transmits the read data to the application processor 614.

[0953] In the application processor 614, the CCI processing unit 652 receives read data and supplies it to the CCI-FS switch 655 via the CCI-FS processing unit 653.

[0954] The CCI-FS switch 655 sends a NACK response and stop command to the CCI processing unit 645. The CCI processing unit 645 provides the NACK response and stop command to the A-PHY processing unit 641. The A-PHY processing unit 641 adds the A-PHY header and A-PHY trailer to the NACK response and stop command, and executes the A-PHY transmission to the slave-side SerDes device 612.

[0955] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and A-PHY trailer, and provides the NACK response and stop command to the CCI processing unit 635 (Slave). The CCI processing unit 635 (Slave) provides the NACK response and stop command to the CCI processing unit 635 (Master), and the CCI processing unit 635 (Master) sends the NACK response and stop command to the image sensor 611.

[0956] In the image sensor 611, the CCI processing unit 622 receives the NACK response and the stop command and provides them to the CS12-FS processing unit 623.

[0957] It should be noted that, Figure 72 In the described stream, I2C control commands (such as start, repeat start, ACK response, NACK response, or stop) set the control code indicator of the extended packet header ePH0 to 1 and indicate each code assigned to the 1-byte payload.

[0958] <Detailed Examples of Image Sensor and Application Processor Configuration>

[0959] (Detailed example of the structure of an image sensor)

[0960] Figure 73 This shows the CCI-FS processor 1001. Figure 25 The block diagram shown illustrates an example of the configuration of the image sensor 211. It should be noted that... Figure 73 In the image sensor 211 shown, with Figure 25 The configuration of the image sensor 211 shown is the same as that shown, and is indicated by the same reference numerals, and its description is omitted.

[0961] like Figure 73 As shown, the CCI-FS processor 1001 is disposed between the CCI slave device 224 and the register 47, and the MUX units 1002-1 and 1002-2 are configured to sandwich the CCI-FS processor 1001 in between. When the CCI-FS processor 1001 is enabled according to the cci_fs_en signal provided by the slave register 47, the MUX units 1002-1 and 1002-2 transmit and receive data through the CCI-FS processor 1001. On the other hand, when the CCI-FS processing unit 1001 is disabled, the MUX units 1002-1 and 1002-2 do not transmit or receive data through the CCI-FS processing unit 1001 according to the cci_fs_en signal supplied by the slave register 47.

[0962] (Detailed example of application processor configuration)

[0963] Figure 74 This is a block diagram illustrating the aforementioned CCI-FS processor 1101. Figure 26 The example shown illustrates the configuration of the application processor 214. It should be noted that... Figure 74 In the application processor 214 shown, and in Figure 26 The configuration of the application processor 214 shown is the same as that shown, and is indicated by the same reference numerals, and its description is omitted.

[0964] like Figure 74As shown, the CCI-FS processor 1101 is arranged between the CCI master device 254 and register 73, and MUX units 1102-1 and 1102-2 are arranged to sandwich the CCI-FS processor 1101 in between. When the CCI-FS processor 1101 is enabled according to the cci_fs_en signal provided from register 73, MUX units 1102-1 and 1102-2 send and receive data through the CCI-FS processor 1101. Conversely, when the CCI-FS processor 1101 is disabled according to the cci_fs_en signal provided from register 73, MUX units 1102-1 and 1102-2 do not send or receive data via the CCI-FS processor 1101.

[0965] It should be noted that the following configuration can also be used to implement the various fields in the extended packet header (ePH). The extended VC is not used in the secure CCI (similar configurations are used to match header fields and extensions in MIPI). In the extended DT, data can be embedded in bus command information from the upper layer, or it can use an implementation from register settings for setting signal lines. Although the protocol is described in I2C, a similar method can also be performed in I3C's SDR mode.

[0966] <Examples of Communication System Structure>

[0967] refer to Figures 75 to 117 A description of a fourth embodiment of a communication system applying this technology is given.

[0968] Figure 75 This is a block diagram illustrating a communication system according to a fourth embodiment. Figure 75 A illustrates the communication system 1201 of the first variant. Figure 75 B illustrates a second variant of the communication system 1201A.

[0969] Figure 75 The communication system 1201 shown in A has a configuration in which an image sensor 1211 and an application processor 1212 are directly coupled to each other.

[0970] The image sensor 1211 has the following configuration: an ALL layer 1222 is arranged on the A-PHY layer 1221, and a CSI-2 transmitter 1223, a CSI extension unit 1224, a CCI slave device 1225, and a CCI extension unit 1226 are arranged on the ALL layer 1222. The CSI-2 transmitter 1223 is equipped with the CSI extension unit 1224, and the CCI slave device 1225 is equipped with the CCI extension unit 1226, thereby enabling the image sensor 1211 to adapt to the corresponding extension standard.

[0971] The application processor 1212 has the following configuration: an ALL layer 1232 is arranged on the A-PHY layer 1231; a CSI-2 receiver 1233 and a CSI extension unit 1234, as well as a CCI host 1235 and a CCI extension unit 1236 are arranged on the ALL layer 1232. The CSI-2 receiver 1233 is equipped with the CSI extension unit 1234, and the CCI host 1235 is equipped with the CCI extension unit 1236, thereby enabling the application processor 1212 to adapt to corresponding extension standards. It should be noted that CSI extensions can also be referred to as Camera Service Extensions (CSE).

[0972] Figure 75 The communication system 1201A shown in Figure B has a configuration in which a display 1213 and an application processor 1212A are coupled to each other. It should be noted that the application processor 1212A includes a DSI-2 transmitter 1233A and a DSI extension unit 1234A, replacing... Figure 75 The application processor 1212 in A has a CSI-2 receiving unit 1233 and a CSI extension unit 1234, and the other blocks are configured in the same way as the application processor 1212.

[0973] The display 1213 has the following configuration: an ALL layer 1242 is arranged on the A-PHY layer 1241; a DSI-2 receiver 1243 and a DSI extension unit 1244, as well as a CCI slave device 1245 and a CCI extension unit 1246 are arranged on the ALL layer 1242. The DSI-2 receiver 1243 has the DSI extension unit 1244, and the CCI slave device 1245 has the CCI extension unit 1246, thereby enabling the display 1213 to adapt to their respective extension standards. It should be noted that DSI extensions can also be referred to as Display Service Extensions (DSE).

[0974] The communication systems 1201 and 1201A configured in this way are capable of at least performing high-speed data transmission for sending frames of data including image data in one direction, and low-speed command transmission for sending commands related to high-speed data transmission in the opposite direction (however, sending a command itself may be referred to as command transmission, or sending a response to a command may be referred to as command transmission). For example, in low-speed command transmission, at least a high-speed data transmission start command for requesting the start of high-speed data transmission is sent, but this may not be the case. Furthermore, high-speed data transmission is faster than low-speed command transmission and begins in response to receiving a high-speed data transmission start command; however, this may not be the case.

[0975] However, the communication partners of application processor 1212 (the image sensor 1211's communication system 1201) and application processor 1212A (the display 1213's communication system 1201A) differ in their directions of high-speed data transmission and low-speed command transmission. That is, in communication system 1201, image data is transmitted from image sensor 1211 to application processor 1212, while in communication system 1201A, image data is transmitted from application processor 1212A to display 1213.

[0976] In the physical layer standard A-PHY, high-speed data transmission and low-speed command transmission are carried out via part or all of a shared communication path. Additionally, A-PHY supports options that enable some or all of the power supply from application processor 1212 to image sensor 1211 and from application processor 1212A to display 1213 to be carried out via the shared communication path.

[0977] Incidentally, for example, low-speed command transmission conforms to the CSI-2 standard for CCI and performs communication based on the I2C or I3C standard. In this case, low-speed command transmission can send commands by sharing not only the independent I2C or I3C physical layer but also some or all of the physical layers of D-PHY, C-PHY, and A-PHY. Meanwhile, high-speed data transmission sends data through some or all of the physical layers of any one of D-PHY, C-PHY, and A-PHY.

[0978] It is important to note that, in the case of a Unified Serial Link (USL) conforming to the CSI-2 standard, for example, low-speed command transmissions can send commands via some or all of the physical layers of either the D-PHY or C-PHY. That is, high-speed data transmission and low-speed command transmissions can send commands via some or all of the physical layers of any of the D-PHY, C-PHY, A-PHY, I2C, and I3C.

[0979] It should be noted that, although in Figure 75 The description already provides an example of the configuration including application processors 1212 and 1201A, but communication systems 1201 and 1201A can be configured to include, for example, an electronic control unit (ECU). That is, there is no limitation on the application processor 1212, as long as it is a processor capable of communicating with the image sensor 1211, display 1213, etc., via direct or indirect coupling. Furthermore, it is also possible to use a configuration including various sensors other than the image sensor 1211.

[0980] The communication systems 1201 and 1201A thus constituted employ the method described below, which involves transmitting random values ​​or initialization vectors including random values.

[0981] Specifically, special public-key encryption algorithms (e.g., AES-GCM / GMAC) require an initialization vector that includes a random value. Therefore, rules for setting the initialization vector and random value are pre-agreed between the image sensor 1211 and the application processor 1212, or between the display 1213 and the application processor 1212A.

[0982] However, when a random value is misidentified or tampered with within each of the image sensors 1211, application processors 1212 and 1201A, and display 1213, subsequent decryption of encrypted image data, message authentication, etc., fail. Therefore, to avoid failures in the normal transmission of image data, countermeasures against misidentification and tampering of random values ​​are needed.

[0983] Simultaneously, it is necessary to define the initialization vector suitable for the CSI or DSI standards as a new security provision for the MIPI Camera Serial Interface (CSI) or MIPI Display Serial Interface (DSI) standards. Therefore, this technology discloses a method for sending random values ​​or initialization vectors comprising random values, adaptable to imaging devices conforming to the CSI standard including image sensor 1211 or display devices conforming to the DSI standard including display 1213.

[0984] It should be noted that although the following description describes the processing performed between the image sensor 1211 and the application processor 1212, similar processing may also be performed between the display 1213 and the application processor 1212A.

[0985] < Figure 75 Detailed examples of the structure of image sensors in [the context of image sensors] >

[0986] Figure 76 This is a block diagram showing a detailed configuration example of the image sensor 1211.

[0987] The image sensor 1211 includes pixels 1301, an AD converter 1302, an image processing unit 1303, an extended mode adaptive CSI-2 transmission circuit 1304, a physical layer processing unit 1305, an I2C / I3C slave device 1306, a storage unit 1307, a message counter 1308, a random number update unit 1309, and a security unit 1310. It should be noted that pixels 1301, AD converter 1302, image processing unit 1303, extended mode adaptive CSI-2 transmission circuit 1304, physical layer processing unit 1305, I2C / I3C slave device 1306, and storage unit 1307 are configured in the same manner as the corresponding blocks in the other embodiments described above, and their detailed description is omitted.

[0988] Whenever an extended packet that meets a predetermined counting condition is sent, message counter 1308 updates the message count value inside image sensor 1211.

[0989] Security unit 1310 derives a session key within image sensor 1211 and uses the session key to generate first protected data (e.g., a complete arithmetic value subjected to arithmetic operations to protect integrity or encrypted data to protect confidentiality) that will undergo high-speed data transmission.

[0990] Each time the security unit 1310 generates the first protection data, the random number update unit 1309 updates the random number (number used once) value inside the image sensor 1211.

[0991] The image sensor 1211 thus configured performs high-speed data transmission of some or all of the random values ​​and some or all of the message count values ​​to the application processor 1212. For example, some or all of the random values ​​may each be a count value or a random number. In addition, some or all of the random values ​​are stored outside the extended packet header for transmission, and the image data is stored inside the packet data for transmission.

[0992] In the image sensor 1211, the message counter 1308 and the random number update unit 1309 can be configured separately or as a whole. For example, if the message counter 1308 and the random number update unit 1309 are configured separately, the random value and the message count value can be updated asynchronously. This enhances the flexibility of the random value and the message count value.

[0993] Simultaneously, when the message counter 1308 and the random number update unit 1309 are configured as a whole, the random value and the message count value can be updated synchronously. In this case, when the count value is used as the random value, the message count value shares some or all of the random value, thereby saving the bit width of the message counter 1308. That is, the message counter 1308 can be part or all of the random number update unit 1309, and part or all of it can be common to the random number update unit 1309.

[0994] < Figure 75 Detailed examples of application processor configuration in [the application processor] >

[0995] Figure 77 This is a block diagram illustrating a detailed configuration example of the application processor 1212.

[0996] Application processor 1212 includes a physical layer processing unit 1321, an extended mode adaptive CSI-2 receiver circuit 1322, an I2C / I3C host device 1323, a storage unit 1324, a data verification unit 1325, a security unit 1326, and a controller 1327. It should be noted that the physical layer processing unit 1321, the extended mode adaptive CSI-2 receiver circuit 1322, the I2C / I3C host device 1323, and the storage unit 1324 are configured in the same manner as their corresponding blocks in the other embodiments described above, and their detailed descriptions are omitted.

[0997] The data verification unit 1325 verifies the validity of random numerical values ​​or message count values ​​sent from the image sensor 1211 to the application processor 1212.

[0998] Security unit 1326 derives a session key corresponding to the session key inside the image sensor 1211 from the session key inside the application processor 1212, and uses the session key inside the application processor 1212 to verify (verify integrity) or decrypt the first protected data of the image data.

[0999] In the application processor 1212 configured as described above, when the data to be verified is a count value, the data verification unit 1325 can verify its continuity. Furthermore, the data verification unit 1325 can be configured to include a counter, updating the count value in the same manner as the image sensor 1211, thereby performing comparison and verification. It should be noted that when the data to be verified is a random number, the data verification unit 1325 can verify its randomness. It should also be noted that the data verification unit 1325 includes a random number update unit 1309 (or message counter), which can be used to verify or decrypt the first protected data, or to verify the data to be verified.

[1000] The image sensor 1211 and application processor 1212 can each be configured for installation on a desired mobile device. For example, the mobile device can be a portable mobile device, such as a mobile phone, smartphone, digital camera, game console, etc. The mobile device can be a propulsion device, such as a vehicle, robot, drone, etc., capable of propulsion (any of moving, walking, or flying). The mobile device can be any autonomous vehicle, autonomous robot, autonomous drone, etc., equipped with AI (artificial intelligence) functionality to achieve autonomous propulsion. The propulsion of the propulsion device can be controlled by the user of the propulsion device, and the propulsion device can notify the user of instructions or alarms as needed. Alternatively, the propulsion of the aforementioned propulsion device can be automatically controlled.

[1001] For example, security units 1310 and 1326 may both include a secure arithmetic unit that performs arithmetic operations for protecting image data. Therefore, security units 1310 and 1326 can enable the secure arithmetic unit to perform any processing such as encryption arithmetic, decryption arithmetic, hash value arithmetic, message authentication code arithmetic, digital signature arithmetic, ID (identification) authentication, firmware measurement, encrypted session key establishment, key exchange, key update, etc.

[1002] Simultaneously, any one of the security units 1310 and 1326, the random number update unit 1309, the message counter 1308, and the data verification unit 1325 can be configured for direct electrical coupling to memory. The memory can be directly electrically coupled to a register. Any one of the security units 1310 and 1326, the random number update unit 1309, the message counter 1308, and the data verification unit 1325 can be directly electrically coupled to a register. The memory can be a memory protected from information leakage or tampering within the memory. Such a memory and such a register are respectively used as storage units 1307 and 1324.

[1003] Storage units 1307 and 1324 can store any key information (e.g., pre-shared key, private key, public key, or session key), certificates (e.g., root certificate, intermediate certificate, or leaf certificate), encryption algorithm information, etc. Storage units 1307 and 1324 can store any of the following: functional information about the image sensor 1211 or application processor 1212; ID information about the image sensor 1211 or application processor 1212 (e.g., source ID, destination ID, final destination ID, etc.); firmware information about the image sensor 1211 or application processor 1212. Storage units 1307 and 1324 can store any of the following: session information (e.g., session ID), arithmetic values ​​from the secure arithmetic unit (e.g., initial value, intermediate value, or final value), initialization vectors, random values, message count values, frame numbers (frame count values), etc.

[1004] For example, if the image sensor 1211 or the application processor 1212 stores multiple random values, count values, completeness arithmetic values, and encrypted information in the storage unit 1307 or 1324, then any one of the security units 1310 and 1326, the random number update unit 1309, the message counter 1308, and the data verification unit 1325 can determine whether a fault exists or not, and respond accordingly (e.g., requesting retransmission of data at the fault location, sending an exception message). Furthermore, if any one of the random values, count values, completeness arithmetic values, and encrypted information is periodically stored in the protected storage unit 1307 or 1324, then the analysis of the protected storage unit 1307 or 1324 when an accident occurs in the mobile device also facilitates the identification of the cause of the accident.

[1005] <Conversation>

[1006] The requester and responder (i.e., application processor 1212 and image sensor 1211) can have one or more communication channels through a session. In the following description, the session is illustrated by showing an application processor 1212 as the requester and the image sensor 1211 as the responder. Undoubtedly, the application processor 1212 can be the responder and the image sensor 1211 can be the requester.

[1007] Furthermore, the requester and responder can establish a secure communication channel using temporarily fixed encrypted information. Specifically, the session provides one or both encryption or message authentication. A session includes, for example, three phases: a session handshake phase, an application phase, and a session termination phase.

[1008] The session handshake phase begins with a key exchange request (either PSK_EXCHANGE or KEY_EXCHANGE) from the requesting party, for example, deriving a session key (e.g., a session secret or encryption key) and using the session key to secure communication. The purpose of this phase is, for example, to establish trust between the responding and requesting parties before either side sends application data (e.g., image data). Furthermore, it ensures a certain degree of integrity in the handshake and synchronization with the derived handshake secret.

[1009] In the event of an error during this phase, the session can be terminated immediately and the session termination process can continue. When the handshake is successful, for example, by a termination response (FINISH_RSP or PSK_FINISH_RSP) from the responder, the session terminates, and the application phase begins. Once the handshake is complete and all authentications are passed, the session reaches the application phase, where either the responder or the requester can transmit application data.

[1010] For example, the application phase ends if an end request (END_SESSION) is issued by the requester or in the event of an error. The next phase is the session termination phase.

[1011] The session termination phase may be merely an internal phase, where no explicit message needs to be sent or received. When the session ends, both the requester and responder discard or clear all exported session keys, such as session secrets and encryption keys. The requester and responder may also have other internal data associated with the session, which they may also wish to clear.

[1012] Session secrets are used, for example, to derive the encryption key and salt to be used in AEAD (Authentication Encryption with Additional Data) functionality. The derivation of the encryption key may frequently use HMAC as defined in HKDF-Expand and RFC2104 as described in RFC5869. A session secret can consist of a single secret or multiple secrets of different types. A session key can consist of a single key or multiple keys of different types.

[1013] <Examples of high-speed data transmission and low-speed command transmission>

[1014] refer to Figures 78 to 80 A description is given of the communication processing that performs high-speed data transmission and low-speed command transmission between the image sensor 1211 and the application processor 1212.

[1015] Figure 78 This is a flowchart describing the first processing instance of the communication process.

[1016] Here, the extended mode adaptive CSI-2 receiving circuit 1322 of the application processor 1212 functions as both a CCI host (requester) and a CSI-2 host. The extended mode adaptive CSI-2 transmitting circuit 1304 of the image sensor 1211 functions as both a CCI device (responder) and a CSI-2 device. The CCI host sends a request message to the CCI device, and in response to its reception, the CCI device sends a response message to the CCI host.

[1017] In step S501, a GET_VERSION request and VERSION response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the Extended Mode Adaptive CSI-2 Receive Circuit 1322 to obtain the SPDM (Security Protocol and Data Model) version of the endpoint.

[1018] In step S502, a GET_CAPABILITIES request and CAPABILITIES response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the Extended Mode Adaptive CSI-2 Receive Circuit 1322 to acquire the SPDM functionality of the endpoint.

[1019] In step S503, a NEGOTIATE_ALGORITHMS request and ALGORITHMS response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the Extended Mode Adaptive CSI-2 Transmit Circuit 1304 to negotiate the encryption algorithm.

[1020] In step S504, a PSK_EXCHANGE request and a PSK_EXCHANGE_RSP response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the Extended Mode Adaptive CSI-2 Transmit Circuit 1304 to derive CCI-oriented session keys, such as session secrets or encryption keys.

[1021] In step S505, a PSK_FINISH request and a PSK_FINISH_RSP response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This confirms to the responding party that the Extended Mode Adaptive CSI-2 Receive Circuit 1322 knows the PSK (Pre-Shared Key) and that the CCI-oriented session key derived in step S504 is correct.

[1022] In step S506, a PSK_EXCHANGE request and a PSK_EXCHANGE_RSP response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the Extended Mode Adaptive CSI-2 Transmit Circuit 1304 to derive CSI-2-oriented session keys, such as session secrets or encryption keys.

[1023] In step S507, a PSK_FINISH request and a PSK_FINISH_RSP response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This confirms to the responding party that the Extended Mode Adaptive CSI-2 Receive Circuit 1322 knows the PSK (Pre-Shared Key) and that the CSI-2-oriented session key derived in step S506 is correct.

[1024] Here, the session key certificate in steps S505 and S507 is obtained by calculating the MAC value using the requester's finish_key and the messages of the session. Then, the session key derived in steps S504 and S506 is used to secure subsequent CCI and CSI-2 communications.

[1025] In step S508, in the extended mode adaptive CSI-2 receiving circuit 1322, the CCI host provides the CSI-2-oriented session secret or session key, algorithm and other parameters from the CCI host to the CSI-2 host.

[1026] In step S509, in the extended mode adaptive CSI-2 transmission circuit 1304, the session secret or session key, algorithm and other parameters for CSI-2 are provided from the CCI device to the CSI-2 device.

[1027] In step S510, the CSI-2 device of the Extended Mode Adaptive CSI-2 Transmitting Circuit 1304 transmits image data to the CSI-2 host of the Extended Mode Adaptive CSI-2 Receiving Circuit 1322 via high-speed data communication. For example, high-speed data communication continues until a timer is reached to update the CSI-2 session key.

[1028] In step S511, in the extended mode adaptive CSI-2 receiving circuit 1322, a trigger for updating the CSI-2-oriented session key is provided from the CSI-2 host to the CCI host. However, this trigger can be provided to the CCI host from either the CSI-2 device or the CCI device, or it can be self-triggered and provided to the CCI host from the CCI host.

[1029] In step S512, a KEY_UPDATE request and a KEY_UPDATE_ACK response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the session key to be updated and a portion of the old session key to be discarded. Note that if the session key consists of multiple keys (request direction key or response direction key, etc.), some or all of them can also be updated. Furthermore, the KEY_UPDATE request can be issued from the responding party using the GET_Encapsulated_REQUEST mechanism described later.

[1030] In step S513, a process similar to that in step S512 is performed, and the KEY_UPDATE request and KEY_UPDATE_ACK response are executed twice. This allows the remaining (all) portion of the old session key that was not discarded through the process in step S512 to be discarded.

[1031] In step S514, in the extended mode adaptive CSI-2 receiving circuit 1322, the CCI host provides the CSI-2-oriented session secret or session key (after update), algorithm and other parameters from the CCI host to the CSI-2 host.

[1032] In step S515, in the extended mode adaptive CSI-2 transmission circuit 1304, the CCI device provides the CSI-2-oriented session secret or session key (after update), algorithm and other parameters from the CCI device to the CSI-2 device.

[1033] In step S516, image data transmission via high-speed data communication begins in the same manner as in step S510; thereafter, the same process as in steps S510 to S515 is repeated.

[1034] It should be noted that in the first processing example of communication processing, the session key for CCI is different from the session key for CSI-2, the session ID for CCI is different from the session ID for CSI-2, and the session secret for CCI is different from the session secret for CSI-2. This is not restrictive; as in the second processing example of communication processing, the session key for CCI and the session key for CSI-2 can be the same, the session ID for CCI and the session ID for CSI-2 can be the same, and the session secret for CCI and the session secret for CSI-2 can be the same.

[1035] Figure 79 This is a flowchart describing the second processing instance of the communication process.

[1036] In steps S521 to S523, the following steps are performed: Figure 78 The similar processes in steps S501 to S503.

[1037] In step S524, a PSK_EXCHANGE request and a PSK_EXCHANGE_RSP response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. Here, in the second processing example of the communication process, the same CCI-oriented session secret and CSI-2-oriented session secret are derived.

[1038] In other words, both the CCI-oriented session key and the CSI-2-oriented session key can be derived from the same session secret. Alternatively, the uplink-oriented session key and the downlink-oriented (opposite direction to the uplink) session key can be derived from the same session secret. Alternatively, the common session key for both CCI and CSI-2 can be derived from the same session secret. It should be noted that even if the CCI-oriented and CSI-2-oriented sessions are the same, the CCI-oriented session secret, session key, etc., may differ from the CSI-2-oriented session secret, session key, etc.

[1039] Subsequently, in steps S525 to S534, the following steps are performed: Figure 78 The processing in steps S507 to S516 is similar to that in the previous process.

[1040] Here, the pre-shared key (PSK) key exchange scheme provides the requester and responder with the option to perform mutual authentication and session key establishment using symmetric key encryption. This option is particularly useful for endpoints that do not support asymmetric key encryption or certificate processing. Even when asymmetric key encryption is supported, this option can be used to accelerate session key establishment. This option requires the requester and responder to know the public PSK in advance before the handshake.

[1041] Essentially, the PSK serves as the foundation for establishing mutual authentication credentials and session keys. Therefore, only the two endpoints and the potential trusted third party providing the PSK to those endpoints can know the PSK value. A requester can be paired with multiple responders. Similarly, a responder can be paired with multiple requesters. One or more PSKs can be provided to a pair of requesters and responders.

[1042] An endpoint can act as a requester for one device and simultaneously as a responder for another device. Before PSK-based session key exchange can begin, the sending layer needs to identify the peer and establish communication between the two endpoints.

[1043] A PSK can be provided in a trusted environment, such as during a secure manufacturing process. In an untrusted environment, the PSK can be agreed upon between two endpoints using a secure protocol. The size of the specified PSK depends on the security strength requirements of the application and should be 128 bits or more, ideally 256 bits or more. During the PSK specification process, endpoint functionalities and supported algorithms can be transmitted to the peer. Therefore, the SPDM commands GET_CAPABILITIES and NEGOTIATE_ALGORITHMS are not required during session key establishment using the PSK option.

[1044] This option defines two message pairs: PSK_EXCHANGE / PSK_EXCHANGE_RSP and PSK_FINISH / PSK_FINISH_RSP. The PSK_EXCHANGE message has three roles: prompting the responder to obtain a specific PSK; exchanging context between the requester and responder; and proving to the requester that the responder knows the correct PSK and has derived the correct session key.

[1045] Figure 80 This is a flowchart describing a third processing example of communication processing.

[1046] In steps S541 to S543, the following steps are performed: Figure 78 The similar processes in steps S501 to S503.

[1047] In step S544, a GET_DIGESTS request and a DIGESTS response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the Extended Mode Adaptive CSI-2 Receive Circuit 1322 to obtain a certificate chain digest from the Extended Mode Adaptive CSI-2 Transmit Circuit 1304.

[1048] In step S545, a GET_CERTIFICATE request and CERTIFICATE response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the Extended Mode Adaptive CSI-2 Receive Circuit 1322 to obtain the certificate chain from the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. It should be noted that the acquisition of the certificate chain can be performed multiple times.

[1049] In step S546, a CHALLENGE request and CHALLENGE_AUTH response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the Extended Mode Adaptive CSI-2 Receive Circuit 1322 to authenticate the Extended Mode Adaptive CSI-2 Transmit Circuit 1304 via a challenge-response protocol.

[1050] In step S547, a KEY_EXCHANGE request (channel = CCI, session ID = D) and a KEY_EXCHANGE_RSP response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the handshake between the requester and responder to begin for the purpose of authenticating the responder (or optionally authenticating both parties). Then, in addition to the content negotiated in the final NEGOTIATE_ALGORITHMS / ALGORITHMS exchange, encryption parameters are negotiated, and shared key information is established.

[1051] In step S548, the CCI host of the extended mode adaptive CSI-2 receiving circuit 1322 sends GET_ENCAPSULATED_REQUEST to the CCI device of the extended mode adaptive CSI-2 transmitting circuit 1304.

[1052] In step S549, the CCI device of the Extended Mode Adaptive CSI-2 Transmitting Circuit 1304 sends an ENCAPSULATED_REQUEST (GET_DIGESTS request) to the CCI host of the Extended Mode Adaptive CSI-2 Receiving Circuit 1322.

[1053] In step S550, the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 sends a DELIVER_ENCAPSULATED_RESPONSE (DIGESTS response) to the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This allows the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304 to obtain the certificate chain digest from the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322.

[1054] In step S551, the CCI device of the Extended Mode Adaptive CSI-2 Transmitting Circuit 1304 sends an ENCAPSULATED_RESPONSE_ACK (GET_CERTIFICATE request) to the CCI host of the Extended Mode Adaptive CSI-2 Receiving Circuit 1322.

[1055] In step S552, the CCI host of the Extended Mode Adaptive CSI-2 receiving circuit 1322 sends a DELIVER_ENCAPSULATED_RESPONSE (CERTIFICATE response) to the CCI device of the Extended Mode Adaptive CSI-2 transmitting circuit 1304. This allows the CCI device (responding party) to obtain the certificate chain from the CCI host (requesting party). Furthermore, this process can be performed multiple times.

[1056] In step S553, the CCI device of the extended mode adaptive CSI-2 transmitting circuit 1304 sends ENCAPSULATED_RESPONSE_ACK to the CCI host of the extended mode adaptive CSI-2 receiving circuit 1322.

[1057] In step S554, a FINISH request and a FINISH_RSP response are performed between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304. This completes the handshake between the CCI host of the Extended Mode Adaptive CSI-2 Receive Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmit Circuit 1304, which was initiated by the KEY_EXCHANGE request in step S547.

[1058] In step S555, a GET_MEASUREMENTS request and MEASUREMENTS response are executed between the CCI host of the Extended Mode Adaptive CSI-2 Receiver Circuit 1322 and the CCI device of the Extended Mode Adaptive CSI-2 Transmitter Circuit 1304. This allows the CCI host of the Extended Mode Adaptive CSI-2 Receiver Circuit 1322 to acquire measurement data from the CCI device of the Extended Mode Adaptive CSI-2 Transmitter Circuit 1304. It should be noted that the GET_ENCAPSULATED_REQUEST mechanism described above can be used to issue a GET_MEASUREMENTS request from the responder. Similarly, the GET_ENCAPSULATED_REQUEST mechanism described above can also be used to issue another request from the responder.

[1059] Next, in step S556, the KEY_EXCHANGE request (channel = CSI-2, session ID = E) and KEY_EXCHANGE_RSP response are executed in the same manner as in step S547. In step S557, the FINISH request and FINISH_RSP response are executed in the same manner as in step S554. Then, in steps S558 to S566, the same... Figure 78 The same processes as those in steps S508 to S516.

[1060] <Data Validation Processing>

[1061] Reference Figures 81 to 83 Provide a description of the data validation process using validation groups and groups to be validated.

[1062] like Figure 81 and Figure 82As shown, an extended packet includes a packet header (PH), an extended packet header (ePH), packet data, an extended packet trailer (ePF), and a packet trailer PF. This type of extended packet can constitute a start of frame, embedded data, image data, user-defined data, end of frame, write command (CCI write), read command (CCI read), and read response (CCI read return value). It should be noted that some or all of the packet header (PH), extended packet header (ePH), packet data, extended packet trailer (ePF), and packet trailer PF can be omitted. That is, a packet structure that includes at least the extended packet header (ePH) and packet data is defined as an extended packet.

[1063] Incidentally, there is a possibility that any of the extended packet header (ePH), packet data, and extended packet trailer (ePF) may fail to be received properly due to noise, interference, or attacks (the message may be lost). Therefore, it is desirable to store a verification packet within the extended packet trailer (ePF0) to verify the integrity of the extended packet header (ePH), packet data, and the remaining portion of the extended packet trailer (ePF1). To verify integrity, for example, CRC32 with cyclic redundancy check is used; CRC32 is a type of error detection code. Furthermore, the generator polynomial of CRC32 is, for example, X32 + X26 + X23 + X22 + X16 + X12 + X11 + X10 + X8 + X7 + X5 + X4 + X2 + X+1.

[1064] Packet data can be used for the packet to be verified. Alternatively, the extended packet header and packet data can be used for the packet to be verified. Alternatively, packet data and the extended packet trailer remnant (ePF1) can be used for the packet to be verified. Alternatively, the extended packet header, packet data, and the extended packet trailer remnant (ePF1) can be used for the packet to be verified. This packet to be verified allows at least the packet data to be protected.

[1065] That is, the image sensor 1211 includes a second protection unit (e.g., a CRC arithmetic unit) that generates second protected data (e.g., a CRC arithmetic value) for packet data without using a session key. For example, the second protected data is stored in the extended packet trailer (ePF) of high-speed data transmission. Specifically, the second stored data is stored in any one of the following: start of frame, embedded data, image data, user-defined data, end of frame, write command (CCI Write), read command (CCI Read), and read response (CCI Read return value).

[1066] The extended packet trailer ePF1 or ePF0 may have security features defined therein. That is, the image sensor 1211 may include a secure arithmetic unit (e.g., an encryption arithmetic unit, a decryption arithmetic unit, a hash value arithmetic unit, a message authentication code arithmetic unit, or a digital signature arithmetic unit). Furthermore, the result of the secure arithmetic operation (e.g., the hash value of the digital signature, the message authentication code) may be stored in the extended packet trailer ePF.

[1067] The result of secure arithmetic operations can be stored only within the extended packet tail ePF1 instead of within the extended packet tail ePF0, and can be stored outside the extended packet tail (e.g., inside the embedded data or inside the read response) instead of inside the extended packet tail. The secure arithmetic unit included in the image sensor 1211 is included in the secure unit 1310.

[1068] As a Message Authentication Code (MAC), any of the following can be used: GMAC (GaloisMAC), CMAC (Cipher-based MAC), HMAC (Hash-based MAC), etc. For example, AES (Advanced Encryption Standard) or SHA (Secure Hash Algorithm) can be used, specifically AES-GMAC, AES-CMAC, SHA2-HMAC, SHA3-HMAC, etc. It should be noted that AES has a 128-bit block length, and the key length can be any of 128 bits, 192 bits, or 256 bits.

[1069] The extended packet trailer can store security information such as hash (specifically, cryptographic hash) values, message authentication codes, digital signatures, etc., where the packet data serves as the packet to be verified, or the extended packet header and packet data serve as the packet to be verified. In this case, it is possible to further resist malicious tampering from attackers. It should be noted that the extended packet trailer "ePF1" or "ePF1 and ePF0" can store a Cyclic Redundancy Check (CRC), which is a type of error detection code.

[1070] In other words, the image sensor 1211 may include a completeness arithmetic unit (e.g., a first protection unit = a secure arithmetic unit, a second protection unit = a CRC arithmetic unit), and may store completeness arithmetic values ​​(e.g., first protected data or second protected data) generated by completeness arithmetic operations in the extended packet trailer. It is important to note that CRC can be used for functional safety, and completeness can be used to prevent undetected hardware failures. Simultaneously, the completeness of security features can be used to detect intentional interference or attacks. That is, the secure arithmetic unit performs arithmetic operations based on cryptographically based completeness arithmetic values, while the CRC arithmetic unit performs arithmetic operations on non-cryptographically based completeness arithmetic values.

[1071] Application processor 1212 can verify the integrity of the packet to be verified, for example, by using a verification packet. For example, in the case of an anomaly, the following processes can also be performed, such as sending a request message to retransmit the packet containing the packet to be verified and the verification packet, sending a request message to inquire whether there is an anomaly in image sensor 1211, sending a request message to request image sensor 1211 to stop some or all of its functions, stopping the propulsion of the propulsion device, changing the propulsion control of the propulsion device, and changing the priority data used for propulsion control.

[1072] It is important to note that the completeness arithmetic value can be stored within any of, for example, embedded data, image data (packet data), user-defined data, write commands, read commands, read responses, etc. In this case, the completeness arithmetic value may not be stored in the extended packet trailer. For example, the completeness arithmetic value can be stored on a frame-by-frame basis rather than on a line-by-line basis, in which case the arithmetic operation is performed effectively on the completeness. In this case, for example, the completeness arithmetic value is stored within the read response or embedded data after image data is sent.

[1073] Figure 81 The extended packet shown in A is a constitutive example, wherein the extended packet header ePH, packet data, and extended packet trailer remainder ePF1 are used as the packet to be verified, and the extended packet trailer ePF0, which stores the calculated value determined by using secure arithmetic operations on the packet to be verified, is used as the verification packet.

[1074] exist Figure 81 The extended packet shown in B is a packet to be verified that uses the packet data and the extended packet tail remainder ePF1 as the packet to be verified, and uses the extended packet tail ePF0, which stores the calculated value determined by using the secure arithmetic operation of the packet to be verified, as a constituent instance of the verification packet.

[1075] exist Figure 81The extended packet shown in C is a constitutive instance in which the extended packet header ePH, the packet data used as the packet to be verified, and the extended packet trailer ePF0, which stores the calculated value determined by using secure arithmetic operations on the packet to be verified, is used as the verification packet.

[1076] exist Figure 81 The extended packet shown in D is a packet that uses the packet data as the packet to be verified, and uses the extended packet message tail ePF0, which stores the calculated value determined by using the secure arithmetic operation of the packet to be verified, as a constituent instance of the verification packet.

[1077] exist Figure 82 The extended packet shown in A is a construction example, wherein the extended packet header ePH and packet data are used as the packet to be verified, and the extended packet tail remainder ePF1, which stores the calculated value determined by using secure arithmetic operations on the packet to be verified, is used as the verification packet.

[1078] exist Figure 82 The extended packet shown in B is a construction example, wherein the extended packet header ePH and packet data are used as the packet to be verified, and the extended packet tail ePF0 and the extended packet tail remainder ePF1, which store the calculated value determined by using secure arithmetic operations on the packet to be verified, are used as the verification packet.

[1079] exist Figure 82 The extended packet shown in C is a packet that uses the packet data as the packet to be verified, and uses the extended packet tail remainder ePF1, which stores the calculated value determined by using the secure arithmetic operation of the packet to be verified, as a constitutive instance of the verification packet.

[1080] exist Figure 82 The extended packet shown in D is a packet that uses the packet data as the packet to be verified, and uses the extended packet tail ePF0 and the extended packet tail remainder ePF1, which store the calculated value determined by using the secure arithmetic operation of the packet to be verified, as a constituent instance of the verification packet.

[1081] Figure 83 This is a flowchart describing the data verification process performed in application processor 1212.

[1082] In step S601, when the extended packet sent from the image sensor 1211 is received by the extended mode adaptive CSI-2 receiving circuit 1322, the security unit 1326 receives the packet to be verified from the extended packet. Then, when the security unit 1326 finishes receiving the packet to be verified, the process proceeds to step S602. It should be noted that even if the reception of all packets to be verified has not been completed, the process can proceed to step S602 as long as the reception of at least a portion (e.g., 128 bits) that enables the commencement of secure arithmetic operations has been completed. In this case, the remaining portion of the packet to be verified continues to be received until the reception of all packets to be verified is completed.

[1083] In step S602, the security unit 1326 uses at least a portion of the packet to be verified received in step S601 to begin calculating the value to be determined by security arithmetic operations.

[1084] In step S603, the security unit 1326 receives the verification packet sent from the image sensor 1211 via the extended mode adaptive CSI-2 receiving circuit 1322. Then, when the security unit 1326 completes the reception of the verification packet and obtains the received value (the calculated value calculated by the image sensor 1211) stored in the verification packet, the process proceeds to step S604.

[1085] In step S604, when the security unit 1326 completes the calculation of the calculated value determined by the security arithmetic operation using the verification packets that started in step S602 (i.e., receives all verification packets and completes the calculation using all verification packets), the process proceeds to step S605.

[1086] In step S605, the security unit 1326 determines whether the received value received in step S603 is consistent with the calculated value determined in step S604.

[1087] If the security unit 1326 determines in step S605 that the received value and the calculated value are consistent, the process proceeds to step S606. In this case, in step S606, the security unit 1326 determines that the extended packet received by the extended mode adaptive CSI-2 receiving circuit 1322 is normal, and the process ends.

[1088] Meanwhile, if the security unit 1326 determines in step S605 that the received value and the calculated value are inconsistent, the process proceeds to step S607. In this case, in step S607, the security unit 1326 determines that an anomaly has occurred in the extended packet received by the extended mode adaptive CSI-2 receiving circuit 1322, and the process ends.

[1089] <Use message count values ​​to ensure functional safety>

[1090] To ensure functional safety (e.g., detecting and properly resolving lost messages), image sensor 1211 can store message count values ​​to be counted by message counter 1308 within the extended packet header or extended packet trailer. For example, message counter 1308 included in image sensor 1211 can store message count values ​​that increment or decrement each time a message is sent from image sensor 1211. It should be noted that image sensor 1211 may have a configuration in which an independent message counter 1308 is provided for each virtual channel (virtual channel), or a configuration in which a message counter 1308 is provided common to all virtual channels.

[1091] Message counter 1308 sets the message count value to an initial value (e.g., zero or the maximum value) in the first packet including the extended packet header of a virtual channel, so as to increment or decrement the message count value each time data including the extended packet header of a virtual channel is transmitted. Furthermore, for example, if data without the extended packet header is transmitted, message counter 1308 does not increment or decrement the message count, and the count is resumed when data including the extended packet header is next transmitted.

[1092] Message counter 1308 can continue counting regardless of whether a frame begins or ends. Then, when the message count value is counted to a specified value (e.g., the maximum value or 0), message counter 1308 returns the next message count value to the initial value (e.g., 0 or the maximum value) and performs counting. It should be noted that a portion of the extended packet header may store some random numerical value.

[1093] It should be noted that in the event of message loss, the receiving side (image sensor 1211 or application processor 1212) that receives the message count value can detect the loss immediately. For example, a DoS (denial of Service) attack that intentionally combines a large number of messages to violate the availability of image sensor 1211 or application processor 1212 is also detected immediately at the receiving side. Therefore, the message count value is ideally stored in the extended packet header. This ability to detect such loss or attack within a short period of time allows the receiving side to begin resolving these losses or attacks within a short period of time, which is particularly suitable for applications such as high-speed moving or propulsion devices that can move at high speeds.

[1094] It should be noted that for write commands (CCI Write), read commands (CCI Read), or read responses (CCI Readreturn value), message count values ​​or integrity arithmetic values ​​can also be stored and applied to elements associated with the extended group. In this case, for example, functional safety or protection integrity can also be addressed for write commands, read commands, or read responses.

[1095] Figure 84 This is a flowchart describing the message count value transmission process of the image sensor 1211.

[1096] In step S611, message counter 1308 initializes the message count value to zero.

[1097] In step S612, the Extended Mode Adaptive CSI-2 Transmitting Circuit 1304 determines whether to transmit the extended packet header and waits until it determines that the extended packet header should be transmitted. Then, in step S612, if the Extended Mode Adaptive CSI-2 Transmitting Circuit 1304 determines that the extended packet header should be transmitted, the process proceeds to step S613.

[1098] In step S613, the extended mode adaptive CSI-2 transmission circuit 1304 obtains the message count value from the message counter 1308 and stores it in the extended packet header.

[1099] In step S614, the extended mode adaptive CSI-2 transmitting circuit 1304 transmits the extended packet header that has stored the message count value in step S613.

[1100] In step S615, message counter 1308 determines whether the message count value has been counted to the maximum value. If message counter 1308 determines in step S615 that the message count value has not yet been counted to the maximum value, the process proceeds to step S616.

[1101] In step S616, message counter 1308 increments the message count value. Afterward, the process returns to step S612, and similar processing is subsequently repeated.

[1102] Meanwhile, if message counter 1308 determines in step S615 that the message count value has been counted to the maximum value, the process returns to step S611 and the message count value is initialized, and then similar processing is repeated.

[1103] It is important to note that, in addition to this incrementing of the message count value, the message count value can also be initialized to a maximum value for decrementing.

[1104] <About Embedded Data>

[1105] Reference Figures 85 to 88 Provide a description of the embedded data.

[1106] Image sensor 1211 can use embedded data to include additional information, such as device setting information, in the data stream. The embedded data is arranged in one or more rows and may include any of the following: image sensor 1211 configuration data, standard-compliant register values, vendor-specific register values, frame format descriptions, statistics, etc.

[1107] Figure 85 A in the diagram shows a line of embedded data; in its structure, the embedded data of the desired amount of data is arranged consecutively after the embedded data format code, and the padding characters are arranged in the remainder.

[1108] Embedded data includes information related to image data or user-defined data. Therefore, while image data or user-defined data can be compressed, embedded data is expected to be uncompressed. Consequently, in the case of data compression, compressed data (image data or user-defined data) and uncompressed data (embedded data) exist in a mixed manner within frames of high-speed data transmission.

[1109] Depending on the number of register values ​​to be added to the embedded data, the embedded data can have multiple lines (rows). Furthermore, the row number of the embedded data can be specified by a portion of the description within the frame format in the first embedded data line within the frame. The line length of the embedded data can be shorter than the line length of the image data or user-defined data, but preferably not exceeding it, and ideally the same as it. The first pixel value of the embedded data can be in the format to be used for the embedded data.

[1110] like Figure 85 As shown in B, some or all of the random values ​​can be stored in at least a portion of the embedded data's manufacturer-specific code (manufacturer-specific) or reserved code (reserved for future use), and said random values ​​can be transmitted. Within a frame, the embedded data is stored between the frame start and the first image data or user-defined data, or between the last image data or user-defined data and the frame end. However, the embedded data between the last image data or user-defined data and the frame end can be omitted.

[1111] Figure 86 An example of the data structure for two frames of image data transmitted from image sensor 1211 is shown.

[1112] like Figure 86As shown, after sending the first virtual channel frame start (VC1 FS), and after the read command and read response, the second virtual channel frame start (VC2 FS) is sent. Next, the first virtual channel first embedded data (VC1 EmbData) and the second virtual channel first embedded data (VC2 EmbData) are sent. Then, one frame of first virtual channel image data (VC1 Img Data) and second virtual channel user-defined data (VC2 UD Data) is sent. Upon completion of one frame transmission, the first virtual channel second embedded data (VC1 EmbData) and the second virtual channel second embedded data (VC2 EmbData) are sent. Subsequently, after sending the first virtual channel frame end (VC1 FE), and after the read command and read response, the second virtual channel frame end (VC2 FE) is sent.

[1113] Figure 86 An example is shown where the message count value is common to both the first and second virtual channels. In this case, a configuration can be adopted where independent message counters are set for the first and second virtual channels. Furthermore, user-defined data can be image data, etc.

[1114] Here, for example, some or all of the random values ​​may be stored during the time period from the start of a frame to the end of a frame, or during the time period from the end of a frame to the start of a frame (frame blanking period). Furthermore, during the time period from the start of a frame to the end of a frame, the random values ​​may be stored, for example, in embedded data, image data, non-image data, or during the line blanking period. Additionally, the random values ​​may be stored in a second virtual channel.

[1115] Defining frame start and frame end points allows, for example, an image sensor to notify the processor of the start and end of high-speed data transmission. Furthermore, the image sensor can maintain a constant frame transmission period. It should be noted that embedded data is data that stores attributes representing image data, information associated with the image data (metadata), etc.

[1116] In this embodiment, an example of performing high-speed data transmission of random values ​​without suppressing high-speed data transmission of image data is described. That is, an example is described where high-speed data transmission of image data and high-speed data transmission of random values ​​are performed serially rather than in parallel. However, when different communication paths are used for high-speed data transmission of image data and transmission of random values ​​(high-speed data transmission or low-speed command transmission), the transmission can be performed in parallel.

[1117] It should be noted that frequency separation of the filter is possible for high-speed data transmission and low-speed command transmission, and therefore, some or all transmissions can be duplicated (executed in parallel) unless there are issues with power consumption. Some or all of the random values ​​can be sent for each of multiple frames, but due to reasons such as frame loss, it is desirable to send some or all of the random values ​​for each frame. For example, the Start of Frame (FS) packet includes a start-of-frame code (DataType = 0x00) and the End of Frame (FE) packet includes an end-of-frame code (DataType = 0x01).

[1118] Figure 87 This is a flowchart describing the image data transmission process of the image sensor 1211.

[1119] In step S621, the extended mode adaptive CSI-2 transmitting circuit 1304 determines whether a start command for high-speed data transmission has been received and waits for processing until it is determined that a start command for high-speed data transmission has been received. Then, if the extended mode adaptive CSI-2 transmitting circuit 1304 determines in step S621 that a start command for high-speed data transmission has been received, the processing proceeds to step S622.

[1120] In step S622, pixel 1301 begins imaging, and the image data output from pixel 1301 is provided to extended mode adaptive CSI-2 transmission circuit 1304 via AD converter 1302 and image processing unit 1303.

[1121] In step S623, the extended mode adaptive CSI-2 transmission circuit 1304 begins transmitting the frame of the first virtual channel.

[1122] In step S624, the extended mode adaptive CSI-2 transmission circuit 1304 begins transmitting frames for the second virtual channel.

[1123] In step S625, the extended mode adaptive CSI-2 transmitting circuit 1304 transmits the first embedded data of the first virtual channel.

[1124] In step S626, the extended mode adaptive CSI-2 transmitting circuit 1304 transmits the first embedded data of the second virtual channel.

[1125] In step S627, the extended mode adaptive CSI-2 transmitting circuit 1304 transmits image data from the first virtual channel.

[1126] In step S628, the extended mode adaptive CSI-2 transmitting circuit 1304 transmits user-defined data for the second virtual channel.

[1127] In step S629, the extended mode adaptive CSI-2 transmission circuit 1304 determines whether the transmission of image data for one frame has been completed.

[1128] If the extended mode adaptive CSI-2 transmission circuit 1304 determines in step S629 that the transmission of one frame of image data has not been completed, the process returns to step S627, and then repeats a similar process. Conversely, if the extended mode adaptive CSI-2 transmission circuit 1304 determines in step S629 that the transmission of one frame of image data has been completed, the process proceeds to step S630.

[1129] In step S630, the extended mode adaptive CSI-2 transmitting circuit 1304 transmits the second embedded data of the first virtual channel.

[1130] In step S631, the extended mode adaptive CSI-2 transmitting circuit 1304 transmits the second virtual channel second embedded data.

[1131] In step S632, the extended mode adaptive CSI-2 transmitting circuit 1304 finishes transmitting the first virtual channel frame.

[1132] In step S633, the extended mode adaptive CSI-2 transmitting circuit 1304 finishes transmitting the second virtual channel frame.

[1133] In step S634, the extended mode adaptive CSI-2 transmitting circuit 1304 determines whether an end command for high-speed data transmission has been received.

[1134] If the extended mode adaptive CSI-2 transmitting circuit 1304 determines in step S634 that no end command for high-speed data transmission has been received, the process returns to step S622, and then repeats a similar process. Conversely, if the extended mode adaptive CSI-2 transmitting circuit 1304 determines in step S634 that an end command for high-speed data transmission has been received, the process ends.

[1135] Imaging can continue until a termination command for high-speed data transmission is received, and imaging can be performed whenever a start command for high-speed data transmission is received.

[1136] Figure 88 This is a flowchart describing the complete arithmetic value transmission process, in which image se...

Claims

1. An information processor, comprising a protection unit protecting at least a portion of first communication, wherein, In the first communication, a frame is sent or received, wherein... The frame includes multiple lines of first packet data, which includes: image data, embedded data, or user-defined data transmitted in the frame. The protection unit performs arithmetic operations on a first message authentication code to protect at least a portion or all of the first packet data, based on first communication message authentication strategy information indicating which first communication message authentication strategy to select from an option including a first message authentication strategy, a second message authentication strategy, a third message authentication strategy, and a fourth message authentication strategy. The first message authentication strategy is a strategy that applies protection to all of the first packet data using the first message authentication code. The second message authentication strategy is a strategy that excludes a portion of the first packet data in the vertical direction of the row from protection by the first message authentication code. The third message authentication strategy is a strategy that excludes a portion of the first packet data in the horizontal row direction from protection by the first message authentication code, and The fourth message authentication strategy is a strategy that excludes all first packet data from protection that has passed the first message authentication code.

2. The information processor according to claim 1, wherein, The frame includes at least one row of second grouped data, and The second group of data includes: the source ID value, the virtual channel value, and part or all of the value that changes for each frame.

3. The information processor according to claim 1, wherein, The protection unit uses an initialization vector in the arithmetic operation of the first message authentication code, and The initialization vector includes the source ID value, the virtual channel value, and the frame count value.

4. The information processor according to claim 1, wherein, The protection unit uses an initialization vector in the arithmetic operation of the first message authentication code, and The initialization vector includes the source ID value, the extended virtual channel value, and the message count value.

5. The information processor according to claim 1, wherein, The protection unit uses a first initialization vector including the value of a pre-counter and performs an arithmetic operation on at least one of the encryption or decryption of the first block of data. The first message authentication code undergoes arithmetic operations using a second initialization vector that does not include the value of the pre-counter, and Some or all of the elements constituting the second initialization vector are the same as some of the elements constituting the first initialization vector.

6. The information processor according to claim 1, wherein, The protection unit selects the first communication message authentication strategy from at least two of the first message authentication strategy, the second message authentication strategy, and the third message authentication strategy, and performs arithmetic operations on the first message authentication code.

7. The information processor according to claim 1, wherein, The protection unit performs arithmetic operations on the second message authentication code to protect at least a portion of the second communication, and The first communication message authentication policy information is protected by the second message authentication code and sent or received.

8. The information processor according to claim 1, wherein, The protection unit performs arithmetic operations on the second message authentication code to protect at least a portion of the second communication, and The second message authentication code protects at least the coupled destination address, register address, written data, the value of an implicit additional message counter that changes in response to the message counter, and the value of the message counter.

9. The information processor according to claim 1, wherein, The protection unit performs arithmetic operations on the second message authentication code to protect at least a portion of the second communication, and The second message authentication code protects at least the coupled destination address, register address, read data, the value of an implicit additional message counter that changes in response to the message counter, and the value of the message counter.

10. The information processor according to claim 1, wherein, The protection unit uses the second session key in the arithmetic operation of the second message authentication code to protect at least a portion of the second communication. The protection unit uses the first session key in the arithmetic operation of the first message authentication code, and In the second communication, the message requesting to begin using the first session key is protected by the second message authentication code and is sent or received.

11. The information processor according to claim 1, wherein, The protection unit uses the second session key in the arithmetic operation of the second message authentication code to protect at least a portion of the second communication. The protection unit uses the first session key in the arithmetic operation of the first message authentication code, and In the second communication, the message requesting the termination of the use of the first session key is protected by the second message authentication code and sent or received.

12. The information processor according to claim 1, wherein, The protection unit uses an initialization vector in the arithmetic operations that protect at least a portion of the second communication, and The initialization vector includes information indicating whether to use a write mode or a read mode.

13. The information processor according to claim 1, wherein, The protection unit uses an initialization vector in the arithmetic operations that protect at least a portion of the second communication, and In response to sending at least one of a request message and a read command from the communication partner of the second communication, at least a portion of the initialization vector is sent to the communication partner of the second communication, or In response to sending at least one of a request message and a read command to the communication partner of the second communication, at least a portion of the initialization vector is sent from the communication partner of the second communication.

14. The information processor according to claim 1, wherein, The protection unit performs arithmetic operations on the second message authentication code, which protects at least a portion of the write data sent or received via the second communication. The protection unit performs processing in response to the write data based on a second communication message authentication policy selected from at least one of a fifth message authentication policy and a sixth message authentication policy. The fifth message authentication policy is a policy that always allows the processing in response to the write data, and The sixth message authentication policy is a policy that allows processing of the write data only if the second message authentication code is successfully verified.

15. The information processor according to claim 1, wherein, The information processor includes function registers, and The function register stores at least information indicating whether the first message authentication strategy can be selected.

16. The information processor according to claim 1, wherein, The information processor includes function registers, and The function register stores information related to the upper limit of the amount of data that the protection unit can retain during the arithmetic operation of the second message authentication code for protecting at least a portion of the second communication, and The protection unit verifies the second message authentication code for data groups within a range not exceeding the upper limit of the data volume.

17. The information processor according to claim 1, wherein, The protection unit performs an arithmetic operation on at least one of a second message authentication code or a CRC, wherein the second message authentication code or CRC protects at least a portion of the second communication. In the second communication, send or receive message groups, and The final message in the message group includes information indicating whether an arithmetic operation of at least one of the second message authentication code or CRC can be completed.

18. The information processor according to claim 1, wherein, The protection unit selects from an option that includes at least CBC mode and CTR mode. The protection unit uses a second session key in the arithmetic operation of at least one of the encryption and decryption processes in the second communication. The protection unit is configured to perform: Regarding the switching of whether to use the second communication to receive the first encrypted data and the first initialization vector, and the switching of using the second session key and the first initialization vector to decrypt the first encrypted data in the CBC mode, and Switching between whether to generate a second initialization vector, using the second initialization vector and the second session key to perform arithmetic operations on the second encrypted data using encryption in the CBC mode, and using the second communication to send the second encrypted data and the second initialization vector.

19. A mobile device, comprising a protective section protecting at least a portion of a first communication, wherein, In the first communication, a frame is sent or received, wherein... The frame includes multiple lines of first packet data, which includes: image data, embedded data, or user-defined data transmitted in the frame. The protection unit performs arithmetic operations on a first message authentication code to protect at least a portion or all of the first packet data, based on first communication message authentication strategy information indicating which first communication message authentication strategy to select from an option including a first message authentication strategy, a second message authentication strategy, a third message authentication strategy, and a fourth message authentication strategy. The first message authentication strategy is a strategy that applies protection to all of the first packet data using the first message authentication code. The second message authentication strategy is a strategy that excludes a portion of the first packet data in the vertical direction of the row from protection by the first message authentication code. The third message authentication strategy is a strategy that excludes a portion of the first packet data in the horizontal row direction from protection by the first message authentication code, and The fourth message authentication strategy is a strategy that excludes all first packet data from protection that has passed the first message authentication code.

20. A communication system, comprising a protection unit protecting at least a portion of a first communication, wherein, In the first communication, a frame is sent or received, wherein... The frame includes multiple lines of first packet data, which includes: image data, embedded data, or user-defined data transmitted in the frame. The protection unit performs arithmetic operations on a first message authentication code to protect at least a portion or all of the first packet data, based on first communication message authentication strategy information indicating which first communication message authentication strategy to select from an option including a first message authentication strategy, a second message authentication strategy, a third message authentication strategy, and a fourth message authentication strategy. The first message authentication strategy is a strategy that applies protection to all of the first packet data using the first message authentication code. The second message authentication strategy is a strategy that excludes a portion of the first packet data in the vertical direction of the row from protection by the first message authentication code. The third message authentication strategy is a strategy that excludes a portion of the first packet data in the horizontal row direction from protection by the first message authentication code, and The fourth message authentication strategy is a strategy that excludes all first packet data from protection that has passed the first message authentication code.

Images