Information processing device, mobile device, and communication system

Abstract

The present disclosure relates to an information processing device, a mobile body device, and a communication system capable of achieving higher safety. When performing high-speed transmission of data of a frame including image data by establishing communication with another information processing device, an expanded packet including an expanded packet header and packet data is transmitted to the other information processing device. A session key is derived, first protection data of the packet data is generated by using the session key, and a nonce value for the session key is updated each time the first protection data is generated. Then, the image data is stored in the packet data. The high-speed data transmission includes transmission of some or all of the nonce values. Some or all of the nonce values are stored outside the expanded packet header and are transmitted. The present technology is applicable to, for example, a communication system compliant with the MIPI standard.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a mobile apparatus, and a communication system, and more particularly to an information processing apparatus, a mobile apparatus, and a communication system that can enhance safety more effectively.

Background Art

[0002] Currently, in CSI (Camera Serial Interface)-2 ver4.0 which is in the process of standardization, two types of packet structures are defined: a packet structure using C-PHY in the physical layer and a packet structure using D-PHY in the physical layer.

[0003] In recent years, as the CSI-2 standard has come to be widely used not only for mobile devices but also for various applications such as in-vehicle and IoT (Internet of Things), it is assumed that the existing packet structures cannot meet the requirements of these applications. Therefore, the MIPI (Mobile Industry Processor Interface) Alliance is considering extended packets with extended packet structures such as existing packet headers and packet footers to support various applications.

[0004] For example, as disclosed in Patent Document 1, a communication system that uses an extended packet header separately from the conventional packet header has been proposed.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] By the way, as described above, the message count value for functional safety use stored in the extended packet under consideration can be diverted as a nonce value for security use. However, for example, since the nonce value for the same session key is prohibited from rolling over for attack countermeasures, the 16-bit message count value is not suitable for directly applying it as the nonce value. Therefore, it is required to make the nonce value usable in a more specific and secure manner.

[0007] The present disclosure has been made in view of such a situation and aims to enhance security.

Means for Solving the Problem

[0008] An information processing apparatus according to an aspect of the present disclosure includes a communication unit that transmits an extended packet including an extended packet header and packet data to another information processing apparatus when performing communication with the other information processing apparatus and transmitting data of a frame including image data at high speed; a first protection unit that derives a session key and generates first protection data for the packet data using the session key; and a nonce update unit that updates a nonce value for the session key every time the first protection data is generated. The image data is stored in the packet data, the high-speed data transmission includes transmission of part or all of the nonce value, and part or all of the nonce value is stored outside the extended packet header and transmitted.

[0009] A mobile device according to an aspect of the present disclosure includes pixels that output image data, and a communication unit that communicates with another information processing device and transmits an extended packet including an extended packet header and packet data to the other information processing device when transmitting data of a frame including the image data at high speed data transmission. The mobile device further includes a first protection unit that derives a session key and generates first protected data of the packet data using the session key, and a nonce update unit that updates a nonce value for the session key each time the first protected data is generated. The image data is stored in the packet data, the high speed data transmission includes transmission of part or all of the nonce value, and part or all of the nonce value is stored outside the extended packet header and transmitted.

[0010] A communication system according to an aspect of the present disclosure includes a communication unit that communicates with another information processing device and transmits an extended packet including an extended packet header and packet data to the other information processing device when transmitting data of a frame including the image data at high speed data transmission. The communication system further includes a first protection unit that derives a session key and generates first protected data of the packet data using the session key, and a nonce update unit that updates a nonce value for the session key each time the first protected data is generated. The image data is stored in the packet data, the high speed data transmission includes transmission of part or all of the nonce value, and part or all of the nonce value is stored outside the extended packet header and transmitted.

[0011] In an aspect of the present disclosure, when communicating with another information processing device and transmitting data of a frame including image data at high speed data transmission, an extended packet including an extended packet header and packet data is transmitted to the other information processing device. A session key is derived, first protected data of the packet data is generated using the session key, and a nonce value for the session key is updated each time the first protected data is generated. The image data is stored in the packet data, the high speed data transmission includes transmission of part or all of the nonce value, and part or all of the nonce value is stored outside the extended packet header and transmitted.

Brief Description of Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Figure 47

Figure 48

Figure 49

Figure 50

Figure 51

Figure 52

Figure 53

Figure 54

Figure 55

Figure 56

Figure 57

Figure 58

Figure 59

Figure 60

Figure 61

Figure 62

Figure 63

Figure 64

Figure 65

Figure 66

Figure 67

Figure 68

Figure 69

Figure 70

Figure 71

Figure 72

Figure 73

Figure 74

Figure 75

Figure 76

Figure 77

Figure 78

Figure 79

Figure 80

Figure 81

Figure 82

Figure 83

Figure 84

Figure 85

Figure 86

Figure 87

Figure 88

Figure 89

Figure 90

Figure 91

Figure 92

Figure 93

Figure 94

Figure 95

Figure 96

Figure 97

Figure 98

Figure 99

Figure 100

Figure 101

Figure 102

Figure 103

Figure 104

Figure 105

Figure 106

Figure 107

Figure 108

Figure 109

Figure 110

Figure 111

Figure 112

Figure 113

Figure 114

Figure 115

Figure 116

Figure 117

Figure 118

Embodiments for Carrying Out the Invention

[0013] Hereinafter, specific embodiments to which this technology is applied will be described in detail with reference to the drawings.

[0014] <Configuration Example of Communication System> FIG. 1 is a block diagram showing a configuration example of a first embodiment of a communication system to which this technology is applied.

[0015] As shown in FIG. 1, the communication system 11 is configured by connecting an image sensor 21 and an application processor 22 via a bus 23. For example, the communication system 11 is used for CSI-2 connection inside an existing mobile device such as a so-called smartphone.

[0016] The image sensor 21 is configured by incorporating, for example, an extended mode compatible CSI-2 transmission circuit 31 together with a lens and an imaging element (both not shown). For example, the image sensor 21 transmits the image data of the image acquired by the imaging element to the application processor 22 by the extended mode compatible CSI-2 transmission circuit 31.

[0017] The application processor 22 is configured by incorporating an LSI (Large Scale Integration) that performs processing according to various applications executed on a mobile device equipped with the communication system 11, along with an extended mode compatible CSI-2 receiving circuit 32. For example, the application processor 22 can receive image data transmitted from the image sensor 21 by the extended mode compatible CSI-2 receiving circuit 32, and perform processing according to the application on the image data by the LSI.

[0018] The bus 23 is a communication path that transmits signals in accordance with the CSI-2 standard. For example, the transmission distance capable of transmitting signals is about 30 cm. Also, as shown in the figure, the bus 23 connects the image sensor 21 and the application processor 22 by a plurality of signal lines (I2C, CLKP / N, D0P / N, D1P / N, D2P / N, D3P / N).

[0019] The extended mode compatible CSI-2 transmitting circuit 31 and the extended mode compatible CSI-2 receiving circuit 32 support communication in an extended mode in which the CSI-2 standard is extended, and can transmit and receive signals to and from each other. The detailed configuration of the extended mode compatible CSI-2 transmitting circuit 31 and the extended mode compatible CSI-2 receiving circuit 32 will be described later with reference to FIGS. 9 and 10.

[0020] FIG. 2 is a block diagram showing a configuration example of a second embodiment of a communication system to which the present technology is applied.

[0021] As shown in FIG. 2, the communication system 11A is configured such that the image sensor 21 and the SerDes device 25 are connected via the bus 24-1, the application processor 22 and the SerDes device 26 are connected via the bus 24-2, and the SerDes device 25 and the SerDes device 26 are connected via the bus 27. For example, the communication system 11A is used for connection in an existing in-vehicle camera.

[0022] Here, the image sensor 21 and the application processor 22 are configured in the same manner as the image sensor 21 and the application processor 22 in FIG. 1, and a detailed description thereof will be omitted.

[0023] The buses 24-1 and 24-2 are communication paths that transmit signals in accordance with the CSI-2 standard, similar to the bus 23 in FIG. 1, and are configured with a plurality of signal lines (HS-GPIO, I2C / I3C, CLKP / N, D0P / N, D1P / N, D2P / N, D3P / N) as shown in the figure.

[0024] The SerDes device 25 is configured to include a CSI-2 receiving circuit 33 and a SerDes (Serializer Deserializer) transmitting circuit 34. For example, in the SerDes device 25, the CSI-2 receiving circuit 33 communicates with the extended mode-compatible CSI-2 transmitting circuit 31 in accordance with the normal CSI-2 standard to acquire the bit-parallel signal transmitted from the image sensor 21. Then, the SerDes device 25 converts the acquired signal into bit-serial and transmits the signal to the SerDes device 26 by the SerDes transmitting circuit 34 communicating with the SerDes receiving circuit 35 in one lane.

[0025] The SerDes device 26 is configured to include a SerDes receiving circuit 35 and a CSI-2 transmitting circuit 36. For example, in the SerDes device 26, the SerDes receiving circuit 35 acquires the bit-serial signal transmitted by communicating with the SerDes transmitting circuit 34 in one lane. Then, the SerDes device 26 converts the acquired signal into bit-parallel and transmits it to the application processor 22 by the CSI-2 transmitting circuit 36 communicating with the extended mode-compatible CSI-2 receiving circuit 32 in accordance with the normal CSI-2 standard.

[0026] Bus 27 is a communication path that transmits signals in accordance with standards such as A-PHY and FPD (Flat Panel Display)-LINK III. For example, the transmission distance capable of transmitting signals is about 15 m, which is a long distance.

[0027] With these physical layer interfaces capable of long-distance transmission, the automotive industry can utilize advanced driver assistance systems (ADAS), autonomous driving systems (ADS), and other surround sensor applications including cameras and in-vehicle infotainment (IVI) displays. MIPI A-PHY has an asymmetric data link layer (asymmetric upper layer) of a point-to-point topology and enables sharing the same physical wiring for high-speed data transmission, control data, and power. It functions as the basis of an end-to-end system designed to simplify the integration of cameras, sensors, and displays, and at the same time, it is also possible to incorporate functional safety and security.

[0028] The communication systems 11 and 11A configured in this way can transmit and receive data in packets with an extended packet structure by the extended mode-compatible CSI-2 transmission circuit 31 and the extended mode-compatible CSI-2 reception circuit 32, as will be described later. Thereby, it is possible to support more diverse applications, for example, RAW24, SmartROI (Region of Interest), GLD (Graceful Link Degradation), etc., as will be described later.

[0029] <First Structural Example of Packet Structure> Referring to FIGS. 3 to 8, a first structural example of the packet structure of the packet used in the communication between the extended mode-compatible CSI-2 transmission circuit 31 and the extended mode-compatible CSI-2 reception circuit 32 will be described.

[0030] FIG. 3 shows the overall packet structure of a packet (hereinafter referred to as an extended packet for D-PHY) used in the extended mode of CSI-2 when the physical layer is D-PHY.

[0031] As shown in FIG. 3, for the extended packet of D-PHY, the packet header and the packet footer have the same packet structure as the existing CSI-2 standard. For example, in the packet header, VC (Virtual Channel) indicating the number of lines of the virtual channel, DataType (Data Type) indicating the type of data, WC (Word Count) indicating the data length of the payload, and VCX / ECC are stored. Also, CRC (Cyclic Redundancy Check) is stored in the packet footer.

[0032] Here, in the existing CSI-2 standard, the data types transmitted in the packet header are defined as reserved for 0x38 to 0x3F. Therefore, in the extended packet for D-PHY, setting information for identifying the extended mode is newly defined using the data types that were previously reserved.

[0033] For example, as the data type, · When DataType[5:3] = 3’b111, the extended mode · DataType[2] = Reserve (RES: Reserved for future extension) · DataType[1:0] = extension mode type (Four extended modes are prepared) is defined.

[0034] That is, among 0x38 to 0x3F of the data types defined as reserved in the existing CSI-2 standard, for example, DataType[5:3] is defined as extended mode setting information, and DataType[1:0] is defined as extended type setting information. The extended mode setting information indicates whether it is an extended mode. For example, when DataType[5:3] is 3’b111, it indicates that it is an extended mode. Also, when four types of extended modes, namely extended mode 0, extended mode 1, extended mode 2, and extended mode 3, are prepared, the extended type setting information indicates which of these types it is. For example, when DataType[1:0] is 2’b00, it indicates that the type of the extended mode is extended mode 0.

[0035] And in extended mode 0 (DataType[1:0] = 2’b00), for example, a packet structure in which the payload is separated into four is defined. That is, the payload in extended mode 0 is separated into an extended packet header (ePH: extended Packet Header), an optional extended packet header (OePH: Optional extended Packet Header), a legacy payload, and an optional extended packet footer (OePF: Optional extended Packet Footer) as shown in FIG. 3. Note that the extended packet header may be transmitted repeatedly.

[0036] The extended packet header is arranged at the head corresponding to the payload of the existing CSI-2 standard and must be transmitted necessarily in the extended mode. For example, as shown in the figure, the extended packet header is composed of setting information such as an SROI identification flag, an extended VC (Virtual Channel), an extended DataType, an OePH selection flag, and an OePF selection flag. Here, due to the extended VC, the VC which was 4 bits in the existing CSI-2 standard is extended to 8 bits, and due to the extended DataType, the DataType which was 4 bits in the existing CSI-2 standard is extended to 8 bits.

[0037] For example, in the packet for D-PHY, since there are already 4 bits of VC in the existing packet header, by defining the extended VC of the extended packet header as 4 bits, the total can be 8 bits. Specifically, it can be defined as OePH[7:0] = {5’h00,RSID,XY_POS,MC}, OePF[3:0] = {3’h0,pCRC}, and the ON / OFF of packet transmission required for each application can be controlled.

[0038] The optional extended packet header and the optional extended packet footer are selectively transmitted according to the application.

[0039] The legacy payload corresponds to the same payload as the existing CSI-2 standard.

[0040] In this way, by setting the extended packet header, the optional extended packet header, and the optional extended packet footer as needed, data corresponding to various applications can be transmitted. Also, the data transmitted in the extended packet header, the optional extended packet header, and the optional extended packet footer is 26bit+6bit of ECC (Error Correction Code). This can reuse the circuit of the existing packet header to suppress the increase in circuit scale and improve error tolerance.

[0041] As a specific application example of such an extended packet for D-PHY, Figure 4 shows the packet structure of a short packet (hereinafter referred to as an extended short packet for D-PHY) used in the extended mode of CSI-2 when the physical layer is D-PHY. Similarly, Figure 5 shows the packet structure of a long packet (hereinafter referred to as an extended long packet for D-PHY) used in the extended mode of CSI-2 when the physical layer is D-PHY.

[0042] In the extended short packet for D-PHY as shown in FIG. 4, the extended type setting information of the data type stored in the packet header indicates that the type of the extended mode is extended mode 0 (DT[5:0]=0x1C (5’b111_0_0)). Also, the short packet setting information of the data type stored in the extended packet header indicates that it is a short packet (DT[7:0]=0x00 (Frame Start Code(Short Packet))).

[0043] In this way, when it is in the extended mode and the data type stored in the extended packet header is DT[7:0]=0x00 to 0x0F, it is an extended short packet, and data including the Short Packet Data Field of the extended short packet is always transmitted to the optional extended packet header. This Short Packet Data Field is the same as that defined in the existing CSI-2 standard.

[0044] Note that when transmitting the extended short packet, among the optional extended packet headers, MC (MessageCount for GLD) and RSID (in-vehicle line number and SourceID) may be transmitted, but since the legacy payload and pCRC are not required, transmission is prohibited. If they are transmitted erroneously, they will be ignored on the receiving side.

[0045] And the extended short packet with the packet structure as shown in FIG. 4 can extend the bit widths of the data type and the virtual channel compared with the extended short packet according to the existing CSI-2 standard, and can support various applications defined by the optional extended packet header. Also, when these functions are not required, the extended short packet according to the existing CSI-2 standard may be transmitted together with the extended long packet.

[0046] In the extended long packet for D-PHY as shown in FIG. 5, the extended type setting information of the data type stored in the packet header indicates that the type of the extended mode is extended mode 0 (DT[5:0]=0x1C (5’b111_0_0)). Also, the short packet setting information of the data type stored in the extended packet header indicates that it is other than a short packet (DT[7:0] is other than 0x00 to 0x0F (= extended LongPackt)). Therefore, in the extended long packet, data including the Short Packet Data Field is not transmitted.

[0047] Also, according to the setting of the extended packet header, the optional extended packet header, the legacy payload, and the optional extended packet footer are stored and transmitted in the payload in the existing CSI-2 standard. In this way, since it is stored and transmitted in the existing payload, the existing SerDes transmission circuit 34 and SerDes reception circuit 35 (FIG. 2) recognize it in the same way as the image data transmitted in the existing payload and transmit it to the subsequent stage as it is.

[0048] And the last-stage application processor 22 can determine that it is an extended mode based on the data type DT[5:0] of the packet header. Therefore, the application processor 22 can interpret the content of the payload in order from the extended packet header and extract the data of the desired extended mode.

[0049] FIG. 6 shows the overall packet structure of a packet (hereinafter referred to as an extended packet for C-PHY) used in the extended mode of CSI-2 when the physical layer is C-PHY. In the extended packet for C-PHY shown in FIG. 6, the description of the configuration common to the extended packet for D-PHY in FIG. 3 is omitted, and the different configurations will be described.

[0050] For example, in the extended packet for C-PHY, similar to the extended packet for D-PHY in FIG. 3, the data type is used to identify the extended mode, and all data corresponding to each application executed by the application processor 22 is embedded in the payload and transmitted.

[0051] As shown in FIG. 6, the extended packet for C-PHY, similar to the packet for C-PHY according to the existing CSI-2 standard, transmits the packet header twice, and arranges the data in 16-bit units for the convenience of C-PHY converting 16 bits into 7 symbols. Also, an extended packet header is arranged at the beginning of the payload. Regarding the virtual channel, in the case of C-PHY, since the beginning of the existing packet header was reserved for that purpose, the virtual channel is not stored in the extended packet header. Of course, similar to the extended packet for D-PHY, the virtual channel may be stored in the extended packet header.

[0052] In addition, since the optional extended packet header and the optional extended packet footer have a large number of bits, a flag called OePHF is prepared. When this flag is 1, the OePH / OePF information is transmitted next. Then, after the ePH information and the OePH information, CRC is transmitted as the extended packet header, and the packet header with the same configuration is transmitted twice repeatedly. By making the mechanism and structure in which the existing packet header is transmitted twice the same, circuit reusability and error tolerance can be achieved at the same time.

[0053] As a specific application example of such an extended packet for C-PHY, FIG. 7 shows the packet structure of a short packet (hereinafter referred to as an extended short packet for C-PHY) used in the extended mode of CSI-2 when the physical layer is C-PHY. Similarly, FIG. 8 shows the packet structure of a long packet (hereinafter referred to as an extended long packet for C-PHY) used in the extended mode of CSI-2 when the physical layer is C-PHY.

[0054] Note that the extended short packet for C-PHY shown in FIG. 7 has no significant difference in packet structure from the extended short packet for D-PHY shown in FIG. 4, and the extended long packet for C-PHY shown in FIG. 8 has no significant difference in packet structure from the extended long packet for D-PHY shown in FIG. 5.

[0055] <Configuration Example of Image Sensor and Application Processor> FIG. 9 is a block diagram showing a configuration example of an image sensor 21 including an extended mode-compatible CSI-2 transmission circuit 31.

[0056] As shown in FIG. 9, in addition to the extended mode-compatible CSI-2 transmission circuit 31, the image sensor 21 includes a pixel 41, an AD converter 42, an image processing unit 43, a pixel CRC calculation unit 44, a physical layer processing unit 45, an I2C / I3C slave 46, and a register 47. Further, the extended mode-compatible CSI-2 transmission circuit 31 includes a packing unit 51, a packet header generation unit 52, an extended packet header generation unit 53, an extended packet footer generation unit 54, selection units 55 and 56, a CRC calculation unit 57, a lane distribution unit 58, a CCI slave 59, and a controller 60.

[0057] The pixel 41 outputs an analog pixel signal corresponding to the amount of light received, and the AD converter (ADC: Analog-to-Digital Converter) 42 digitally converts the pixel signal output from the pixel 41 and supplies it to the image processing unit 43. The image processing unit (ISP: Image Signal Processor) 43 performs various image processes on the image based on the pixel signal and supplies the obtained image data to the pixel CRC calculation unit 44 and the packing unit 51. Further, the image processing unit 43 supplies a data enable signal data_en indicating whether the image data is valid to the packing unit 51 and the controller 60.

[0058] The pixel CRC calculation unit 44 calculates the CRC for each pixel in the image data supplied from the image processing unit 43 and supplies the CRC to the extended packet footer generation unit 54.

[0059] The physical layer processing unit 45 can execute physical layer processing for both C-PHY and D-PHY. For example, when the C layer enable signal cphy_en supplied from the controller 60 is valid, the physical layer processing unit 45 executes the physical layer processing of C-PHY, and when the C layer enable signal cphy_en is invalid, it executes the physical layer processing of D-PHY. Then, the physical layer processing unit 45 transmits the packet divided into 4 lanes by the lane distribution unit 58 to the application processor 22.

[0060] The I2C / I3C slave 46 communicates according to the leadership of the I2C / I3C master 72 (FIG. 10) of the application processor 22 based on the I2C (Inter-Integrated Circuit) or I3C (Improved Inter Integrated Circuits) standard.

[0061] Various settings transmitted from the application processor 22 are written into the register 47 via the I2C / I3C slave 46 and the CCI slave 59. Here, the settings written into the register 47 include, for example, communication settings according to the CSI-2 standard, an extended mode setting indicating the use or non-use of the extended mode, and fixed communication settings required for communication in the extended mode.

[0062] The packing unit 51 performs a packing process of storing the image data supplied from the image processing unit 43 in the payload of the packet, and supplies the payload to the selection unit 55 and the lane distribution unit 58.

[0063] When the generation of the packet header is instructed according to the packet header generation instruction signal ph_go supplied from the controller 60, the packet header generation unit 52 generates a packet header and supplies it to the selection unit 55 and the lane distribution unit 58.

[0064] That is, the packet header generation unit 52 generates a packet header that stores setting information indicating conditions set for data transmitted in a packet, for example, a data type indicating the type of data, in accordance with the existing CSI-2 standard. Further, the packet header generation unit 52 stores, in an unused area defined as unused in the existing CSI-2 standard in the data type, which is setting information indicating the type of data transmitted in a packet, extension mode setting information indicating whether it is an extension mode that uses an extended header. Furthermore, the packet header generation unit 52 stores, in the unused area, extension type setting information indicating which type among a plurality of types of extension modes prepared as the extension mode.

[0065] The extended packet header generation unit 53 generates an extended packet header and an optional extended packet header according to the extended packet header generation instruction signal eph_go and the extended packet header enable signal ePH_en supplied from the controller 60, and supplies them to the selection unit 56 and the lane distribution unit 58. In addition, in accordance with the use of the image sensor 21, vehicle-mounted line numbers, source IDs (identifications), etc. are supplied to the extended packet header generation unit 53, and if necessary, they are stored in the extended packet header or the optional extended packet header.

[0066] That is, the extended packet header generation unit 53 generates an extended packet header that stores setting information such as that shown in FIG. 3 separately from the packet header generated by the packet header generation unit 52. Further, when transmitting the optional extended packet header, the extended packet header generation unit 53 stores, in the extended packet header, optional extended packet header setting information (OePH[7:0]) indicating whether to transmit the optional extended packet header, and generates the optional extended packet header following the extended packet header.

[0067] The extended packet footer generation unit 54 generates an optional extended packet footer according to the extended packet footer generation instruction signal epf_go and the extended packet header enable signal ePF_en supplied from the controller 60, and supplies it to the selection unit 56 and the lane distribution unit 58.

[0068] That is, when the packet transmitted in the extended mode is an extended long packet that stores data transmitted as a payload in the existing CSI-2 standard, the extended packet footer generation unit 54 generates an optional extended packet footer that is arranged following the legacy payload in which the data is stored.

[0069] In addition, a C layer enable signal cphy_en is supplied from the controller 60 to the packet header generation unit 52, the extended packet header generation unit 53, and the extended packet footer generation unit 54. When the C layer enable signal cphy_en indicates valid, the packet header generation unit 52 generates a packet header for C-PHY, the extended packet header generation unit 53 generates an extended packet header for C-PHY and an optional extended packet header, and the extended packet footer generation unit 54 generates an optional extended packet footer for C-PHY. On the other hand, when the C layer enable signal cphy_en indicates invalid, the packet header generation unit 52 generates a packet header for D-PHY, the extended packet header generation unit 53 generates an extended packet header for D-PHY and an optional extended packet header, and the extended packet footer generation unit 54 generates an optional extended packet footer for D-PHY.

[0070] The selection unit 55 selects the packet header supplied from the packet header generation unit 52 and supplies it to the selection unit 56 when the C layer enable signal cphy_en is valid according to the C layer enable signal cphy_en supplied from the controller 60. On the other hand, the selection unit 55 selects the payload supplied from the packing unit 51 and supplies it to the selection unit 56 when the C layer enable signal cphy_en is invalid.

[0071] The selection unit 56 selectively supplies, according to the data selection signal data_sel supplied from the controller 60, either the packet header or payload selectively supplied via the selection unit 55, the extended packet header supplied from the extended packet header generation unit 53, the optional extended packet header, or the optional extended packet footer supplied from the extended packet footer generation unit 54, to the CRC calculation unit 57.

[0072] The CRC calculation unit 57 calculates the CRC of the packet header, payload, extended packet header, optional extended packet header, or optional extended packet footer selectively supplied via the selection unit 56, and supplies the calculated CRC to the lane distribution unit 58.

[0073] The lane distribution unit 58 distributes, according to the control of the controller 60, the payload supplied from the packing unit 51, the packet header supplied from the packet header generation unit 52, the extended packet header and the optional extended packet header supplied from the extended packet header generation unit 53, the optional extended packet footer supplied from the extended packet footer generation unit 54, and the CRC supplied from the CRC calculation unit 57, to four lanes according to the CSI-2 standard, and supplies them to the physical layer processing unit 45.

[0074] The CCI (Camera Control Interface) slave 59 communicates according to the leadership of the CCI master 88 (Fig. 10) of the application processor 22 based on the CSI-2 standard.

[0075] The controller 60 reads out various settings stored in the register 47, and controls each block constituting the CSI-2 transmission circuit 31 corresponding to the extended mode according to those settings. For example, the controller 60 controls the switching between the transmission of a packet having a packet structure according to the existing CSI-2 standard and the transmission of a packet having a packet structure in the extended mode according to the content of the data to be transmitted.

[0076] The image sensor 21 is configured in this way, and can generate an extended packet with the packet structure as described with reference to FIGS. 3 to 8, and transmit it to the application processor 22.

[0077] FIG. 10 is a block diagram showing a configuration example of an application processor 22 including an extended mode-compatible CSI-2 reception circuit 32.

[0078] As shown in FIG. 10, in addition to the extended mode-compatible CSI-2 reception circuit 32, the application processor 22 includes a physical layer processing unit 71, an I2C / I3C master 72, a register 73, and a controller 74. Further, the extended mode-compatible CSI-2 reception circuit 32 includes a packet header detection unit 81, a lane merging unit 82, an interpretation unit 83, selection units 84 and 85, a CRC calculation unit 86, an unpacking unit 87, and a CCI master 88.

[0079] The physical layer processing unit 71 can perform physical layer processing for both C-PHY and D-PHY. As described above, in the physical layer processing unit 45 of the image sensor 21, physical layer processing for either C-PHY or D-PHY is performed, and the physical layer processing unit 71 performs the same physical layer processing as that performed in the physical layer processing unit 45.

[0080] The I2C / I3C master 72 takes the lead in communicating with the I2C / I3C slave 46 (FIG. 9) of the image sensor 21 based on the I2C or I3C standard.

[0081] In the register 73, various settings to be written to the register 47 of the image sensor 21 are recorded by the controller 74.

[0082] The controller 74 controls each block constituting the application processor 22.

[0083] The packet header detection unit 81 detects a packet header from the packet supplied from the physical layer processing unit 71, and checks the data type stored in the packet header. Then, when the data type in the packet header indicates that the extended mode setting information is in the extended mode (DataType[5:3]=3’b111), the packet header detection unit 81 supplies an extended mode detection flag indicating the extended mode to the interpretation unit 83, the selection unit 84, and the selection unit 85. Also, the packet header detection unit 81 supplies a merge enable signal mrg_en indicating whether to enable the merging of the four lanes that are split based on the packet header to the lane merging unit 82.

[0084] That is, the packet header detection unit 81 detects a packet header in which setting information (such as data type) indicating conditions set for the data transmitted in the packet is stored according to the existing CSI-2 standard. At this time, the packet header detection unit 81 outputs an extended mode detection flag according to the extended mode setting information indicating whether it is an extended mode using an extended header stored in the unused area defined as unused in the existing CSI-2 standard in the data type which is the setting information indicating the type of data transmitted in the packet, thereby enabling switching between receiving a packet with a packet structure according to the existing CSI-2 standard and receiving a packet with a packet structure in the extended mode. Also, the packet header detection unit 81 recognizes which type of extended mode among the multiple types of extended modes prepared as the extended mode according to the extended mode type information stored in the unused area of the data type defined as unused in the existing CSI-2 standard.

[0085] When the merge enable signal mrg_en supplied from the packet header detection unit 81 is valid, the lane merging unit 82 merges the packet split into four lanes supplied from the physical layer processing unit 71. Then, the lane merging unit 82 supplies the packet of one lane to the interpretation unit 83, the selection unit 84, and the selection unit 85.

[0086] When the extended mode detection flag supplied from the packet header detection unit 81 indicates that it is in the extended mode, the interpretation unit 83 reads out an extended packet header, an optional extended packet header, and an optional extended packet footer from the packet supplied from the lane merging unit 82 based on the packet structure of the extended mode. Then, the interpretation unit 83 interprets the setting information stored in the extended packet header, the optional extended packet header, and the optional extended packet footer.

[0087] That is, the interpretation unit 83 receives an extended packet header arranged at the head of the payload according to the existing CSI-2 standard as an extended header, and interprets the setting information stored in the extended packet header. Further, when the optional extended packet header setting information stored in the extended packet header indicates that an optional extended packet header selectively transmitted according to the application is to be transmitted, the interpretation unit 83 receives the optional extended packet header following the extended packet header, and interprets the setting information stored in the optional extended packet header. Furthermore, when the packet transmitted in the extended mode is an extended long packet storing data transmitted as a payload in the existing CSI-2 standard, the interpretation unit 83 receives an optional extended packet footer arranged following the legacy payload in which the data is stored, and interprets the optional extended packet footer.

[0088] Then, the interpretation unit 83 reads out, for example, the in-vehicle line number, the source ID, etc. stored in the optional extended packet header, and outputs them to a subsequent LSI (not shown).

[0089] Note that when the extended mode detection flag supplied from the packet header detection unit 81 does not indicate that it is in the extended mode, that is, when a packet having the existing packet structure is supplied, the interpretation unit 83 stops without performing the above-described processing.

[0090] The selection unit 84 selectively supplies data to the unpacking unit 87 based on the packet structure of the existing packet or the packet structure of the extended packet according to the extended mode detection flag supplied from the packet header detection unit 81.

[0091] The selection unit 85 selectively supplies data to the CRC calculation unit 86 based on the packet structure of the existing packet or the packet structure of the extended packet according to the extended mode detection flag supplied from the packet header detection unit 81.

[0092] The CRC calculation unit 86 calculates the CRC of the packet header, payload, extended packet header, optional extended packet header, or optional extended packet footer selectively supplied via the selection unit 85. And when a CRC error is detected, the CRC calculation unit 86 outputs a crc error detection signal indicating that to the subsequent LSI (not shown).

[0093] The unpacking unit 87 performs an unpacking process to extract the image data stored in the payload selectively supplied via the selection unit 84, and outputs the acquired image data to the subsequent LSI (not shown).

[0094] The CCI master 88 takes the lead in communicating with the CCI slave 59 (Fig. 9) of the image sensor 21 based on the CSI-2 standard.

[0095] The application processor 22 is configured in this way, and can receive the extended packet transmitted from the image sensor 21, interpret the setting information stored in the extended packet header, optional extended packet header, and optional extended packet footer, and acquire the image data.

[0096] <Communication Processing> With reference to Figs. 11 to 14, the communication processing performed by the image sensor 21 and the application processor 22 will be described.

[0097] FIG. 11 is a flowchart for explaining the process in which the image sensor 21 transmits a packet.

[0098] For example, when the image sensor 21 is connected to the application processor 22 via the bus 23, the process starts. In step S11, the controller 60 determines whether to use the extended mode when starting communication with the application processor 22. For example, the controller 60 checks the extended mode setting stored in the register 47, and if the extended mode setting indicating the use of the extended mode is written by the application processor 22, it determines to use the extended mode.

[0099] If the controller 60 determines in step S11 not to use the extended mode, the process proceeds to step S12.

[0100] In step S12, the I2C / I3C slave 46 receives a transmission start command for image data transmitted from the application processor 22 (in step S54 of FIG. 13 described later). Further, the I2C / I3C slave 46 receives the communication settings conforming to the CSI-2 standard transmitted together with the transmission start command and writes them to the register 47 via the CCI slave 59.

[0101] In step S13, in the image sensor 21, based on the communication settings stored in the register 47, a conventional packet transmission process of transmitting a packet having a packet structure conforming to the existing CSI-2 standard to the application processor 22 is executed.

[0102] On the other hand, if the controller 60 determines in step S11 to use the extended mode, the process proceeds to step S14.

[0103] In step S14, the I2C / I3C slave 46 receives fixed communication settings (such as copy for each PH / PF lane during GLD) required for communication in the extended mode, and writes them to the register 47 via the CCI slave 59.

[0104] In step S15, the I2C / I3C slave 46 receives a transmission start command for image data transmitted from the application processor 22 (at step S57 in FIG. 13 described later). Further, the I2C / I3C slave 46 receives communication settings conforming to the CSI-2 standard transmitted together with the transmission start command, and writes them to the register 47 via the CCI slave 59.

[0105] In step S16, the controller 60 determines whether to start packet transmission, and waits for processing until it is determined to start packet transmission.

[0106] Then, in step S16, if it is determined to start packet transmission, the process proceeds to step S17, and the controller 60 determines whether the data to be transmitted is data to be transmitted in the extended mode. Here, the controller 60 determines that the data is data to be transmitted in the extended mode when, for example, in the use case of an application example as described later, the data is data to be transmitted.

[0107] In step S17, if the controller 60 determines that the data is data to be transmitted in the extended mode, the process proceeds to step S18, and extended mode transmission processing (see FIG. 12) for transmitting an extended packet corresponding to the extended mode is performed.

[0108] On the other hand, in step S17, if the controller 60 determines that the data is not data to be transmitted in the extended mode, the process proceeds to step S19.

[0109] In step S19, the controller 60 determines whether to transmit a short packet. For example, the controller 60 determines to transmit a short packet at the start and end of a frame.

[0110] If, in step S19, the controller 60 determines to transmit a short packet, the process proceeds to step S20. In step S20, the packet header generation unit 52 generates a packet header and transmits a short packet with a conventional packet structure to the application processor 22.

[0111] On the other hand, if, in step S19, the controller 60 determines not to transmit a short packet (i.e., to transmit a long packet), the process proceeds to step S21. In step S21, the packing unit 51 stores the image data in the payload, and the CRC calculation unit 57 obtains the CRC, thereby generating a long packet with a conventional packet structure and transmitting it to the application processor 22.

[0112] After the processing of step S18, step S20, or step S21, the process proceeds to step S22, and the controller 60 ends the packet transmission process. Thereafter, the process returns to step S16, and hereinafter, the process of transmitting a packet in the same manner is repeatedly performed for the next packet.

[0113] FIG. 12 is a flowchart for explaining the extended mode transmission process performed in the process of step S18 in FIG. 11.

[0114] In step S31, the packet header generation unit 52 generates a packet header storing VC, data type, WC, etc., and transmits it to the application processor 22. At this time, the packet header generation unit 52 writes extension mode setting information indicating that it is in the extension mode (DataType[5:3]=3’b111) and extension type setting information identifying that the mode setting of the extension mode is extension mode 0 (DataType[1:0] =2’b00) into the data type of the packet header.

[0115] In step S32, the application processor 22 determines whether to transmit an extended short packet. For example, the controller 60 determines to transmit an extended short packet at the start and end of a frame.

[0116] In step S32, if the application processor 22 determines to transmit an extended short packet, the process proceeds to step S33.

[0117] In step S33, the extended packet header generation unit 53 transmits an extended packet header with the data type (DataType[7:0]) set to a short packet in the first byte of the payload. At this time, the extended packet header generation unit 53 performs various settings (such as OePH[7:0] and OePF[3:0], etc.) stored in the extended packet header.

[0118] In step S34, the extended packet header generation unit 53 stores and transmits the frame number (FN:FrameNumber) in the second byte of the payload.

[0119] In step S35, the extended packet header generation unit 53 generates and transmits an optional extended packet header as shown in FIG. 4 according to the setting (OePH[7:0]) performed in step S33.

[0120] In step S36, the CRC calculation unit 57 obtains the CRC and transmits it as a packet footer.

[0121] On the other hand, in step S32, when the application processor 22 determines not to transmit an extended short packet (i.e., to transmit a long packet), the process proceeds to step S37.

[0122] In step S37, the extended packet header generation unit 53 transmits an extended packet header with the data type (DataType[7:0]) set to other than a short packet at the first byte of the payload. At this time, the extended packet header generation unit 53 performs various settings (for example, OePH[7:0], OePF[3:0], etc.) to be stored in the extended packet header.

[0123] In step S38, the extended packet header generation unit 53 generates and transmits an optional extended packet header as shown in FIG. 5 according to the setting (OePH[7:0]) performed in step S37.

[0124] In step S39, the packing unit 51 packs the image data supplied from the image processing unit 43 and generates and transmits a legacy payload.

[0125] In step S40, the extended packet footer generation unit 54 generates and transmits an optional extended packet footer as shown in FIG. 4 according to the setting (OePF[3:0]) performed in step S37.

[0126] In step S41, the CRC calculation unit 57 obtains the CRC and transmits it as a packet footer.

[0127] Then, after the processing of step S36 or S41, the extended mode transmission process ends.

[0128] As described above, the image sensor 21 can generate and transmit an extended short packet or an extended long packet.

[0129] FIG. 13 is a flowchart for explaining the process in which the application processor 22 receives a packet.

[0130] For example, when the image sensor 21 is connected to the application processor 22 via the bus 23, the process starts. In step S51, the controller 74 writes the initial settings of the image sensor 21 (for example, which of C-PHY and D-PHY is used as the physical layer) to the register 73 and transmits it to the image sensor 21 through the I2C / I3C master 72 via the CCI master 88. As a result, the initial settings are written to the register 47 of the image sensor 21.

[0131] In step S52, the controller 74 recognizes whether the image sensor 21 supports the extended mode. For example, the controller 74 can recognize whether the image sensor 21 supports the extended mode by acquiring the set value (for example, extended PH / PF support capability) stored in the register 47 of the image sensor 21 by the I2C / I3C master 72. Alternatively, the controller 74 can recognize in advance whether the image sensor 21 supports the extended mode based on an input such as a manual.

[0132] In step S53, the controller 74 determines whether the image sensor 21 supports the extended mode and whether the use of the extended mode is required by the application executed by the application processor 22.

[0133] In step S53, if the controller 74 determines that the image sensor 21 does not support the extended mode or the use of the extended mode is not required, the process proceeds to step S54.

[0134] In step S54, the controller 74 causes the I2C / I3C master 72 to send an image data transmission start command to the image sensor 21. At this time, the controller 74 also causes the communication settings according to the CSI-2 standard to be transmitted.

[0135] In step S55, in the application processor 22, based on the communication settings transmitted in step S54, a conventional packet reception process is performed to receive a packet with a packet structure according to the existing CSI-2 standard.

[0136] On the other hand, in step S53, when the controller 74 determines that the image sensor 21 supports the extended mode and the use of the extended mode is required by the application executed by the application processor 22, the process proceeds to step S56.

[0137] In step S56, before the communication in the extended mode is started, the I2C / I3C master 72 transmits the fixed communication settings required for the communication in the extended mode. As a result, the fixed communication settings are written into the register 47 of the image sensor 21 (step S14 in FIG. 11).

[0138] In step S57, the controller 74 causes the I2C / I3C master 72 to send an image data transmission start command to the image sensor 21. At this time, the controller 74 also causes the communication settings according to the CSI-2 standard to be transmitted.

[0139] In step S58, the packet header detection unit 81 determines whether the reception of the packet has started by checking the data supplied from the physical layer processing unit 71, and waits for the process until it is determined that the reception of the packet has started. For example, when the packet header detection unit 81 detects a packet header from the data supplied from the physical layer processing unit 71, it determines that the reception of the packet has started.

[0140] In step S58, when the packet header detection unit 81 determines that the reception of a packet has started, the process proceeds to step S59.

[0141] In step S59, the packet header detection unit 81 checks the data type of the packet header detected in step S58 and determines whether the packet for which reception has started is an extended packet corresponding to the extended mode. For example, when the extended mode setting information indicates the extended mode in the data type of the packet header (DataType[5:3]=3’b111), the packet header detection unit 81 determines that the packet for which reception has started is an extended packet.

[0142] In step S59, when the packet header detection unit 81 determines that the packet for which reception has started is an extended packet, the process proceeds to step S60, and extended mode reception processing for receiving the extended packet (see FIG. 14) is performed.

[0143] On the other hand, in step S59, when the packet header detection unit 81 determines that the packet for which reception has started is not an extended packet, the process proceeds to step S61.

[0144] In step S61, the packet header detection unit 81 checks the data type (DataType[5:0]) of the packet header detected in step S58 and determines whether the packet for which reception has started is a short packet.

[0145] In step S61, when the packet header detection unit 81 determines that the packet for which reception has started is a short packet, the process proceeds to step S62. In step S62, the packet header detection unit 81 receives a short packet having a conventional packet structure transmitted from the image sensor 21.

[0146] On the one hand, in step S61, when the packet header detection unit 81 determines that the received packet is not a short packet (i.e., the reception of a long packet has started), the process proceeds to step S63. In step S63, the unpacking unit 87 receives the payload of the long packet with the conventional packet structure transmitted from the image sensor 21 and extracts the image data, and the CRC calculation unit 86 receives the (WC + 1)-th byte transmitted following the packet header as the CRC.

[0147] After the processing of step S60, step S62, or step S63, the process proceeds to step S64, and the controller 74 ends the packet reception process. Then, the process returns to step S58, and hereinafter, the process of receiving packets in the same manner for the next packet is repeatedly performed.

[0148] FIG. 14 is a flowchart for explaining the extended mode reception process performed in the process of step S60 in FIG. 13.

[0149] In step S71, the packet header detection unit 81 determines whether the mode setting of the extended mode is extended mode 0. For example, when the extended type setting information indicates that the extended type setting information is extended mode 0 in the data type of the packet header (DataType[1:0] = 2'b00), the packet header detection unit 81 determines that the mode setting of the extended mode is extended mode 0.

[0150] In step S71, when the packet header detection unit 81 determines that the mode setting of the extended mode is extended mode 0, the process proceeds to step S72. In step S72, the interpretation unit 83 receives the first byte of the payload as the extended packet header.

[0151] In step S73, the interpretation unit 83 checks the data type (DataType[7:0]) of the extended packet header received in step S72 and determines whether the received packet is an extended short packet.

[0152] In step S73, when the interpretation unit 83 determines that it is an extended short packet, the process proceeds to step S74. In step S74, the interpretation unit 83 receives an optional extended packet header according to the setting (OePH[7:0]) stored in the extended packet header received in step S72.

[0153] In step S75, the CRC calculation unit 86 receives the WC+1 byte following the optional extended packet header as the CRC.

[0154] On the other hand, in step S73, when the interpretation unit 83 determines that it is not an extended short packet (i.e., the reception of an extended long packet has started), the process proceeds to step S76. In step S76, the interpretation unit 83 receives an optional extended packet header according to the setting (OePH[7:0]) stored in the extended packet header received in step S72.

[0155] In step S77, the unpacking unit 87 receives the legacy payload of the extended long packet transmitted from the image sensor 21 and extracts the image data.

[0156] In step S78, the interpretation unit 83 receives an optional extended packet footer according to the setting (OePF[3:0]) stored in the extended packet header received in step S72.

[0157] In step S79, the CRC calculation unit 86 receives the WC+1 byte following the optional extended packet footer as the CRC.

[0158] And when it is determined in step S71 that the mode setting in the extended mode is not extended mode 0, after the process of step S75 or after the process of step S79, the extended mode reception process ends.

[0159] As described above, the application processor 22 can receive an extended short packet or an extended long packet to obtain data.

[0160] <Second Structural Example of Packet Structure> Referring to FIGS. 15 to 18, a second structural example of the packet structure used in the communication between the extended mode-compatible CSI-2 transmission circuit 31 and the extended mode-compatible CSI-2 reception circuit 32 will be described.

[0161] In the first structural example shown in FIGS. 3 to 8 described above, emphasis was placed on maintaining compatibility with the existing CSI-2 standard, and the packet header and packet footer had the same packet structure as the existing CSI-2 standard, and the packet structure was extended by the extended packet header, the optional extended packet header, and the optional extended packet footer. On the other hand, in the second structural example described below, the packet header and packet footer are different from the existing CSI-2 standard, and the packet structure is extended by the extended packet header and the extended packet footer.

[0162] FIG. 15 shows the packet structure of a short packet (hereinafter, an extended short packet for D-PHY) used in the extended mode of CSI-2 when the physical layer is D-PHY.

[0163] Similar to the extended short packet for D-PHY in the first structural example shown in FIG. 4, the extended mode of the extended short packet for D-PHY shown in FIG. 15 is identified by the data type stored in the packet header that is the same as the existing CSI-2 standard.

[0164] On the other hand, in the extended short packet for D-PHY shown in FIG. 15, a frame number is stored in the short packet data field in the next 16 bits following the data type of the packet header, similar to the short packet according to the existing CSI-2 standard. Then, following the packet header, an extended packet header configured in the same manner as the extended packet header shown in FIG. 4 is transmitted.

[0165] Therefore, the receiving application processor 22 can interpret the data type stored in the extended packet header and determine that the frame number is stored in the data field of the packet header when it is an extended short packet.

[0166] Note that the optional extended packet header in the extended short packet for D-PHY shown in FIG. 15 is configured in the same manner as the optional extended packet header in the extended short packet for D-PHY in the first structural example shown in FIG. 4. However, since the optional extended packet header has a packet structure that is not embedded in the payload, it is not necessary to append a CRC at the end.

[0167] FIG. 16 shows the packet structure of a long packet (hereinafter, the extended long packet for D-PHY) used in the extended mode of CSI-2 when the physical layer is D-PHY.

[0168] In the extended long packet for D-PHY shown in FIG. 16, the extended data is not embedded in the payload but is transmitted as part of the packet header or packet footer. Therefore, the WC of the first packet header indicates only the byte length of the payload, as in the existing standard.

[0169] FIG. 17 shows the packet structure of a short packet (hereinafter, the extended short packet for C-PHY) used in the extended mode of CSI-2 when the physical layer is C-PHY.

[0170] In the extended part of the extended short packet for C-PHY shown in FIG. 17, since it is transmitted only as an extension of the packet header according to the existing CSI-2 standard, an extended part such as an extended packet header is inserted after the frame number. And, similar to the existing CSI-2 standard, the packet header ends with a CRC. Further, the packet structure that transmits these twice with SYNC in between is the same as the short packet according to the existing CSI-2 standard.

[0171] FIG. 18 shows the packet structure of a long packet (hereinafter, extended long packet for C-PHY) used in the extended mode of CSI-2 when the physical layer is C-PHY.

[0172] As described above, the WC of the packet header at the head of the extended long packet for C-PHY shown in FIG. 18 is the same as the existing standard in that it only indicates the byte length of the payload, and there is a difference from the extended long packet for C-PHY of the first structural example shown in FIG. 8.

[0173] As described above, due to the packet structure of the extended packet of the second structural example shown in FIGS. 15 to 18, similar to the packet structure of the extended packet of the first structural example (FIGS. 3 to 8), it becomes possible to support more diverse applications than before.

[0174] However, the extended packet of the second structural example has a packet structure in which the existing packet header and footer are extended without embedding extended data in the existing payload. Therefore, when adopting the packet structure of the extended packet of the second structural example, compared with the case of adopting the packet structure of the extended packet of the first structural example, it is impossible to minimize the influence that requires a change from the communication system that has been conventionally used. That is, for example, the existing SerDes transmission circuit 34 needs to be changed for the SerDes reception circuit 35 (FIG. 2).

[0175] As described above, by adopting the extended packet of the first structural example, it is possible to cope with various applications such as in-vehicle use, and an in-vehicle system can be constructed while minimizing the impact that requires changes from the conventionally used communication system.

[0176] Also, by adopting the extended packet of the second structural example, although changes are required from the conventionally used communication system, it is possible to cope with various applications such as in-vehicle use.

[0177] <Modification Examples of Image Sensor and Application Processor> With reference to FIG. 19, modification examples of the image sensor and the application processor will be described.

[0178] Each block constituting the image sensor 21 in FIG. 9 and the application processor 22 in FIG. 10 described above was configured to be able to perform processing corresponding to both D-PHY and C-PHY packets. On the other hand, for example, it may be provided with both a block that exclusively processes D-PHY packets and a block that exclusively processes C-PHY packets, and the processing may be switched between them.

[0179] The image sensor 21A shown in A of FIG. 19 includes a D-layer processing block section 101, a C-layer processing block section 102, a switching section 103, and a controller 60.

[0180] The D-layer processing block section 101 has a block that exclusively processes packets for D-PHY among the blocks constituting the image sensor 21 in FIG. 9. The C-layer processing block section 102 has a block that exclusively processes packets for C-PHY among the blocks constituting the image sensor 21 in FIG. 9. The switching section 103 outputs the D-PHY packets generated in the D-layer processing block section 101 when D-PHY is used in the physical layer, and outputs the C-PHY packets generated in the C-layer processing block section 102 when C-PHY is used in the physical layer, according to the control by the controller 60.

[0181] The application processor 22A shown in B of FIG. 19 includes a switching section 111, a D-layer processing block section 112, a C-layer processing block section 113, and a controller 74.

[0182] The switching section 111 switches to supply the packets transmitted from the image sensor 21A to either the D-layer processing block section 112 or the C-layer processing block section 113 according to the control by the controller 74. The D-layer processing block section 112 has a block that exclusively processes packets for D-PHY among the blocks constituting the application processor 22 in FIG. 10. The C-layer processing block section 113 has a block that exclusively processes packets for C-PHY among the blocks constituting the application processor 22 in FIG. 10.

[0183] In the image sensor 21A and the application processor 22A configured as described above, before starting communication, the physical layer to be used can be set between the controller 60 and the controller 74. For example, when D-PHY is used for the physical layer, the packet for D-PHY generated in the D layer processing block unit 101 is transmitted via the switching unit 103, and is supplied to the D layer processing block unit 112 via the switching unit 111 for processing. Also, for example, when C-PHY is used for the physical layer, the packet for C-PHY generated in the C layer processing block unit 102 is transmitted via the switching unit 103, and is supplied to the C layer processing block unit 113 via the switching unit 111 for processing.

[0184] <Examples of application of extended packets> The extended packets described above are being considered for application to, for example, the following use cases.

[0185] For example, it is considered to apply the extended packet to a use case where a higher-definition image (RAW24) is transmitted.

[0186] For example, when transmitting image data in RAW format, RAW6, RAW7, RAW8, RAW10, RAW12, RAW14, RAW16, and RAW20 are defined as the data types stored in the packet header according to the existing CSI-2 standard. On the other hand, in recent years, in order to support autonomous driving using in-vehicle cameras, transmission of higher-definition images has been expected. Therefore, by applying the extended packet to expand the number of bits of the data type, for example, it becomes possible to define a higher-definition RAW24 in the data type of the extended packet header.

[0187] Also, it is considered to apply the extended packet to SmartROI, which is a technology for transmitting only the image area of interest on the screen.

[0188] For example, currently, a large number of cameras are installed in stadiums, airports, etc. When the entire images captured by these cameras are transmitted from the cameras to a cloud server via a network such as the Internet, it is assumed that problems such as insufficient Internet bandwidth, increased computational load or data volume on the cloud side may occur. Therefore, it is expected to cut out only the target image area at the edge (camera side) and transmit the target image area to suppress insufficient Internet bandwidth, increased computational load or data volume on the cloud side, etc.

[0189] When transmitting such SROI, in order to tell the receiving side where the target image area corresponds to on the entire screen, it is necessary to transmit the coordinates of the upper left corner of the rectangular area (ROI) together. Also, it is necessary to send the data of the entire imaging screen at a predetermined timing according to an instruction from the receiving side. Therefore, for example, SROI images and data of the entire image (existing packet headers) are mixed in units of frames.

[0190] Therefore, by applying an extended packet, it becomes possible to transmit coordinate data with 16 bits or more for each of the X coordinate and the Y coordinate, for example.

[0191] Furthermore, the use case of applying the extended packet to GLD, which continues communication by reducing the bandwidth and the number of lanes even when the channel deteriorates, is considered. Note that GLD is a proposal being considered in CSI-2 ver3.0.

[0192] For example, in the case of autonomous driving, even if a part of the cable connecting the camera during a collision is disconnected, it is required to continue communication using the non-disconnected cable and automatically stop the vehicle after evacuating to the safety belt. Therefore, an in-vehicle camera interface must at least have a disconnection detection function, and information such as the line number (16 bits) indicating which line of information on the screen, the SourceID (8 bits) indicating from which camera the data was sent, and the message counter (16 bits) indicating the transmission number is required. Furthermore, when used in combination with the SROI as described above, it is conceivable that this information is transmitted in frame units.

[0193] Therefore, by applying an extended packet, it becomes possible to transmit this information.

[0194] <The first configuration example adapted to E2E protection> With reference to FIGS. 20 to 26, a configuration example adapted to the regulation prohibiting packet modification and the like on the transmission path will be described.

[0195] For example, in the communication system 11A having the configuration described with reference to FIG. 2 above, when the interfaces between the image sensor 21 and the application processor 22 are different, it is necessary to convert the packets on the transmission path. That is, in the case where the physical layer of the image sensor 21 is D-PHY and the physical layer of the application processor 22 is C-PHY, for example, it is necessary to convert the packets from D-PHY to C-PHY in the SerDes device 26.

[0196] In a configuration where packet conversion is performed in the SerDes device 26 in this way, for example, it violates the regulations defined in ISO26262 (Functional Safety), that is, the regulations prohibiting packet modification and the like on the transmission path (hereinafter referred to as E2E (End-to-End) protection).

[0197] FIG. 20 is a block diagram showing a configuration example of a communication system 201 adapted to E2E protection as a third embodiment of the present technology.

[0198] As shown in FIG. 20, the communication system 201 is configured by connecting an image sensor 211, a SerDes device 212, a SerDes device 213, and an application processor 214. Note that FIG. 20 describes the case where the SERDES is A-PHY as an example, but it also includes the case where it is connected using other SERDES standards such as FPD-LINK3. In addition, in the SERDES standard, communication may be performed based on the SERDES standard while maintaining the CIS-2 format (at least the Application Specific payload). Also, in the SERDES, the physical layer processing units 237 and 247 may include a plurality of physical layer processing units of other SERDES standards in addition to A-PHY, and the physical layer processing unit can be switched according to the application.

[0199] The image sensor 211 has at least an extended mode-compatible CSI-2 transmission circuit 221, a physical layer processing unit (hereinafter referred to as a C / D-PHY physical layer processing unit) 222 corresponding to C-PHY or D-PHY, or both, a slave (hereinafter referred to as an I2C / I3C slave) 223 corresponding to I2C or I3C, or both, and a CCI slave 224.

[0200] The SerDes device 212 has at least a CSI-2 reception circuit 231, a C / D-PHY physical layer processing unit 232, an I2C / I3C master 233, a CCI master 234, a CSI-2 A-PHY packet generation unit 235, a CCI A-PHY packet transmission / reception unit 236, and a physical layer processing unit 237 corresponding to A-PHY. For example, in the SerDes device 212, a packet for C-PHY or D-PHY is converted into a packet for A-PHY, and this conversion is determined based on register settings and the like.

[0201] The SerDes device 213 includes at least a CSI-2 transmission circuit 241, a C / D-PHY physical layer processing unit 242, an I2C / I3C slave 243, a CCI slave 244, a CSI-2 A-PHY packet receiving unit 245, a CCI A-PHY packet transceiver 246, and a physical layer processing unit 247 corresponding to A-PHY. For example, in the SerDes device 213, a packet for A-PHY is converted into a packet for C-PHY or D-PHY, and this conversion is determined based on register settings or the like.

[0202] The application processor 214 includes at least an extended mode-compatible CSI-2 receiving circuit 251, a C / D-PHY physical layer processing unit 252, an I2C / I3C master 253, and a CCI master 254.

[0203] In this way, the communication system 201 is configured, and an extended packet having the structure as described above is transmitted from the image sensor 211 and received by the application processor 214. Here, even if the communication system 201 is configured such that the physical layer processing unit 222 of the image sensor 211 corresponds to D-PHY and the physical layer processing unit 252 of the application processor 22 corresponds to C-PHY, it is necessary to ensure that it does not violate E2E protection.

[0204] Therefore, in order for the communication system 201 to be adaptable to E2E protection, the protection range of E2E protection is limited to an Application Specific payload (hereinafter referred to as AS payload), which is a payload specific to the application. That is, it is prohibited to modify the AS payload during the conversion from a packet for A-PHY to a packet for C-PHY or D-PHY, or during the conversion from a packet for C-PHY or D-PHY to a packet for A-PHY.

[0205] FIG. 21 shows an example of the structure of an extended packet for D-PHY extended to support E2E protection.

[0206] As shown, the extended packet for D-PHY has an AS payload consisting of an extended packet header (ePH), packet data, and an extended packet footer (ePF), which is limited as the protection range of E2E protection.

[0207] And, the extended packet header describes predetermined information required when the protection range of E2E protection is limited to the AS payload. For example, as the predetermined information described in the extended packet header, a packet count PC (Packet Count) indicating the data length of the data stored in the AS payload is added to enable identification of the data length of the packet data. That is, the packet data has the number of bytes determined by the packet count PC. Also, as the predetermined information described in the extended packet header, a virtual channel VC (Virtual Channel) indicating the number of lines of the virtual channel is copied to the existing packet header.

[0208] FIG. 22 shows an example of the structure of an extended packet for C-PHY extended to support E2E protection.

[0209] As shown, the extended packet for C-PHY, similar to the extended packet for D-PHY, has an AS payload consisting of an extended packet header (ePH), packet data, and an extended packet footer (ePF), which is limited as the protection range of E2E protection. And, in the extended packet header, similar to the extended packet for D-PHY, a packet count PC and a virtual channel VC are described as the predetermined information required when the protection range of E2E protection is limited to the AS payload.

[0210] FIG. 23 shows an example of the structure of an extended packet for A-PHY extended to support E2E protection.

[0211] As shown in the figure, in the extended packet for A-PHY as well, the AS payload consisting of an extended packet header (ePH), packet data, and an extended packet footer (ePF) is limited as the protection scope of E2E protection.

[0212] Here, as described with reference to FIG. 20, in the communication system 201, an extended packet for A-PHY is generated from an extended packet for D-PHY or C-PHY transmitted from the image sensor 211 to the SerDes device 212. Therefore, in the extended packet header of the extended packet for A-PHY, the packet count PC and the virtual channel VC are already described.

[0213] By adopting such a packet structure, the communication system 201 can avoid the AS payload being modified on the transmission path and comply with E2E protection. Note that the packet structures shown in FIGS. 21 to 23 can be partially replaced with the corresponding packets of the packet structures shown in FIGS. 3 to 8 and FIGS. 15 to 18 and used, and a part of the packet generation is replaced.

[0214] FIG. 24 is a flowchart for explaining packet transmission / reception processing adapted to E2E protection.

[0215] For example, when data to be stored in the packet data (e.g., image data, etc.) is supplied to the extended mode-compatible CSI-2 transmission circuit 221, the processing starts. Then, in step S101, in the image sensor 211, the extended mode-compatible CSI-2 transmission circuit 221 stores the supplied data in the packet data. Further, the extended mode-compatible CSI-2 transmission circuit 221 generates an extended packet header describing the virtual channel VC and the packet count PC as shown in FIG. 21 or FIG. 22 described above. Then, the extended mode-compatible CSI-2 transmission circuit 221 generates an AS payload by adding the extended packet header to the packet data and adding an extended packet footer.

[0216] In step S102, the extended mode - compliant CSI - 2 transmission circuit 221 generates an extended packet for C - PHY or D - PHY by adding a packet header for C - PHY or D - PHY and a packet footer for C - PHY or D - PHY to the AS payload generated in step S101. Then, the extended mode - compliant CSI - 2 transmission circuit 221 transmits the extended packet for C - PHY or D - PHY to the SerDes device 212 via the C / D - PHY physical layer processing unit 222.

[0217] In step S103, in the SerDes device 212, the CSI - 2 reception circuit 231 receives the extended packet for C - PHY or D - PHY transmitted from the image sensor 211 in step S102 via the C / D - PHY physical layer processing unit 232. Then, the CSI - 2 reception circuit 231 obtains the AS payload excluding the packet header and the packet footer from the received extended packet, and supplies the AS payload as it is to the CSI - 2 A - PHY packet generation unit 235.

[0218] In step S104, in the SerDes device 212, the CSI - 2 A - PHY packet generation unit 235 generates an extended packet for A - PHY by adding a packet header for A - PHY and a packet footer for A - PHY to the AS payload supplied from the CSI - 2 reception circuit 231. Then, the CSI - 2 A - PHY packet generation unit 235 transmits the extended packet for A - PHY to the SerDes device 213 via the physical layer processing unit 237 corresponding to A - PHY.

[0219] In step S105, in the SerDes device 213, the A-PHY packet receiver 245 for CSI-2 receives, via the physical layer processing unit 247 corresponding to A-PHY, the extended packet for A-PHY transmitted from the SerDes device 212 in step S104. Then, the A-PHY packet receiver 245 for CSI-2 obtains the AS payload excluding the packet header and the packet footer from the received extended packet, and supplies the AS payload as it is to the CSI-2 transmission circuit 241.

[0220] In step S106, the CSI-2 transmission circuit 241 generates an extended packet for C-PHY or D-PHY by adding a packet header for C-PHY or D-PHY and a packet footer for C-PHY or D-PHY to the AS payload supplied from the A-PHY packet receiver 245 for CSI-2 in step S105. Then, the CSI-2 transmission circuit 241 transmits the extended packet for C-PHY or D-PHY to the application processor 214 via the C / D-PHY physical layer processing unit 242.

[0221] In step S107, in the application processor 214, the CSI-2 receiver 251 compatible with the extended mode receives, via the C / D-PHY physical layer processing unit 252, the extended packet for C-PHY or D-PHY transmitted from the SerDes device 213 in step S106. Then, the CSI-2 receiver 251 compatible with the extended mode obtains the AS payload excluding the packet header and the packet footer from the received extended packet, and outputs various data stored in the packet data of the AS payload to a subsequent LSI (not shown). Thereafter, the packet transmission / reception process adapted to E2E protection is terminated, and the same process is repeatedly performed for the next extended packet.

[0222] As described above, by executing packet transmission and reception processing adapted to E2E protection, the communication system 201 can transmit and receive extended packets without modifying the AS payload on the transmission path. At this time, for example, even when the physical layer of the image sensor 211 is D-PHY and the physical layer of the application processor 214 is C-PHY, that is, even when the respective interfaces are different, E2E protection can be complied with.

[0223] FIG. 25 is a block diagram showing a detailed configuration example of the image sensor 211. In the image sensor 211 shown in FIG. 25, the same reference numerals are given to the configurations common to the image sensor 21 in FIG. 9, and detailed descriptions thereof are omitted.

[0224] That is, the image sensor 211 is configured to include a pixel 41, an AD converter 42, an image processing unit 43, a register 47, and a controller 60, similar to the image sensor 21 in FIG. 9. Further, the I2C / I3C slave 223 and the CCI slave 224 included in the image sensor 211 respectively correspond to the I2C / I3C slave 46 and the CCI slave 59 in FIG. 9.

[0225] The image sensor 211 includes a CSI-2 transmission circuit 221 compatible with the extended mode and a physical layer processing unit 222, and the physical layer processing unit 222 is compatible with A-PHY, C-PHY, and D-PHY.

[0226] The CSI-2 transmission circuit 221 compatible with the extended mode includes, in addition to the controller 60 and the CCI slave 224, an AS payload generation unit 301, a selector 302, an A-PHY packet generation unit 303, a C-PHY packet generation unit 304, a D-PHY packet generation unit 305, and a selector 306.

[0227] The AS payload generation unit 301 generates an AS payload limited as the protection range of E2E protection and outputs it to the selector 302. For example, the AS payload generation unit 301 includes a packing unit 311, an extended packet header generation unit 312, and an extended packet footer generation unit 313.

[0228] The packing unit 311 packs the image data supplied from the image processing unit 43 as the data to be transmitted and generates packet data with the number of bytes determined by the packet count PC. For example, the controller 60 can control the number of bytes of the packet data generated by the packing unit 311 according to the set value (e.g., image size, etc.) stored in the register 47.

[0229] The extended packet header generation unit 312 generates an extended packet header describing the packet count PC and the virtual channel VC, for example, as described with reference to FIGS. 21 to 23, and adds it to the packet data. The extended packet footer generation unit 313 generates an extended packet footer and adds it to the packet data.

[0230] The selector 302 selects one of the A-PHY packet generation unit 303, the C-PHY packet generation unit 304, and the D-PHY packet generation unit 305 provided in parallel as the output destination of the AS payload supplied from the AS payload generation unit 301 according to the control of the controller 60.

[0231] The A-PHY packet generation unit 303 generates an extended packet for A-PHY from the AS payload supplied via the selector 302 and outputs it to the selector 306. For example, the A-PHY packet generation unit 303 includes an AAL generation unit 321, an A-PHY packet header generation unit 322, and an A-PHY packet footer generation unit 323.

[0232] For example, the AAL (A-PHY Adaptation Layer) generation unit 321 divides the AS payload generated by the AS payload generation unit 301 into 380-byte segments at a layer called the Adaptation Layer. Then, for the divided AS payload, the A-PHY packet header generation unit 322 adds an A-PHY packet header, and the A-PHY packet footer generation unit 323 adds an A-PHY packet footer.

[0233] The C-PHY packet generation unit 304 generates an extended packet for C-PHY from the AS payload supplied via the selector 302 and outputs it to the selector 306. For example, the C-PHY packet generation unit 304 includes a C-PHY packet header generation unit 331, a C-PHY packet footer generation unit 332, and a C-PHY lane distribution unit 333.

[0234] For example, for the AS payload generated by the AS payload generation unit 301, the C-PHY packet header generation unit 331 adds a C-PHY packet header, and the C-PHY packet footer generation unit 332 adds a C-PHY packet footer. Then, the C-PHY lane distribution unit 333 distributes the C-PHY extended packet to three lanes according to the CSI-2 standard.

[0235] The D-PHY packet generation unit 305 generates an extended packet for D-PHY from the AS payload supplied via the selector 302 and outputs it to the selector 306. For example, the D-PHY packet generation unit 305 includes a D-PHY packet header generation unit 341, a D-PHY packet footer generation unit 342, and a D-PHY lane distribution unit 343.

[0236] For example, for the AS payload generated by the AS payload generation unit 301, the D-PHY packet header generation unit 341 adds a packet header for D-PHY, and the D-PHY packet footer generation unit 342 adds a packet footer for D-PHY. Then, the D-PHY lane allocation unit 343 allocates the D-PHY extended packet to four lanes according to the CSI-2 standard.

[0237] The selector 306 selects, according to the control of the controller 60, one of the A-PHY packet generation unit 303, the C-PHY packet generation unit 304, and the D-PHY packet generation unit 305 provided in parallel as the output source of the extended packet supplied to the physical layer processing unit 222.

[0238] Then, when an extended packet for A-PHY is supplied from the A-PHY packet generation unit 303, the physical layer processing unit 222 transmits the extended packet for A-PHY on one lane. Also, when an extended packet for C-PHY is supplied from the C-PHY packet generation unit 304, the physical layer processing unit 222 transmits the extended packet for C-PHY on three lanes. Also, when an extended packet for D-PHY is supplied from the D-PHY packet generation unit 305, the physical layer processing unit 222 transmits the extended packet for D-PHY on four lanes.

[0239] The image sensor 211 configured as described above has an extended mode - compatible CSI - 2 transmission circuit 221 configured such that the AS payload generation unit 301 is connected to the A - PHY packet generation unit 303, the C - PHY packet generation unit 304, and the D - PHY packet generation unit 305 via the selector 302. As a result, the image sensor 211 can generate the AS payload common to the extended packet for A - PHY, the extended packet for C - PHY, and the extended packet for D - PHY in one AS payload generation unit 301. That is, the A - PHY packet generation unit 303, the C - PHY packet generation unit 304, and the D - PHY packet generation unit 305 can share the AS payload generation unit 301, thereby reducing the circuit scale. Therefore, miniaturization of the image sensor 211 can be realized.

[0240] FIG. 26 is a block diagram showing a detailed configuration example of the application processor 214. In the application processor 214 shown in FIG. 26, components common to the application processor 22 in FIG. 10 are denoted by the same reference numerals, and detailed descriptions thereof are omitted.

[0241] That is, the application processor 214 is configured to include a register 73 and a controller 74, similar to the application processor 22 in FIG. 10. Note that the controller 74 may be implemented by software. Also, the I2C / I3C master 253 and the CCI master 254 included in the application processor 214 respectively correspond to the I2C / I3C master 72 and the CCI master 88 in FIG. 10.

[0242] The application processor 214 includes an extended mode - compatible CSI - 2 reception circuit 251 and a physical layer processing unit 252, and the physical layer processing unit 252 is compatible with A - PHY, C - PHY, and D - PHY.

[0243] The CSI-2 reception circuit 251 compatible with the extended mode includes, in addition to the CCI master 254, a selector 401, an A-PHY packet reception unit 402, a C-PHY packet reception unit 403, a D-PHY packet reception unit 404, a selector 405, and an AS payload reception unit 406.

[0244] The selector 401 selects one of the A-PHY packet reception unit 402, the C-PHY packet reception unit 403, and the D-PHY packet reception unit 404 provided in parallel as the output destination of the extended packet supplied from the physical layer processing unit 252.

[0245] The A-PHY packet reception unit 402 receives the extended packet for A-PHY supplied via the selector 401 and outputs it to the selector 405. For example, the A-PHY packet reception unit 402 has an A-PHY packet header interpretation unit 411, an A-PHY packet footer verification unit 412, and an AAL processing unit 413.

[0246] For example, the A-PHY packet header interpretation unit 411 interprets the content described in the A-PHY packet header and performs processing necessary for receiving the extended packet for A-PHY, and the A-PHY packet footer verification unit 412 verifies the presence or absence of an error using the A-PHY packet footer. Then, the AAL processing unit 413 performs processing to combine the Adaptation Layer divided in the AAL generation unit 321 of FIG. 25.

[0247] The C-PHY packet reception unit 403 receives the extended packet for C-PHY supplied via the selector 401 and outputs it to the selector 405. For example, the C-PHY packet reception unit 403 has a C-PHY lane merging unit 421, a C-PHY packet header interpretation unit 422, and a C-PHY packet footer verification unit 423.

[0248] For example, the C-PHY lane merger 421 merges the extended packets for C-PHY that are distributed into three lanes according to the CSI-2 standard and supplied via the physical layer processing unit 252. Then, the C-PHY packet header interpretation unit 422 interprets the content described in the C-PHY packet header, performs the processing necessary for receiving the extended packets for C-PHY, and the C-PHY packet footer verification unit 423 verifies the presence or absence of errors using the C-PHY packet footer.

[0249] The D-PHY packet receiver 404 receives the extended packets for D-PHY supplied via the selector 401 and outputs them to the selector 405. For example, the D-PHY packet receiver 404 includes a D-PHY lane merger 431, a D-PHY packet header interpretation unit 432, and a D-PHY packet footer verification unit 433.

[0250] For example, the D-PHY lane merger 431 merges the extended packets for D-PHY that are distributed into four lanes according to the CSI-2 standard and supplied via the physical layer processing unit 252. Then, the D-PHY packet header interpretation unit 432 interprets the content described in the D-PHY packet header, performs the processing necessary for receiving the extended packets for D-PHY, and the D-PHY packet footer verification unit 433 verifies the presence or absence of errors using the D-PHY packet footer.

[0251] The selector 405 selects one of the A-PHY packet receiver 402, the C-PHY packet receiver 403, and the D-PHY packet receiver 404 provided in parallel as the output source of the extended packets supplied to the AS payload receiver 406.

[0252] The AS payload receiving unit 406 has an unpacking unit 441, an extended packet header interpretation unit 442, and an extended packet footer verification unit 443 corresponding to the AS payload generation unit 301 in FIG. 25. The unpacking unit 441 unpacks the image data packed by the packing unit 311. The extended packet header interpretation unit 442 interprets the extended packet header generated by the extended packet header generation unit 312, and reads out, for example, the packet count PC and the virtual channel VC. The extended packet footer verification unit 443 verifies the presence or absence of an error using the extended packet footer added by the extended packet footer generation unit 313. Then, the AS payload receiving unit 406 outputs various data stored in the packet data supplied via the selector 405, such as image data, in-vehicle line numbers, SourceID, etc., and CRC errors, to a subsequent LSI (not shown).

[0253] The application processor 214 configured as described above is such that the extended mode-compatible CSI-2 receiving circuit 251 is configured so that the AS payload receiving unit 406 is connected to the A-PHY packet receiving unit 402, the C-PHY packet receiving unit 403, and the D-PHY packet receiving unit 404 via the selector 405. Thereby, the application processor 214 can receive the AS payload common to the extended packet for A-PHY, the extended packet for C-PHY, and the extended packet for D-PHY with one AS payload receiving unit 406. That is, the A-PHY packet receiving unit 402, the C-PHY packet receiving unit 403, and the D-PHY packet receiving unit 404 can share the AS payload receiving unit 406, thereby reducing the circuit scale. Therefore, miniaturization of the application processor 214 can be achieved.

[0254] <Second Configuration Example Adapted to E2E Protection> With reference to FIGS. 27 to 74, a second configuration example adapted to E2E Protection will be described.

[0255] <Configuration Example of A-PHY Direct Connection Structure> The communication system 501 shown in FIG. 27 has a direct connection structure in which the image sensor 511 and the application processor 512 are directly connected by A-PHY (without passing through a SerDes device as described with reference to FIG. 40 to be described later).

[0256] The image sensor 511 includes an A-PHY processing unit 521, a CSIA processing unit 522, a CSI2 processing unit 523, a CSI2-FS processing unit 524, a CCI processing unit 525, a CCI-FS processing unit 526, and a register 527.

[0257] In the A-PHY processing unit 521, the CCI processing unit 525 is implemented as an upper layer, and it transmits and receives data including an extended packet header ePH and an extended packet footer ePF by connecting to the A-PHY processing unit 531 of the application processor 512 via MIPI A-PHY.

[0258] The CCI-FS processing unit 526 compares, for example, the Destination ID included in the extended packet header ePH with the ID (Source ID) of the image sensor 511 to determine whether or not to access the image sensor 511.

[0259] The application processor 512 includes an A-PHY processing unit 531, a CSIA processing unit 532, a CSI2 processing unit 533, a CSI2-FS processing unit 534, a CCI processing unit 535, a CCI-FS processing unit 536, a register 537, and a CCI-FS switch 538.

[0260] In the A-PHY processing unit 531, the CCI processing unit 535 is implemented as an upper layer, and it transmits and receives data including an extended packet header ePH and an extended packet footer ePF by connecting to the A-PHY processing unit 521 of the image sensor 511 via MIPI A-PHY.

[0261] The CCI-FS processing unit 536 compares, for example, the Destination ID included in the extended packet header ePH with the ID (Source ID) that the application processor 512 has, and determines whether or not to access the application processor 512.

[0262] When the CCI-FS processing unit 536 is valid, the CCI-FS switch 538 switches so that data is transmitted and received via the CCI-FS processing unit 536, and when the CCI-FS processing unit 536 is invalid, data is transmitted and received without going through the CCI-FS processing unit 536.

[0263] With reference to FIGS. 28 to 32, the transfer of the read command and read data in the communication system 501 will be described.

[0264] FIG. 28 shows an example of the packet configuration of the read command generated in the CCI-FS processing unit 536 of the application processor 512 at the time of read access.

[0265] As shown in FIG. 28, the read command is composed of an extended packet header ePH* (* = n), an AP(CCI) payload, an extended packet footer ePF1, and an extended packet footer ePF0.

[0266] The extended packet header ePH* (* = n) consists of extended packet headers ePH0 to ePH3 as shown in the figure.

[0267] In the extended packet header ePH0, an extended VC, an extended DT, an extended PFEN, and an extended PHEN are stored. For example, the extended DT is information indicating the CCI protocol (I2C), and routing processing is performed using the extended DT.

[0268] The extended packet header ePH1 stores Source ID[7:1] and Packet Length. For example, Source ID is information indicating the source of the CCI protocol (I2C), and response processing is performed based on the Source ID. Packet Length is information indicating the data length.

[0269] The extended packet header ePH2 stores a Security Descriptor and a Message Counter. The Security Descriptor indicates whether security is used, and indicates "8'h0" when security is not used. The Message Counter is information indicating the packet order, shows the count value obtained by counting messages, and indicates "16'h5" when the message is the fifth one.

[0270] The extended packet header ePH3 stores Destination ID[7:1], Read / Write, and Destination Address. Destination ID[7:1] indicates the slave address of the CCI processing unit 525 of the image sensor 511, and is "7'h0D" in the illustrated example. For example, Destination ID is information indicating the destination of the CCI protocol (I2C), routing is performed based on the Destination ID, and the communication path is referenced. Read / Write indicates data reading or writing, and indicates "1'b1" in the case of read. Destination Address indicates the address of the register 527 of the image sensor 511 that is the final destination, and is "0x0200" in the illustrated example.

[0271] The AP (CCI) payload stores various types of data (Data0[7:0]), for example. The AP (CCI) payload is not transmitted when security is off, and may be transmitted with dummy data stored when security is on.

[0272] The extended packet footer ePF1 is not transmitted when security is off.

[0273] The CRC calculated value is stored in the extended packet footer ePF0.

[0274] In the application processor 512, such a read command with the packet structure is generated in the CCI-FS processing unit 536 and supplied to the A-PHY processing unit 531.

[0275] FIG. 29 shows an example of the packet configuration of the read command output from the A-PHY processing unit 531 of the application processor 512 during a read access.

[0276] As shown in FIG. 29, the A-PHY processing unit 531 adds an A-PHY header and an A-PHY footer to the read command supplied from the CCI-FS processing unit 536 as the protection range of E2E Protection.

[0277] Such a read command with the packet structure is A-PHY transferred by the A-PHY processing unit 531 of the application processor 512. Then, in the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and the A-PHY footer from the read command. After that, the read command is supplied to the CCI-FS processing unit 526 via the CCI processing unit 525 of the slave address "7'h0D" indicated by the Destination ID.

[0278] FIG. 30 shows an example of the packet configuration of the read command supplied to the CCI-FS processing unit 526 and the read data generated in the CCI-FS processing unit 526 during a read access.

[0279] As shown in FIG. 30, the read command with the packet structure shown in FIG. 28, that is, the read command that is the protection range of E2E Protection in the A-PHY transfer, is supplied to the CCI-FS processing unit 526.

[0280] As shown in the figure, the read data is composed of an extended packet header ePH*(*=n), an AP(CCI) payload, an extended packet footer ePF1, and an extended packet footer ePF0. And in the AP(CCI) payload, the read data value read from the address "0x0200" of register 527 indicated by the source address information (Destination Address) of the extended packet header ePH of the read command is stored.

[0281] In the image sensor 511, the read data with such a packet structure is generated in the CCI-FS processing unit 526 and supplied to the A-PHY processing unit 521.

[0282] FIG. 31 shows an example of the packet structure of the read data output from the A-PHY processing unit 521 of the image sensor 511 at the time of read access.

[0283] As shown in FIG. 31, the A-PHY processing unit 521 adds an A-PHY header and an A-PHY footer to the read data supplied from the CCI-FS processing unit 526 as the protection range of E2E Protection.

[0284] The read data with such a packet structure is A-PHY transferred by the A-PHY processing unit 521 of the image sensor 511. And in the application processor 512, the A-PHY processing unit 531 removes the A-PHY header and the A-PHY footer from the read data, and the read data is supplied to the CCI-FS processing unit 536.

[0285] FIG. 32 shows an example of the packet structure of the read data supplied to the CCI-FS processing unit 536 at the time of read access.

[0286] As shown in FIG. 32, the read data in the packet structure shown in FIG. 30, that is, the read data whose protection range is E2E Protection in A-PHY transfer, is supplied to the CCI-FS processing unit 536.

[0287] With reference to FIGS. 33 to 35, the transfer of write data in the communication system 501 will be described. Note that the description will be made assuming access from the state where the CCI-FS processing unit 526 on the image sensor 511 side is enabled.

[0288] FIG. 33 shows an example of the packet configuration of write data generated in the CCI-FS processing unit 536 of the application processor 512 during a write access.

[0289] As shown in FIG. 33, the write data is composed of an extended packet header ePH*(* = n), an AP(CCI) payload (write data), an extended packet footer ePF1, and an extended packet footer ePF0.

[0290] The extended packet header ePH*(* = n) consists of extended packet headers ePH0 to ePH3 as shown in the figure.

[0291] The extended packet header ePH0 stores an extended VC, an extended DT, an extended PFEN, and an extended PHEN.

[0292] The extended packet header ePH1 stores a Source ID[7:1] and a Packet Length.

[0293] The extended packet header ePH2 stores a Security Descriptor and a Message Counter. The Security Descriptor indicates whether security is used or not, and indicates "8'h0" when security is not used. The Message Counter indicates the count value obtained by counting messages, and indicates "16'h4" when the message is the fourth one.

[0294] In the extended packet header ePH3, Destination ID[7:1], Read / Write, and Destination Address are stored. Destination ID[7:1] indicates the slave address of the CCI processing unit 525 of the image sensor 511, which is "7'h0D" in the illustrated example. Read / Write indicates data reading or writing, and indicates "1'b0" in the case of write. Destination Address indicates the address of the register 527 of the image sensor 511 that is the final destination, which is "0x1234" in the illustrated example.

[0295] In the AP(CCI) payload, data (Data0[7:0]) to be written to the image sensor 511 is stored, and the 0xFF value becomes the write data.

[0296] The extended packet footer ePF1 is not transmitted when security is off.

[0297] In the extended packet footer ePF0, the CRC calculated value is stored.

[0298] In the application processor 512, write data with such a packet structure is generated in the CCI-FS processing unit 536 and supplied to the A-PHY processing unit 531.

[0299] FIG. 34 shows an example of the packet configuration of write data output from the A-PHY processing unit 531 of the application processor 512 during a write access.

[0300] As shown in FIG. 34, the A-PHY processing unit 531 adds an A-PHY header and an A-PHY footer to the write data supplied from the CCI-FS processing unit 536 as the protection range of E2E Protection.

[0301] Such write data with the packet structure is A-PHY transferred by the A-PHY processing unit 531 of the application processor 512. Then, in the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY footer from the write data. Thereafter, the write data is supplied to the CCI-FS processing unit 526 via the CCI processing unit 525 with the slave address "7'h0D" indicated by the Destination ID.

[0302] FIG. 35 shows an example of the packet structure of the write data supplied to the CCI-FS processing unit 526 during a write access.

[0303] As shown in FIG. 35, the write data with the packet structure shown in FIG. 33, that is, the write data that is the protection range of E2E Protection in the A-PHY transfer, is supplied to the CCI-FS processing unit 526. Then, the CCI-FS processing unit 526 writes the data stored in the AP (CCI) payload from the address "0x1234" of the register 527 indicated by the CCI command ID information, that is, the source address information (Destination Address) of the extended packet header ePH of the read command.

[0304] With reference to FIG. 36, the outline of the extended packet header ePH and the extended packet footer ePF will be described.

[0305] As shown in FIG. 36, the CCI-FS E2E packet is composed of an extended packet header ePH, packet data, and an extended packet footer ePF, and its packet length is Length = Byte Count × Data Byte width.

[0306] For the extended packet header ePH, fields such as an extended VC, an extended DT, and a Message Counter are used. The length of the extended packet header ePH can be changed with the field value (epFEN field) of the extended packet header ePH.

[0307] The packet data is composed of, for example, PL pieces of data (Data 0 to Data PL-1), and its length is Length = Packet Length (PL) × Data Byte Width. In the case of a read command, no data is stored when security is off, and 1-byte dummy data is stored when security is on in the packet data. In the case of a write access, write data for the payload data is stored in the packet data. In the case of a read access, read data for the payload data is stored in the packet data. When using Clock Stretch (Control Code Indicator of ePH0 = 1), a 1-byte data payload indicating the type of control is attached to the packet data.

[0308] The extended packet footer ePF1 is a field setting value (epFEN field) of the extended packet header ePH, and its length can be changed. Also, security-related information can be added.

[0309] The extended packet footer ePF0 is a field setting value of the extended packet header ePH, and CRC-32 calculated from the packet data can be added.

[0310] <Example of communication processing> Referring to the flowcharts of FIGS. 37 to 39, the communication processing using CCI-FS performed in the communication system 501 shown in FIG. 27 will be described.

[0311] As shown in FIG. 37, in steps S211 to S222, initial setting and confirmation operations are performed.

[0312] In step S211, two read accesses are performed on the Capability register of the CCI-FS processing unit 526 from the application processor 512 to the image sensor 511. Note that the number of read accesses is not limited to two, and for example, it can be arbitrarily set in a functionally safe manner, and it may be one time or multiple times such as three times or more.

[0313] In step S212, in the application processor 512, the CSI2-FS processing unit 524 determines whether the Capability register value of the CCI-FS processing unit 526 is 1'b1 both times for the result of the read access in step S211. In step S212, if it is determined that the Capability register value of the CCI-FS processing unit 526 is not 1'b1 both times, the process proceeds to step S213.

[0314] In step S213, in the application processor 512, the CSI2-FS processing unit 524 determines whether the number of retransmissions is three or more. Note that the number of retransmissions is not limited to three and can be set to any number, and the same applies to the number of retransmissions described below. In step S213, if it is determined that the number of retransmissions is less than three (one or two), the process returns to step S211, and the following similar processing is repeated.

[0315] On the other hand, in step S212, if it is determined that the Capability register value of the CCI-FS processing unit 526 is 1'b1 both times, the process proceeds to 214.

[0316] In step S214, one write access is performed on the Enable register of the CCI-FS processing unit 526 from the application processor 512 to the image sensor 511.

[0317] In step S215, in the image sensor 511, the CCI-FS processing unit 526 performs one write access to the Enable register of the CCI-FS processing unit 536 of the application processor 512.

[0318] In step S216, the slave address of the opposing image sensor 511 is set in the Destination SID register of the CCI-FS processing unit 536 of the application processor 512.

[0319] In step S217, the setting of the ePH register of the CCI-FS processing unit 536 of the application processor 512 is performed.

[0320] In step S218, the setting of the ePH register of the CCI-FS processing unit 526 is performed from the application processor 512 to the image sensor 511.

[0321] In step S219, read accesses to the Enable register and the Error register of the CCI-FS processing unit 526 are performed from the application processor 512 to the image sensor 511.

[0322] In step S220, in the application processor 512, the CCI-FS processing unit 536 determines whether the Enable register value of the CCI-FS processing unit 526 is 1'b1 and the Error register value is 0 based on the result of the read access in step S219.

[0323] In step S220, if it is determined that the Enable register value of the CCI-FS processing unit 526 is not 1'b1 or the Error register value is not 0, the process proceeds to step S221.

[0324] In step S221, in the application processor 512, the CSI2-FS processing unit 524 determines whether the retransmission count is 3 or more. In step S221, if it is determined that the retransmission count is 3 or more, the process returns to step S211, and thereafter, the same process is repeated.

[0325] On the other hand, in step S213, if it is determined that the retransmission count is 3 or more, or in step S221, if it is determined that the retransmission count is less than 3 (1 or 2), the process proceeds to step S222.

[0326] In step S222, communication is performed using CCI without using CCI-FS, and then the communication process ends.

[0327] On the other hand, in step S220, if it is determined that the Enable register value of the CCI-FS processing unit 526 is 1'b1 and the Error register value is 0, the process proceeds to step S223.

[0328] As shown in FIG. 38, in steps S223 to S234, a write operation using CCI-FS is performed.

[0329] In step S223, the CCI-FS processing unit 536 of the application processor 512 sets the ePH register so as to perform a write operation.

[0330] In step S224, the CCI-FS processing unit 536 of the application processor 512 sets the write data register.

[0331] In step S225, the CCI-FS processing unit 536 of the application processor 512 sets the command execution register to 1.

[0332] In step S226, in the application processor 512, the A-PHY processing unit 531 adds an A-PHY header and an A-PHY footer to the write data generated by the CCI-FS processing unit 536 as the protection range of E2E Protection, and performs A-PHY transfer, as shown in FIG. 34 described above.

[0333] In step S227, in the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and the A-PHY footer from the write data, and supplies the protection range of E2E Protection to the CCI-FS processing unit 526.

[0334] In step S228, in the image sensor 511, the CCI-FS processing unit 526 checks the Source ID of the image sensor 511 and the Destination SID of the extended packet header ePH from the content of the extended packet header ePH.

[0335] In step S229, in the image sensor 511, the CCI-FS processing unit 526 determines whether the Source ID of the image sensor 511 checked in step S228 matches the Destination SID of the extended packet header ePH.

[0336] If it is determined in step S229 that the Source ID of the image sensor 511 matches the Destination SID of the extended packet header ePH, the process proceeds to step S230.

[0337] In step S230, in the image sensor 511, the CCI-FS processing unit 526 checks the Message Counter from the content of the extended packet header ePH.

[0338] In step S231, in image sensor 511, the CCI-FS processing unit 526 determines whether the Message Counter (received) of the image sensor 511 confirmed in step S230 matches the Message Counter of the extended packet header ePH.

[0339] In step S231, if it is determined that the Message Counter (received) of the image sensor 511 matches the Message Counter of the extended packet header ePH, the process proceeds to step S232.

[0340] In step S232, in image sensor 511, the CCI-FS processing unit 526 checks the CRC from the content of the extended packet footer ePF.

[0341] In step S233, in image sensor 511, the CCI-FS processing unit 526 determines whether the received value (ePF0) of the extended packet footer ePF confirmed in step S232 matches the CRC calculation result calculated in the CCI-FS processing unit 526.

[0342] In step S233, if it is determined that the received value (ePF0) of the extended packet footer ePF matches the CRC calculation result, the process proceeds to step S234.

[0343] In step S234, in image sensor 511, the CCI-FS processing unit 526 performs a write process of writing write data to the address of register 527 from the contents of the extended packet header ePH and the extended packet footer ePF. Then, the process proceeds to step S235.

[0344] As shown in FIG. 39, in steps S235 to S247, a read operation using CCI-FS is performed.

[0345] In step S235, in the application processor 512, the CCI-FS processing unit 536 sets the ePH register so that a read operation is performed.

[0346] In step S236, in the application processor 512, the CCI-FS processing unit 536 sets the command execution register to 1.

[0347] In step S237, in the application processor 512, as shown in FIG. 29 described above, the A-PHY processing unit 531 adds an A-PHY header and an A-PHY footer to the write data generated by the CCI-FS processing unit 536 as the protection range of E2E Protection, and performs A-PHY transfer.

[0348] In step S238, in the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and the A-PHY footer from the write data, and supplies the protection range of E2E Protection to the CCI-FS processing unit 526.

[0349] In step S239, in the image sensor 511, the CCI-FS processing unit 526 checks the Source ID of the image sensor 511 and the Destination SID of the extended packet header ePH from the content of the extended packet header ePH.

[0350] In step S240, in the image sensor 511, the CCI-FS processing unit 526 determines whether the Source ID of the image sensor 511 checked in step S239 matches the Destination SID of the extended packet header ePH.

[0351] In step S240, if it is determined that the Source ID of the image sensor 511 matches the Destination SID of the extended packet header ePH, the process proceeds to step S241.

[0352] In step S241, in the image sensor 511, the CCI-FS processing unit 526 checks the Message Counter from the content of the extended packet header ePH.

[0353] In step S242, in the image sensor 511, the CCI-FS processing unit 526 determines whether the Message Counter (received) of the image sensor 511 checked in step S241 matches the Message Counter of the extended packet header ePH.

[0354] If it is determined in step S242 that the Message Counter (received) of the image sensor 511 matches the Message Counter of the extended packet header ePH, the process proceeds to step S243.

[0355] In step S243, in the image sensor 511, the CCI-FS processing unit 526 checks the CRC from the content of the extended packet footer ePF.

[0356] In step S244, in the image sensor 511, the CCI-FS processing unit 526 determines whether the received value (ePF0) of the extended packet footer ePF checked in step S243 matches the CRC calculation result calculated in the CCI-FS processing unit 526.

[0357] If it is determined in step S244 that the received value (ePF0) of the extended packet footer ePF matches the CRC calculation result, the process ends.

[0358] On the other hand, if it is determined in step S229 of FIG. 38 or step S240 of FIG. 39 that the Source ID of the image sensor 511 does not match the Destination SID of the extended packet header ePH, the process proceeds to step S245.

[0359] In step S245, the Error register (Routing) on the image sensor 511 side is set to 1, and then the process ends.

[0360] On the other hand, if it is determined in step S231 of FIG. 38 or step S242 of FIG. 39 that the Message Counter (received) of the image sensor 511 does not match the Message Counter of the extended packet header ePH, the process proceeds to step S246.

[0361] In step S246, the Error register (MC) on the image sensor 511 side is set to 1, and then the process ends.

[0362] On the other hand, if it is determined in step S233 of FIG. 38 or step S244 of FIG. 39 that the received value (ePF0) of the extended packet footer ePF does not match the CRC calculation result, the process proceeds to step S247.

[0363] In step S247, the Error register (CRC) on the image sensor 511 side is set to 1, and then the process ends.

[0364] <Configuration example of SerDes connection configuration> In the communication system 601 shown in FIG. 40, the image sensor 611 and the application processor 614 are connected in a SerDes connection configuration via the slave-side SerDes device 612 and the master-side SerDes device 613.

[0365] The image sensor 611 includes an I2C / I3C slave 621, a CCI processing unit 622, a CSI2-FS processing unit 623, and a register 624.

[0366] The slave-side SerDes device 612 includes an A-PHY processing unit 631, a CSIA processing unit 632, a CSI2-FS processing unit 633, an I2C / I3C master 634, a CCI processing unit 635, a CCI-FS processing unit 636, and a register 637.

[0367] The SerDes device 613 on the master side includes an A-PHY processing unit 641, a CSIA processing unit 642, a CSI2-FS processing unit 643, an I2C / I3C slave 644, a CCI processing unit 645, a CCI-FS processing unit 646, and a register 647.

[0368] The application processor 614 includes an I2C / I3C master 651, a CCI processing unit 652, a CCI-FS processing unit 653, a register 654, and a CCI-FS switch 655.

[0369] In the SerDes connection configuration as shown in FIG. 40, when the CCI configuration or the CCI-FS configuration is implemented as a higher-level protocol, other SerDes standards may be used. For example, by implementing the configuration of the extended packet header ePH, the extended packet footer ePF1, and the extended packet footer ePF0 as shown in FIG. 41 in the Payload from the Application Layer or a higher layer corresponding to the layer below it, various SerDes-related standards such as PCIE, USB, DisplayPort, HDMI (registered trademark), LVDS, and FPD-LINK can be applied.

[0370] With reference to FIGS. 41 to 49, the transfer of the read command and the read data in the communication system 601 will be described.

[0371] FIG. 41 shows an example of the packet configuration of the read command generated in the CCI-FS processing unit 653 of the application processor 614 during a read access.

[0372] As shown in FIG. 41, the read command is composed of an extended packet header ePH*(*=n), an extended packet footer ePF1, and an extended packet footer ePF0. Note that the details thereof are the same as the read command described with reference to FIG. 28 above.

[0373] In the application processor 614, a read command with such a packet structure is generated in the CCI-FS processing unit 653 and supplied to the I2C / I3C master 651.

[0374] FIG. 42 shows an example of the packet configuration of the read command output from the I2C / I3C master 651 of the application processor 614 during a read access.

[0375] As shown in FIG. 42, following the start condition S, the I2C / I3C master 651 transmits the address of the destination sensor, that is, the address of the CCI processing unit 645 of the master-side SerDes device 613 (Slave Address+W 8-bit) in the configuration shown in FIG. 40. In the example shown in FIG. 42, the address of the CCI processing unit 645 is Slave Address[7:1]=7'h0F. Following that address, the register addresses (Register Address [15:8] and Register Address [7:0]) of the register 647 of the master-side SerDes device 613 are transmitted. The I2C / I3C master 651 finally transmits the stop condition P following the extended packet header ePH*(*=n), the extended packet footer ePF1, and the extended packet footer ePF0.

[0376] A read command with such a packet structure is transferred from the I2C / I3C master 651 of the application processor 614 to the I2C / I3C. In the master-side SerDes device 613, the I2C / I3C slave 644 acquires the read command (extended packet header ePH*(*=n), extended packet footer ePF1, and extended packet footer ePF0). The read command is supplied to the CCI processing unit 645 with Slave Address[7:1]=7'h0F and then supplied to the A-PHY processing unit 641 via the CCI-FS processing unit 646, the CSI2-FS processing unit 643, and the CSIA processing unit 642.

[0377] FIG. 43 shows an example of the packet configuration of a read command output from the A-PHY processing unit 641 of the master-side SerDes device 613 during read access.

[0378] As shown in FIG. 43, the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the read command acquired by the I2C / I3C slave 644 as the protection range of E2E Protection. Note that the address of the CCI processing unit 635 of the master-side SerDes device 613, for example, Slave Address[7:1]=7'h0E, is added in the CSI2-FS processing unit 643 to the extended packet header ePH*(*=n).

[0379] The read command with such a packet structure is A-PHY transferred by the A-PHY processing unit 641 of the master-side SerDes device 613. In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and the A-PHY footer from the read command. The read command is supplied to the CCI processing unit 635 of the slave address "7'h0E" indicated by the Destination ID via the CSIA processing unit 632, the CSI2-FS processing unit 633, and the CCI-FS processing unit 636, and then supplied to the I2C / I3C master 634.

[0380] FIG. 44 shows an example of the packet configuration of a read command output from the I2C / I3C master 634 during read access.

[0381] As shown in FIG. 44, following the start condition S, the I2C / I3C master 634 transmits the destination sensor address, i.e., the address of the CCI processing unit 622 of the image sensor 611 (Slave Address+W 8-bit) in the configuration shown in FIG. 40. In the example shown in FIG. 44, the address of the CCI processing unit 622 is Slave Address[7:1]=7'h0D. Following that address, the register addresses (Register Address [15:8] and Register Address [7:0]) of the register 624 of the image sensor 611 are transmitted. The I2C / I3C master 634 finally transmits a stop condition P following the extended packet header ePH*(*=n), the extended packet footer ePF1, and the extended packet footer ePF0.

[0382] Such a read command with this packet structure is transferred from the I2C / I3C master 634 of the slave-side SerDes device 612 to I2C / I3C. Then, in the image sensor 611, the I2C / I3C slave 621 acquires the read command (extended packet header ePH*(*=n), extended packet footer ePF1, and extended packet footer ePF0). The read command is supplied to the CSI2-FS processing unit 623 via the CCI processing unit 622 with Slave Address[7:1]=7'h0D.

[0383] FIG. 45 shows an example of the packet structure of the read command supplied to the CSI2-FS processing unit 623 and the read data generated in the CSI2-FS processing unit 623 during a read access.

[0384] As shown in FIG. 45, the read command with the same packet structure as shown in FIG. 41, i.e., the read command whose protection range is E2E Protection in the A-PHY transfer, is supplied to the CSI2-FS processing unit 623.

[0385] As shown in the figure, the read data is composed of an extended packet header ePH*(*=n), an AP(CCI) payload, an extended packet footer ePF1, and an extended packet footer ePF0. And the read data value read from the address "0x0200" of register 624 indicated by the source address information (Destination Address) of the extended packet header ePH of the read command is stored in the AP(CCI) payload.

[0386] In the image sensor 611, the read data with such a packet structure is generated in the CCI-FS processing unit 623 and supplied to the I2C / I3C slave 621 via the CCI processing unit 622.

[0387] FIG. 46 shows an example of the packet structure of the read data output from the I2C / I3C slave 621 of the image sensor 611 during a read access.

[0388] As shown in FIG. 46, following the start condition S, the I2C / I3C slave 621 transmits the destination sensor address, that is, the address (Slave Address+W 8-bit) of the I2C / I3C master 634 of the slave-side SerDes device 612 in the configuration shown in FIG. 40. In the example shown in FIG. 46, the address of the I2C / I3C master 634 is Slave Address[7:1]=7'h0D. Following that address, the storage address of the read data (the address of register 624 of the image sensor 611) is transmitted, and the address (Slave Address+R 8-bit) of the I2C / I3C master 634 of the slave-side SerDes device 612 is transmitted. After the I2C / I3C slave 621 transmits the extended packet header ePH*(*=n), the AP(CCI) payload, the extended packet footer ePF1, and the extended packet footer ePF0, finally the stop condition P is transmitted.

[0389] Such a read command with the packet structure is transferred via I2C / I3C from the I2C / I3C slave 621 of the image sensor 611. In the SerDes device 612 on the slave side, the I2C / I3C master 634 acquires read data (extended packet header ePH*(*=n), AP (CCI) payload, extended packet footer ePF1, and extended packet footer ePF0). The read data is supplied to the CCI processing unit 635 with Slave Address[7:1]=7'h0E, and then supplied to the A-PHY processing unit 631 via the CCI-FS processing unit 636, the CSI2-FS processing unit 633, and the CSIA processing unit 632.

[0390] FIG. 47 shows an example of the packet configuration of the read data output from the A-PHY processing unit 631 of the SerDes device 612 on the slave side during a read access.

[0391] As shown in FIG. 47, the A-PHY processing unit 631 adds an A-PHY header and an A-PHY footer to the read data acquired by the I2C / I3C master 634 as the protection range of E2E Protection.

[0392] Such read data with the packet structure is A-PHY transferred by the A-PHY processing unit 631 of the SerDes device 612 on the slave side. Then, in the SerDes device 613 on the master side, the A-PHY processing unit 641 removes the A-PHY header and the A-PHY footer from the read data. The read data is supplied to the I2C / I3C slave 644 via the CSIA processing unit 642, the CSI2-FS processing unit 643, the CCI-FS processing unit 646, and the CCI processing unit 635.

[0393] FIG. 48 shows an example of the packet configuration of the read data output from the I2C / I3C slave 644 of the SerDes device 613 on the master side during a read access.

[0394] As shown in FIG. 48, following the start condition S, the I2C / I3C slave 644 transmits the destination sensor address, that is, in the configuration shown in FIG. 40, the address of the CCI processing unit 635 of the master-side SerDes device 613 (Slave Address+W 8-bit). In the example shown in FIG. 48, the address of the CCI processing unit 635 is Slave Address[7:1]=7'h0F. Following that address, the register addresses of the register 647 of the master-side SerDes device 613 (Register Address [15:8] and Register Address [7:0]) are transmitted, and the address of the CCI processing unit 635 (Slave Address+R 8-bit) is transmitted. Subsequently, after the I2C / I3C slave 644 transmits the extended packet header ePH*(*=n), the AP(CCI) payload, the extended packet footer ePF1, and the extended packet footer ePF0, finally the stop condition P is transmitted.

[0395] Such read data with the packet structure is transferred from the I2C / I3C slave 644 of the master-side SerDes device 613 to I2C / I3C. Then, in the application processor 614, the I2C / I3C master 651 acquires the read command (extended packet header ePH*(*=n), extended packet footer ePF1, and extended packet footer ePF0) and supplies it to the CCI-FS processing unit 653.

[0396] FIG. 49 shows an example of the packet structure of the read data supplied to the CCI-FS processing unit 653 during a read access.

[0397] As shown in FIG. 49, the read data with the same packet structure as shown in FIG. 45, that is, the read data that is the protection range of E2E Protection in the A-PHY transfer, is supplied to the CCI-FS processing unit 653.

[0398] <Example of Communication Processing> Referring to the flowcharts of FIGS. 50 to 57, the communication processing using CCI-FS performed in the communication system 601 shown in FIG. 40 will be described.

[0399] As shown in FIG. 50, in steps S301 to S317, initial settings and confirmation operations are performed.

[0400] In step S301, the slave address of the opposing image sensor 611 is set in the Destination SID register of the CCI-FS processing unit 653 of the application processor 614.

[0401] In step S302, the setting of the ePH register of the CCI-FS processing unit 653 of the application processor 614 is performed.

[0402] In step S303, the setting of the Destination SID of the Bridge configuration of the CCI-FS processing unit 653 of the application processor 614 is performed, and the master-side SerDes device 613 is registered. Here, the Address, attribution, and Timeout_no1 registers are also set in the same manner, and the following settings are performed in the same manner.

[0403] In step S304, the setting of the ePH register of the CCI-FS processing unit 643 is performed from the application processor 614 to the master-side SerDes device 613.

[0404] In step S305, the setting of the Destination SID of the Bridge configuration of the CCI-FS processing unit 643 is performed from the application processor 614 to the master-side SerDes device 613, and the slave-side SerDes device 612 is registered.

[0405] In step S306, a read access to the Error register of the CCI-FS processing unit 643 is performed from the application processor 614 to the master-side SerDes device 613.

[0406] In step S307, in the application processor 614, the CCI-FS processing unit 653 determines whether the register value of the Error register of the CCI-FS processing unit 643 of the master-side SerDes device 613 is 0 as a result of the read access in step S306.

[0407] In step S307, if it is determined that the register value of the Error register of the CCI-FS processing unit 643 of the master-side SerDes device 613 is not 0 (other than 0), the process proceeds to step S308.

[0408] In step S308, in the application processor 614, the CCI-FS processing unit 653 determines whether the number of retransmissions is 3 or more. If it is determined that the number of retransmissions is less than 3 (1 or 2), the process returns to step S304, and the following similar processing is repeated.

[0409] On the other hand, in step S307, if it is determined that the register value of the Error register of the CCI-FS processing unit 643 of the master-side SerDes device 613 is 0, the process proceeds to step S309.

[0410] In step S309, the setting of the ePH register of the CCI-FS processing unit 636 is performed from the application processor 614 to the slave-side SerDes device 612.

[0411] In step S310, the setting of the Destination SID of the Bridge configuration of the CCI-FS processing unit 636 is performed from the application processor 614 to the slave-side SerDes device 612, and the slave-side SerDes device 612 is registered.

[0412] In step S311, a read access to the Error register of the CCI-FS processing unit 636 is performed from the application processor 614 to the slave-side SerDes device 612.

[0413] In step S312, in the application processor 614, the CCI-FS processing unit 653 determines whether the register value of the Error register of the CCI-FS processing unit 636 of the slave-side SerDes device 612 is 0 as a result of the read access in step S311.

[0414] In step S312, if it is determined that the register value of the Error register of the CCI-FS processing unit 636 of the slave-side SerDes device 612 is not 0 (is other than 0), the process proceeds to step S313.

[0415] In step S313, in the application processor 614, the CCI-FS processing unit 653 determines whether the number of retransmissions is 3 or more. If it is determined that the number of retransmissions is less than 3 (1 or 2), the process returns to step S309, and the following similar processes are repeated.

[0416] On the other hand, in step S312, if it is determined that the register value of the Error register of the CCI-FS processing unit 636 of the slave-side SerDes device 612 is 0, the process proceeds to step S314.

[0417] In step S314, the setting of the ePH register of the CCI-FS processing unit 623 is performed from the application processor 614 to the image sensor 611.

[0418] In step S315, a read access to the Error register of the CCI-FS processing unit 623 is performed from the application processor 614 to the image sensor 611.

[0419] In step S316, in the application processor 614, the CCI-FS processing unit 653 determines whether the register value of the Error register of the CCI-FS processing unit 623 of the image sensor 611 is 0 as a result of the read access in step S315.

[0420] In step S316, if it is determined that the register value of the Error register of the CCI-FS processing unit 623 of the image sensor 611 is not 0 (other than 0), the process proceeds to step S317.

[0421] In step S317, in the application processor 614, the CCI-FS processing unit 653 determines whether the number of retransmissions is 3 or more. If it is determined that the number of retransmissions is less than 3 (1 or 2), the process returns to step S314, and the following similar processes are repeated.

[0422] Here, in step S308, step S313, or step S317, if it is determined that the number of retransmissions is 3 or more, the process returns to step S301, and the following similar processes are repeated.

[0423] On the other hand, in step S316, if it is determined that the register value of the Error register of the CCI-FS processing unit 623 of the image sensor 611 is 0, the process proceeds to step S318.

[0424] As shown in FIG. 51, in steps S318 to S327, a write operation using CCI-FS is performed.

[0425] In step S318, the CCI-FS processing unit 653 of the application processor 614 sets the ePH register so as to perform a write operation.

[0426] In step S319, the CCI-FS processing unit 653 of the application processor 614 sets the write data register.

[0427] In step S320, the CCI-FS processing unit 653 of the application processor 614 sets the command execution register to 1 and issues a write command.

[0428] In step S321, the application processor 614 performs a Sequence A_Write (at AP) process, which will be described later with reference to FIG. 53.

[0429] In step S322, the master-side SerDes device 613 performs a Sequence B (at SerDes (Master)) process, which will be described later with reference to FIG. 56. Although FIG. 56 describes the Sequence B (at SerDes (Slave)) process executed by the slave-side SerDes device 612, the master-side SerDes device 613 can also execute similar processes with corresponding blocks.

[0430] In step S323, from the extended DT of the extended packet header ePH of the master-side SerDes device 613, the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer via the CSI2-FS processing unit 643 and the CSIA processing unit 642 to perform A-PHY transfer.

[0431] In step S324, the slave-side SerDes device 612 performs a Sequence B (at SerDes (Slave)) process, which will be described later with reference to FIG. 56.

[0432] In step S325, the slave-side SerDes device 612 performs a Sequence A_Write (at SerDes (Slave)) process, which will be described later with reference to FIG. 53. Although FIG. 53 describes the Sequence A_Write (at AP) process executed by the application processor 614, the slave-side SerDes device 612 can also execute similar processes with corresponding blocks.

[0433] In step S326, the image sensor 611 performs the Sequence B (Image Sensor time) process described later with reference to FIG. 56. Note that FIG. 56 describes the Sequence B (SerDes (Slave) time) process executed by the SerDes device 612 on the slave side, but the image sensor 611 can also execute similar processes by corresponding blocks.

[0434] In step S327, in the image sensor 611, the CCI-FS processing unit 623 performs a write process of writing write data to the address of the register 624 from the contents of the extended packet header ePH and the extended packet footer ePF. Then, the process proceeds to step S328.

[0435] As shown in FIG. 52, in steps S328 to S344, a read operation using CCI-FS is performed.

[0436] In step S328, the CCI-FS processing unit 653 of the application processor 614 sets the ePH register to perform a read operation.

[0437] In step S329, the CCI-FS processing unit 653 of the application processor 614 sets the read data register.

[0438] In step S330, the CCI-FS processing unit 653 of the application processor 614 sets the command execution register to 1 and issues a read command.

[0439] In step S331, the application processor 614 performs the Sequence A_Read_CMD (AP time) process described later with reference to FIG. 54. Here, in the Sequence A_Read_CMD (AP time) process, two branched processes are performed in parallel, and the process proceeds to step S332 according to branch A, and the process proceeds to step S339 according to branch B.

[0440] In step S332, the master-side SerDes device 613 performs the Sequence B (when SerDes (Master)) process, which will be described later with reference to FIG. 56. Note that in FIG. 56, the Sequence B (when SerDes (Slave)) process executed by the slave-side SerDes device 612 is described, but the master-side SerDes device 613 can also execute similar processes with corresponding blocks.

[0441] In step S333, from the extended DT of the extended packet header ePH of the master-side SerDes device 613, the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer via the CSI2-FS processing unit 643 and the CSIA processing unit 642 to perform A-PHY transfer.

[0442] In step S334, the slave-side SerDes device 612 performs the Sequence B (when SerDes (Slave)) process, which will be described later with reference to FIG. 56.

[0443] In step S355, the slave-side SerDes device 612 performs the Sequence A_Read_CMD (when SerDes (Slave)) process, which will be described later with reference to FIG. 54. Note that in FIG. 54, the Sequence A_Read_CMD (when AP) process executed by the application processor 614 is described, but the slave-side SerDes device 612 can also execute similar processes with corresponding blocks. Here, in the Sequence A_Read_CMD (when SerDes (Slave)) process, among the two branched processes, the process does not proceed to branch A, and the process proceeds to step S336 according to branch B.

[0444] In step S336, the slave-side SerDes device 612 performs the Sequence A_Read_Data (when SerDes (Slave)) process described later with reference to FIG. 57. Note that FIG. 57 describes the Sequence A_Read_Data (when AP) process executed in the application processor 614, but the slave-side SerDes device 612 can also execute similar processes with corresponding blocks.

[0445] In step S337, from the extended DT of the extended packet header ePH of the slave-side SerDes device 612, the A-PHY processing unit 631 adds an A-PHY header and an A-PHY footer via the CSI2-FS processing unit 633 and the CSIA processing unit 632 to perform A-PHY transfer.

[0446] In step S338, the master-side SerDes device 613 performs the Sequence B (when SerDes (Master)) process described later with reference to FIG. 56. Note that FIG. 56 describes the Sequence B (when SerDes (Slave)) process executed in the slave-side SerDes device 612, but the master-side SerDes device 613 can also execute similar processes with corresponding blocks.

[0447] In step S339, the application processor 614 performs the Sequence A_Read_Data (when AP) process described later with reference to FIG. 57.

[0448] In step S340, the application processor 614 performs the Sequence B (when AP) process described later with reference to FIG. 56. Note that FIG. 56 describes the Sequence B (when SerDes (Slave)) process executed in the slave-side SerDes device 612, but the application processor 614 can also execute similar processes with corresponding blocks.

[0449] In step S341, in the application processor 614, the CCI-FS processing unit 653 stores read data at the address of the register 654 from the contents of the extended packet header ePH and the extended packet footer ePF.

[0450] In step S342, the above-described read process is performed by the image sensor 611, the slave-side SerDes device 612, the master-side SerDes device 613, and the application processor 614 to check the Error register.

[0451] In step S343, the image sensor 611 and each device (the slave-side SerDes device 612, the master-side SerDes device 613, and the application processor 614) determine whether the register value of the Error register of each CCI-FS processing unit is 0.

[0452] In step S343, if it is determined that the register values of all CCI-FS processing units are not 0 (there is a register value other than 0 in any of them), the process proceeds to step S344.

[0453] In step S344, the Error-related register value of the CCI-FS processing unit whose register value is not 0 is checked, and the Error register is cleared by one write and retransmission processing is performed.

[0454] On the other hand, in step S343, if it is determined that the register values of all CCI-FS processing units are 0, or after the processing of step S344, the process ends.

[0455] FIG. 53 is a flowchart for explaining the Sequence A_Write (at AP) process performed in step S321 of FIG. 51. In FIG. 53, the process performed by the application processor 614 is described as an example, but the Sequence A_Write (at SerDes (Slave)) process in step S325 of FIG. 51 is performed in the same manner.

[0456] In step S351, in the application processor 614, the I2C / I3C master 651 issues a start command and a slave address (Slave Address+W 8-bit shown in FIG. 42).

[0457] In step S352, in the application processor 614, the I2C / I3C master 651 determines whether it has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613. In step S352, if it is determined that an ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S353.

[0458] In step S353, in the application processor 614, the I2C / I3C master 651 issues a register address (Register Address [15:8] shown in FIG. 42). Here, each time the process of step S353 is repeated, as shown in FIG. 42, the payload below this register address is transmitted.

[0459] In step S354, in the application processor 614, the I2C / I3C master 651 determines whether it has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613. In step S354, if it is determined that an ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S355.

[0460] In step S355, in the application processor 614, the I2C / I3C master 651 determines whether the transfer of the final data has been completed. In step S355, if it is determined that the transfer of the final data has not been completed, the process returns to step S353, and the following similar process is repeated.

[0461] On the other hand, in step S355, if it is determined that the transfer of the final data has been completed, the process proceeds to step S356. In step S356, in the application processor 614, the I2C / I3C master 651 issues a stop command. As a result, the Sequence A_Write (at AP) process ends, and the process returns to step S322 in FIG. 51.

[0462] On the other hand, in step S352 or S354, if it is determined that the ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613 has not been received, the process proceeds to step S357. In step S357, in the application processor 614, the I2C / I3C master 651 issues a stop command. In this case, the Sequence A_Write (at AP) process ends and the communication process itself ends.

[0463] FIG. 54 is a flowchart for explaining the Sequence A_Read_CMD (at AP) process performed in step S331 of FIG. 52. In FIG. 54, the process performed by the application processor 614 is described as an example, but the Sequence A_Read_CMD (at SerDes (Slave)) process in step S335 of FIG. 52 is performed in the same manner.

[0464] In step S361, in the application processor 614, the I2C / I3C master 651 issues a start command and a slave address (Slave Address+W 8-bit shown in FIG. 42) and starts a timer.

[0465] In step S362, in the application processor 614, the I2C / I3C master 651 determines whether it has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613. In step S362, if it is determined that the ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613 has been received, the process proceeds to step S363.

[0466] In step S363, in the application processor 614, the I2C / I3C master 651 issues a register address (Register Address [15:8] shown in FIG. 42). Here, each time the process of step S363 is repeated, as shown in FIG. 42, the transmission of the payload below this register address is transmitted.

[0467] In step S364, in the application processor 614, the I2C / I3C master 651 determines whether it has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613.

[0468] In step S364, if it is determined that an ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S365.

[0469] In step S365, in the application processor 614, the I2C / I3C master 651 determines whether the transfer of the final data has been completed.

[0470] In step S365, if it is determined that the transfer of the final data has been completed, the process proceeds to step S366.

[0471] In step S366, in the application processor 614, the I2C / I3C master 651 issues a stop command. Thereafter, the process branches into two, and according to branch A, the process proceeds to step S332 in FIG. 52. On the other hand, according to branch B, after the Sequence C (at AP) process (see FIG. 55 described later) is performed in step S367, the process proceeds to step S339 in FIG. 52.

[0472] On the other hand, in step S365, if it is determined that the transfer of the final data has not been completed, the process proceeds to step S368.

[0473] In step S368, the application processor 614 determines whether the timer started in step S361 has timed out by the I2C / I3C master 651. If it is determined in step S368 that the timer has not timed out, the process returns to step S363, and the following similar processes are repeated.

[0474] On the other hand, if it is determined in step S368 that the timer has timed out, the process proceeds to step S369.

[0475] In step S369, the application processor 614 sets 1 in the Error register (Timeout) and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related register.

[0476] After the process of step S369, or if it is determined in step S362 or S364 that no ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S370.

[0477] In step S370, the application processor 614 issues a stop command by the I2C / I3C master 651. In this case, the Sequence A_Read_CMD (at AP) process ends and the communication process itself ends.

[0478] FIG. 55 is a flowchart for explaining the Sequence C (at AP) process performed in step S367 of FIG. 54. In FIG. 55, the process performed by the application processor 614 is described as an example, but the same process can also be performed in the slave-side SerDes device 612.

[0479] In step S381, in the application processor 614, the I2C / I3C master 651 determines whether the timer started in step S361 of FIG. 54 has timed out, and the process waits until it is determined that the timer has timed out. In step S381, when it is determined that the timer has timed out, the process proceeds to step S382, and in the application processor 614, the I2C / I3C master 651 performs a polling operation.

[0480] In step S383, in the application processor 614, the I2C / I3C master 651 determines whether the Status register value of the read command is 1.

[0481] In step S383, when it is determined that the Status register value of the read command is 1, the process proceeds to step S384. In step S384, after the application processor 614 performs a read access, the process returns to step S339 of FIG. 52.

[0482] On the other hand, in step S383, when it is determined that the Status register value of the read command is not 1 (other than 1), the process proceeds to step S385. In step S385, the application processor 614 sets 1 in the Error register (Timeout) and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related register.

[0483] In step S386, in the application processor 614, the I2C / I3C master 651 issues a stop command. In this case, the Sequence C (at AP) process ends and the communication process itself ends.

[0484] FIG. 56 is a flowchart for explaining the Sequence B (when SerDes (Slave)) process performed in steps S324 and S334 of FIG. 51. In FIG. 56, the process performed by the SerDes device 612 on the slave side is described as an example. However, the Sequence B (when SerDes (Master)) in step S322 of FIG. 51, the Sequence B (when Image Sensor) process in step S326 of FIG. 51, and the Sequence B (when SerDes (Master)) process in step S332 of FIG. 52 are also performed in the same manner.

[0485] In step S391, in the SerDes device 612 on the slave side, the CCI-FS processing unit 636 checks the Source ID of the SerDes device 612 on the slave side and the Destination SID of the extended packet header ePH.

[0486] In step S392, in the SerDes device 612 on the slave side, the CCI-FS processing unit 636 determines whether the Source ID of the SerDes device 612 on the slave side and the Destination SID of the extended packet header ePH do not match.

[0487] If it is determined in step S392 that the Source ID of the SerDes device 612 on the slave side and the Destination SID of the extended packet header ePH do not match, the process proceeds to step S393.

[0488] In step S393, in the SerDes device 612 on the slave side, the CCI-FS processing unit 636 checks the Destination SID of the SerDes device 612 on the slave side and the Destination SID of the extended packet header ePH.

[0489] In step S394, in the slave-side SerDes device 612, the CCI-FS processing unit 636 determines whether the Source ID of the slave-side SerDes device 612 matches the Destination SID of the extended packet header ePH.

[0490] In step S394, if it is determined that the Source ID of the slave-side SerDes device 612 matches the Destination SID of the extended packet header ePH, the process proceeds to step S395.

[0491] In step S395, in the slave-side SerDes device 612, the CCI-FS processing unit 636 checks the Message Counter from the content of the extended packet header ePH.

[0492] In step S396, in the slave-side SerDes device 612, the CCI-FS processing unit 636 determines whether the Message Counter in the slave-side SerDes device 612 matches the received value of the Message Counter checked from the content of the extended packet header ePH.

[0493] In step S396, if it is determined that the Message Counter in the slave-side SerDes device 612 matches the received value of the Message Counter checked from the content of the extended packet header ePH, the process proceeds to step S397.

[0494] In step S397, in the slave-side SerDes device 612, the CCI-FS processing unit 636 checks the CRC calculation result calculated from the extended packet header ePH in the slave-side SerDes device 612 and the received value (ePF0) of the extended packet footer ePF.

[0495] In step S398, it is determined whether the received value (ePF0) of the extended packet footer ePF matches the CRC calculation result. If it is determined that they match, the process returns to step S325 in FIG. 51.

[0496] On the other hand, in step S392, if it is determined that the Source ID of the slave-side SerDes device 612 and the Destination SID of the extended packet header ePH do not mismatch (match), the process proceeds to step S399.

[0497] In steps S399 to S402, the same processing as in steps S395 to S398 is performed.

[0498] In step S402, if it is determined that the received value (ePF0) of the extended packet footer ePF matches the CRC calculation result, the process proceeds to step S403. In step S403, a write access is performed to the register 637 of the slave-side SerDes device 612.

[0499] In step S394, if it is determined that the Source ID of the slave-side SerDes device 612 and the Destination SID of the extended packet header ePH do not match, the process proceeds to step S404. In step S404, in the slave-side SerDes device 612, the CCI-FS processing unit 636 sets 1 in the Error register [2] (Routing) and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related register.

[0500] In step S398 or S402, if it is determined that the received value (ePF0) of the extended packet footer ePF does not match the CRC calculation result, the process proceeds to step S405. In step S405, in the slave-side SerDes device 612, the CCI-FS processing unit 636 sets 1 in the Error register (CRC) and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related register.

[0501] In step S396 or S400, if it is determined that the Message Counter in the slave-side SerDes device 612 does not match the received value of the Message Counter confirmed from the content of the extended packet header ePH, the process proceeds to step S406. In step S406, in the slave-side SerDes device 612, the CCI-FS processing unit 636 sets 1 in the Error register (MC) and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related register.

[0502] After the processing of steps S403 to S406, the Sequence B (during SerDes (Slave)) processing ends and the communication process itself ends.

[0503] Note that for CRC calculation, it is also possible to perform it only for E2E Protection, detect errors in each device respectively, and discard / not discard packets. Combinations of these are assumed.

[0504] FIG. 57 is a flowchart for explaining the Sequence A_Read_Data (during AP) process performed in step S339 of FIG. 52. In FIG. 57, the process performed by the application processor 614 is described as an example, but the Sequence A_Read_Data (during SerDes (Slave)) process in step S336 of FIG. 52 is also performed in the same manner.

[0505] In step S411, in the application processor 614, the I2C / I3C master 651 issues a start command and a slave address (Slave Address+W 8-bit shown in FIG. 48).

[0506] In step S412, the application processor 614 determines whether the I2C / I3C master 651 has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613. If it is determined in step S412 that an ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S413.

[0507] In step S413, the application processor 614 issues a start command and a slave address (Slave Address+R 8-bit shown in FIG. 48) by the I2C / I3C master 651 and starts a timer.

[0508] In step S414, the application processor 614 determines whether the I2C / I3C master 651 has received an ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613. If it is determined in step S414 that an ACK response has been received from the I2C / I3C slave 644 of the master-side SerDes device 613, the process proceeds to step S415.

[0509] In step S415, the application processor 614 acquires read data from the opposing I2C / I3C slave 644 on the application processor 614 side by the I2C / I3C master 651.

[0510] In step S416, it is determined whether the I2C / I3C master 651 of the application processor 614 has transmitted an ACK and whether the opposing I2C / I3C slave 644 on the application processor 614 side has received the ACK.

[0511] In step S416, if the I2C / I3C master 651 of the application processor 614 performs ACK transmission and it is determined that ACK reception is performed by the opposing I2C / I3C slave 644 on the application processor 614 side, the process proceeds to step S417.

[0512] In step S417, it is determined whether the I2C / I3C master 651 of the application processor 614 performs NACK transmission as the transfer of the final data is completed.

[0513] In step S417, if it is determined that NACK transmission is performed, the process proceeds to step S418. In step S418, in the application processor 614, the I2C / I3C master 651 issues a stop command. As a result, the Sequence A_Read_Data (at AP) process ends, and the process returns to step S340 in FIG. 52.

[0514] On the other hand, in step S417, if it is determined that NACK transmission is not performed, the process proceeds to step S419.

[0515] In step S419, in the application processor 614, the I2C / I3C master 651 determines whether the timer started in step S413 has timed out. In step S419, if it is determined that the timer has not timed out, the process returns to step S415, and the following similar processes are repeatedly performed.

[0516] On the other hand, in step S419, if it is determined that the timer has timed out, the process proceeds to step S420.

[0517] In step S420, the application processor 614 sets 1 in the Error register (Timeout) and stores the data of the extended packet header ePH and the extended packet footer ePF in the Error-related register.

[0518] After the process of step S420, or when it is determined in step S414 that the ACK response from the I2C / I3C slave 644 of the master-side SerDes device 613 has not been received, the process proceeds to step S421. Similarly, in step S416, when it is determined that the I2C / I3C master 651 of the application processor 614 has not sent an ACK, or that the ACK has not been received by the opposing I2C / I3C slave 644 on the application processor 614 side, the process proceeds to step S421.

[0519] In step S421, in the application processor 614, the I2C / I3C master 651 issues a stop command. In this case, the Sequence A_Read_Data (at AP) process ends and the communication process itself ends.

[0520] Here, when the I2C / I3C slave 621 outputs (see Fig. 46), there are three combinations as described below for the access timing from the I2C / I3C master 634 to the I2C / I3C slave 621, and when the I2C / I3C slave 644 of the master-side SerDes device 613 outputs (see Fig. 48), for the access timing from the I2C / I3C master 651 to the I2C / I3C slave 644.

[0521] The first access timing is to poll until the read data is acquired, and after the preparation for reading the read data is complete, the I2C / I3C master starts the read process.

[0522] The second access timing is that after a certain period of time has elapsed, the I2C / I3C master starts the read process.

[0523] The third access timing uses the Clock Stretch method (refer to FIG. 72 described later). After a certain period of time, the I2C / I3C master starts the read process. At this time, there are two forms: sending read data in chunks and sending read data individually (asserting the Clock Stretch Mode signal).

[0524] <Configuration example of the extended packet header ePH> FIGS. 58 to 60 are diagrams showing a configuration example of the extended packet header ePH.

[0525] FIG. 58 shows detailed configuration examples of the extended packet header ePH0, the extended packet header ePH1, and the extended packet header ePH2. The addition of the extended packet header ePH as shown is defined by reusing the ePH structure in C-PHY and D-PHY for the content of the extended packet header ePH for CCI-FS.

[0526] FIG. 59 shows a detailed configuration example of the extended packet header ePH3. The addition of the extended packet header ePH as shown is defined for the content of the extended packet header ePH for CCI-FS.

[0527] FIG. 60 shows a detailed configuration example of the extended DT of the extended packet header ePH. For example, in order to support CCI-FS, "0xC0:For I2C" and "0xC1:For I3C" are added to the data type of the extended packet header ePH.

[0528] <Example of I2C circuit configuration> FIG. 61 shows a configuration example of a conventional I2C in hardware. For example, it shows a configuration example of I2C in the case of a bus connection configuration at the upper level during hardware implementation. The slave side may be configured to receive AKC / NACK from the upper level. Of course, it is just an example shown, and the upper bus configuration does not necessarily match.

[0529] FIG. 62 shows waveforms during data transfer on the I2C bus. Note that the I2C bus standard and CCI (I2C) are equivalent.

[0530] FIG. 63 is a block diagram showing a configuration example related to CCI in a communication system 701 with an A-PHY direct connection configuration, similar to the communication system 501 shown in FIG. 27 described above.

[0531] As shown in FIG. 63, in the communication system 701, an image sensor 711 and an application processor 712 are directly connected by an A-PHY.

[0532] The image sensor 711 includes an A-PHY processing unit 721, a CSIA processing unit 722, a CSI2 processing unit 523, a CSI2-FS processing unit 724, a CCI processing unit 725, a CCI-FS processing unit 726, a register 727, and selectors 728-1 and 728-2. As shown in the figure, the selectors 728-1 and 728-2 are arranged so as to sandwich the CCI-FS processing unit 726, and can switch the enable / disable of the CCI-FS processing unit 726 according to the CCI_FS_Enable signal of the register 727.

[0533] The application processor 712 includes an A-PHY processing unit 731, a CSIA processing unit 732, a CSI2 processing unit 733, a CSI2-FS processing unit 734, a CCI processing unit 735, a CCI-FS processing unit 736, a register 737, and selectors 738-1 and 738-2. As shown in the figure, the selectors 738-1 and 738-2 are arranged so as to sandwich the CCI-FS processing unit 736, and can switch the enable / disable of the CCI-FS processing unit 736 according to the CCI_FS_Enable signal of the register 737.

[0534] For example, when the CCI_FS_Enable signal indicates enabling CCI-FS (CCI_FS_Enable = 1), as shown by the arrow of the dashed line, data is transmitted and received via the CCI-FS processing unit 726 and the CCI-FS processing unit 736. On the other hand, when the CCI_FS_Enable signal indicates disabling CCI-FS (CCI_FS_Enable = 0), as shown by the arrow of the dotted line, data is transmitted and received without passing through the CCI-FS processing unit 726 and the CCI-FS processing unit 736.

[0535] <Network connection form> FIG. 64 shows an example of the connection form (topology) of a network in an A-PHY direct connection configuration and a SerDes connection configuration.

[0536] The application processor 801 is directly connected to the image sensor 802 via A-PHY, and the image sensor 802 can form a connection configuration in which it is connected to the sensor 803 via I2C / I3C.

[0537] The application processor 801 is connected to the master-side SerDes device 804 via I2C / I3C, and the master-side SerDes device 804 and the slave-side SerDes device 805 are connected via A-PHY. The slave-side SerDes device 805 can form a connection configuration in which it is connected to two sensors 806-1 and 806-2 via I2C / I3C.

[0538] <Circuit configuration of CCI-FS processing unit> FIG. 65 is a block diagram showing an example of the circuit configuration of the CCI-FS processing unit. The CCI-FS processing unit 901 and the register 902 shown in FIG. 65 have a common configuration with the CCI-FS processing unit and the register provided in each of the above-described devices.

[0539] As shown in FIG. 65, in the CCI-FS processing unit 901, CCI-FS switches, registers, etc. are provided in the upper layer, and a CCI processing unit is provided in the lower layer. The CCI-FS processing unit 901 includes a CCI-FS transmission unit 911 and a CCI-FS reception unit 912. Various register setting value information is supplied from the register 902 to the CCI-FS processing unit 901, and an Error notification is supplied from the CCI-FS processing unit 901 to the register 902.

[0540] The CCI-FS transmission unit 911 includes an extended packet header ePH generation unit 921, an extended packet footer ePF generation unit 922, and a Destination Address confirmation unit 923.

[0541] The extended packet header ePH generation unit 921 has an MC generation unit 941 that generates a Message Counter and a Packet Length calculation unit 942 that calculates the packet length. The extended packet footer ePF generation unit 922 has an extended packet footer ePF1 generation unit 943 that generates an extended packet footer ePF1 and a CRC calculation unit 944 that calculates the CRC stored in the extended packet footer ePF0.

[0542] The CCI-FS reception unit 912 includes an extended packet header ePH confirmation unit 931, an extended packet footer ePF confirmation unit 932, and a Destination Address confirmation unit 933.

[0543] The extended packet header ePH confirmation unit 931 has an MC confirmation unit 951 that confirms the Message Counter and a Packet Length calculation / confirmation unit 952 that calculates and confirms the packet length. The extended packet footer ePF confirmation unit 932 has an extended packet footer ePF1 confirmation unit 953 that confirms the extended packet footer ePF1 and a CRC calculation unit 954 that calculates the CRC stored in the extended packet footer ePF0.

[0544] The CCI-FS processing unit 901 can, through the CCI-FS transmission unit 911, check the Destination Address of the data from the upper layer, generate the extended packet header ePH and the extended packet footer ePF, add them to the data, and supply the data to the lower layer. The CCI-FS processing unit 901 can, through the CCI-FS reception unit 912, check the Destination Address of the data from the lower layer, check the extended packet header ePH and the extended packet footer ePF, and supply the data to the upper layer.

[0545] Here, the operations of the CCI-FS processing units of the respective devices constituting the communication system 601 with the configuration example of the SerDes connection configuration shown in FIG. 40 described above will be described.

[0546] The application processor 614 has a Source ID indicating its own device in the extended packet header ePH in the application processor 614. Then, the CCI-FS processing unit 653 adds the above information and the information having a Destination ID indicating the device to which the access is to be made.

[0547] The slave-side SerDes device 612 and the master-side SerDes device 613 have a Source ID indicating their own devices either by being preset or as unique values. The CCI-FS processing unit 636 and the CCI-FS processing unit 646 preset the above information and the information having a Destination ID indicating the connection device and the device to which the access is to be made.

[0548] Also, the CCI-FS processing unit 636 and the CCI-FS processing unit 646 compare the received Destination ID in the extended packet header ePH with their own ID (Source ID), and determine whether it is an access to themselves or an indication of the target device (Destination ID). For example, when the Destination ID in the received extended packet header ePH matches their own ID (Source ID), they perform their own register access as an access to the intermediate device (SerDes device). On the other hand, when the Destination ID in the received extended packet header ePH does not match their own ID (Source ID), they perform data transfer toward the connected device (Destination ID) as an access to the subsequent device.

[0549] As described above, based on the Source ID and Destination ID embedded in the extended packet header ePH, the Source ID with a preset or unique value, and the preset destination information in the intermediate device (SerDes device) or the target device, data is transferred and access is performed toward the target device.

[0550] When the Destination ID in the received extended packet header ePH matches the ID (Source ID) of the CSI2-FS processing unit 623 of the image sensor 611, the CSI2-FS processing unit 623 of the image sensor 611 performs its own register access as an access to the image sensor 611.

[0551] In this way, the Source ID possessed by each device can use a value unique to each device, a preset value, or a combination thereof.

[0552] Figures 66 to 68 are diagrams showing detailed configuration examples of the register 902.

[0553] Figure 66 shows the details of register 902 from address 0x000 to address 0x109. Figure 67 shows a configuration example at the time of Bridge configuration as the details of register 902 from address 0x110 to address 0x125.

[0554] Figure 68 shows an Error-related register as the details of address 0x200 of register 902. Figure 68 shows an Error-related register (debug) as the details of addresses 0x300 and 0x400 of register 902. Figure 68 shows an Error Injection-related register (debug) as the details of address 0x800 of register 902.

[0555] <Modification example of extended packet header ePH> With reference to FIGS. 69 and 70, a modification example of the extended packet header ePH will be described.

[0556] As described with reference to FIG. 33 above, FIG. 69 shows a modification example of the extended packet header ePH in the packet configuration of the write data generated by the CCI-FS processing unit 536 of the application processor 512 during a write access. The extended packet header ePH shown in FIG. 69 has a configuration of the extended packet header ePH3 and the extended packet header ePH4 that is different from the configuration example shown in FIG. 33 above.

[0557] As described with reference to FIG. 28 above, FIG. 70 shows a modification example of the extended packet header ePH in the packet configuration of the write data generated by the CCI-FS processing unit 536 of the application processor 512 during a read access. The extended packet header ePH shown in FIG. 70 has a configuration of the extended packet header ePH3 and the extended packet header ePH4 that is different from the configuration example shown in FIG. 28 above.

[0558] For example, in the extended packet header ePH shown in FIGS. 69 and 70, depending on the implementation, the following combinations are assumed.

[0559] The Read address information may be stored in the extended packet header ePH or in the AP(CCI) payload. The Length information may be stored in the extended packet header ePH or in the AP(CCI) payload. The information of CMD may be stored in the CCI Command ID of the extended packet header ePH. Based on the CCI Command ID, the start, resume, and end information of the command is referenced. The CCI information (such as Slave Address, etc.) may be stored in the AP(CCI) payload using the CCI Header Length. The CCI Header Length is information indicating the header length of the CCI protocol (I2C).

[0560] FIG. 71 is a diagram for explaining the flow between the image sensor 511 and the application processor 512 in the A-PHY direct connection configuration as shown in FIG. 27.

[0561] In the application processor 512, the CCI-FS switch 538 issues read commands and write commands. The CCI-FS switch 538 supplies the slave address (Slave Address+W 8bit), the register address (Register Address[15:8], Register Address[7:0]), and the data (Data*(*=N)[7:0]) to the CCI processing unit 535. The CCI processing unit 535 converts them into the AP(CCI) payload and supplies it to the A-PHY processing unit 531. The A-PHY processing unit 531 adds the A-PHY header and the A-PHY footer to the AP(CCI) payload and performs an A-PHY transfer to the image sensor 511.

[0562] In the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and A-PHY footer and supplies the AP(CCI) payload to the CCI processing unit 525. The CCI processing unit 525 converts the AP(CCI) payload, and based on its content, writes data to the register 527 according to the write command and reads data from the register 527 according to the read command.

[0563] At this time, the initial setting of CCI-FS Enable is performed by the CCI processing unit 525, and bus conversion such as the register interface and the AHB bus is performed. Then, the confirmation of the CCI-FS Enable setting is performed via the CCI processing unit 525 or the CCI-FS processing unit 526.

[0564] The CCI processing unit 525 converts the read data (Data*(*=M)[7:0]) read from the register 527 in response to the read command into an AP(CCI) payload and supplies it to the A-PHY processing unit 521. The A-PHY processing unit 521 adds an A-PHY header and an A-PHY footer to the AP(CCI) payload and performs an A-PHY transfer to the application processor 512.

[0565] In the application processor 512, the A-PHY processing unit 531 removes the A-PHY header and A-PHY footer and supplies the AP(CCI) payload to the CCI processing unit 535. The CCI processing unit 535 converts the AP(CCI) payload and supplies the read data (Data*(*=M)[7:0]) to the CCI-FS switch 538.

[0566] The CCI-FS switch 538 performs CCI-FS Enable setting and various CCI-FS related register settings on the register 537. At this time, the register access depends on the implementation. The CCI-FS switch 538 performs various CCI-FS related register settings on the register 527 via the register 537, the CCI-FS processing unit 536, the A-PHY processing unit 531, the A-PHY processing unit 521, and the CCI-FS processing unit 526.

[0567] In the application processor 512, the CCI-FS switch 538 issues a read command. The CCI-FS switch 538 supplies a slave address (Slave Address+W 8bit), a register address (Register Address[15:8], Register Address[7:0]), and data (Data*(*=N)[7:0]) to the register 537. The CCI-FS processing unit 536 converts them into an AP (CCI) payload, adds an extended packet header ePH*(*=n), an extended packet footer ePF1, and an extended packet footer ePF0, and supplies them to the A-PHY processing unit 531. The A-PHY processing unit 531 adds an A-PHY header and an A-PHY footer to them and performs an A-PHY transfer to the image sensor 511.

[0568] In the image sensor 511, the A-PHY processing unit 521 removes the A-PHY header and the A-PHY footer and supplies an extended packet header ePH*(*=n), an AP (CCI) payload, an extended packet footer ePF1, and an extended packet footer ePF0 to the CCI-FS processing unit 526. The CCI-FS processing unit 526 converts the AP (CCI) payload, reads data from the register 527 according to the read command based on its content. At this time, the register access depends on the implementation, and bus conversions such as a register interface, an AHB bus, and a CCI interface are performed.

[0569] The CCI-FS processing unit 526 converts the read data (Data*(*=M)[7:0]) read from the register 527 according to the read command into an AP (CCI) payload, adds an extended packet header ePH*(*=n), an extended packet footer ePF1, and an extended packet footer ePF0, and supplies them to the A-PHY processing unit 521. The A-PHY processing unit 521 adds an A-PHY header and an A-PHY footer to them and performs an A-PHY transfer to the application processor 512.

[0570] In the application processor 512, the A-PHY processing unit 531 removes the A-PHY header and A-PHY footer, and supplies the extended packet header ePH*(*=n), the AP(CCI) payload, the extended packet footer ePF1, and the extended packet footer ePF0 to the CCI-FS processing unit 536. The CCI-FS processing unit 536 converts the AP(CCI) payload and supplies the read data (Data*(*=M)[7:0]) to the CCI-FS switch 538.

[0571] Note that the above flow was described by taking the generation of I2C / I3C commands in hardware as an example, and there are other combinations as follows.

[0572] In the case of software, as the generation of I2C / I3C in software, the Slave Address, Register address, Payload, ACK response reception, transmission, and various control codes (S, Sr, ACK, NACK, P) are generated in software (for example, the image of GPIO control). As the generation of I2C / I3C commands in software, in response to ACK reception with the CPU bus setting, the CPU issues the Slave Address, Register address, and Payload.

[0573] In the case of hardware, as the generation of I2C / I3C in hardware, transfer settings and data are set in the I2C / I3C HW IP. Various control codes are automatically responded to by the hardware. As the generation of I2C / I3C commands in hardware, data is set in the I2C / I3C HW IP with transfer settings, and transmission is performed with commands. Various control codes are automatically responded to by the hardware.

[0574] FIG. 72 is a diagram for explaining a flow using the Clock Stretch method in the Write access and Read access between the image sensor 611 and the application processor 614 in the SerDes connection configuration as shown in FIG. 40.

[0575] The CCI-FS switch 655 of the application processor 614 supplies the start command and the write command (Slave Address+W 8bit) to the CCI processing unit 645 of the SerDes device 613 on the master side and asserts the Scl_enb signal. In the SerDes device 613 on the master side, the CCI processing unit 645 supplies the write command to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the write command and performs an A-PHY transfer to the SerDes device 612 on the slave side.

[0576] In the SerDes device 612 on the slave side, the A-PHY processing unit 631 removes the A-PHY header and the A-PHY footer and supplies the write command to the CCI processing unit 635 (Slave). The CCI processing unit 635 (Slave) negates the Scl_enb signal and supplies the write command to the CCI processing unit 635 (Master). Here, the CCI processing unit 635 that communicates with the SerDes device 613 on the master side and functions as a slave is referred to as the CCI processing unit 635 (Slave), and the CCI processing unit 635 that communicates with the image sensor 611 side and functions as a master is referred to as the CCI processing unit 635 (Master).

[0577] The CCI processing unit 635 (Master) transmits the start command and the write command to the image sensor 611.

[0578] In the image sensor 611, the CCI processing unit 622 receives the start command and the write command and supplies them to the CSI2-FS processing unit 623. The CSI2-FS processing unit 623 supplies an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 transmits the ACK response to the SerDes device 612 on the slave side.

[0579] In the SerDes device 612 on the slave side, when the CCI processing unit 635 (Master) receives an ACK response and the Scl_enb signal is negated from the CCI processing unit 635 (Slave), the ACK response is supplied to the CCI-FS processing unit 636. After that, the CCI processing unit 635 (Slave) asserts the Scl_enb signal to the CCI processing unit 635 (Master).

[0580] The CCI-FS processing unit 636 supplies the ACK response to the A-PHY processing unit 631. The A-PHY processing unit 631 adds an A-PHY header and an A-PHY footer to the ACK response and performs an A-PHY transfer to the SerDes device 613 on the master side.

[0581] In the SerDes device 613 on the master side, the A-PHY processing unit 641 removes the A-PHY header and the A-PHY footer and supplies the ACK response to the CCI processing unit 645. When the CCI-FS switch 655 of the application processor 614 negates the Scl_enb signal to the CCI processing unit 645, the CCI processing unit 645 transmits the ACK response to the application processor 614.

[0582] In the application processor 614, the CCI processing unit 652 receives the ACK response and supplies it to the CCI-FS switch 655 via the CCI-FS processing unit 653.

[0583] The CCI-FS switch 655 of the application processor 614 supplies the register address (Register Address[7:0]) to the CCI processing unit 645 of the SerDes device 613 on the master side and asserts the Scl_enb signal. In the SerDes device 613 on the master side, the CCI processing unit 645 supplies the register address to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the register address and performs an A-PHY transfer to the SerDes device 612 on the slave side.

[0584] In the SerDes device 612 on the slave side, the A-PHY processing unit 631 removes the A-PHY header and A-PHY footer and supplies the register address to the CCI processing unit 635 (Slave). The CCI processing unit 635 (Slave) negates the Scl_enb signal and supplies the register address to the CCI processing unit 635 (Master). The CCI processing unit 635 (Master) transmits the register address to the image sensor 611. After that, the CCI processing unit 635 (Slave) asserts the Scl_enb signal to the CCI processing unit 635 (Master).

[0585] In the image sensor 611, the CCI processing unit 622 receives the register address and supplies it to the CSI2-FS processing unit 623. The CSI2-FS processing unit 623 supplies an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 transmits the ACK response to the SerDes device 612 on the slave side.

[0586] After that, in the same manner as the above-described processing, the ACK response is supplied up to the CCI-FS switch 655.

[0587] In the application processor 614, the CCI-FS processing unit 653 transmits an extended packet header ePH* (* = n) to the SerDes device 613 on the master side according to the control of the CCI-FS switch 655.

[0588] In the SerDes device 613 on the master side, the CCI processing unit 645 receives the extended packet header ePH* (* = n), and when the Scl_enb signal is asserted from the CCI-FS switch 655, supplies the extended packet header ePH* (* = n) to the A-PHY processing unit 641. After that, the CCI-FS switch 655 negates the Scl_enb signal to the CCI processing unit 645. The A-PHY processing unit 641 adds an A-PHY header and A-PHY footer to the extended packet header ePH* (* = n) and performs an A-PHY transfer to the SerDes device 612 on the slave side.

[0589] In the SerDes device 612 on the slave side, the A-PHY processing unit 631 removes the A-PHY header and A-PHY footer and supplies the extended packet header ePH*(*=n) to the CCI-FS processing unit 636. The CCI-FS processing unit 636 negates the Scl_enb signal and supplies the extended packet header ePH*(*=n) to the CCI processing unit 635 (Master). The CCI processing unit 635 (Master) transmits the extended packet header ePH*(*=n) to the image sensor 611. After that, the CCI processing unit 635 (Slave) asserts the Scl_enb signal to the CCI processing unit 635 (Master).

[0590] In the image sensor 611, the CSI2-FS processing unit 623 receives the extended packet header ePH*(*=n). The CSI2-FS processing unit 623 supplies an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 transmits the ACK response to the SerDes device 612 on the slave side.

[0591] After that, in the same manner as the above-described processing, the ACK response is supplied to the CCI-FS switch 655.

[0592] The CCI-FS switch 655 of the application processor 614 supplies the write data (Dara0[7:0]) to the CCI processing unit 645 of the SerDes device 613 on the master side and asserts the Scl_enb signal. In the SerDes device 613 on the master side, the CCI processing unit 645 supplies the write data to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds the A-PHY header and A-PHY footer to the write data and performs A-PHY transfer to the SerDes device 612 on the slave side.

[0593] In the master-side SerDes device 613, when the CCI processing unit 645 receives write data and the Scl_enb signal is asserted from the CCI-FS switch 655, it supplies the write data to the A-PHY processing unit 641. Thereafter, the CSI2-FS processing unit 653 negates the Scl_enb signal to the CCI processing unit 645 according to the control of the CCI-FS switch 655. The A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the write data and performs an A-PHY transfer to the slave-side SerDes device 612.

[0594] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and the A-PHY footer and supplies the write data to the CCI processing unit 635. The CCI processing unit 635 negates the Scl_enb signal and supplies the write data to the CCI processing unit 635 (Master). The CCI processing unit 635 (Master) transmits the write data to the image sensor 611. Thereafter, the CCI processing unit 635 (Slave) asserts the Scl_enb signal to the CCI processing unit 635 (Master).

[0595] In the image sensor 611, the CCI processing unit 622 receives the write data and supplies it to the CSI2-FS processing unit 623, and the CSI2-FS processing unit 623 writes the write data to the register 624. The CSI2-FS processing unit 623 supplies an ACK response indicating successful writing of the write data to the CCI processing unit 622, and the CCI processing unit 622 transmits the ACK response to the slave-side SerDes device 612.

[0596] Thereafter, in the same manner as the above-described processing, the ACK response is supplied up to the CCI-FS switch 655.

[0597] In the application processor 614, the CCI-FS processing unit 653 transmits an extended packet footer ePF0 to the master-side SerDes device 613 according to the control of the CCI-FS switch 655.

[0598] In the master-side SerDes device 613, when the CCI processing unit 645 receives the extended packet footer ePF0 and the Scl_enb signal is asserted from the CCI-FS switch 655, the CCI processing unit 645 supplies the extended packet footer ePF0 to the A-PHY processing unit 641. Then, the CCI-FS switch 655 negates the Scl_enb signal to the CCI processing unit 645. The A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the extended packet footer ePF0 and performs an A-PHY transfer to the slave-side SerDes device 612.

[0599] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and the A-PHY footer and supplies the extended packet footer ePF0 to the CCI-FS processing unit 636. The CCI-FS processing unit 636 negates the Scl_enb signal and supplies the extended packet footer ePF0 to the CCI processing unit 635 (Master). The CCI processing unit 635 (Master) transmits the extended packet footer ePF0 to the image sensor 611. Then, the CCI processing unit 635 (Slave) asserts the Scl_enb signal to the CCI processing unit 635 (Master).

[0600] In the image sensor 611, the CSI2-FS processing unit 623 receives the extended packet footer ePF0. The CSI2-FS processing unit 623 supplies an ACK response indicating successful reception to the CCI processing unit 622, and the CCI processing unit 622 transmits the ACK response to the slave-side SerDes device 612.

[0601] After that, in the same manner as the above-described processing, the ACK response is supplied up to the CCI-FS switch 655.

[0602] The CCI-FS switch 655 of the application processor 614 supplies a repeat start command and a read command (Slave Address+R 8bit) to the CCI processing unit 645 of the master-side SerDes device 613 and asserts the Scl_enb signal. In the master-side SerDes device 613, the CCI processing unit 645 supplies the read command to the A-PHY processing unit 641, and the A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the read command and performs an A-PHY transfer to the slave-side SerDes device 612.

[0603] In the slave-side SerDes device 612, the A-PHY processing unit 631 removes the A-PHY header and the A-PHY footer and supplies the read command to the CCI processing unit 635 (Slave). The CCI processing unit 635 (Slave) negates the Scl_enb signal and supplies the read command to the CCI processing unit 635 (Master). The CCI processing unit 635 (Master) transmits the repeat start command and the read command to the image sensor 611.

[0604] In the image sensor 611, the CCI processing unit 622 receives the repeat start command and the read command and accesses the register 624. The CCI processing unit 622 transmits an ACK response indicating the success of the reception to the slave-side SerDes device 612.

[0605] After that, in the same manner as the above-described processing, the ACK response is supplied up to the CCI-FS switch 655.

[0606] In the image sensor 611, the CCI processing unit 622 reads out read data (Data0[7:0]) from the register 624 and transmits it to the slave-side SerDes device 612.

[0607] In the SerDes device 612 on the slave side, the CCI processing unit 635 (Master) receives the read data and supplies it to the CCI processing unit 635 (Slave), and the CCI processing unit 635 (Slave) supplies the read data to the A-PHY processing unit 631. The A-PHY processing unit 631 adds an A-PHY header and an A-PHY footer to the read data and performs an A-PHY transfer to the SerDes device 613 on the master side.

[0608] In the SerDes device 613 on the master side, the A-PHY processing unit 641 removes the A-PHY header and the A-PHY footer and supplies the read data to the CCI processing unit 645, and the CCI processing unit 645 transmits the read data to the application processor 614.

[0609] In the application processor 614, the CCI processing unit 652 receives the read data and supplies it to the CCI-FS switch 655 via the CCI-FS processing unit 653.

[0610] The CCI-FS switch 655 transmits a NACK response and a stop command to the CCI processing unit 645. The CCI processing unit 645 supplies the NACK response and the stop command to the A-PHY processing unit 641. The A-PHY processing unit 641 adds an A-PHY header and an A-PHY footer to the NACK response and the stop command and performs an A-PHY transfer to the SerDes device 612 on the slave side.

[0611] In the SerDes device 612 on the slave side, the A-PHY processing unit 631 removes the A-PHY header and the A-PHY footer and supplies the NACK response and the stop command to the CCI processing unit 635 (Slave). The CCI processing unit 635 (Slave) supplies the NACK response and the stop command to the CCI processing unit 635 (Master), and the CCI processing unit 635 (Master) transmits the NACK response and the stop command to the image sensor 611.

[0612] In the image sensor 611, the CCI processing unit 622 receives a NACK response and a stop command and supplies them to the CSI2-FS processing unit 623.

[0613] In the flow described in FIG. 72, I2C control commands such as start, repeat start, ACK response, NACK response, and stop are indicated by setting the Control Code Indicator of the extended packet header ePH0 to 1 and each code assigned to the 1-byte Payload.

[0614] <Detailed Configuration Example of Image Sensor and Application Processor> FIG. 73 is a block diagram showing a configuration example of the configuration in which the image sensor 211 shown in FIG. 25 described above includes a CCI-FS processing unit 1001. In the image sensor 211 shown in FIG. 73, the same reference numerals are given to the configurations common to the image sensor 211 in FIG. 25, and the description thereof is omitted.

[0615] As shown in FIG. 73, the CCI-FS processing unit 1001 is arranged between the CCI slave 224 and the register 47, and the MUX units 1002-1 and 1002-2 are arranged so as to sandwich the CCI-FS processing unit 1001. When the cci_fs_en signal supplied from the register 47 enables the CCI-FS processing unit 1001, the MUX units 1002-1 and 1002-2 transmit and receive data via the CCI-FS processing unit 1001. On the other hand, when the cci_fs_en signal supplied from the register 47 disables the CCI-FS processing unit 1001, the MUX units 1002-1 and 1002-2 transmit and receive data without passing through the CCI-FS processing unit 1001.

[0616] FIG. 74 is a block diagram showing a configuration example of the configuration in which the application processor 214 shown in FIG. 26 described above includes a CCI-FS processing unit 1101. In the application processor 214 shown in FIG. 74, the same reference numerals are given to the configurations common to the application processor 214 in FIG. 26, and the description thereof is omitted.

[0617] As shown in FIG. 74, the CCI-FS processing unit 1101 is arranged between the CCI master 254 and the register 73, and the MUX units 1102-1 and 1102-2 are arranged so as to sandwich the CCI-FS processing unit 1101. When the cci_fs_en signal supplied from the register 73 enables the CCI-FS processing unit 1101, the MUX units 1102-1 and 1102-2 transmit and receive data via the CCI-FS processing unit 1101. On the other hand, when the cci_fs_en signal supplied from the register 73 disables the CCI-FS processing unit 1101, the MUX units 1102-1 and 1102-2 transmit and receive data without passing through the CCI-FS processing unit 1101.

[0618] Regarding the implementation method of each field in the configuration of the extended packet header ePH, the following configuration may be adopted. · The extended VC is unused in Safe CCI. (The same configuration is used to combine the extended header related in MIPI and the header field) · In the extended DT, it may be embedded in the information related to the command of the bus from the upper part, or may be the implementation configuration of the signal line setting from the register setting. · The Protocol is described in I2C, but the same thing can be implemented in the SDR mode of I3C.

[0619] <Configuration Example of Communication System> Referring to FIGS. 75 to 117, a fourth embodiment of a communication system to which the present technology is applied will be described.

[0620] FIG. 75 is a block diagram of the communication system according to the fourth embodiment. In FIG. 75A, a communication system 1201 as a first variation is shown, and in FIG. 75B, a communication system 1201A as a second variation is shown.

[0621] The communication system 1201 shown in A of FIG. 75 is configured with an image sensor 1211 and an application processor 1212 directly connected.

[0622] In the image sensor 1211, an ALL layer 1222 is arranged on an A-PHY layer 1221, and on top of that, a CSI-2 transmitter 1223, a CSI extension 1224, a CCI slave 1225, and a CCI extension 1226 are arranged. The image sensor 1211 can support the extended standards by providing the CSI extension 1224 for the CSI-2 transmitter 1223 and the CCI extension 1226 for the CCI slave 1225, respectively.

[0623] In the application processor 1212, an ALL layer 1232 is arranged on an A-PHY layer 1231, and on top of that, a CSI-2 receiver 1233, a CSI extension 1234, a CCI master 1235, and a CCI extension 1236 are arranged. The application processor 1212 can support the extended standards by providing the CSI extension 1234 for the CSI-2 receiver 1233 and the CCI extension 1236 for the CCI master 1235, respectively. Note that the CSI extension may also be referred to as Camera Service Extensions (CSE).

[0624] The communication system 1201A shown in B of FIG. 75 is configured with a display 1213 and an application processor 1212A connected. Note that the application processor 1212A is configured with a DSI-2 transmitter 1233A and a DSI extension 1234A instead of the CSI-2 receiver 1233 and the CSI extension 1234 of the application processor 1212 in A of FIG. 75, and the other blocks have the same configuration as the application processor 1212.

[0625] The display 1213 has a configuration in which the ALL layer 1242 is arranged on the A-PHY layer 1241, and on top of that, the DSI-2 receiver 1243, the DSI extension 1244, as well as the CCI slave 1245 and the CCI extension 1246 are arranged. The display 1213 can be made to correspond to the extended standards respectively by providing the DSI extension 1244 for the DSI-2 receiver 1243 and the CCI extension 1246 for the CCI slave 1245. Note that the DSI extension may also be referred to as Display Service Extensions (DSE).

[0626] The communication systems 1201 and 1201A configured in this way can at least perform high-speed data transmission for transmitting data of a frame including image data in one direction and low-speed command transmission for transmitting commands related to the high-speed data transmission in the reverse direction. For example, in the low-speed command transmission, at least the transmission of a high-speed data transmission start command for requesting the start of high-speed data transmission is performed. Also, the high-speed data transmission is fast compared to the low-speed command transmission and is started in response to the reception of the high-speed data transmission start command.

[0627] However, in the communication system 1201 where the communication partner of the application processor 1212 is the image sensor 1211 and the communication system 1201A where the communication partner of the application processor 1212A is the display 1213, the directions of the high-speed data transmission and the low-speed command transmission are different. That is, in the communication system 1201, image data is transmitted from the image sensor 1211 to the application processor 1212, and in the communication system 1201A, image data is transmitted from the application processor 1212A to the display 1213.

[0628] In the A-PHY of the physical layer specification, high-speed data transmission and low-speed command transmission are transmitted through part or all of a common communication path. In addition, A-PHY supports an option that enables part or all of the power supply from the application processor 1212 to the image sensor 1211 and the power supply from the application processor 1212A to the display 1213 to be transmitted through a common communication path.

[0629] By the way, low-speed command transmission complies with, for example, CCI of the CSI-2 standard, and communication is performed based on the I2C or I3C standard. At this time, low-speed command transmission can transmit commands by sharing part or all of not only the independent physical layer of I2C or I3C but also part or all of any one of the physical layers of D-PHY, C-PHY, and A-PHY. On the other hand, high-speed data transmission transmits data through part or all of any one of the physical layers of D-PHY, C-PHY, and A-PHY.

[0630] In addition, when low-speed command transmission complies with, for example, Unified Serial Link (USL) within the CSI-2 standard, it is possible to transmit commands through part or all of any one of the physical layers of D-PHY or C-PHY. That is, high-speed data transmission and low-speed command transmission can be transmitted through part or all of any one of the physical layers of D-PHY, C-PHY, A-PHY, I2C, and I3C.

[0631] In addition, in FIG. 75, a configuration example including the application processors 1212 and 1201A has been described. However, the communication systems 1201 and 1201A may have a configuration including, for example, an electronic control unit (ECU). That is, it is not limited to the application processor 1212 as long as it is a processor that can communicate with the image sensor 1211, the display 1213, etc. directly or indirectly. Also, it may have a configuration including various sensors other than the image sensor 1211.

[0632] The communication systems 1201 and 1201A configured as such adopt a nonce value transmission method or an initialization vector configuration including a nonce value as described below.

[0633] Specifically, for a specific common key encryption algorithm (e.g., AES-GCM / GMAC), an initialization vector including a nonce value is required. Therefore, the initialization vector and nonce value setting rules are pre-agreed between the image sensor 1211 and the application processor 1212, or between the display 1213 and the application processor 1212A.

[0634] However, if misrecognition or forgery of the nonce value occurs inside each of the image sensor 1211, the application processors 1212 and 1201A, and the display 1213, subsequent decryption of encrypted image data and message authentication will fail. Therefore, in order to avoid the problem that the transmission of image data cannot be performed normally, countermeasure technologies for misrecognition and forgery of the nonce value were required.

[0635] On the other hand, as a new security specification for the MIPI Camera Serial Interface (CSI) standard or the MIPI Display Serial Interface (DSI) standard, it was necessary to define an initialization vector suitable for the CSI standard or the DSI standard. Therefore, the present technology discloses a nonce value transmission method or an initialization vector configuration including a nonce value suitable for an imaging device compliant with the CSI standard including the image sensor 1211, or a display device compliant with the DSI standard including the display 1213.

[0636] Note that, in the following, the processing performed between the image sensor 1211 and the application processor 1212 will be described, but the same processing can also be performed between the display 1213 and the application processor 1212A.

[0637] FIG. 76 is a block diagram showing a detailed configuration example of the image sensor 1211.

[0638] The image sensor 1211 includes a pixel 1301, an AD converter 1302, an image processing unit 1303, an extended mode-compatible CSI-2 transmission circuit 1304, a physical layer processing unit 1305, an I2C / I3C slave 1306, a storage unit 1307, a message counter 1308, a nonce update unit 1309, and a security unit 1310. Note that the pixel 1301, the AD converter 1302, the image processing unit 1303, the extended mode-compatible CSI-2 transmission circuit 1304, the physical layer processing unit 1305, the I2C / I3C slave 1306, and the storage unit 1307 are configured in the same manner as the corresponding blocks in the other embodiments described above, and detailed descriptions thereof are omitted.

[0639] The message counter 1308 updates the message count value in the image sensor 1211 each time an extended packet that satisfies a predetermined count condition is transmitted.

[0640] The security unit 1310 derives a session key in the image sensor 1211 and generates first protected data (for example, an integrity calculation value calculated to protect integrity, encrypted cipher data to protect confidentiality) of data transmitted at high speed using the session key.

[0641] The nonce update unit 1309 updates the nonce (number used once) value in the image sensor 1211 each time the security unit 1310 generates the first protected data.

[0642] The image sensor 1211 configured as described above transmits part or all of the nonce value and part or all of the message count value to the application processor 1212 at high speed. For example, part or all of the nonce value may be a count value or a random number. Also, part or all of the nonce value is stored and transmitted outside the extended packet header, and the image data is stored and transmitted in the packet data.

[0643] In the image sensor 1211, the message counter 1308 and the nonce update unit 1309 may be configured separately or integrally. For example, when the message counter 1308 and the nonce update unit 1309 are configured separately, the update of the nonce value and the message count value can be asynchronous. Thereby, the degrees of freedom of the nonce value and the message count value can be increased.

[0644] On the other hand, when the message counter 1308 and the nonce update unit 1309 are configured integrally, the update of the nonce value and the message count value can be synchronized. In that case, if the message count value uses the count value as the nonce value, the bit width of the message counter 1308 can be saved by sharing part or all of the nonce value. That is, the message counter 1308 may be part or all of the nonce update unit 1309, and part or all of the message counter 1308 and the nonce update unit 1309 can be made common.

[0645] FIG. 77 is a block diagram showing a detailed configuration example of the application processor 1212.

[0646] The application processor 1212 includes a physical layer processing unit 1321, an extended mode compatible CSI-2 receiving circuit 1322, an I2C / I3C master 1323, a storage unit 1324, a data verification unit 1325, a security unit 1326, and a controller 1327. Note that the physical layer processing unit 1321, the extended mode compatible CSI-2 receiving circuit 1322, the I2C / I3C master 1323, and the storage unit 1324 are configured in the same manner as the corresponding blocks in the other embodiments described above, and detailed descriptions thereof are omitted.

[0647] The data verification unit 1325 verifies the validity of the nonce value or the message count value transmitted from the image sensor 1211 to the application processor 1212.

[0648] The security unit 1326 derives the session key in the application processor 1212 corresponding to the session key in the image sensor 1211, and verifies (integrity verification) or decrypts the first protected data of the image data using the session key in the application processor 1212.

[0649] When the data to be verified is a count value in the application processor 1212 configured as described above, the data verification unit 1325 can verify its continuity. Also, the data verification unit 1325 may be configured with a counter and perform comparison verification by updating the count value in the same manner as the image sensor 1211. When the data to be verified is a random number, the data verification unit 1325 may verify its randomness. The data verification unit 1325 may include a nonce update unit 1309 (or a message counter), and use this to verify or decrypt the first protected data, or use this to verify the data to be verified.

[0650] The image sensor 1211 and the application processor 1212 can be configured to be mounted on a desired mobile device. For example, the mobile device may be a portable mobile device, such as any of a mobile phone, smartphone, digital camera, game device, etc. The mobile device may be a propulsion device, such as any of a vehicle, robot, drone, etc. that can be propelled (moved, traveled, walked, flown, etc.). The mobile device may be any of an autonomous vehicle, autonomous robot, autonomous drone, etc. equipped with an AI (Artificial Intelligence) function and capable of autonomous propulsion. The propulsion of the propulsion device may be controlled by the user of the propulsion device, and the propulsion device may notify the user of instructions or warnings as necessary. On the other hand, the propulsion device may be configured such that the propulsion device automatically controls the propulsion of the propulsion device.

[0651] Security units 1310 and 1326 may each include a security operation unit that performs operations for protecting, for example, image data. Therefore, security units 1310 and 1326 can process any one of encryption operations, decryption operations, hash value operations, message authentication code operations, digital signature operations, ID (identification) authentication, firmware measurement, encrypted session key establishment, key exchange, key update, etc. by the security operation unit.

[0652] On the other hand, any one of security units 1310 and 1326, nonce update unit 1309, message counter 1308, and data verification unit 1325 can be configured to be directly electrically connected to a memory. This memory may be directly electrically connected to a register, and any one of security units 1310 and 1326, nonce update unit 1309, message counter 1308, and data verification unit 1325 may be directly electrically connected to a register. The memory may be a memory protected from either leakage or tampering of information in the memory. Such a memory and register are used as storage units 1307 and 1324, respectively.

[0653] Any one of key information (e.g., pre-shared key, private key, public key, or session key), certificate (e.g., root certificate, intermediate certificate, or leaf certificate), encryption algorithm information, etc. may be stored in storage units 1307 and 1324. Any one of function information of image sensor 1211 or application processor 1212, ID information of image sensor 1211 or application processor 1212 (e.g., source ID, destination ID, final destination ID, etc.), firmware information of image sensor 1211 or application processor 1212, etc. may be stored in storage units 1307 and 1324. Any one of session information (e.g., session ID) described later, operation values of the security operation unit (e.g., initial value, intermediate value, or final value), initialization vector, nonce value, message count value, frame number (frame count value), etc. may be stored in storage units 1307 and 1324.

[0654] Any of the security units 1310 and 1326, the nonce update unit 1309, the message counter 1308, and the data verification unit 1325 can determine whether there is a malfunction, for example, when the image sensor 1211 or the application processor 1212 saves any of a plurality of nonce values, count values, integrity calculation values, encryption information, etc. to the storage unit 1307 or 1324, and corresponding countermeasures (for example, a request for retransmission of data at the malfunction location, transmission of an abnormal message) are also possible. Also, when any of the nonce value, count value, integrity calculation value, encryption information, etc. is periodically saved to the protected storage unit 1307 or 1324, if an accident occurs in the mobile device, analyzing the protected storage unit 1307 or 1324 makes it easier to identify the cause of the accident.

[0655] <Session> The requester and the responder, that is, the application processor 1212 and the image sensor 1211, can have one or more communication channels by a session. Hereinafter, the session will be described using as an example a configuration in which the application processor 1212 is the requester and the image sensor 1211 is the responder. Of course, the application processor 1212 may be the responder and the image sensor 1211 may be the requester.

[0656] Also, the requester and the responder can construct a secure communication channel using temporarily fixed encryption information. Specifically, the session provides either or both of encryption or message authentication. The session includes, for example, three stages: a session handshake stage, an application stage, and a session termination stage.

[0657] The session handshake phase begins, for example, with a key exchange request (either PSK_EXCHANGE or KEY_EXCHANGE) from the requester, derives session keys such as session secrets and encryption keys, and protects the communication using the session keys. The purpose of this phase is, for example, to be able to build trust between the responder and the requester before either side sends application data (e.g., image data). Further, a certain degree of integrity of the handshake and synchronization with the derived handshake secrets may be guaranteed.

[0658] If an error occurs during this phase, the session may immediately end and proceed to the session termination. When the handshake is successful, it ends, for example, with a finish response (FINISH_RSP or PSK_FINISH_RSP) from the responder, and the application phase begins. Once the handshake is complete and all validations pass, the session reaches the application phase where either the responder or the requester may send application data.

[0659] The application phase ends, for example, when an end request (END_SESSION) is issued from the requester or when an error occurs. The next phase is the session termination phase.

[0660] The session termination phase is, for example, just an internal phase and there are no explicit messages sent or received. Both the requester and the responder discard or clean up all derived session keys such as session secrets and encryption keys when the session ends. The requester and the responder may have other internal data associated with this session, and they may also wish to clean it up.

[0661] The session secret is used, for example, to derive the encryption key and salt used within an AEAD (Authenticated Encryption with Additional Data) function. The derivation of the encryption key may frequently use HMAC as defined in RFC2104 and HKDF-Expand as described in RFC5869. The session secret may be composed of a single secret or multiple types of secrets. The session key may be composed of a single key or multiple types of keys.

[0662] <Example of Processing for High-Speed Data Transmission and Low-Speed Command Transmission> Referring to FIGS. 78 to 80, the communication processing in which high-speed data transmission and low-speed command transmission are performed between the image sensor 1211 and the application processor 1212 will be described.

[0663] FIG. 78 is a flowchart for explaining a first example of the communication processing.

[0664] Here, the extended mode-compatible CSI-2 reception circuit 1322 of the application processor 1212 has functions as a CCI host (requester) and a CSI-2 host. The extended mode-compatible CSI-2 transmission circuit 1304 of the image sensor 1211 has functions as a CCI device (responder) and a CSI-2 device. The CCI host transmits a request message to the CCI device, and in response to its reception, the CCI device transmits a response message to the CCI host.

[0665] In step S501, a GET_VERSION request and a VERSION response are performed between the CCI host of the extended mode-compatible CSI-2 reception circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmission circuit 1304. Thereby, the extended mode-compatible CSI-2 reception circuit 1322 acquires the SPDM (Security Protocol and Data Model) version of the endpoint.

[0666] In step S502, a GET_CAPABILITIES request and a CAPABILITIES response are made between the CCI host of the extended mode - compatible CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compatible CSI - 2 transmitting circuit 1304. Thereby, the extended mode - compatible CSI - 2 receiving circuit 1322 acquires the SPDM function of the endpoint.

[0667] In step S503, a NEGOTIATE_ALGORITHMS request and an ALGORITHMS response are made between the CCI host of the extended mode - compatible CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compatible CSI - 2 transmitting circuit 1304. Thereby, the extended mode - compatible CSI - 2 receiving circuit 1322 negotiates an encryption algorithm with the extended mode - compatible CSI - 2 transmitting circuit 1304.

[0668] In step S504, a PSK_EXCHANGE request and a PSK_EXCHANGE_RSP response are made between the CCI host of the extended mode - compatible CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compatible CSI - 2 transmitting circuit 1304. Thereby, the extended mode - compatible CSI - 2 receiving circuit 1322 and the extended mode - compatible CSI - 2 transmitting circuit 1304 derive a session key for CCI, such as a session secret and an encryption key.

[0669] In step S505, a PSK_FINISH request and a PSK_FINISH_RSP response are made between the CCI host of the extended mode - compatible CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compatible CSI - 2 transmitting circuit 1304. Thereby, the extended mode - compatible CSI - 2 receiving circuit 1322 proves to the responder that it knows the PSK (PSK: Pre - shared key) and that the session key for CCI derived in step S504 is correct.

[0670] In step S506, a PSK_EXCHANGE request and a PSK_EXCHANGE_RSP response are performed between the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304. As a result, the extended mode - compliant CSI - 2 receiving circuit 1322 and the extended mode - compliant CSI - 2 transmitting circuit 1304 derive a session key for CSI - 2, such as a session secret and an encryption key.

[0671] In step S507, a PSK_FINISH request and a PSK_FINISH_RSP response are performed between the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304. As a result, the extended mode - compliant CSI - 2 receiving circuit 1322 knows the PSK (PSK: Pre - shared key) and proves to the responder that the session key for CSI - 2 derived in step S506 is correct.

[0672] Here, the proof of the session key in steps S505 and S507 is realized by the MAC value calculated from the finished_key of the requester and the messages of this session. Then, the session keys derived in steps S504 and S506 are used to protect subsequent CCI communication and CSI - 2 communication.

[0673] In step S508, in the extended mode - compliant CSI - 2 receiving circuit 1322, a CSI - 2 session secret, a session key, an algorithm, and other parameters are supplied from the CCI host to the CSI - 2 host.

[0674] In step S509, in the extended mode - compliant CSI - 2 transmitting circuit 1304, a CSI - 2 session secret, a session key, an algorithm, and other parameters are supplied from the CCI device to the CSI - 2 device.

[0675] In step S510, the CSI-2 device of the extended mode-compatible CSI-2 transmission circuit 1304 transmits image data to the CSI-2 host of the extended mode-compatible CSI-2 reception circuit 1322 by high-speed data communication. For example, the high-speed data communication continues until the timing to update the CSI-2 session key.

[0676] In step S511, in the extended mode-compatible CSI-2 reception circuit 1322, a trigger to update the CSI-2 session key is supplied from the CSI-2 host to the CCI host. However, a trigger may be supplied from the CSI-2 device or the CCI device to the CCI host, or a self-trigger may be supplied from the CCI host to the CCI host.

[0677] In step S512, a KEY_UPDATE request and a KEY_UPDATE_ACK response are performed between the CCI host of the extended mode-compatible CSI-2 reception circuit 1322 and the CCI device of the extended mode-compatible CSI-2 transmission circuit 1304. Thereby, the session key is updated and a part of the old session key is discarded. When the session key is composed of multiple types of keys (such as a request direction key and a response direction key), a part or all of them may be updated. Also, the KEY_UPDATE request may be issued from the responder using the GET_ENCAPSULATED_REQUEST mechanism described later.

[0678] In step S513, the same processing as in step S512 is performed, and the KEY_UPDATE request and the KEY_UPDATE_ACK response are performed twice. Thereby, the remaining (all) of the old session key that was not discarded only by the processing of step S512 is discarded.

[0679] In step S514, in the extended mode-compatible CSI-2 reception circuit 1322, the CSI-2 session secret, the session key (after update), the algorithm, and other parameters are supplied from the CCI host to the CSI-2 host.

[0680] In step S515, in the extended mode - compatible CSI - 2 transmission circuit 1304, the CCI device supplies the CSI - 2 device with session secrets for CSI - 2, session keys (after update), algorithms, and other parameters.

[0681] In step S516, similar to step S510, the transmission of image data by high - speed data communication is started, and hereinafter, the same processing as steps S510 to S515 is repeatedly performed.

[0682] Note that in the first processing example of the communication process, the session key for CCI and the session key for CSI - 2 are different, the session IDs are different for CCI and CSI - 2, and the session secrets are different for CCI and CSI - 2. However, this is not limited to this. As in the second processing example of the communication process, the session key for CCI and the session key for CSI - 2 may be the same, the session IDs may be the same for CCI and CSI - 2, and the session secrets may be the same for CCI and CSI - 2.

[0683] FIG. 79 is a flowchart for explaining the second processing example of the communication process.

[0684] In steps S521 to S523, the same processing as steps S501 to S503 in FIG. 78 is performed.

[0685] In step S524, a PSK_EXCHANGE request and a PSK_EXCHANGE_RSP response are made between the CCI host of the extended mode - compatible CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compatible CSI - 2 transmission circuit 1304. Here, in the second processing example of the communication process, the same session secret is derived for both the CCI - oriented and CSI - 2 - oriented session secrets.

[0686] That is, from the same session secret, a session key for CCI and a session key for CSI-2 can be derived. Alternatively, from the same session secret, an uplink session key and a downlink (opposite to the uplink) session key may be derived. Alternatively, from the same session secret, a session key common to both CCI and CSI-2 may be derived. Note that even if the sessions for CCI and CSI-2 are the same, the session secrets and session keys for CCI and CSI-2 may be different.

[0687] Thereafter, in steps S525 to S534, the same processing as steps S507 to S516 in FIG. 78 is performed.

[0688] Here, the pre-shared key (PSK) key exchange scheme provides an option for a requester and a responder to perform mutual authentication and session key establishment using symmetric key cryptography. This option is particularly useful for endpoints that do not support public key cryptography or certificate processing. Even when public key cryptography is supported, this option can also be utilized to speed up session key establishment. For this option, the requester and the responder need to know a common PSK in advance before the handshake.

[0689] Basically, the PSK functions as qualification information for mutual authentication and a basis for session key establishment. Therefore, only two endpoints and a potentially trusted third party that provisions the PSK to the two endpoints may know the value of the PSK. The requester may be paired with multiple responders. Similarly, the responder may be paired with multiple requesters. One or more PSKs may be provisioned for the pair of the requester and the responder.

[0690] An endpoint may operate as a requester for one device and at the same time as a responder for another device. The transport layer needs to identify the peer and establish communication between two endpoints before the PSK-based session key exchange starts.

[0691] The PSK may be provisioned in a trusted environment, such as during a secure manufacturing process. In an untrusted environment, the PSK may be agreed upon between two endpoints using a secure protocol. The size of the provisioned PSK is determined by the security strength requirements of the application, but it should be at least 128 bits and preferably 256 bits or more. During PSK provisioning, the endpoint capabilities and supported algorithms may be communicated to the peer. Therefore, during session key establishment using the PSK option, the SPDM commands GET_CAPABILITIES and NEGOTIATE_ALGORITHMS are not necessary.

[0692] This option defines two message pairs: PSK_EXCHANGE / PSK_EXCHANGE_RSP and PSK_FINISH / PSK_FINISH_RSP. The PSK_EXCHANGE message has three roles: prompting the responder to obtain a specific PSK, exchanging context between the requester and the responder, and proving to the requester that the responder knows the correct PSK and has derived the correct session key.

[0693] Figure 80 is a flowchart for explaining a third example of communication processing.

[0694] In steps S541 to S543, the same processing as steps S501 to S503 in FIG. 78 is performed.

[0695] In step S544, a GET_DIGESTS request and a DIGESTS response are made between the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304. As a result, the extended mode - compliant CSI - 2 receiving circuit 1322 acquires the certificate chain digest from the extended mode - compliant CSI - 2 transmitting circuit 1304.

[0696] In step S545, a GET_CERTIFICATE request and a CERTIFICATE response are made between the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304. As a result, the extended mode - compliant CSI - 2 receiving circuit 1322 acquires the certificate chain from the extended mode - compliant CSI - 2 transmitting circuit 1304. Note that the acquisition of the certificate chain may be executed multiple times.

[0697] In step S546, a CHALLENGE request and a CHALLENGE_AUTH response are made between the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304. As a result, the extended mode - compliant CSI - 2 receiving circuit 1322 can authenticate the extended mode - compliant CSI - 2 transmitting circuit 1304 through the challenge - response protocol.

[0698] In step S547, a KEY_EXCHANGE request (channel = CCI, sessionID = D) and a KEY_EXCHANGE_RSP response are made between the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322 and the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304. As a result, a handshake between the requester and the responder for the purpose of authenticating the responder (or optionally both parties) is started. Then, in addition to the content negotiated in the last NEGOTIATE_ALGORITHMS / ALGORITHMS exchange, encryption parameters are negotiated and shared key information is established.

[0699] In step S548, the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322 transmits a GET_ENCAPSULATED_REQUEST to the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304.

[0700] In step S549, the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304 transmits an ENCAPSULATED_REQUEST (GET_DIGESTS request) to the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322.

[0701] In step S550, the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322 transmits a DELIVER_ENCAPSULATED_RESPONSE (DIGESTS response) to the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304. Thereby, the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304 acquires the certificate chain digest from the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322.

[0702] In step S551, the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304 transmits an ENCAPSULATED_RESPONSE_ACK (GET_CERTIFICATE request) to the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322.

[0703] In step S552, the CCI host of the extended mode - compliant CSI - 2 receiving circuit 1322 transmits a DELIVER_ENCAPSULATED_RESPONSE (CERTIFICATE response) to the CCI device of the extended mode - compliant CSI - 2 transmitting circuit 1304. Thereby, the CCI device (responder) may acquire the certificate chain from the CCI host (requester). Note that this process may be executed multiple times.

[0704] In step S553, the CCI device of the extended mode - compliant CSI - 2 transmission circuit 1304 transmits ENCAPSULATED_RESPONSE_ACK to the CCI host of the extended mode - compliant CSI - 2 reception circuit 1322.

[0705] In step S554, a FINISH request and a FINISH_RSP response are performed between the CCI host of the extended mode - compliant CSI - 2 reception circuit 1322 and the CCI device of the extended mode - compliant CSI - 2 transmission circuit 1304. As a result, the handshake between the CCI host of the extended mode - compliant CSI - 2 reception circuit 1322 and the CCI device of the extended mode - compliant CSI - 2 transmission circuit 1304, which was started by the KEY_EXCHANGE request in step S547, is completed.

[0706] In step S555, a GET_MEASUREMENTS request and a MEASUREMENTS response are performed between the CCI host of the extended mode - compliant CSI - 2 reception circuit 1322 and the CCI device of the extended mode - compliant CSI - 2 transmission circuit 1304. As a result, the CCI host of the extended mode - compliant CSI - 2 reception circuit 1322 acquires measurement data from the CCI device of the extended mode - compliant CSI - 2 transmission circuit 1304. Note that the GET_MEASUREMENTS request may be issued from the responder using the GET_ENCAPSULATED_REQUEST mechanism described above. Similarly, other requests may also be issued from the responder using the GET_ENCAPSULATED_REQUEST mechanism described above.

[0707] Thereafter, in step S556, a KEY_EXCHANGE request (channel = CSI - 2, sessionID = E) and a KEY_EXCHANGE_RSP response are performed in the same manner as in step S547, and in step S557, a FINISH request and a FINISH_RSP response are performed in the same manner as in step S554. Then, in steps S558 to S566, the same processing as in steps S508 to S516 in FIG. 78 is performed.

[0708] <Data Verification Process> Referring to FIGS. 81 to 83, a data verification process using a verification packet and a packet to be verified will be described.

[0709] As shown in FIGS. 81 and 82, the extended packet is composed of a packet header PH, an extended packet header ePH, packet data, an extended packet footer ePF, and a packet footer PF. With an extended packet having such a configuration, a frame start, embedded data, image data, user-defined data, frame end, write command (CCI Write), read command (CCI Read), and read response (CCI Read return value) can be configured. Note that the packet header PH, extended packet header ePH, packet data, extended packet footer ePF, and packet footer PF may omit some or all of them. That is, a packet configuration including at least the extended packet header ePH and packet data is defined as an extended packet.

[0710] By the way, any one of the extended packet header ePH, packet data, and extended packet footer ePF may not be received normally (the message disappears) due to noise, interference, or an attack. Therefore, it is desirable that a verification packet for verifying the integrity of the extended packet header ePH, packet data, and the remaining extended packet footer ePF1 is stored inside the extended packet footer end ePF0. For integrity verification, for example, CRC32 of cyclic redundancy check, which is a kind of error detection code, is used. Also, the generating polynomial of CRC32 is, for example, X 32 +X 26 +X 23 +X 22 +X 16 +X 12 +X 11 +X 10 +X 8 +X 7 +X 5 +X 4 +X 2 +X + 1 is used.

[0711] For the packet to be verified, packet data can be used. Or, for the packet to be verified, an extended packet header and packet data can be used. Or, for the packet to be verified, packet data and the remaining extended packet footer (ePF1) can be used. Or, for the packet to be verified, an extended packet header, packet data, and the remaining extended packet footer (ePF1) can be used. With such a packet to be verified, at least the packet data is protected.

[0712] That is, the image sensor 1211 includes a second protection unit (e.g., a CRC calculation unit) that generates second protection data (e.g., a CRC calculation value) for the packet data without using a session key. The second protection data is stored, for example, in the extended packet footer ePF for high-speed data transmission. That is, it is stored in any of the frame start, embedded data, image data, user-defined data, frame end, write command (CCI Write), read command (CCI Read), read response (CCI Read return value), etc.

[0713] The extended packet footers ePF1 and ePF0 may have a security feature defined. That is, the image sensor 1211 may include a security calculation unit (e.g., an encryption calculation unit, a decryption calculation unit, a hash value calculation unit, a message authentication code calculation unit, a digital signature calculation unit). And the result of the security calculation (e.g., a hash value, a message authentication code, a digital signature) may be stored in the extended packet footer ePF.

[0714] The result of the security calculation may be stored only in the extended packet footer ePF1 and not in the extended packet footer ePF0, or may be outside the extended packet footer rather than inside the extended packet footer (e.g., in the embedded data or the read response). The security calculation unit included in the image sensor 1211 is included in the security unit 1310.

[0715] As the message authentication code (MAC), any one of GMAC (Galois MAC), CMAC (Cipher-based MAC), HMAC (Hash-based MAC), etc. may be used. For example, any one of AES-GMAC, AES-CMAC, SHA2-HMAC, SHA3-HMAC, etc., to which AES (Advanced Encryption Standard) or SHA (Secure Hash Algorithm) is applied, may be used. Note that the block length of AES is 128 bits, and the key length of AES is selected from any one of 128 bits, 192 bits, and 256 bits.

[0716] Inside the extended packet footer, for example, as the packet data as the packet to be verified, or as the extended packet header and the packet data as the packet to be verified, any security information such as a hash (especially a cryptographic hash) value, a message authentication code, or a digital signature may be stored. In that case, it is possible to provide further resistance against malicious tampering by an attacker. Note that a cyclic redundancy check (CRC), which is a kind of error detection code, may be stored inside the extended packet footer "ePF1" or inside "ePF1 and ePF0".

[0717] That is, the image sensor 1211 may include an integrity calculation unit (for example, the first protection unit = security calculation unit, the second protection unit = CRC calculation unit), and an integrity calculation value (for example, the first protection data, the second protection data), which is the result of calculating the integrity, may be stored inside the extended packet footer. Note that CRC can be used for functional safety, and its integrity can be used to prevent the detection of hardware failures. On the other hand, the integrity of the security function can be used to detect intentional interference or attacks. That is, the security calculation unit calculates an integrity calculation value based on encryption, and the CRC calculation unit calculates an integrity calculation value not based on encryption.

[0718] The application processor 1212 can verify the integrity of the packet to be verified by using, for example, verification packets. When an abnormality is determined, for example, any of the following processes may be executed: transmission of a request message requesting retransmission of a packet including the packet to be verified and the verification packet, transmission of a request message to inquire of the image sensor 1211 whether there is an abnormality in the image sensor 1211, transmission of a request message to request the image sensor 1211 to stop some or all of the functions of the image sensor 1211, propulsion stop of the propulsion device, change of the propulsion control of the propulsion device, change of the priority data used for the propulsion control, etc.

[0719] Note that the integrity calculation value may be stored, for example, in any of embedded data, image data (packet data), user-defined data, write instructions, read instructions, read responses, etc. In that case, the integrity calculation value may not be stored in the extended packet footer. For example, the integrity calculation value may be stored in units of image frames instead of in units of image lines, and in that case, the integrity can be calculated efficiently. In that case, the integrity calculation value is stored, for example, in the embedded data or the read response after the image data is transmitted.

[0720] The extended packet shown in A of FIG. 81 has a configuration example in which the extended packet header ePH, the packet data, and the remaining extended packet footer ePF1 are used as the packet to be verified, and the extended packet footer end ePF0 in which the calculated value obtained by the security calculation using the packet to be verified is stored is used as the verification packet.

[0721] The extended packet shown in B of FIG. 81 has a configuration example in which the packet data and the remaining extended packet footer ePF1 are used as the packet to be verified, and the extended packet footer end ePF0 in which the calculated value obtained by the security calculation using the packet to be verified is stored is used as the verification packet.

[0722] The extended packet shown in C of FIG. 81 is a configuration example in which the extended packet header ePH and the packet data are used as the packet to be verified, and the end ePF0 of the extended packet footer in which the calculated value obtained by the security operation using the packet to be verified is stored is used as the verification packet.

[0723] The extended packet shown in D of FIG. 81 is a configuration example in which the packet data is used as the packet to be verified, and the end ePF0 of the extended packet footer in which the calculated value obtained by the security operation using the packet to be verified is stored is used as the verification packet.

[0724] The extended packet shown in A of FIG. 82 is a configuration example in which the extended packet header ePH and the packet data are used as the packet to be verified, and the remaining ePF1 of the extended packet footer in which the calculated value obtained by the security operation using the packet to be verified is stored is used as the verification packet.

[0725] The extended packet shown in B of FIG. 82 is a configuration example in which the extended packet header ePH and the packet data are used as the packet to be verified, and the remaining ePF1 of the extended packet footer and the end ePF0 of the extended packet footer in which the calculated value obtained by the security operation using the packet to be verified is stored are used as the verification packet.

[0726] The extended packet shown in C of FIG. 82 is a configuration example in which the packet data is used as the packet to be verified, and the remaining ePF1 of the extended packet footer in which the calculated value obtained by the security operation using the packet to be verified is stored is used as the verification packet.

[0727] The extended packet shown in D of FIG. 82 is a configuration example in which the packet data is used as the packet to be verified, and the remaining ePF1 of the extended packet footer and the end ePF0 of the extended packet footer in which the calculated value obtained by the security operation using the packet to be verified is stored are used as the verification packet.

[0728] FIG. 83 is a flowchart for explaining data verification processing performed in the application processor 1212.

[0729] In step S601, when the extended packet transmitted from the image sensor 1211 is received by the extended mode compatible CSI-2 receiving circuit 1322, the security unit 1326 receives the packet to be verified of the extended packet. Then, when the security unit 1326 completes the reception of the packet to be verified, the process proceeds to step S602. Note that even if the reception of all the packets to be verified is not completed, if at least a part (for example, 128 bits) of the reception that enables the start of the calculation of the security operation is completed, the process may proceed to step S602. In that case, until the reception of all the packets to be verified is completed, the remaining packets to be verified are continuously received.

[0730] In step S602, the security unit 1326 starts calculating a calculated value obtained by a security operation using at least a part of the packet to be verified received in step S601.

[0731] In step S603, the security unit 1326 receives a verification packet transmitted from the image sensor 1211 via the extended mode compatible CSI-2 receiving circuit 1322. Then, when the security unit 1326 completes the reception of the verification packet and acquires the received value (calculated value calculated by the image sensor 1211) stored in the verification packet, the process proceeds to step S604.

[0732] In step S604, when the calculation of the calculated value obtained by the security operation using the packet to be verified started in step S602 is completed (that is, when all of the packet to be verified is received and the calculation using all of it is completed), the process proceeds to step S605.

[0733] In step S605, the security unit 1326 determines whether the received value received in step S603 matches the calculated value obtained in step S604.

[0734] In step S605, if the security unit 1326 determines that the received value and the calculated value match, the process proceeds to step S606. In this case, in step S606, the security unit 1326 determines that the extended packet received by the extended mode - compliant CSI - 2 receiving circuit 1322 is normal, and the process ends.

[0735] On the other hand, in step S605, if the security unit 1326 determines that the received value and the calculated value do not match, the process proceeds to step S607. In this case, in step S607, the security unit 1326 determines that an abnormality has occurred in the extended packet received by the extended mode - compliant CSI - 2 receiving circuit 1322, and the process ends.

[0736] <Ensuring Functional Safety Using Message Count Values> The image sensor 1211 can store the message count value counted by the message counter 1308 in the extended packet header or the extended packet footer in order to ensure functional safety (for example, detecting message loss and taking appropriate measures). For example, the message counter 1308 provided in the image sensor 1211 can store the message count value incremented or decremented each time a message is transmitted from the image sensor 1211. Note that the image sensor 1211 may be configured to provide an independent message counter 1308 for each virtual channel or a common message counter 1308 for virtual channels.

[0737] The message counter 1308 sets the message count value to an initial value (e.g., 0 or the maximum value) in the first packet including the extended packet header of a certain virtual channel, and increments or decrements the message count value each time data including the extended packet header of a certain virtual channel is transmitted. Also, the message counter 1308 does not increment or decrement the message count value when, for example, data not including the extended packet header is transmitted, and resumes counting when data including the extended packet header is transmitted next.

[0738] The message counter 1308 may continue counting regardless of the frame start or frame end. And when the message count value is counted up to a specified value (e.g., the maximum value or 0), the message counter 1308 returns the next message count value to the initial value (e.g., 0 or the maximum value) and performs counting. Note that a part of the extended packet header may store a part of the random value.

[0739] Note that the receiving side (image sensor 1211 or application processor 1212) that receives the message count value can immediately detect the loss of the message if the message is lost. For example, a DoS (Denial-of-service) attack that infringes the availability of the image sensor 1211 or the application processor 1212 by intentionally mixing a huge amount of messages is also immediately detected on the receiving side. For this reason, it is desirable that the message count value be stored in the extended packet header. By making it possible to detect such losses and attacks in a shorter time, the receiving side can start corresponding to them in a short time, which is particularly suitable for a propulsion device capable of high-speed movement or high-speed movement.

[0740] Note that for a write command (CCI Write), a read command (CCI Read), or a read response (CCI Read return value), the message count value or the integrity operation value may be configured to be stored, and elements related to the extended packet may be applied. In that case, it becomes possible to handle functional safety and protect integrity for the write command, the read command, or the read response as well.

[0741] FIG. 84 is a flowchart for explaining a message count value transmission process in which the image sensor 1211 transmits a message count value.

[0742] In step S611, the message counter 1308 initializes the message count value and sets it to 0.

[0743] In step S612, the extended mode - compliant CSI - 2 transmission circuit 1304 determines whether to transmit an extended packet header, and the process waits until it is determined to transmit the extended packet header. Then, in step S612, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines to transmit the extended packet header, the process proceeds to step S613.

[0744] In step S613, the extended mode - compliant CSI - 2 transmission circuit 1304 acquires the message count value from the message counter 1308 and stores it in the extended packet header.

[0745] In step S614, the extended mode - compliant CSI - 2 transmission circuit 1304 transmits the extended packet header that stored the message count value in step S613.

[0746] In step S615, the message counter 1308 determines whether the message count value has been counted up to the maximum value. In step S615, if the message counter 1308 determines that the message count value has not been counted up to the maximum value, the process proceeds to step S616.

[0747] In step S616, the message counter 1308 increments the message count value. Thereafter, the process returns to step S612, and the same process is repeated hereinafter.

[0748] On the other hand, in step S615, when the message counter 1308 determines that the message count value has been counted up to the maximum value, the process returns to step S611, the message count value is initialized, and then the same process is repeated hereinafter.

[0749] In addition to incrementing the message count value in this way, for example, the message count value may be initialized and set to the maximum value, and then decremented.

[0750] <Regarding Embedded Data> With reference to FIGS. 85 to 88, the embedded data will be described.

[0751] The image sensor 1211 can include additional information such as device setting information in the data stream by using the embedded data. The embedded data is composed of one or more lines, and can include any of the configuration data of the image sensor 1211, register values conforming to the standard, vendor-specific register values, description of the frame format, statistical values, and the like.

[0752] In A of FIG. 85, one line of embedded data is shown. Following the embedded data format code, the embedded data of the desired data amount is continuously arranged, and padding characters are arranged in the rest.

[0753] The embedded data includes information related to image data or user-defined data. Therefore, the image data or user-defined data may be compressed data, but the embedded data is preferably uncompressed data (non-compressed data). Accordingly, when data compression is used, compressed data (image data or user-defined data) and uncompressed data (embedded data) will be mixed within the frame for high-speed data transmission.

[0754] The embedded data can have multiple lines according to the number of register values added to the embedded data. Also, the number of lines of the embedded data can be specified by a part of the description within the frame format in the first embedded data line within the frame. The line length of the embedded data may be shorter than the line length of the image data or user-defined data, but it is not preferable to exceed the line length of the image data or user-defined data, and it is desirable to be the same as the line length of the image data or user-defined data. The first pixel value of the embedded data may indicate the format used for the embedded data.

[0755] Some or all of the nuisance values may be stored and transmitted into at least a part of the embedded data indicating vendor-specific code or reserved code for future use as shown in B of FIG. 85. Within the frame, the embedded data is stored either between the frame start and the first image data or user-defined data, or between the last image data or user-defined data and the frame end. However, the embedded data between the last image data or user-defined data and the frame end may be omitted.

[0756] FIG. 86 shows an example of the data structure of two frames of image data transmitted from the image sensor 1211.

[0757] As shown in FIG. 86, after the frame start (VC1 FS) of the first virtual channel is transmitted, following the read command and read response, the frame start (VC2 FS) of the second virtual channel is transmitted. Next, the first embedded data (VC1 Emb Data) of the first virtual channel and the first embedded data (VC2 Emb Data) of the second virtual channel are transmitted. Then, the image data (VC1 Img Data) of the first virtual channel for one frame and the user-defined data (VC2 UD Data) of the second virtual channel are transmitted. When the transmission for one frame is completed, the second embedded data (VC1 Emb Data) of the first virtual channel and the second embedded data (VC2 Emb Data) of the second virtual channel are transmitted. After that, after the frame end (VC1 FE) of the first virtual channel is transmitted, following the read command and read response, the frame end (VC2 FE) of the second virtual channel is transmitted.

[0758] In FIG. 86, an example is shown in which the message count values are shared between the first virtual channel and the second virtual channel. At this time, a configuration may be adopted in which independent message counters are provided for the first virtual channel and the second virtual channel. Also, the user-defined data may be image data or the like.

[0759] Here, part or all of the dummy value is stored, for example, within the period from the frame start to the frame end, or within the period from the frame end to the frame start (frame blanking period). Also, the dummy value can be stored within the period from the frame start to the frame end, for example, in any of the embedded data, image data, non-image data, and line blanking period. Also, it may be stored in the second virtual channel.

[0760] By defining a frame start and a frame end, for example, it becomes possible to notify the start and end of high-speed data transmission from an image sensor to a processor. Also, the image sensor can keep the frame transmission period constant. Note that the embedded data is data in which attributes representing image data and information related to the image data (metadata) are stored.

[0761] In the present embodiment, an example in which high-speed data transmission of nuisance values is executed without inhibiting high-speed data transmission of image data will be described. That is, an example in which high-speed data transmission of image data and high-speed data transmission of nuisance values are serially executed instead of being executed in parallel will be described. However, if the communication paths are different between high-speed data transmission of image data and transmission of nuisance values (high-speed data transmission or low-speed command transmission), they may be executed in parallel.

[0762] Note that since high-speed data transmission and low-speed command transmission can be frequency-separated by a filter, if there is no problem with power consumption, part or all of the transmission may be overlapped (executed in parallel). Part or all of the nuisance values may be transmitted every multiple frames, but it is desirable to transmit them every frame, for example, due to reasons such as frame dropout. For example, a Frame Start (FS) packet includes a Frame Start Code (Data Type = 0x00), and a Frame End (FE) packet includes a Frame End Code (Data Type = 0x01).

[0763] FIG. 87 is a flowchart for explaining an image data transmission process in which the image sensor 1211 transmits image data.

[0764] In step S621, the extended mode - compliant CSI - 2 transmission circuit 1304 determines whether it has received a start command for high - speed data transmission, and the process waits until it is determined that the start command for high - speed data transmission has been received. Then, in step S621, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines that it has received the start command for high - speed data transmission, the process proceeds to step S622.

[0765] In step S622, the pixel 1301 starts imaging, and the image data output from the pixel 1301 is supplied to the extended mode - compliant CSI - 2 transmission circuit 1304 via the AD converter 1302 and the image processing unit 1303.

[0766] In step S623, the extended mode - compliant CSI - 2 transmission circuit 1304 transmits the frame start of the first virtual channel.

[0767] In step S624, the extended mode - compliant CSI - 2 transmission circuit 1304 transmits the frame start of the second virtual channel.

[0768] In step S625, the extended mode - compliant CSI - 2 transmission circuit 1304 transmits the first embedded data of the first virtual channel.

[0769] In step S626, the extended mode - compliant CSI - 2 transmission circuit 1304 transmits the first embedded data of the second virtual channel.

[0770] In step S627, the extended mode - compliant CSI - 2 transmission circuit 1304 transmits the image data of the first virtual channel.

[0771] In step S628, the extended mode - compliant CSI - 2 transmission circuit 1304 transmits the user - defined data of the second virtual channel.

[0772] In step S629, the CSI-2 transmission circuit 1304 corresponding to the extended mode determines whether the transmission of the image data for one frame has been completed.

[0773] In step S629, if the CSI-2 transmission circuit 1304 corresponding to the extended mode determines that the transmission of the image data for one frame has not been completed, the process returns to step S627, and the following similar processes are repeated. On the other hand, in step S629, if the CSI-2 transmission circuit 1304 corresponding to the extended mode determines that the transmission of the image data for one frame has been completed, the process proceeds to step S630.

[0774] In step S630, the CSI-2 transmission circuit 1304 corresponding to the extended mode transmits the second embedded data of the first virtual channel.

[0775] In step S631, the CSI-2 transmission circuit 1304 corresponding to the extended mode transmits the second embedded data of the second virtual channel.

[0776] In step S632, the CSI-2 transmission circuit 1304 corresponding to the extended mode transmits the frame end of the first virtual channel.

[0777] In step S633, the CSI-2 transmission circuit 1304 corresponding to the extended mode transmits the frame end of the second virtual channel.

[0778] In step S634, the CSI-2 transmission circuit 1304 corresponding to the extended mode determines whether it has received an end command for high-speed data transmission.

[0779] In step S634, if the CSI-2 transmission circuit 1304 corresponding to the extended mode determines that it has not received an end command for high-speed data transmission, the process returns to step S622, and the following similar processes are repeated. On the other hand, in step S634, if the CSI-2 transmission circuit 1304 corresponding to the extended mode determines that it has received an end command for high-speed data transmission, the process ends.

[0780] The imaging start may be continuously executed until a termination command for high-speed data transmission is received, or may be executed each time a start command for high-speed data transmission is received.

[0781] FIG. 88 is a flowchart for explaining the integrity calculation value transmission process in which the image sensor 1211 transmits an integrity calculation value.

[0782] In step S641, the security unit 1310 derives a session key for the first virtual channel.

[0783] In step S642, the security unit 1310 derives a session key for the second virtual channel.

[0784] In step S643, the message counter 1308 initializes the upper count value of the message count value and sets it to 0.

[0785] In step S644, the message counter 1308 initializes the lower count value of the message count value and sets it to 0.

[0786] In step S645, the extended mode compliant CSI-2 transmission circuit 1304 determines whether to end the session. If it is determined not to end the session, the process proceeds to step S646.

[0787] In step S646, the extended mode compliant CSI-2 transmission circuit 1304 determines whether to transmit an extended packet for the first virtual channel.

[0788] In step S646, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines not to transmit the extended packet of the first virtual channel, the process returns to step S645, and the following similar processes are repeated. On the other hand, in step S646, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines to transmit the extended packet of the first virtual channel, the process proceeds to step S647.

[0789] In step S647, the security unit 1310 calculates the integrity operation value of the first virtual channel using the session key of the first virtual channel derived in step S641.

[0790] In step S648, the extended mode - compliant CSI - 2 transmission circuit 1304 places the integrity operation value calculated in step S647 in the extended packet of the first virtual channel and transmits the extended packet of the first virtual channel.

[0791] In step S649, the extended mode - compliant CSI - 2 transmission circuit 1304 determines whether to transmit the extended packet of the second virtual channel and waits for the process until it is determined to transmit the extended packet of the second virtual channel. Then, in step S649, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines to transmit the extended packet of the second virtual channel, the process proceeds to step S650.

[0792] In step S650, the security unit 1310 calculates the integrity operation value of the second virtual channel using the session key of the second virtual channel derived in step S642.

[0793] In step S651, the extended mode - compliant CSI - 2 transmission circuit 1304 places the integrity operation value calculated in step S650 in the extended packet of the second virtual channel and transmits the extended packet of the second virtual channel.

[0794] In step S652, the message counter 1308 determines whether the lower count value of the message count value has been counted up to the maximum value.

[0795] In step S652, if the message counter 1308 determines that the lower count value of the message count value has not been counted up to the maximum value, the process proceeds to step S653. In step S653, after the message counter 1308 increments the lower count value of the message count value, the process returns to step S645, and the following similar process is repeated.

[0796] On the other hand, in step S652, if the message counter 1308 determines that the lower count value of the message count value has been counted up to the maximum value, the process proceeds to step S654. In step S654, after the message counter 1308 increments the upper count value of the message count value, the process returns to step S644, and the following similar process is repeated.

[0797] Then, in step S645, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines to end the session, the process proceeds to step S655.

[0798] In step S655, the security unit 1310 discards or cleans up the session key of the first virtual channel and the session key of the second virtual channel, and then the process ends.

[0799] <Modification Example of Data Structure of Image Data> With reference to FIGS. 89 to 91, the data structure of the image data will be described.

[0800] FIG. 89 shows a first modification example of the data structure of the image data.

[0801] In the data structure of the image data shown in FIG. 89, a message count value shared by the first virtual channel and the second virtual channel is used.

[0802] However, the session key or the message counter may be shared by the first virtual channel and the second virtual channel. Also, the image data or the embedded data may be replaced with other data. For example, the embedded data may be replaced with image data. On the other hand, the message counter may be shared by counting across virtual channels (VC).

[0803] FIG. 90 shows a second modification example of the data structure of the image data.

[0804] In the data structure of the image data shown in FIG. 90, independent message count values are used for Write (CCI write instruction), Read1 (CCI read instruction), and Read2 (CCI read response), respectively.

[0805] FIG. 91 shows a third modification example of the data structure of the image data.

[0806] In the data structure of the image data shown in FIG. 91, independent message count values are provided for CCI uplink (Write and Read1) and CCI downlink (Read2), respectively. That is, the message count value may be shared between Write (CCI write instruction) and Read1 (CCI read instruction).

[0807] <Regarding the nuisance value> The nonce value is used, for example, as part or all of the initialization vector for the encryption operation or decryption operation using the session key because it is number used once for the same session key. Therefore, the nonce used by the image sensor 1211 for the encryption operation can be transmitted from the image sensor 1211 and received by the application processor 1212, so that the application processor 1212 can obtain the nonce value required for the decryption operation.

[0808] That is, it is desirable for the image sensor 1211 to transmit the nonce value before transmitting the image data. Specifically, part or all of the nonce value corresponding to the image data in a certain frame is stored in any of the read response, user-defined data, embedded data (immediately after the image data), frame end, frame start, embedded data (immediately before the image data), etc. from after the transmission of the last image data in the previous frame is completed until before the transmission of the first image data in a certain frame is started.

[0809] For example, the application processor 1212, which is the master for low-speed command transmission, may transmit a read command that requests to read the nonce value in the image sensor 1211 to the application processor 1212 by low-speed command transmission in response to the start or completion of reception of any of the frame start, embedded data, image data, user-defined data, frame end, etc. transmitted by high-speed data transmission from the image sensor 1211, which is the slave for low-speed command transmission.

[0810] The image sensor 1211 receives the read command transmitted from the application processor 1212 and transmits the corresponding nonce value by high-speed data transmission. Then, when the application processor 1212 receives the read response, the nonce value can be notified from the image sensor 1211 to the application processor 1212.

[0811] Since the nuisance value notified from the image sensor 1211 is used within the application processor 1212, it is desirable that part or all of the nuisance value be transmitted during the frame blanking period when the image data between the frame end and the next frame start is not transmitted. However, for the first frame (Frame Number = 1), it is sufficient to pre - agree on the first nuisance value (initial value) between the image sensor 1211 and the application processor 1212, or that part or all of the first nuisance value has been received by the application processor 1212 before the start of the transmission of the image data.

[0812] This read command corresponds to Read in Read / Write in the I2C or I3C standard, for example. On the other hand, the read response corresponds to the Read return value. Note that a timer may be provided to wait for a predetermined time between when the application processor 1212 receives high - speed data transmission and when it transmits the read command in order to adjust the timing of the read response.

[0813] <Regarding I2C and I3C> The integrated - circuit - to - integrated - circuit serial bus, sometimes called the I2C bus or I 2 C bus, is a serial single - ended computer bus intended for connecting low - speed peripheral devices to the application processor 1212. The I2C bus is a multi - master bus in which each device can act as a master and a slave for various messages transmitted on the I2C bus.

[0814] The I2C bus can transmit data using only two bidirectional open-drain connectors that include a serial data line (SDA) and a serial clock line (SCL). Those connectors usually include signal lines terminated by pull-up resistors. The protocol that manages the operation of the I2C bus defines the basic types of messages, and those messages each start with START and end with STOP. The I2C bus uses 7-bit addressing and defines two types of nodes.

[0815] The master node is the node that generates the clock and initiates communication with the slave nodes. The slave node is the node that receives the clock and responds when addressed by the master. The I2C bus is a multi-master bus, which means that any number of master nodes can exist. Further, the roles of master and slave may change between messages (i.e., after STOP is sent). In this embodiment which is a camera implementation, unidirectional transmission may be used to capture an image from the sensor and send such image data to the memory in the baseband processor, while control data may be exchanged between the baseband processor and the sensor as well as other peripheral devices.

[0816] In one example, the Camera Control Interface (CCI) protocol may be used for such control data between the baseband processor and the image sensor (or one or more slave nodes). In one example, the CCI protocol may be implemented via the I2C serial bus between the image sensor and the baseband processor. Conventional I2C systems, i.e., camera control interface-based camera systems, use a separate interrupt (IRQ) line for each slave device to enable the slave node to indicate to the master node that it desires to use the bus.

[0817] On one hand, the I3C communication standard is a standard for communication via two signal lines, namely the SDA line for transmitting data and the SCL line for transmitting clock signals. In this standard, devices (such as processors) are classified into devices that operate as masters or slaves and devices that operate only as slaves. For example, a processor can operate as a master or a slave, while a sensor operates only as a slave.

[0818] Here, a master is a device that controls slaves, and a slave is a device that operates according to the control of the master. Also, in I3C, multiple slaves can be connected to one master. Additionally, multiple masters can send signals to one slave, and this communication is hereinafter referred to as "multi-master communication". Furthermore, slaves can communicate with each other without going through a master, and this communication is called "peer-to-peer communication". Also, a slave can interrupt the communication and perform communication while the SDA line is busy (in use) due to the communication of other devices, and this interruption is called "In-Band Interrupt".

[0819] In the above multi-master communication, in-band interrupt, and peer-to-peer communication, there is a possibility that signals simultaneously transmitted by multiple devices may collide on the SDA line. For example, while a master is sending a signal to a certain slave, if another slave performs an in-band interrupt and sends a signal to the master, the signal from the master and the signal from the slave will collide. Therefore, devices in I3C have a function to detect collisions and arbitrate between devices.

[0820] When using the above-described interrupt function, it is possible to easily synchronize with the application processor 1212. Therefore, by executing an interrupt at the timing determined by the image sensor 1211, nuisance-related information is transmitted according to the timing determined by the image sensor 1211. However, the image sensor 1211 may trigger a read command by an in-band interrupt and transmit a read response accordingly, or may omit the read command by an in-band interrupt and transmit a read response.

[0821] <Integrity operation value processing> With reference to FIGS. 92 to 95, integrity operation value processing will be described.

[0822] FIG. 92 is a flowchart for explaining a first processing example of integrity operation value processing in which the image sensor 1211 transmits an integrity operation value.

[0823] In step S661, the security unit 1310 derives a session key.

[0824] In step S662, the message counter 1308 initializes the message count value and sets it to 0.

[0825] In step S663, the extended mode-compatible CSI-2 transmission circuit 1304 determines whether to end the session. If it is determined that the session is not ended, the process proceeds to step S664.

[0826] In step S664, the extended mode-compatible CSI-2 transmission circuit 1304 determines whether to transmit an extended packet.

[0827] In step S664, if the extended mode-compatible CSI-2 transmission circuit 1304 determines not to transmit an extended packet, the process returns to step S663, and the following similar processing is repeated. On the other hand, in step S664, if the extended mode-compatible CSI-2 transmission circuit 1304 determines to transmit an extended packet, the process proceeds to step S665.

[0828] In step S665, the security unit 1310 calculates a integrity operation value using the message count value.

[0829] In step S666, the extended mode - compliant CSI - 2 transmission circuit 1304 places the integrity operation value calculated in step S665 in the extended packet and transmits the extended packet.

[0830] In step S667, the message counter 1308 determines whether the message count value has been counted up to the maximum value. In step S667, if the message counter 1308 determines that the message count value has not been counted up to the maximum value, the process proceeds to step S668.

[0831] In step S668, the message counter 1308 increments the message count value. Then, the process returns to step S662, and the following similar processes are repeated.

[0832] On the other hand, in step S667, if the message counter 1308 determines that the message count value has been counted up to the maximum value, the process proceeds to step S669. In step S669, after the security unit 1310 updates the session key, the process returns to step S663, and the following similar processes are repeated.

[0833] And in step S663, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines to end the session, the process proceeds to step S670.

[0834] In step S670, the security unit 1310 discards or cleans up the session key, and then the process ends.

[0835] When calculating the MAC value for each image line, storing it in the extended packet footer, and then transmitting it, the message count value is incremented by 1 each time an extended packet is transmitted. Therefore, 16 the message count value will cycle back to zero after 2 16 times. For example, when transmitting 4K data with a frame rate of 60 fps and a pixel count of 4096×2160 (horizontal × vertical), assuming that 2163 lines of extended packets, which is the sum of 3 lines including the frame start, embedded data, and frame end, are transmitted within one frame, the message count value will cycle back to zero in approximately (2

[0836] ) / (60×2163) ≈ 0.5 seconds.

[0837] FIG. 93 is a flowchart for explaining a second processing example of integrity operation value processing in which the image sensor 1211 transmits an integrity operation value.

[0838] In step S681, the security unit 1310 derives a session key.

[0839] In step S682, the message counter 1308 initializes the upper count value of the message count value and sets it to 0.

[0840] In step S683, the message counter 1308 initializes the lower count value of the message count value and sets it to 0.

[0841] In step S684, the extended mode - compliant CSI - 2 transmission circuit 1304 determines whether to end the session. If it is determined that the session is not ended, the process proceeds to step S685.

[0842] In step S685, the extended mode - compliant CSI - 2 transmission circuit 1304 determines whether to transmit an extended packet.

[0843] In step S685, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines not to transmit an extended packet, the process returns to step S684, and the following similar processing is repeated. On the other hand, in step S685, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines to transmit an extended packet, the process proceeds to step S686.

[0844] In step S686, the security unit 1310 calculates an integrity operation value using the upper count value and the lower count value of the message count value.

[0845] In step S687, the extended mode - compliant CSI - 2 transmission circuit 1304 places the integrity operation value calculated in step S686 in the extended packet and transmits the extended packet.

[0846] In step S688, the message counter 1308 determines whether the lower count value of the message count value has been counted up to the maximum value. In step S688, if the message counter 1308 determines that the lower count value of the message count value has not been counted up to the maximum value, the process proceeds to step S689.

[0847] In step S689, the message counter 1308 increments the lower count value of the message count value. Thereafter, the process returns to step S684, and the following similar processes are repeatedly performed.

[0848] On the other hand, in step S688, when the message counter 1308 determines that the lower count value of the message count value has been counted up to the maximum value, the process proceeds to step S690. In step S690, after the message counter 1308 increments the upper count value of the message count value, the process returns to step S683, and the following similar processes are repeatedly performed.

[0849] Then, in step S684, when the extended mode - compliant CSI - 2 transmission circuit 1304 determines to end the session, the process proceeds to step S691.

[0850] In step S691, the security unit 1310 discards or cleans up the session key, and then the process ends.

[0851] In this way, when using the message count value as part of the initialization vector, that is, part of the nonce value (for example, the lower count value), by also using the rest of the nonce value (for example, the upper count value), it is possible to eliminate the need for updating the session key or reduce the frequency of session key updates.

[0852] For example, when transmitting 4K data with a frame rate of 60 fps and a pixel count of 4096×2160 (horizontal × vertical), the time it takes for the nonce value to cycle through is: · When using the upper count value with a 16 - bit width in combination, it is 2 32 ÷60÷2163≒9 hours · When using the upper count value with a 20 - bit width in combination, it is 2 36 ÷60÷2163≒6 days · When using the upper count value with a 24 - bit width in combination, it is 2 40 ÷60÷2163≒98 days · When using the upper count value with a 28-bit width, 2 44 ÷60÷2163 ≈ 4 years · When using the upper count value with a 32-bit width, 2 48 ÷60÷2163 ≈ 69 years This results in

[0853] Here, when the power supply of the image sensor 1211 or the application processor 1212 is restarted (turned on after being off), key exchange is required before retransmitting the protected image data, so the session key is updated accordingly. For example, in general in-vehicle applications, the possibility of not restarting the power supply for more than 6 days is low, and the possibility of not restarting the power supply for more than 4 years is extremely low. Therefore, an upper count value with a width of 20 - 28 bits is sufficient. Of course, it is not limited to this, and a larger bit width can also be used.

[0854] For example, in the case of a fuel-powered vehicle, the power supply can be turned off during refueling. Even for a fuel-powered or rechargeable vehicle, if the power supply is turned off during vehicle inspection, key exchange is required before retransmitting the protected image data, so the session key is updated accordingly. For example, when an image sensor for IoT (Internet of Things or Intelligence of Things) is assumed, it is also assumed that the power supply will not be restarted, so an upper count value with a 32-bit width is sufficient. Of course, it is not limited to this, and a larger bit width can also be used.

[0855] Figure 94 is a flowchart for explaining a third processing example of the integrity operation value processing in which the image sensor 1211 transmits the integrity operation value.

[0856] In step S701, the security unit 1310 derives the session key.

[0857] In step S702, the message counter 1308 initializes the frame count value and sets it to 1.

[0858] In step S703, the extended mode - compliant CSI - 2 transmission circuit 1304 determines whether to end the session. If it is determined not to end the session, the process proceeds to step S704.

[0859] In step S704, the extended mode - compliant CSI - 2 transmission circuit 1304 determines whether to transmit an extended packet.

[0860] In step S704, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines not to transmit the extended packet, the process returns to step S703, and the following similar processing is repeated. On the other hand, in step S704, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines to transmit the extended packet, the process proceeds to step S705.

[0861] In step S705, the security unit 1310 prepares for the calculation of the integrity calculation value performed using the frame count value.

[0862] In step S706, the extended mode - compliant CSI - 2 transmission circuit 1304 transmits the extended packet.

[0863] In step S707, the extended mode - compliant CSI - 2 transmission circuit 1304 determines whether the transmission other than the frame end within the frame is completed. In step S707, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines that the transmission other than the frame end within the frame is not completed, the process returns to step S703, and the following similar processing is repeated. On the other hand, in step S707, if the extended mode - compliant CSI - 2 transmission circuit 1304 determines that the transmission other than the frame end within the frame is completed, the process proceeds to step S708.

[0864] In step S708, the security unit 1310 completes the calculation of the integrity calculation value performed using the frame count value.

[0865] In step S709, the extended mode - compliant CSI - 2 transmission circuit 1304 transmits the integrity operation value together with the frame end.

[0866] In step S710, the message counter 1308 determines whether the frame count value has been counted up to a specified value. In step S710, if the message counter 1308 determines that the frame count value has not been counted up to the specified value, the process proceeds to step S711.

[0867] In step S711, the message counter 1308 increments the frame count value. Then, the process returns to step S703, and the following similar processes are repeated.

[0868] On the other hand, in step S710, if the message counter 1308 determines that the frame...

Claims

1. a communication unit that, when communicating with another information processing device and communicating frame data including image data, transmits a packet including a packet header and packet data to the other information processing device or receives a packet from the other information processing device; a protection unit that generates, verifies, or decrypts protection data of the packet data using a session key; a nonce updating unit that updates a nonce value for the session key every time the protected data is generated, verified, or decrypted; Equipped with The image data is stored in the packet data; said data communication including transmitting or receiving a part or all of said nonce value; A part or all of the nonce value is transmitted or received stored outside the packet header. Information processing device.

2. At least one of the communication unit, the protection unit, and the nonce update unit selects one mode from at least two modes of a first MAC mode, a second MAC mode, and a non-MAC mode. The information processing device according to claim 1 .

3. the communication unit performs command communication to transmit a command related to the data communication to the other information processing device or to receive a command related to the data communication from the other information processing device; the command communication includes transmitting or receiving a read command requesting to read information in the information processing device or information in the other information processing device; the data communication includes sending or receiving a frame start signal indicating that the frame is beginning, and sending or receiving a frame end signal indicating that the frame is ending; The frames may be transmitted or received periodically; A part or all of the nonce value is transmitted or received as a read response in response to the read command within a period between the frame end and the next frame start. The information processing device according to claim 1 .

4. the frame includes embedded data stored within the packet data; The embedded data includes attributes representing the image data or information related to the image data; A part or all of the nonce value is transmitted or received stored within the embedded data. The information processing device according to claim 1 .

5. the communication unit performs command communication to transmit a command related to the data communication to the other information processing device or to receive a command related to the data communication from the other information processing device; the command communication includes transmitting or receiving a read command requesting to read information in the information processing device or information in the other information processing device; the data communication includes sending or receiving a frame start signal indicating that the frame is beginning, and sending or receiving a frame end signal indicating that the frame is ending; A part or all of the nonce value is transmitted or received within a period between the frame start and the frame end as a read response in response to the read command. The information processing device according to claim 1 .

6. the frame includes non-image data stored within the packet data; Some or all of the nonce value is transmitted or received stored within the non-image data. The information processing device according to claim 1 .

7. the packet is an extended packet, the packet header is an extended packet header, A part or all of the remaining part of the nonce value is stored in the extended packet header and transmitted or received. The information processing device according to claim 1 .

8. the data communication includes sending or receiving a frame start signal indicating that the frame is beginning, and sending or receiving a frame end signal indicating that the frame is ending; A part or all of the nonce value is stored in the frame start or the frame end as a part or all of the packet data and transmitted or received. The information processing device according to claim 1 .

9. A part or all of the nonce value is used as part of an initialization vector when computing the protection data; The initialization vector includes information related to an extended virtual channel and source ID of the communication. The information processing device according to claim 1 .

10. A part or all of the nonce value is used as part of an initialization vector when computing the protection data; The initialization vector includes information related to a virtual channel, a data type, an extended data type, a session ID, or a final destination ID for the communication. The information processing device according to claim 1 .

11. A part or all of the nonce value includes a frame number and an additional frame number. The information processing device according to claim 1 .

12. A part or all of the nonce value is used as part of an initialization vector when computing the protection data; The initialization vector includes information identifying a video stream or an audio stream. The information processing device according to claim 1 .

13. the frame includes embedded data stored within the packet data indicating a vendor specific code; A part or all of the nonce value is transmitted or received stored in embedded data indicating the vendor-specific code. The information processing device according to claim 1 .

14. the frame includes second image data or user defined data for a second virtual channel that is different from the image data for the first virtual channel; The number of lines of the image data and the second image data or the user-defined data is unified. The information processing device according to claim 1 .

15. the packet is an extended packet, the packet header is an extended packet header, the nonce value includes a message count value that is updated every time the extended packet that satisfies a predetermined count condition is transmitted or received, The message count value is stored in the extended packet header. The information processing device according to claim 1 .

16. the packet is an extended packet, the packet header is an extended packet header, a message counter that updates a message count value every time the extended packet that satisfies a predetermined count condition is transmitted or received, the message count value is stored in the extended packet header; The nonce update unit and the message counter are separate entities. The information processing device according to claim 1 .

17. the packet is an extended packet, the packet header is an extended packet header, the extended packet includes an extended packet footer, and the protector generates, verifies, or decrypts an integrity operation value using the session key; The integrity calculation value protects at least the packet data and is stored in part or in whole within the extended packet footer. The information processing device according to claim 1 .

18. The protection unit generates, verifies, or decrypts an integrity operation value using the session key; The integrity calculation value protects at least the image data and is stored in a packet data different from the image data. The information processing device according to claim 1 .

19. A pixel that outputs image data; a communication unit that, when communicating with another information processing device and communicating data of a frame including the image data, transmits a packet including a packet header and packet data to the other information processing device or receives a packet from the other information processing device; a protection unit that generates, verifies, or decrypts protection data of the packet data using a session key; a nonce updating unit that updates a nonce value for the session key every time the protected data is generated, verified, or decrypted; having The image data is stored in the packet data; said data communication including transmitting or receiving a part or all of said nonce value; A part or all of the nonce value is transmitted or received stored outside the packet header. A mobile device including an information processing device.

20. a communication unit that, when communicating with another information processing device and communicating frame data including image data, transmits a packet including a packet header and packet data to the other information processing device or receives a packet from the other information processing device; a protection unit that generates, verifies, or decrypts protection data of the packet data using a session key; a nonce updating unit that updates a nonce value for the session key every time the protected data is generated, verified, or decrypted; having The image data is stored in the packet data; said data communication including transmitting or receiving a part or all of said nonce value; A part or all of the nonce value is transmitted or received stored outside the packet header. A communication system including an information processing device.

Images