Transmitting device, receiving device, and communication system

Abstract

The present invention relates to a transmitter, a receiver and a communication system which can be used in more diverse applications and can be adapted to rules that prohibit packet modification during the transmission path. This transmitter adds an extension packet header different from the physical layer packet header to packet data in which transmission data is packaged, and generates an Application Specific payload restricted as a protected range that is to be protected with prohibition against modification during the transmission path, and adds at least a prescribed physical layer packet header to the Application Specific payload to generate a physical layer packet. The receiver receives the sent physical layer packet and acquires the Application Specific payload from the packet. This invention may be applied for example to communication systems for use inside of mobile devices or for use in connections to vehicle-mounted cameras.

Description

Transmitting device, receiving device, and communication system

[0001] The present disclosure relates to a transmitting device, a receiving device, and a communication system, and in particular to a transmitting device, a receiving device, and a communication system that can accommodate a wider variety of applications and can comply with regulations prohibiting packet modification on transmission paths.

[0002] CSI (Camera Serial Interface)-2 ver. 4.0, which is currently undergoing standardization, defines two types of packet structures: one that uses C-PHY in the physical layer, and one that uses D-PHY in the physical layer.

[0003] In recent years, the CSI-2 standard has been used not only in mobile devices but also in a wide range of applications, including automotive and IoT (Internet of Things), and as a result, it is expected that the existing packet structure will not be able to support these applications. Therefore, the MIPI (Mobile Industry Processor Interface) Alliance is considering extending the packet structure, such as the existing packet header and packet footer, to support a wider range of applications.

[0004] Furthermore, Patent Document 1 proposes a system that utilizes the CSI-2 standard to reduce the number of data buses when connecting a processing device and a plurality of image sensors.

[0005] JP 2017-211864 A

[0006] As mentioned above, there are plans to expand the packet structure of the CSI-2 standard, but in doing so, it is necessary to maintain compatibility with the existing CSI-2 standard while enabling it to transmit more information and accommodate a wider range of applications. At the same time, it is necessary to ensure that the regulations prohibiting packet modification along the transmission path are not violated.

[0007] The present disclosure has been made in light of the above circumstances, and is intended to accommodate a wider variety of applications and to comply with regulations prohibiting packet alteration on transmission paths.

[0008] A transmitting device according to a first aspect of the present disclosure includes an Application Specific payload generating unit that adds an extended packet header different from that for the physical layer to packet data that packs data to be transmitted, and generates an Application Specific payload that is limited as a protection range that should be protected by prohibiting modification on the transmission path, and a packet generating unit that adds at least a packet header for a predetermined physical layer to the Application Specific payload, and generates a packet for that physical layer.

[0009] In a first aspect of the present disclosure, an extended packet header different from that for the physical layer is added to packet data that packs data to be transmitted, an Application Specific payload is generated that is limited as a protection range that should be protected by prohibiting modification on the transmission path, and at least a packet header for a specified physical layer is added to the Application Specific payload to generate a packet for that physical layer.

[0010] A receiving device according to a second aspect of the present disclosure includes a packet receiving unit that receives a packet for a physical layer in which an extended packet header different from that for the physical layer is added to packet data that packs data to be transmitted, and in which at least a packet header for a predetermined physical layer is added to an Application Specific payload that is limited as a protection range that should be protected by prohibiting modification on the transmission path, and an Application Specific payload acquisition unit that acquires the Application Specific payload from the packet.

[0011] In a second aspect of the present disclosure, an extended packet header different from that for the physical layer is added to packet data that packs data to be transmitted, and a packet for the physical layer to which at least a packet header for a predetermined physical layer is added is received for an Application Specific payload that is limited as a protection range that should be protected by prohibiting modification on the transmission path, and the Application Specific payload is obtained from the packet.

[0012] A communication system according to a third aspect of the present disclosure includes a transmitting device having a packet generating unit that adds an extended packet header different from that for the physical layer to packet data that is obtained by packing data to be transmitted, thereby generating an Application Specific payload that is limited as a protection range that should be protected by prohibiting modification on the transmission path, and adds at least a packet header for a predetermined physical layer to the Application Specific payload to generate a packet for the physical layer; a packet receiving unit that receives the packet for the physical layer transmitted from the packet generating unit; and an Application Specific payload acquiring unit that acquires the Application Specific payload from the packet.

[0013] In a third aspect of the present disclosure, an extended packet header different from that for the physical layer is added to packet data obtained by packing data to be transmitted, an Application Specific Payload is generated that is limited as a protection range that should be protected by prohibiting alteration on a transmission path, and at least a predetermined packet header for the physical layer is added to the Application Specific Payload to generate a packet for the physical layer.The transmitted physical layer packet is then received, and the Application Specific Payload is obtained from the packet.

[0014] 1 is a block diagram showing a configuration example of a first embodiment of a communication system to which the present technology is applied. FIG. 2 is a block diagram showing a configuration example of a second embodiment of a communication system to which the present technology is applied. FIG. 3 is a diagram showing a first structural example of the overall packet structure of an extended packet for D-PHY. FIG. 4 is a diagram showing a first structural example of the packet structure of an extended short packet for D-PHY. FIG. 5 is a diagram showing a first structural example of the packet structure of an extended long packet for D-PHY. FIG. 6 is a diagram showing a first structural example of the overall packet structure of an extended packet for C-PHY. FIG. 7 is a diagram showing a first structural example of the packet structure of an extended short packet for C-PHY. FIG. 8 is a block diagram showing a configuration example of an image sensor. FIG. 9 is a block diagram showing a configuration example of an application processor. FIG. 10 is a flowchart illustrating processing for transmitting packets by an image sensor. FIG. 11 is a flowchart illustrating extended mode transmission processing. FIG. 12 is a flowchart illustrating processing for receiving packets by an application processor. FIG. 13 is a flowchart illustrating extended mode reception processing. FIG. 14 is a diagram showing a second structural example of the overall packet structure of an extended packet for D-PHY. FIG. 15 is a diagram showing a second structural example of the packet structure of an extended long packet for D-PHY. FIG. 10 is a diagram showing a second structural example of the packet structure of an extended long packet for C-PHY. FIG. 11 is a block diagram showing a modified example of the configuration for switching between D-PHY and C-PHY. FIG. 12 is a block diagram showing a structural example of a third embodiment of a communication system to which the present technology is applied. FIG. 13 is a diagram showing a structural example of an extended packet for D-PHY that complies with the provision for prohibiting packet modification. FIG. 14 is a diagram showing a structural example of an extended packet for C-PHY that complies with the provision for prohibiting packet modification. FIG. 15 is a flowchart illustrating packet transmission and reception processing that complies with the provision for prohibiting packet modification. FIG. 16 is a block diagram showing a structural example of an image sensor that complies with the provision for prohibiting packet modification. FIG. 17 is a block diagram showing a structural example of an application processor that complies with the provision for prohibiting packet modification.1 is a block diagram illustrating an example of the configuration of an embodiment of a computer to which the present technology is applied.

[0015] Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

[0016] <Configuration Example of Communication System> FIG. 1 is a block diagram showing a configuration example of a first embodiment of a communication system to which the present technology is applied.

[0017] 1, the communication system 11 is configured by connecting an image sensor 21 and an application processor 22 via a bus 23. For example, the communication system 11 is used for CSI-2 connection inside an existing mobile device such as a smartphone.

[0018] The image sensor 21 is configured by incorporating, for example, a lens, an imaging element (neither of which are shown), and an extended mode-compatible CSI-2 transmission circuit 31. For example, the image sensor 21 transmits image data of an image captured by the imaging element to the application processor 22 via the extended mode-compatible CSI-2 transmission circuit 31.

[0019] The application processor 22 is configured by incorporating an extended mode-compatible CSI-2 receiver circuit 32 together with an LSI (Large Scale Integration) that performs processing according to various applications executed in a mobile device that includes the communication system 11. For example, the application processor 22 receives image data transmitted from the image sensor 21 using the extended mode-compatible CSI-2 receiver circuit 32, and can process the image data according to the application using the LSI.

[0020] The bus 23 is a communication path that transmits signals in accordance with the CSI-2 standard, and the possible transmission distance for signals is, for example, approximately 30 cm. The bus 23 also connects the image sensor 21 and the application processor 22 via multiple signal lines (I2C, CLKP / N, D0P / N, D1P / N, D2P / N, D3P / N) as shown in the figure.

[0021] The extended-mode-compatible CSI-2 transmitter circuit 31 and the extended-mode-compatible CSI-2 receiver circuit 32 support communication in extended mode, which is an extension of the CSI-2 standard, and can transmit and receive signals to and from each other. Note that the detailed configurations of the extended-mode-compatible CSI-2 transmitter circuit 31 and the extended-mode-compatible CSI-2 receiver circuit 32 will be described later with reference to FIGS. 9 and 10.

[0022] FIG. 2 is a block diagram showing a configuration example of a second embodiment of a communication system to which the present technology is applied.

[0023] 2, the communication system 11A is configured such that the image sensor 21 and the serializer 25 are connected via a bus 24-1, the application processor 22 and the deserializer 26 are connected via a bus 24-2, and the serializer 25 and the deserializer 26 are connected via a bus 27. For example, the communication system 11A is used for connection in an existing vehicle-mounted camera.

[0024] Here, the image sensor 21 and the application processor 22 are configured in the same manner as the image sensor 21 and the application processor 22 in FIG. 1, and a detailed description thereof will be omitted.

[0025] Like the bus 23 in FIG. 1, the buses 24-1 and 24-2 are communication paths that transmit signals in accordance with the CSI-2 standard, and are configured with multiple signal lines (HS-GPIO, I2C, CLKP / N, D0P / N, D1P / N, D2P / N, D3P / N) as shown in the figure.

[0026] The serializer 25 is configured to include a CSI-2 receiver circuit 33 and a SerDes (Serializer Deserializer) transmitter circuit 34. For example, the serializer 25 acquires a bit-parallel signal transmitted from the image sensor 21 by the CSI-2 receiver circuit 33 performing communication conforming to the normal CSI-2 standard with the extended mode-compatible CSI-2 transmitter circuit 31. The serializer 25 then converts the acquired signal into a bit-serial signal, and transmits the signal to the deserializer 26 by the SerDes transmitter circuit 34 performing one-lane communication with the SerDes receiver circuit 35.

[0027] The deserializer 26 is configured to include a SerDes receiver circuit 35 and a CSI-2 transmitter circuit 36. For example, the deserializer 26 acquires a bit-serial signal transmitted by the SerDes receiver circuit 35 performing one-lane communication with the SerDes transmitter circuit 34. The deserializer 26 then converts the acquired signal into a bit-parallel signal, and transmits the signal to the application processor 22 by the CSI-2 transmitter circuit 36 ​​performing communication with the extended mode-compatible CSI-2 receiver circuit 32 in accordance with the normal CSI-2 standard.

[0028] The bus 27 is a communication path that transmits signals in accordance with standards such as A-PHY and FPD (Flat Panel Display)-LINK III, and the transmission distance over which signals can be transmitted is, for example, approximately 15 m.

[0029] The communication systems 11 and 11A configured in this manner can transmit and receive data in packets with an extended packet structure, as described below, using the extended mode-compatible CSI-2 transmitter circuit 31 and the extended mode-compatible CSI-2 receiver circuit 32. This allows them to support a wider variety of applications, such as RAW24, SmartROI (Region of Interest), and GLD (Graceful Link Degradation), as described below.

[0030] <First structural example of packet structure> A first structural example of the packet structure of a packet used in communication between the extended mode-compatible CSI-2 transmitter circuit 31 and the extended mode-compatible CSI-2 receiver circuit 32 will be described with reference to Figures 3 to 8.

[0031] FIG. 3 shows the overall packet structure of a packet used in the extended mode of CSI-2 when the physical layer is D-PHY (hereinafter referred to as an extended packet for D-PHY).

[0032] As shown in Figure 3, the packet header and footer of the D-PHY extended packet have the same packet structure as the existing CSI-2 standard. For example, the packet header stores VC (Virtual Channel), which indicates the number of virtual channels, DataType, which indicates the type of data, WC (Word Count), which indicates the data length of the payload, and VCX / ECC. The packet footer stores CRC (Cyclic Redundancy Check).

[0033] In the existing CSI-2 standard, data types transmitted in the packet header are defined as reserved from 0x38 to 0x3F. Therefore, in the D-PHY extended packet, new setting information is defined to identify the extended mode on the receiving side, using the existing reserved data types.

[0034] For example, the following data types are defined: ・When DataType[5:3]=3'b111, it is extension mode ・DataType[2]=Reserve (RES: reserved for future extension) ・DataType[1:0]=extension mode type (four extension modes are available)

[0035] That is, among the data types 0x38 to 0x3F defined as reserved in the existing CSI-2 standard, for example, DataType[5:3] is defined as extended mode setting information, and DataType[1:0] is defined as extended type setting information. The extended mode setting information indicates whether or not the mode is extended. For example, if DataType[5:3] is 3'b111, it indicates extended mode. Furthermore, when four types of extended mode are provided, namely extended mode 0, extended mode 1, extended mode 2, and extended mode 3, the extended type setting information indicates which of these types is used. For example, if DataType[1:0] is 2'b00, it indicates that the type of extended mode is extended mode 0.

[0036] In extended mode 0 (DataType[1:0]=2'b00), for example, a packet structure in which the payload is separated into four parts is defined. That is, as shown in FIG. 3, the payload in extended mode 0 is separated into an extended packet header (ePH), an optional extended packet header (OePH), a legacy payload, and an optional extended packet footer (OePF). Note that the extended packet header may be transmitted repeatedly.

[0037] The extended packet header is placed at the beginning of the payload in the existing CSI-2 standard and must be transmitted in extended mode. For example, as shown in the figure, the extended packet header consists of configuration information such as an SROI identification flag, an extended VC (Virtual Channel), an extended DataType, an OePH selection flag, and an OePF selection flag. Here, the extended VC extends the VC, which was 4 bits in the existing CSI-2 standard, to 8 bits, and the extended DataType extends the DataType, which was 4 bits in the existing CSI-2 standard, to 8 bits.

[0038] For example, in a D-PHY packet, the VC of the existing packet header already has 4 bits, and by defining the extended VC of the extended packet header as 4 bits, a total of 8 bits can be achieved. Specifically, OePH[7:0] = {5'h00,RSID,XY_POS,MC} and OePF[3:0] = {3'h0,pCRC} can be defined, allowing for on / off control of packet transmission as required for each application.

[0039] The optional extended packet header and optional extended packet footer are selectively transmitted depending on the application.

[0040] The legacy payload corresponds to the same payload as the existing CSI-2 standard.

[0041] In this way, by setting the extended packet header, optional extended packet header, and optional extended packet footer as needed, it becomes possible to transmit data for a variety of purposes. Furthermore, the data transmitted in the extended packet header, optional extended packet header, and optional extended packet footer is encoded with a 26-bit + 6-bit ECC (Error Correction Code). This allows the existing packet header circuit to be reused, suppressing increases in circuit size and improving error tolerance.

[0042] As a specific application example of such an extended packet for D-PHY, Fig. 4 shows the packet structure of a short packet (hereinafter referred to as an extended short packet for D-PHY) used in the extended mode of CSI-2 when the physical layer is D-PHY. Similarly, Fig. 5 shows the packet structure of a long packet (hereinafter referred to as an extended long packet for D-PHY) used in the extended mode of CSI-2 when the physical layer is D-PHY.

[0043] 4, in the D-PHY extended short packet, the data type extended type setting information stored in the packet header indicates that the type of extended mode is extended mode 0 (DT[5:0]=0x1C(5'b111_0_0)).Furthermore, the data type short packet setting information stored in the extended packet header indicates that it is a short packet (DT[7:0]=0x00 (Frame Start Code (Short Packet))).

[0044] In this way, when the extended mode is selected and the data type stored in the extended packet header is DT[7:0]=0x00-0x0F, the packet is treated as an extended short packet, and the optional extended packet header always contains data including the Short Packet Data Field of the extended short packet. This Short Packet Data Field is the same as that defined in the existing CSI-2 standard.

[0045] When transmitting an extended short packet, the MC (MessageCount for GLD) and RSID (Vehicle Line Number and SourceID) of the optional extended packet header may be transmitted, but the legacy payload and pCRC are not required and therefore must not be transmitted. If they are transmitted by mistake, they will be ignored by the receiving side.

[0046] 4, the extended short packet can expand the bit width of the data type and virtual channel compared to the extended short packet conforming to the existing CSI-2 standard, and can support various applications defined by the optional extension packet header. Furthermore, if these functions are not required, the extended short packet conforming to the existing CSI-2 standard may be transmitted together with the extended long packet.

[0047] In an extended long packet for D-PHY such as that shown in Figure 5, the data type extension type setting information stored in the packet header indicates that the type of extension mode is extended mode 0 (DT[5:0] = 0x1C(5'b111_0_0)). Also, the data type short packet setting information stored in the extended packet header indicates that it is a type other than a short packet (DT[7:0] is other than 0x00 to 0x0F (= extended long packet)). Therefore, data including a short packet data field is not transmitted in an extended long packet.

[0048] Furthermore, in accordance with the settings of the extension packet header, the optional extension packet header, legacy payload, and optional extension packet footer are stored in a payload conforming to the existing CSI-2 standard and transmitted. Because they are stored in the existing payload and transmitted in this manner, the existing SerDes transmitter circuit 34 and SerDes receiver circuit 35 (FIG. 2) recognize them as the same as image data transmitted in the existing payload, and transmit them as is to the subsequent stages.

[0049] The application processor 22 at the final stage can determine the extended mode based on the data type DT[5:0] in the packet header. Therefore, the application processor 22 can interpret the contents of the payload, starting from the extended packet header, and extract the desired extended mode data.

[0050] Fig. 6 shows the overall packet structure of a packet used in the extended mode of CSI-2 when the physical layer is C-PHY (hereinafter referred to as an extended packet for C-PHY). Note that in the extended packet for C-PHY shown in Fig. 6, a description of the configuration common to the extended packet for D-PHY shown in Fig. 3 will be omitted, and only different configurations will be described.

[0051] For example, in an extended packet for C-PHY, the extended mode is identified by the data type, as in the extended packet for D-PHY in Figure 3, and all data corresponding to each application executed by the application processor 22 is embedded in the payload and transmitted.

[0052] As shown in Figure 6, the C-PHY extended packet transmits the packet header twice, just like a C-PHY packet conforming to the existing CSI-2 standard, and arranges data in 16-bit units because C-PHY converts 16 bits to 7 symbols. An extended packet header is placed at the beginning of the payload, but with regard to virtual channels, the beginning of the existing packet header in the case of C-PHY was reserved for this purpose, so virtual channels are not stored in the extended packet header. Of course, virtual channels may be stored in the extended packet header, just like a D-PHY extended packet.

[0053] Furthermore, because the optional extended packet header and optional extended packet footer have a large number of bits, a flag called OePHF is prepared, and when this flag is 1, OePH / OePF information is transmitted next. After the ePH information and OePH information, a CRC is transmitted as an extended packet header, and a packet header configured in the same way is transmitted twice. In this way, by maintaining the same structure as the existing mechanism for transmitting packet headers twice, it is possible to achieve both circuit reuse and error resistance.

[0054] As a specific application example of such an extended packet for C-PHY, Fig. 7 shows the packet structure of a short packet (hereinafter referred to as an extended short packet for C-PHY) used in the extended mode of CSI-2 when the physical layer is C-PHY. Similarly, Fig. 8 shows the packet structure of a long packet (hereinafter referred to as an extended long packet for C-PHY) used in the extended mode of CSI-2 when the physical layer is C-PHY.

[0055] The extended short packet for C-PHY shown in FIG. 7 has a packet structure that is not significantly different from that of the extended short packet for D-PHY shown in FIG. 4, and the extended long packet for C-PHY shown in FIG. 8 has a packet structure that is not significantly different from that of the extended long packet for D-PHY shown in FIG. 5.

[0056] <Configuration Example of Image Sensor and Application Processor> FIG. 9 is a block diagram showing a configuration example of an image sensor 21 including an extended mode-compatible CSI-2 transmission circuit 31.

[0057] 9 , the image sensor 21 includes, in addition to an extended mode-compatible CSI-2 transmission circuit 31, pixels 41, an AD converter 42, an image processing unit 43, a pixel CRC calculation unit 44, a physical layer processing unit 45, an I2C / I3C slave 46, and a register 47. The extended mode-compatible CSI-2 transmission circuit 31 also includes a packing unit 51, a packet header generation unit 52, an extended packet header generation unit 53, an extended packet footer generation unit 54, selection units 55 and 56, a CRC calculation unit 57, a lane distribution unit 58, a CCI slave 59, and a controller 60.

[0058] The pixels 41 output analog pixel signals corresponding to the amount of light received, and an analog-to-digital converter (ADC) 42 converts the pixel signals output from the pixels 41 into digital signals and supplies them to an image processing unit 43. The image processing unit (ISP) 43 supplies image data obtained by performing various image processes on an image based on the pixel signals to a pixel CRC calculation unit 44 and a packing unit 51. The image processing unit 43 also supplies a data enable signal data_en indicating whether the image data is valid to the packing unit 51 and the controller 60.

[0059] The pixel CRC calculation unit 44 calculates and obtains a CRC for each pixel in the image data supplied from the image processing unit 43, and supplies the CRC to an extended packet footer generation unit 54.

[0060] The physical layer processing unit 45 can perform physical layer processing for both C-PHY and D-PHY. For example, the physical layer processing unit 45 performs physical layer processing for C-PHY when a C-layer enable signal cphy_en supplied from the controller 60 is enabled, and performs physical layer processing for D-PHY when the C-layer enable signal cphy_en is disabled. The physical layer processing unit 45 then transmits the packets divided into four lanes by the lane distribution unit 58 to the application processor 22.

[0061] The I2C / I3C slave 46 performs communication under the initiative of an I2C / I3C master 72 (FIG. 10) of the application processor 22, based on the I2C (Inter-Integrated Circuit) or I3C (Improved Inter Integrated Circuits) standard.

[0062] Various settings transmitted from the application processor 22 are written to the register 47 via the I2C / I3C slave 46 and the CCI slave 59. The settings written to the register 47 include, for example, communication settings in accordance with the CSI-2 standard, extended mode settings indicating whether or not extended mode is used, and fixed communication settings required for communication in extended mode.

[0063] The packing unit 51 performs packing processing to store the image data supplied from the image processing unit 43 in the payload of a packet, and supplies the payload to the selection unit 55 and the lane distribution unit 58 .

[0064] When the packet header generation unit 52 is instructed to generate a packet header in accordance with a packet header generation instruction signal ph_go supplied from the controller 60 , the packet header generation unit 52 generates a packet header and supplies it to the selection unit 55 and the lane distribution unit 58 .

[0065] That is, the packet header generator 52 generates a packet header that stores setting information indicating conditions set for data transmitted in a packet, such as a data type indicating the type of data, in accordance with the existing CSI-2 standard. Furthermore, the packet header generator 52 stores extended mode setting information indicating whether the extended mode uses an extension header in an unused area of ​​the data type, which is setting information indicating the type of data transmitted in a packet and is defined as unused in the existing CSI-2 standard. Furthermore, the packet header generator 52 stores extended type setting information indicating which of multiple types of extended modes is used in the unused area.

[0066] The extension packet header generator 53 generates an extension packet header and an optional extension packet header in accordance with an extension packet header generation instruction signal eph_go and an extension packet header enable signal ePH_en supplied from the controller 60, and supplies them to the selector 56 and the lane distributor 58. In addition, the extension packet header generator 53 is supplied with an in-vehicle row number, a source ID (identification), and the like depending on the application of the image sensor 21, and stores them in the extension packet header or the optional extension packet header as necessary.

[0067] 3, for example, in addition to the packet header generated by the packet header generator 52. Furthermore, when transmitting an optional extension packet header, the extension packet header generator 53 stores optional extension packet header setting information indicating that an optional extension packet header is to be transmitted in the extension packet header as optional extension packet header setting information (OePH[7:0]) indicating whether or not an optional extension packet header is to be transmitted, and generates the optional extension packet header following the extension packet header.

[0068] The extended packet footer generator 54 generates an optional extended packet footer in accordance with the extended packet footer generation instruction signal epf_go and the extended packet header enable signal ePF_en supplied from the controller 60 , and supplies the generated optional extended packet footer to the selector 56 and the lane distributor 58 .

[0069] That is, when a packet transmitted in extended mode is an extended long packet that stores data transmitted as a payload in the existing CSI-2 standard, the extended packet footer generation unit 54 generates an optional extended packet footer that is placed following the legacy payload in which the data is stored.

[0070] The controller 60 also supplies the packet header generator 52, the extended packet header generator 53, and the extended packet footer generator 54 with a C-layer enable signal cphy_en. When the C-layer enable signal cphy_en indicates valid, the packet header generator 52 generates a packet header for C-PHY, the extended packet header generator 53 generates an extended packet header and optional extended packet header for C-PHY, and the extended packet footer generator 54 generates an optional extended packet footer for C-PHY. On the other hand, when the C-layer enable signal cphy_en indicates invalid, the packet header generator 52 generates a packet header for D-PHY, the extended packet header generator 53 generates an extended packet header and optional extended packet header for D-PHY, and the extended packet footer generator 54 generates an optional extended packet footer for D-PHY.

[0071] If the C-layer enable signal cphy_en is valid in accordance with the C-layer enable signal cphy_en supplied from the controller 60, the selector 55 selects the packet header supplied from the packet header generator 52 and supplies it to the selector 56. On the other hand, if the C-layer enable signal cphy_en is invalid, the selector 55 selects the payload supplied from the packing unit 51 and supplies it to the selector 56.

[0072] In accordance with a data selection signal data_sel supplied from the controller 60, the selector 56 selects one of the packet header or payload selectively supplied via the selector 55, the extended packet header and optional extended packet header supplied from the extended packet header generator 53, and the optional extended packet footer supplied from the extended packet footer generator 54, and supplies the selected one to the CRC calculator 57.

[0073] The CRC calculation unit 57 calculates the CRC of the packet header, payload, extended packet header, optional extended packet header, or optional extended packet footer selectively supplied via the selection unit 56, and supplies the CRC to the lane distribution unit 58.

[0074] Under the control of the controller 60, the lane distribution unit 58 distributes the payload supplied from the packing unit 51, the packet header supplied from the packet header generation unit 52, the extended packet header and optional extended packet header supplied from the extended packet header generation unit 53, the optional extended packet footer supplied from the extended packet footer generation unit 54, and the CRC supplied from the CRC calculation unit 57 to four lanes in accordance with the CSI-2 standard, and supplies them to the physical layer processing unit 45.

[0075] A CCI (Camera Control Interface) slave 59 performs communication based on the CSI-2 standard under the initiative of a CCI master 88 (FIG. 10) of the application processor 22 .

[0076] The controller 60 reads out various settings stored in the register 47 and, according to those settings, controls each block constituting the extended mode-compatible CSI-2 transmission circuit 31. For example, the controller 60 controls switching between transmitting packets with a packet structure conforming to the existing CSI-2 standard and transmitting packets with a packet structure in extended mode, depending on the content of the data to be transmitted.

[0077] The image sensor 21 is configured in this way, and can generate an extended packet having the packet structure as described with reference to FIGS. 3 to 8 and transmit it to the application processor 22.

[0078] FIG. 10 is a block diagram showing an example of the configuration of an application processor 22 including an extended mode-compatible CSI-2 receiving circuit 32.

[0079] 10 , in addition to the extended mode-compatible CSI-2 receiver circuit 32, the application processor 22 is configured to include a physical layer processing unit 71, an I2C / I3C master 72, a register 73, and a controller 74. The extended mode-compatible CSI-2 receiver circuit 32 is configured to include a packet header detection unit 81, a lane merging unit 82, an interpretation unit 83, selection units 84 and 85, a CRC calculation unit 86, an unpacking unit 87, and a CCI master 88.

[0080] The physical layer processing unit 71 can perform physical layer processing for both C-PHY and D-PHY. As described above, the physical layer processing unit 45 of the image sensor 21 performs physical layer processing for either C-PHY or D-PHY, and the physical layer processing unit 71 performs the same physical layer processing as that performed in the physical layer processing unit 45.

[0081] The I2C / I3C master 72 takes the lead in communicating with the I2C / I3C slave 46 (FIG. 9) of the image sensor 21 based on the I2C or I3C standard.

[0082] The register 73 records various settings to be written by the controller 74 to the register 47 of the image sensor 21 .

[0083] The controller 74 controls each block that constitutes the application processor 22 .

[0084] The packet header detection unit 81 detects a packet header from the packet supplied from the physical layer processing unit 71 and checks the data type stored in the packet header. If the extension mode setting information in the data type of the packet header indicates the extension mode (DataType[5:3]=3'b111), the packet header detection unit 81 supplies an extension mode detection flag indicating the extension mode to the interpretation unit 83, the selection unit 84, and the selection unit 85. The packet header detection unit 81 also supplies a merge enable signal mrg_en indicating whether or not to enable merging of the four divided lanes to the lane merging unit 82 based on the packet header.

[0085] That is, the packet header detection unit 81 detects a packet header that stores setting information (such as a data type) indicating conditions set for data transmitted in the packet in accordance with the existing CSI-2 standard. At this time, the packet header detection unit 81 outputs an extended mode detection flag in accordance with extended mode setting information indicating whether the extended mode uses an extended header, and the extended mode setting information is stored in an unused area defined as unused in the existing CSI-2 standard in the data type, which is setting information indicating the type of data transmitted in the packet. This allows the unit 81 to switch between receiving packets with a packet structure conforming to the existing CSI-2 standard and receiving packets with a packet structure in the extended mode. Furthermore, the packet header detection unit 81 recognizes which of multiple extended modes provided as extended modes the current mode is, in accordance with extended mode type information stored in an unused area of ​​the data type defined as unused in the existing CSI-2 standard.

[0086] When the merge enable signal mrg_en supplied from the packet header detection unit 81 is valid, the lane merging unit 82 merges the packets divided into four lanes supplied from the physical layer processing unit 71. Then, the lane merging unit 82 supplies the packet of one lane to the interpretation unit 83, the selection unit 84, and the selection unit 85.

[0087] If the extension mode detection flag supplied from the packet header detection unit 81 indicates the extension mode, the interpretation unit 83 reads the extension packet header, optional extension packet header, and optional extension packet footer from the packet supplied from the lane merging unit 82 based on the packet structure of the extension mode. Then, the interpretation unit 83 interprets the setting information stored in the extension packet header, optional extension packet header, and optional extension packet footer.

[0088] That is, the interpretation unit 83 receives an extension packet header placed at the beginning of a payload conforming to the existing CSI-2 standard as an extension header and interprets the setting information stored in the extension packet header. Furthermore, when the optional extension packet header setting information stored in the extension packet header indicates that an optional extension packet header that is selectively transmitted depending on the application is to be transmitted, the interpretation unit 83 receives the optional extension packet header following the extension packet header and interprets the setting information stored in the optional extension packet header. Furthermore, when a packet transmitted in extended mode is an extended long packet that stores data transmitted as a payload according to the existing CSI-2 standard, the interpretation unit 83 receives an optional extension packet footer placed following the legacy payload in which the data is stored and interprets the optional extension packet footer.

[0089] The interpretation unit 83 then reads out, for example, the vehicle row number and source ID stored in the optional extension packet header, and outputs them to a downstream LSI (not shown).

[0090] In addition, if the extended mode detection flag supplied from the packet header detection unit 81 does not indicate extended mode, that is, if a packet with an existing packet structure is supplied, the interpretation unit 83 stops without performing the above-mentioned processing.

[0091] The selector 84 selectively supplies data to an unpacking unit 87 based on the packet structure of the existing packet or the packet structure of the extended packet in accordance with the extension mode detection flag supplied from the packet header detector 81 .

[0092] The selector 85 selectively supplies data to a CRC calculator 86 based on the packet structure of the existing packet or the packet structure of the extended packet in accordance with the extension mode detection flag supplied from the packet header detector 81 .

[0093] The CRC calculation unit 86 calculates the CRC of the packet header, payload, extended packet header, optional extended packet header, or optional extended packet footer selectively supplied via the selection unit 85. If a CRC error is detected, the CRC calculation unit 86 outputs a CRC error detection signal indicating this to a downstream LSI (not shown).

[0094] The unpacking unit 87 performs an unpacking process to extract image data stored in the payload selectively supplied via the selection unit 84, and outputs the acquired image data to a downstream LSI (not shown).

[0095] The CCI master 88 takes the lead in communicating with the CCI slave 59 (FIG. 9) of the image sensor 21 based on the CSI-2 standard.

[0096] The application processor 22 is configured in this manner, and is able to receive an extension packet sent from the image sensor 21, interpret the setting information stored in the extension packet header, optional extension packet header, and optional extension packet footer, and acquire image data.

[0097] <Communication Processing> The communication processing performed by the image sensor 21 and the application processor 22 will be described with reference to FIGS.

[0098] FIG. 11 is a flowchart illustrating a process in which the image sensor 21 transmits a packet.

[0099] For example, processing begins when the image sensor 21 is connected to the application processor 22 via the bus 23. In step S11, the controller 60 determines whether or not to use the extended mode when starting communication with the application processor 22. For example, the controller 60 checks the extended mode setting stored in the register 47, and determines that the extended mode will be used if the extended mode setting indicating the use of the extended mode has been written by the application processor 22.

[0100] If the controller 60 determines in step S11 that the extended mode will not be used, the process proceeds to step S12.

[0101] In step S12, the I2C / I3C slave 46 receives the image data transmission start command transmitted from the application processor 22 (in step S54 of FIG. 13 , which will be described later). Furthermore, the I2C / I3C slave 46 receives communication settings conforming to the CSI-2 standard transmitted together with the transmission start command, and writes them into the register 47 via the CSI slave 59.

[0102] In step S13, the image sensor 21 performs conventional packet transmission processing, transmitting a packet with a packet structure conforming to the existing CSI-2 standard to the application processor 22 based on the communication settings stored in the register 47.

[0103] On the other hand, if the controller 60 determines in step S11 that the extended mode is to be used, the process proceeds to step S14.

[0104] In step S14, the I2C / I3C slave 46 receives fixed communication settings required for communication in extended mode (e.g., a copy of each PH / PF lane in GLD mode) and writes them to the register 47 via the CCI slave 59.

[0105] In step S15, the I2C / I3C slave 46 receives the image data transmission start command transmitted from the application processor 22 (in step S57 of FIG. 13 , which will be described later). Furthermore, the I2C / I3C slave 46 receives communication settings conforming to the CSI-2 standard transmitted together with the transmission start command, and writes them into the register 47 via the CSI slave 59.

[0106] In step S16, the controller 60 determines whether or not to start transmitting packets, and waits until it determines to start transmitting packets.

[0107] If it is determined in step S16 that packet transmission should be started, the process proceeds to step S17, where the controller 60 determines whether the data should be transmitted in the extended mode. Here, the controller 60 determines that the data should be transmitted in the extended mode depending on the content of the data to be transmitted, for example, if the data is data that would be transmitted in a use case of an application example described below.

[0108] If the controller 60 determines in step S17 that the data should be transmitted in extended mode, the process proceeds to step S18, where an extended mode transmission process (see FIG. 12) is performed to transmit an extended packet corresponding to the extended mode.

[0109] On the other hand, if the controller 60 determines in step S17 that the data is not to be transmitted in the extended mode, the process proceeds to step S19.

[0110] In step S19, the controller 60 determines whether or not to transmit a short packet. For example, the controller 60 determines to transmit a short packet at the start and end of a frame.

[0111] If the controller 60 determines in step S19 that a short packet is to be transmitted, the process proceeds to step S20. In step S20, the packet header generator 52 generates a packet header and transmits a short packet having a conventional packet structure to the application processor 22.

[0112] On the other hand, if the controller 60 determines in step S19 not to transmit a short packet (i.e., to transmit a long packet), the process proceeds to step S21. In step S21, the packing unit 51 stores the image data in the payload, and the CRC calculation unit 57 calculates the CRC to generate a long packet with a conventional packet structure, which is then transmitted to the application processor 22.

[0113] After the processing of step S18, step S20, or step S21, the process proceeds to step S22, where the controller 60 ends the packet transmission process. Thereafter, the process returns to step S16, and the process of transmitting the next packet is repeated in the same manner.

[0114] FIG. 12 is a flowchart illustrating the extended mode transmission process performed in step S18 of FIG.

[0115] In step S31, the packet header generation unit 52 generates a packet header that stores the VC, data type, WC, etc., and transmits it to the application processor 22. At this time, the packet header generation unit 52 writes, to the data type of the packet header, extended mode setting information (DataType[5:3]=3'b111) indicating the extended mode, and extended type setting information (DataType[1:0]=2'b00) identifying that the mode setting of the extended mode is extended mode 0.

[0116] In step S32, the application processor 22 determines whether to transmit an extended short packet. For example, the controller 60 determines to transmit an extended short packet at the start and end of a frame.

[0117] If the application processor 22 determines in step S32 that an extended short packet is to be transmitted, the process proceeds to step S33.

[0118] In step S33, the extension packet header generator 53 transmits an extension packet header in which the data type (DataType[7:0]) is set to short packet in the first byte of the payload. At this time, the extension packet header generator 53 performs various settings (e.g., OePH[7:0], OePF[3:0], etc.) to be stored in the extension packet header.

[0119] In step S34, the extended packet header generator 53 stores a frame number (FN: Frame Number) in the second byte of the payload and transmits it.

[0120] In step S35, the extension packet header generator 53 generates and transmits an optional extension packet header as shown in FIG. 4 in accordance with the setting (OePH[7:0]) made in step S33.

[0121] In step S36, the CRC calculation unit 57 calculates the CRC and transmits it as a packet footer.

[0122] On the other hand, if the application processor 22 determines in step S32 not to transmit an extended short packet (that is, to transmit a long packet), the process proceeds to step S37.

[0123] In step S37, the extension packet header generator 53 transmits an extension packet header in which the data type (DataType[7:0]) is set to other than a short packet in the first byte of the payload. At this time, the extension packet header generator 53 performs various settings (e.g., OePH[7:0], OePF[3:0], etc.) to be stored in the extension packet header.

[0124] In step S38, the extension packet header generator 53 generates and transmits an optional extension packet header as shown in FIG. 5 in accordance with the setting (OePH[7:0]) made in step S37.

[0125] In step S39, the packing unit 51 packs the image data supplied from the image processing unit 43, generates a legacy payload, and transmits it.

[0126] In step S40, the extended packet footer generating unit 54 generates and transmits an optional extended packet footer such as that shown in FIG. 4 in accordance with the setting (OePF[3:0]) made in step S37.

[0127] In step S41, the CRC calculation unit 57 calculates a CRC and transmits it as a packet footer.

[0128] After the processing of step S36 or S41, the extended mode transmission processing ends.

[0129] As described above, the image sensor 21 can generate and transmit an extended short packet or an extended long packet.

[0130] FIG. 13 is a flowchart illustrating the process of the application processor 22 receiving a packet.

[0131] For example, the process starts when the image sensor 21 is connected to the application processor 22 via the bus 23. In step S51, the controller 74 writes initial settings for the image sensor 21 (e.g., whether C-PHY or D-PHY is to be used as the physical layer) to the register 73, and transmits the settings to the image sensor 21 via the I2C / I3C master 72 and the CCI master 88. As a result, the initial settings are written to the register 47 of the image sensor 21.

[0132] In step S52, the controller 74 determines whether the image sensor 21 supports the extended mode. For example, the controller 74 can determine whether the image sensor 21 supports the extended mode by acquiring a setting value (e.g., extended PH / PF capability) stored in the register 47 of the image sensor 21 via the I2C / I3C master 72. Alternatively, the controller 74 can determine in advance whether the image sensor 21 supports the extended mode based on, for example, a manual input.

[0133] In step S53, the controller 74 determines whether the image sensor 21 supports the extended mode and whether the application executed by the application processor 22 requests the use of the extended mode.

[0134] If the controller 74 determines in step S53 that the image sensor 21 does not support the extended mode or that use of the extended mode is not desired, the process proceeds to step S54.

[0135] In step S54, the controller 74 transmits an image data transmission start command to the image sensor 21 via the I2C / I3C master 72. At this time, the controller 74 also transmits communication settings in accordance with the CSI-2 standard.

[0136] In step S55, the application processor 22 performs conventional packet reception processing to receive packets with a packet structure conforming to the existing CSI-2 standard, based on the communication settings transmitted in step S54.

[0137] On the other hand, if the controller 74 determines in step S53 that the image sensor 21 supports the extended mode and that the application executed by the application processor 22 requests the use of the extended mode, processing proceeds to step S56.

[0138] In step S56, before communication in the extended mode is started, the I2C / I3C master 72 transmits fixed communication settings required for communication in the extended mode, which are then written to the register 47 of the image sensor 21 (step S14 in FIG. 11).

[0139] In step S57, the controller 74 transmits an image data transmission start command to the image sensor 21 via the I2C / I3C master 72. At this time, the controller 74 also transmits communication settings in accordance with the CSI-2 standard.

[0140] In step S58, the packet header detection unit 81 determines whether packet reception has started by checking the data supplied from the physical layer processing unit 71, and waits until it determines that packet reception has started. For example, when the packet header detection unit 81 detects a packet header from the data supplied from the physical layer processing unit 71, it determines that packet reception has started.

[0141] If the packet header detection unit 81 determines in step S58 that packet reception has started, the process proceeds to step S59.

[0142] In step S59, the packet header detection unit 81 checks the data type of the packet header detected in step S58 and determines whether the packet that has started to be received is an extended packet that supports the extended mode. For example, if the extended mode setting information in the data type of the packet header indicates the extended mode (DataType[5:3]=3'b111), the packet header detection unit 81 determines that the packet that has started to be received is an extended packet.

[0143] In step S59, if the packet header detection unit 81 determines that the packet that has started to be received is an extended packet, the process proceeds to step S60, and extended mode reception processing (see FIG. 14) is performed to receive the extended packet.

[0144] On the other hand, if the packet header detection unit 81 determines in step S59 that the packet that has started to be received is not an extended packet, the process proceeds to step S61.

[0145] In step S61, the packet header detection unit 81 checks the data type (DataType[5:0]) of the packet header detected in step S58, and determines whether the packet that has started to be received is a short packet.

[0146] In step S61, if the packet header detection unit 81 determines that the packet that has started to be received is a short packet, the process proceeds to step S62. In step S62, the packet header detection unit 81 receives a short packet with a conventional packet structure transmitted from the image sensor 21.

[0147] On the other hand, if the packet header detection unit 81 determines in step S61 that the packet that has started to be received is not a short packet (i.e., reception of a long packet has started), the process proceeds to step S63. In step S63, the unpacking unit 87 receives the payload of the long packet with the conventional packet structure transmitted from the image sensor 21 and extracts the image data, and the CRC calculation unit 86 receives the WC+1 byte transmitted following the packet header as a CRC.

[0148] After step S60, step S62, or step S63, the process proceeds to step S64, where the controller 74 ends the packet reception process. Thereafter, the process returns to step S58, and the packet reception process is repeated in the same manner for the next packet.

[0149] FIG. 14 is a flowchart illustrating the extended mode reception process performed in step S60 of FIG.

[0150] In step S71, the packet header detection unit 81 determines whether the mode setting of the extended mode is extended mode 0. For example, if the extended type setting information in the data type of the packet header indicates extended mode 0 (DataType[1:0] = 2'b00), the packet header detection unit 81 determines that the mode setting of the extended mode is extended mode 0.

[0151] In step S71, if the packet header detection unit 81 determines that the mode setting of the extended mode is extended mode 0, the process proceeds to step S72. In step S72, the interpretation unit 83 receives the first byte of the payload as an extended packet header.

[0152] In step S73, the interpretation unit 83 checks the data type (DataType[7:0]) of the extension packet header received in step S72 to determine whether the packet that has started to be received is an extension short packet.

[0153] If the interpretation unit 83 determines in step S73 that the packet is an extended short packet, the process proceeds to step S74, where the interpretation unit 83 receives an optional extension packet header in accordance with the setting (OePH[7:0]) stored in the extension packet header received in step S72.

[0154] In step S75, the CRC calculation unit 86 receives the WC+1 byte transmitted following the optional extension packet header as a CRC.

[0155] On the other hand, if the interpretation unit 83 determines in step S73 that the packet is not an extended short packet (i.e., reception of an extended long packet has started), the process proceeds to step S76, in which the interpretation unit 83 receives an optional extension packet header in accordance with the setting (OePH[7:0]) stored in the extension packet header received in step S72.

[0156] In step S77, the unpacking unit 87 receives the legacy payload of the extended long packet transmitted from the image sensor 21 and extracts the image data.

[0157] In step S78, the interpretation unit 83 receives the optional extension packet footer in accordance with the setting (OePF[3:0]) stored in the extension packet header received in step S72.

[0158] In step S79, the CRC calculation unit 86 receives the WC+1 byte transmitted following the optional extension packet footer as a CRC.

[0159] If it is determined in step S71 that the mode setting of the extended mode is not extended mode 0, the extended mode reception process is terminated after the process of step S75 or the process of step S79.

[0160] As described above, the application processor 22 can receive an extended short packet or an extended long packet and acquire data.

[0161] <Second structural example of packet structure> A second structural example of the packet structure of the packet used in communication between the extended mode-compatible CSI-2 transmission circuit 31 and the extended mode-compatible CSI-2 reception circuit 32 will be described with reference to Figures 15 to 18.

[0162] 3 to 8, emphasis is placed on maintaining compatibility with the existing CSI-2 standard, and the packet header and packet footer have the same packet structure as the existing CSI-2 standard, with the packet structure being extended by an extended packet header, optional extended packet header, and optional extended packet footer.In contrast, in the second structure example described below, the packet header and packet footer are different from those of the existing CSI-2 standard, and the packet structure is extended by an extended packet header and extended packet footer.

[0163] FIG. 15 shows the packet structure of a short packet used in the extended mode of CSI-2 when the physical layer is D-PHY (hereinafter referred to as an extended short packet for D-PHY).

[0164] In the extended short packet for D-PHY shown in FIG. 15, the extended mode is identified by the data type stored in the packet header, which is the same as that of the existing CSI-2 standard, as in the extended short packet for D-PHY of the first structure example shown in FIG.

[0165] On the other hand, in the extended short packet for D-PHY shown in Fig. 15, the frame number is stored in the short packet data field in the 16 bits following the data type of the packet header, just like a short packet conforming to the existing CSI-2 standard. Then, following the packet header, an extended packet header configured in the same way as the extended packet header shown in Fig. 4 is transmitted.

[0166] Therefore, the receiving application processor 22 can interpret the data type stored in the extended packet header and determine that, if it is an extended short packet, the frame number is stored in the data field of the packet header.

[0167] The optional extension packet header in the D-PHY extended short packet shown in Fig. 15 has the same structure as the optional extension packet header in the D-PHY extended short packet of the first structure example shown in Fig. 4. However, since the optional extension packet header has a packet structure that is not embedded in the payload, there is no need to add a CRC to the end.

[0168] FIG. 16 shows the packet structure of a long packet used in the extended mode of CSI-2 when the physical layer is D-PHY (hereinafter referred to as an extended long packet for D-PHY).

[0169] In the D-PHY extended long packet shown in Figure 16, the extended data is not embedded in the payload but is transmitted as part of the packet header or packet footer. Therefore, the WC in the first packet header simply indicates the byte length of the payload, as in the existing standard.

[0170] FIG. 17 shows the packet structure of a short packet used in the extended mode of CSI-2 when the physical layer is C-PHY (hereinafter referred to as an extended short packet for C-PHY).

[0171] The extended portion of the C-PHY extended short packet shown in Figure 17 is transmitted as an extension of the packet header in accordance with the existing CSI-2 standard, so the extended portion, such as the extended packet header, is inserted after the frame number. As with the existing CSI-2 standard, the packet header ends with a CRC. Furthermore, the packet structure, which is transmitted twice with a SYNC in between, is the same as that of a short packet in accordance with the existing CSI-2 standard.

[0172] FIG. 18 shows the packet structure of a long packet used in the extended mode of CSI-2 when the physical layer is C-PHY (hereinafter referred to as an extended long packet for C-PHY).

[0173] As described above, the extended long packet for C-PHY shown in FIG. 18 differs from the extended long packet for C-PHY of the first structure example shown in FIG. 8 in that the WC in the first packet header indicates the byte length of the payload, as in the existing standard.

[0174] As described above, the packet structure of the extended packet of the second structure example shown in Figures 15 to 18 can accommodate a wider variety of applications than conventional ones, similar to the packet structure of the extended packet of the first structure example (Figures 3 to 8).

[0175] However, the extended packet of the second structure example has a packet structure in which the existing packet header and footer are extended without embedding the extended data in the existing payload. Therefore, when the packet structure of the extended packet of the second structure example is adopted, the impact of requiring changes to the conventional communication system cannot be minimized compared to when the packet structure of the extended packet of the first structure example is adopted. That is, for example, the existing SerDes transmitter circuit 34 would need to be changed to the SerDes receiver circuit 35 ( FIG. 2 ).

[0176] As described above, by adopting the extended packet of the first structural example, it is possible to accommodate a variety of applications, such as in-vehicle use, and to construct an in-vehicle system while minimizing the impact that would require changes to the communication system that has been used traditionally.

[0177] Furthermore, by adopting the extended packet of the second structural example, although modifications are required from conventionally used communication systems, it is possible to accommodate a variety of uses, such as in-vehicle use.

[0178] <Modifications of Image Sensor and Application Processor> A modification of the image sensor and application processor will be described with reference to FIG.

[0179] The blocks constituting the image sensor 21 in Fig. 9 and the application processor 22 in Fig. 10 are configured to be able to process both D-PHY and C-PHY packets. However, for example, both a block that processes D-PHY packets exclusively and a block that processes C-PHY packets exclusively may be provided, and processing may be switched between them.

[0180] The image sensor 21A shown in FIG. 19A is configured to include a D-layer processing block unit 101, a C-layer processing block unit 102, a switching unit 103, and a controller 60.

[0181] The D-layer processing block unit 101 includes a block that processes D-PHY packets exclusively among the blocks that make up the image sensor 21 in Fig. 9. The C-layer processing block unit 102 includes a block that processes C-PHY packets exclusively among the blocks that make up the image sensor 21 in Fig. 9. Under the control of the controller 60, the switching unit 103 switches so as to output D-PHY packets generated in the D-layer processing block unit 101 when D-PHY is used for the physical layer, and to output C-PHY packets generated in the C-layer processing block unit 102 when C-PHY is used for the physical layer.

[0182] The application processor 22A shown in FIG. 19B is configured to include a switching unit 111, a D-layer processing block unit 112, a C-layer processing block unit 113, and a controller 74.

[0183] The switching unit 111 switches, under the control of the controller 74, so as to supply packets transmitted from the image sensor 21A to either the D-layer processing block unit 112 or the C-layer processing block unit 113. The D-layer processing block unit 112 has a block that processes packets for D-PHY exclusively, among the blocks constituting the application processor 22 in Fig. 10. The C-layer processing block unit 113 has a block that processes packets for C-PHY exclusively, among the blocks constituting the application processor 22 in Fig. 10.

[0184] In the image sensor 21A and application processor 22A configured as described above, before starting communication, the physical layer to be used can be set between the controller 60 and the controller 74. For example, when D-PHY is used for the physical layer, packets for D-PHY generated in the D-layer processing block unit 101 are transmitted via the switching unit 103 and supplied to the D-layer processing block unit 112 via the switching unit 111 for processing. For example, when C-PHY is used for the physical layer, packets for C-PHY generated in the C-layer processing block unit 102 are transmitted via the switching unit 103 and supplied to the C-layer processing block unit 113 via the switching unit 111 for processing.

[0185] <Application Examples of Extended Packets> The application of the above-described extended packets to the following use cases is being considered, for example.

[0186] For example, the extended packet is being considered for use in a use case such as transmitting higher resolution images (RAW24).

[0187] For example, when transmitting image data in RAW format, the existing CSI-2 standard defines RAW6, RAW7, RAW8, RAW10, RAW12, RAW14, RAW16, and RAW20 as data types to be stored in the packet header. However, in recent years, there has been a demand for the transmission of higher-resolution images to support autonomous driving using in-vehicle cameras. Therefore, by applying extended packets to extend the number of bits in the data type, it is possible to define, for example, the higher-resolution RAW24 as the data type in the extended packet header.

[0188] In addition, the extended packet is being considered for application to SmartROI, a technology that transmits only the image area of ​​interest on the screen.

[0189] For example, many cameras are currently installed in stadiums, airports, and other locations. If the entire image captured by these cameras is transmitted from the camera to a cloud server via a network such as the Internet, it is expected that the Internet bandwidth will be insufficient and the amount of calculation or data on the cloud side will increase. Therefore, by extracting only the image region of interest at the edge (camera side) and transmitting that image region, it is expected that the Internet bandwidth will be insufficient and the amount of calculation or data on the cloud side will be reduced.

[0190] When transmitting such an SROI, the coordinates of the upper left corner of the rectangular region of interest (ROI) must also be transmitted to inform the receiving side of where on the entire screen the image region of interest corresponds. Furthermore, the receiving side must send data for the entire captured image at a specific timing in response to a command. Therefore, for example, the SROI image and the data for the entire image (existing packet header) will be mixed on a frame-by-frame basis.

[0191] Therefore, by applying an extended packet, it becomes possible to transmit coordinate data of 16 bits or more for each of the X and Y coordinates, for example.

[0192] Furthermore, the use case of the extended packet is being considered for application to GLD, which reduces the bandwidth and number of lanes to continue communication even when the channel is degraded. GLD is a proposal under consideration for CSI-2 ver. 3.0.

[0193] For example, in autonomous driving, even if part of the cable connecting the camera breaks during a collision, communication is required to continue using the remaining cable, and the vehicle is required to automatically stop after retreating to a safety belt. Therefore, the in-vehicle camera interface must have at least a break detection function and must provide information such as a line number (16 bits) indicating which line of information on the screen the information is on, a Source ID (8 bits) indicating which camera it was sent from, and a message counter (16 bits) indicating the transmission number. Furthermore, when used in combination with the SROI described above, this information may be transmitted on a frame-by-frame basis.

[0194] Therefore, by applying extended packets, it becomes possible to transmit this information.

[0195] <Configuration Example Adapted to E2E Protection> A configuration example adapted to a regulation prohibiting packet alteration or the like on a transmission path will be described with reference to FIGS. 20 to 26. FIG.

[0196] For example, in the communication system 11A configured as described above with reference to Fig. 2, if the image sensor 21 and the application processor 22 have different interfaces, it is necessary to convert packets on the transmission path. That is, if the image sensor 21 has a D-PHY physical layer and the application processor 22 has a C-PHY physical layer, it is necessary to convert packets from D-PHY to C-PHY in the deserializer 26, for example.

[0197] In this way, a configuration in which packet conversion is performed in the deserializer 26 violates, for example, the regulations defined by ISO26262 (Functional Safety), that is, the regulations prohibiting packet modification, etc. on the transmission path (hereinafter referred to as E2E (End-to-End) protection).

[0198] FIG. 20 is a block diagram showing an example of the configuration of a communication system 201 adapted to E2E protection as a third embodiment of a communication system to which the present technology is applied.

[0199] As shown in FIG. 20 , the communication system 201 is configured by connecting an image sensor 211, a serializer 212, a deserializer 213, and an application processor 214. While FIG. 20 illustrates an example in which the SERDES is A-PHY, other SERDES standards, such as FPD-LINK3, may also be used for connection. In addition, with respect to other SERDES standards, communication may be performed based on the SERDES standard while maintaining the CIS-2 format (at least the application specific payload). Furthermore, in the SERDES, the physical layer processors 237 and 247 may include multiple physical layer processors of other SERDES standards in addition to A-PHY, and the physical layer processors may be switched depending on the application.

[0200] The image sensor 211 has at least an extended mode compatible CSI-2 transmitter circuit 221, a physical layer processing unit (hereinafter referred to as a C / D-PHY physical layer processing unit) 222 compatible with C-PHY or D-PHY, or both, a slave (hereinafter referred to as an I2C / I3C slave) 223 compatible with I2C or I3C, or both, and a CCI slave 224.

[0201] The serializer 212 has at least a CSI-2 receiving circuit 231, a C / D-PHY physical layer processing unit 232, an I2C / I3C master 233, a CCI master 234, a CSI-2 A-PHY packet generating unit 235, a CCI A-PHY packet transmitting / receiving unit 236, and an A-PHY-compatible physical layer processing unit 237. For example, the serializer 212 converts packets for C-PHY or D-PHY into packets for A-PHY, and this conversion is determined based on register settings, etc.

[0202] The deserializer 213 has at least a CSI-2 transmission circuit 241, a C / D-PHY physical layer processing unit 242, an I2C / I3C slave 243, a CCI slave 244, a CSI-2 A-PHY packet receiving unit 245, a CCI A-PHY packet transmitting / receiving unit 246, and an A-PHY-compatible physical layer processing unit 247. For example, the deserializer 213 converts A-PHY packets into C-PHY or D-PHY packets, and this conversion is determined based on register settings, etc.

[0203] The application processor 214 includes at least an extended mode compatible CSI-2 receiving circuit 251 , a C / D-PHY physical layer processing unit 252 , an I2C / I3C master 253 , and a CCI master 254 .

[0204] The communication system 201 is configured in this manner, and an extended packet having the above-described structure is transmitted from the image sensor 211 and received by the application processor 214. Even if the communication system 201 is configured so that the physical layer processing unit 222 of the image sensor 211 supports D-PHY and the physical layer processing unit 252 of the application processor 22 supports C-PHY, it is necessary to ensure that E2E protection is not violated.

[0205] Therefore, in order to be able to apply E2E protection, the communication system 201 limits the protection scope of E2E protection to an application specific payload (hereinafter referred to as AS payload), which is a payload specific to an application. That is, the AS payload is prohibited from being changed when converting an A-PHY packet to a C-PHY or D-PHY packet, or when converting a C-PHY or D-PHY packet to an A-PHY packet.

[0206] FIG. 21 shows an example of the structure of an extended packet for D-PHY that is extended to support E2E protection.

[0207] As shown in the figure, the AS payload of an extended packet for D-PHY, which is made up of an extended packet header (ePH), packet data, and an extended packet footer (ePF), is limited as the protection scope of E2E protection.

[0208] The extended packet header contains predetermined information that is required when the scope of E2E protection is limited to the AS payload. For example, a packet count PC (Packet Count) indicating the data length of the data stored in the AS payload is added as predetermined information to the extended packet header so that the data length of the packet data can be identified. In other words, the packet data has the number of bytes determined by the packet count PC. Also, a virtual channel VC (Virtual Channel) indicating the number of virtual channel lines is copied from the existing packet header as predetermined information to be written in the extended packet header.

[0209] FIG. 22 shows an example of the structure of an extended packet for C-PHY that is extended to support E2E protection.

[0210] As shown in the figure, in the case of an extended packet for C-PHY, the protection scope of E2E protection is limited to the AS payload, which consists of an extended packet header (ePH), packet data, and an extended packet footer (ePF), just like in the case of an extended packet for D-PHY. Similarly to the case of an extended packet for D-PHY, the extended packet header contains a packet count PC and a virtual channel VC as predetermined information required when the protection scope of E2E protection is limited to the AS payload.

[0211] FIG. 23 shows an example of the structure of an extended packet for A-PHY that is extended to support E2E protection.

[0212] As shown in the figure, even in an extended packet for A-PHY, the protection scope of E2E protection is limited to the AS payload consisting of an extended packet header (ePH), packet data, and extended packet footer (ePF).

[0213] 20, in the communication system 201, an extended packet for A-PHY is generated from an extended packet for D-PHY or C-PHY transmitted from the image sensor 211 to the serializer 212. Therefore, the packet count PC and the virtual channel VC are already written in the extended packet header of the extended packet for A-PHY.

[0214] By adopting such a packet structure, the communication system 201 can avoid the AS payload from being altered on the transmission path and comply with E2E protection. Note that the packet structures shown in Figures 21 to 23 can be partially replaced with corresponding packets having the packet structures shown in Figures 3 to 8 and 15 to 18, and part of the packet generation is replaced.

[0215] FIG. 24 is a flowchart illustrating a packet transmission / reception process adapted to E2E protection.

[0216] For example, processing begins when data to be stored in packet data (e.g., image data) is supplied to the extended mode-compatible CSI-2 transmitter circuit 221. Then, in step S101, the extended mode-compatible CSI-2 transmitter circuit 221 in the image sensor 211 stores the supplied data in packet data. Furthermore, the extended mode-compatible CSI-2 transmitter circuit 221 generates an extended packet header that describes a virtual channel VC and a packet count PC, as shown in FIG. 21 or 22 above. The extended mode-compatible CSI-2 transmitter circuit 221 then generates an AS payload by adding an extended packet header and an extended packet footer to the packet data.

[0217] In step S102, the extended mode-compatible CSI-2 transmitter circuit 221 generates an extended packet for C-PHY or D-PHY by adding a packet header for C-PHY or D-PHY and a packet footer for C-PHY or D-PHY to the AS payload generated in step S101. The extended mode-compatible CSI-2 transmitter circuit 221 then transmits the extended packet for C-PHY or D-PHY to the serializer 212 via the C / D-PHY physical layer processing unit 222.

[0218] In step S103, in the serializer 212, the CSI-2 receiving circuit 231 receives the C-PHY or D-PHY extended packet transmitted from the image sensor 211 in step S102 via the C / D-PHY physical layer processing unit 232. Then, the CSI-2 receiving circuit 231 obtains the AS payload, excluding the packet header and packet footer, from the received extended packet, and supplies the AS payload as is to the CSI-2 A-PHY packet generating unit 235.

[0219] In step S104, in the serializer 212, the CSI-2 A-PHY packet generator 235 generates an extended packet for A-PHY by adding an A-PHY packet header and an A-PHY packet footer to the AS payload supplied from the CSI-2 receiving circuit 231. Then, the CSI-2 A-PHY packet generator 235 transmits the extended packet for A-PHY to the deserializer 213 via the physical layer processor 237 corresponding to A-PHY.

[0220] In step S105, in the deserializer 213, the CSI-2 A-PHY packet receiver 245 receives the A-PHY extended packet transmitted from the serializer 212 in step S104, via the A-PHY-compatible physical layer processor 247. The CSI-2 A-PHY packet receiver 245 then acquires the AS payload, excluding the packet header and packet footer, from the received extended packet, and supplies the AS payload to the CSI-2 transmitter circuit 241 as is.

[0221] In step S106, the CSI-2 transmission circuit 241 generates an extended packet for C-PHY or D-PHY by adding a packet header for C-PHY or D-PHY and a packet footer for C-PHY or D-PHY to the AS payload supplied in step S105 from the CSI-2 A-PHY packet reception unit 245. The CSI-2 transmission circuit 241 then transmits the extended packet for C-PHY or D-PHY to the application processor 214 via the C / D-PHY physical layer processing unit 242.

[0222] In step S107, in the application processor 214, the extended mode-compatible CSI-2 receiver circuit 251 receives the C-PHY or D-PHY extended packet transmitted from the deserializer 213 in step S106 via the C / D-PHY physical layer processor 252. The extended mode-compatible CSI-2 receiver circuit 251 then acquires the AS payload, excluding the packet header and packet footer, from the received extended packet and outputs various data stored in the packet data of the AS payload to a downstream LSI (not shown). After that, the packet transmission / reception process compatible with E2E protection ends, and the same process is repeated for the next extended packet.

[0223] As described above, the communication system 201 can transmit and receive extended packets without modifying the AS payload on the transmission path by executing packet transmission and reception processing that conforms to E2E protection. In this case, even if, for example, the physical layer of the image sensor 211 is D-PHY and the physical layer of the application processor 214 is C-PHY, that is, even if the interfaces of the respective devices are different, E2E protection can be observed.

[0224] Fig. 25 is a block diagram showing a detailed configuration example of the image sensor 211. In the image sensor 211 shown in Fig. 25, components common to the image sensor 21 in Fig. 9 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0225] 9, the image sensor 211 is configured to include pixels 41, an AD converter 42, an image processing unit 43, a register 47, and a controller 60. The I2C / I3C slave 223 and the CCI slave 224 included in the image sensor 211 correspond to the I2C / I3C slave 46 and the CCI slave 59 in FIG.

[0226] The image sensor 211 includes an extended mode compatible CSI-2 transmission circuit 221 and a physical layer processing unit 222, and the physical layer processing unit 222 supports A-PHY, C-PHY, and D-PHY.

[0227] The extended mode compatible CSI-2 transmission circuit 221 is configured to include, in addition to the controller 60 and the CCI slave 224, an AS payload generation unit 301, a selector 302, an A-PHY packet generation unit 303, a C-PHY packet generation unit 304, a D-PHY packet generation unit 305, and a selector 306.

[0228] The AS payload generator 301 generates an AS payload that is limited as the protection range of E2E protection, and outputs it to the selector 302. For example, the AS payload generator 301 has a packing unit 311, an extended packet header generator 312, and an extended packet footer generator 313.

[0229] The packing unit 311 packs image data supplied from the image processing unit 43 as data to be transmitted, and generates packet data of the number of bytes determined by the packet count PC. For example, the controller 60 can control the number of bytes of packet data generated by the packing unit 311 in accordance with a setting value (e.g., image size) stored in the register 47.

[0230] The extended packet header generator 312 generates an extended packet header describing a packet count PC and a virtual channel VC, and adds it to the packet data, as described with reference to Figures 21 to 23. The extended packet footer generator 313 generates an extended packet footer and adds it to the packet data.

[0231] The selector 302, under the control of the controller 60, selects one of the A-PHY packet generator 303, the C-PHY packet generator 304, and the D-PHY packet generator 305, which are arranged in parallel, as the output destination for the AS payload supplied from the AS payload generator 301.

[0232] The A-PHY packet generator 303 generates an extended packet for A-PHY from the AS payload supplied via the selector 302, and outputs the packet to the selector 306. For example, the A-PHY packet generator 303 includes an AAL generator 321, an A-PHY packet header generator 322, and an A-PHY packet footer generator 323.

[0233] For example, the AAL (A-PHY Adaptive Layer) generation unit 321 divides the AS payload generated by the AS payload generation unit 301 into 380-byte pieces at a layer called an Adaptive Layer. Then, the A-PHY packet header generation unit 322 adds an A-PHY packet header to the divided AS payloads, and the A-PHY packet footer generation unit 323 adds an A-PHY packet footer.

[0234] The C-PHY packet generator 304 generates an extended packet for C-PHY from the AS payload supplied via the selector 302, and outputs the packet to the selector 306. For example, the C-PHY packet generator 304 includes a C-PHY packet header generator 331, a C-PHY packet footer generator 332, and a C-PHY lane distributor 333.

[0235] For example, a C-PHY packet header generator 331 adds a C-PHY packet header, and a C-PHY packet footer generator 332 adds a C-PHY packet footer to the AS payload generated by the AS payload generator 301. Then, a C-PHY lane distributor 333 distributes the C-PHY extended packet to three lanes in accordance with the CSI-2 standard.

[0236] The D-PHY packet generator 305 generates an extended packet for D-PHY from the AS payload supplied via the selector 302, and outputs the packet to the selector 306. For example, the D-PHY packet generator 305 has a D-PHY packet header generator 341, a D-PHY packet footer generator 342, and a D-PHY lane distributor 343.

[0237] For example, a D-PHY packet header generator 341 adds a D-PHY packet header, and a D-PHY packet footer generator 342 adds a D-PHY packet footer to the AS payload generated by the AS payload generator 301. Then, a D-PHY lane distributor 343 distributes the D-PHY extended packets to four lanes in accordance with the CSI-2 standard.

[0238] The selector 306, under the control of the controller 60, selects one of the A-PHY packet generator 303, the C-PHY packet generator 304, and the D-PHY packet generator 305, which are arranged in parallel, as the output source of the extended packet to be supplied to the physical layer processing unit 222.

[0239] When the physical layer processing unit 222 receives an A-PHY extension packet from the A-PHY packet generation unit 303, it transmits the A-PHY extension packet on lane 1. When the physical layer processing unit 222 receives a C-PHY extension packet from the C-PHY packet generation unit 304, it transmits the C-PHY extension packet on lane 3. When the physical layer processing unit 222 receives a D-PHY extension packet from the D-PHY packet generation unit 305, it transmits the D-PHY extension packet on lane 4.

[0240] In the image sensor 211 configured as described above, the extended mode-compatible CSI-2 transmission circuit 221 is configured so that the AS payload generator 301 is connected to the A-PHY packet generator 303, the C-PHY packet generator 304, and the D-PHY packet generator 305 via the selector 302. This allows the image sensor 211 to generate an AS payload common to the A-PHY extended packet, the C-PHY extended packet, and the D-PHY extended packet using a single AS payload generator 301. In other words, the A-PHY packet generator 303, the C-PHY packet generator 304, and the D-PHY packet generator 305 can share the AS payload generator 301, thereby reducing the circuit scale. This allows the image sensor 211 to be made smaller.

[0241] Fig. 26 is a block diagram showing a detailed configuration example of the application processor 214. In the application processor 214 shown in Fig. 26, components common to the application processor 22 in Fig. 10 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0242] 10, the application processor 214 is configured to include a register 73 and a controller 74. The controller 74 may be implemented by software. The I2C / I3C master 253 and CCI master 254 included in the application processor 214 correspond to the I2C / I3C master 72 and CCI master 88 in FIG. 10, respectively.

[0243] The application processor 214 includes an extended mode compatible CSI-2 receiving circuit 251 and a physical layer processing unit 252, and the physical layer processing unit 252 supports A-PHY, C-PHY, and D-PHY.

[0244] The extended mode compatible CSI-2 receiver circuit 251 is configured to include a CCI master 254, a selector 401, an A-PHY packet receiver 402, a C-PHY packet receiver 403, a D-PHY packet receiver 404, a selector 405, and an AS payload receiver 406.

[0245] The selector 401 selects one of the A-PHY packet receiver 402, C-PHY packet receiver 403, and D-PHY packet receiver 404, which are arranged in parallel, as the output destination of the extended packet supplied from the physical layer processing unit 252.

[0246] The A-PHY packet receiving unit 402 receives an A-PHY extended packet supplied via the selector 401 and outputs it to the selector 405. For example, the A-PHY packet receiving unit 402 includes an A-PHY packet header interpreting unit 411, an A-PHY packet footer verifying unit 412, and an AAL processing unit 413.

[0247] For example, an A-PHY packet header interpretation unit 411 interprets the contents written in the A-PHY packet header and performs processing required to receive an A-PHY extended packet, and an A-PHY packet footer verification unit 412 verifies the presence or absence of an error using the A-PHY packet footer. An AAL processing unit 413 then performs processing to combine the adaptive layers divided by the AAL generation unit 321 in FIG. 25 .

[0248] The C-PHY packet receiving unit 403 receives the C-PHY extended packet supplied via the selector 401 and outputs it to the selector 405. For example, the C-PHY packet receiving unit 403 includes a C-PHY lane merging unit 421, a C-PHY packet header interpretation unit 422, and a C-PHY packet footer verification unit 423.

[0249] For example, the C-PHY lane merging unit 421 merges C-PHY extended packets that are distributed to three lanes in accordance with the CSI-2 standard and supplied via the physical layer processing unit 252. The C-PHY packet header interpretation unit 422 then interprets the contents written in the C-PHY packet header and performs processing required to receive the C-PHY extended packets, and the C-PHY packet footer verification unit 423 verifies the presence or absence of errors using the C-PHY packet footer.

[0250] The D-PHY packet receiving unit 404 receives the D-PHY extended packet supplied via the selector 401 and outputs it to the selector 405. For example, the D-PHY packet receiving unit 404 has a D-PHY lane merging unit 431, a D-PHY packet header interpretation unit 432, and a D-PHY packet footer verification unit 433.

[0251] For example, the D-PHY lane merging unit 431 merges D-PHY extended packets that are distributed to four lanes in accordance with the CSI-2 standard and supplied via the physical layer processing unit 252. Then, the D-PHY packet header interpretation unit 432 interprets the contents written in the D-PHY packet header and performs processing required to receive the D-PHY extended packets, and the D-PHY packet footer verification unit 433 verifies the presence or absence of errors using the D-PHY packet footer.

[0252] The selector 405 selects one of the A-PHY packet receiver 402, C-PHY packet receiver 403, and D-PHY packet receiver 404, which are arranged in parallel, as the output source of the extended packet to be supplied to the AS payload receiver 406.

[0253] The AS payload receiver 406 includes an unpacking unit 441, an extended packet header interpretation unit 442, and an extended packet footer verification unit 443, corresponding to the AS payload generator 301 in FIG. 25 . The unpacking unit 441 unpacks the image data packed by the packing unit 311. The extended packet header interpretation unit 442 interprets the extended packet header generated by the extended packet header generator 312 and reads, for example, the packet count PC and the virtual channel VC. The extended packet footer verification unit 443 verifies the presence or absence of errors using the extended packet footer added by the extended packet footer generator 313. The AS payload receiver 406 then outputs various data stored in the packet data supplied via the selector 405, such as image data, vehicle row number, Source ID, CRC error, etc., to a downstream LSI (not shown).

[0254] In the application processor 214 configured as described above, the extended mode-compatible CSI-2 receiver circuit 251 is configured so that the AS payload receiver 406 is connected to the A-PHY packet receiver 402, the C-PHY packet receiver 403, and the D-PHY packet receiver 404 via the selector 405. This allows the application processor 214 to receive an AS payload common to the A-PHY extended packet, the C-PHY extended packet, and the D-PHY extended packet using a single AS payload receiver 406. In other words, the A-PHY packet receiver 402, the C-PHY packet receiver 403, and the D-PHY packet receiver 404 can share the AS payload receiver 406, thereby reducing the circuit scale. This allows the application processor 214 to be made smaller.

[0255] <Example of Computer Configuration> Next, the above-described series of processes (communication method) can be performed by hardware or software. When the series of processes is performed by software, a program constituting the software is installed in a general-purpose computer or the like.

[0256] FIG. 27 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0257] In the computer, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, and an EEPROM (Electronically Erasable and Programmable Read Only Memory) 504 are interconnected by a bus 505. An input / output interface 506 is further connected to the bus 505, and the input / output interface 506 is connected to the outside.

[0258] In a computer configured as described above, the CPU 501 performs the above-described series of processes by loading programs stored in, for example, the ROM 502 and EEPROM 504 into the RAM 503 via the bus 505 and executing the programs. Furthermore, the programs executed by the computer (CPU 501) can be written in advance in the ROM 502, or can be installed or updated in the EEPROM 504 from outside via the input / output interface 506.

[0259] In this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order according to the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing that is executed in parallel or individually (for example, parallel processing or object-based processing).

[0260] The program may be processed by a single computer (processor), or may be distributed among multiple computers. Furthermore, the program may be transferred to and executed on a remote computer.

[0261] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0262] Also, for example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0263] Furthermore, for example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0264] Furthermore, for example, the above-described program can be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0265] Also, for example, each step described in the above flowchart can be executed by one device or can be shared and executed by multiple devices. Furthermore, if one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0266] In addition, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0267] It should be noted that the present technologies described in this specification can be implemented independently and singly, unless a contradiction arises. Of course, any two or more of the present technologies can also be implemented in combination. For example, part or all of the present technologies described in any embodiment can be implemented in combination with part or all of the present technologies described in other embodiments. Furthermore, part or all of any of the present technologies described above can also be implemented in combination with other technologies not described above.

[0268] <Examples of Combinations of Configurations> The present technology can also be configured as follows. (1) A transmitting device comprising: an Application Specific payload generation unit that adds an extension packet header different from that for a physical layer to packet data obtained by packing data to be transmitted, and generates an Application Specific payload that is limited as a protection range to be protected by prohibiting modification on a transmission path; and a packet generation unit that adds at least a packet header for a predetermined physical layer to the Application Specific payload, and generates a packet for the physical layer. (2) The transmitting device described in (1) above, in which the extension packet header describes predetermined information required for transmitting the Application Specific payload as a protection range. (3) The transmitting device described in (2) above, in which the predetermined information is a packet count indicating the data length of the packet data. (4) The transmitting device described in any of (1) to (3) above, in which a plurality of the packet generation units are provided in parallel for each of a plurality of types of physical layers, and further comprising: a selector that switches the supply of the Application Specific payload from the Application Specific payload generation unit to a plurality of the packet generation units. (5) The transmitting device according to any one of (1) to (4), wherein the packet generating unit generates the packets for C-PHY or D-PHY and transmits them to a serializer via the corresponding physical layer, wherein the serializer obtains the Application Specific payload from the packets for C-PHY or D-PHY, generates the packets for A-PHY, and transmits them to a deserializer, and wherein the deserializer obtains the Application Specific payload from the packets for A-PHY, and generates the packets for C-PHY or D-PHY.(6) A receiving device comprising: a packet receiving unit that receives packets for a physical layer, in which an extension packet header different from that for the physical layer is added to packet data obtained by packing data to be transmitted, and in which at least a predetermined packet header for the physical layer is added to an Application Specific payload that is limited as a protection range to be protected by prohibiting alteration on a transmission path, and an Application Specific payload acquisition unit that acquires the Application Specific payload from the packets. (7) The receiving device according to (6), in which the extension packet header describes predetermined information required for transmitting the Application Specific payload as a protection range. (8) The receiving device according to (7), in which the predetermined information is a packet count that indicates the data length of the packet data. (9) The receiving device according to any of (6) to (8), in which a plurality of the packet receiving units are provided in parallel for each of a plurality of types of physical layers, and further comprising: a selector that switches the supply of the Application Specific payload from the plurality of packet receiving units to the Application Specific payload acquisition unit. (10) The receiving device according to any one of (6) to (9) above, wherein a serializer acquires the Application Specific payload from the packet for C-PHY or D-PHY, generates the packet for A-PHY, and transmits it to a deserializer; the deserializer acquires the Application Specific payload from the packet for A-PHY, generates the packet for C-PHY or D-PHY; and the packet receiving unit for C-PHY or D-PHY receives the packet via the corresponding physical layer.(11) A communication system comprising: a transmitting device having an Application Specific payload generating unit that adds an extended packet header different from that for the physical layer to packet data that is packed with data to be transmitted, and generates an Application Specific payload that is limited as a protection range to be protected by prohibiting modification on the transmission path; and a packet generating unit that adds at least a packet header for a predetermined physical layer to the Application Specific payload, and generates a packet for the physical layer; a packet receiving unit that receives the packet for the physical layer transmitted from the packet generating unit; and an Application Specific payload obtaining unit that obtains the Application Specific payload from the packet.

[0269] It should be noted that the present embodiment is not limited to the above-described embodiment, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, the effects described in this specification are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0270] 11 Communication system, 21 Image sensor, 22 Application processor, 23 and 24 Bus, 25 Serializer, 26 Deserializer, 27 Bus, 31 Extended mode compatible CSI-2 transmitter circuit, 32 Extended mode compatible CSI-2 receiver circuit, 33 CSI-2 receiver circuit, 34 SerDes transmitter circuit, 35 SerDes receiver circuit, 36 CSI-2 transmitter circuit, 41 Pixel, 42 AD converter, 43 Image processing unit, 44 Pixel CRC calculation unit, 45 Physical layer processing unit, 46 I2C / I3C slave, 47 Register, 51 Packing unit, 52 Packet header generator, 53 Extended packet header generator, 54 Extended packet footer generator, 55 and 56 Selection unit, 57 CRC calculation unit, 58 Lane distribution unit, 59 CCI slave, 60 Controller, 71 Physical layer processing unit, 72 I2C / I3C master, 73 Register, 74 Controller, 81 Packet header detection unit, 82 Lane merging unit, 83 Interpretation unit, 84 and 85 Selection unit, 86 CRC calculation unit, 87 Unpacking unit, 88 CCI master

Claims

1. A transmitting device comprising: an application specific payload generating unit that adds an extended packet header different from that for the physical layer to packet data that is packed with data to be transmitted, and generates an application specific payload that is limited as a protection range that must be protected by prohibiting modification on the transmission path; and a packet generating unit that adds at least a packet header for a predetermined physical layer to the application specific payload, and generates a packet for that physical layer.

2. The transmitting device according to claim 1, wherein the packet header for extension contains predetermined information required for transmitting the application specific payload as a protected area.

3. The transmitting device according to claim 2, wherein the predetermined information is a packet count indicating the data length of the packet data.

4. The transmitting device according to claim 1, further comprising: a plurality of packet generators arranged in parallel for each of a plurality of types of physical layers; and a selector for switching the supply of the Application Specific payload from the Application Specific payload generator to the plurality of packet generators.

5. The transmitting device according to claim 1, wherein the packet generating unit generates the packets for C-PHY or D-PHY and transmits them to a serializer via the corresponding physical layer, the serializer obtains the Application Specific payload from the packets for C-PHY or D-PHY, generates the packets for A-PHY and transmits them to a deserializer, and the deserializer obtains the Application Specific payload from the packets for A-PHY and generates the packets for C-PHY or D-PHY.

6. A receiving device comprising: a packet receiving unit that receives packets for a physical layer in which an extended packet header different from that for the physical layer is added to packet data that is packed with data to be transmitted, and in which at least a predetermined packet header for the physical layer is added to an Application Specific payload that is limited as a protection range that must be protected by prohibiting modification on the transmission path; and an Application Specific payload acquisition unit that acquires the Application Specific payload from the packet.

7. The receiving device according to claim 6, wherein the packet header for extension contains predetermined information required for transmitting the application specific payload as a protected range.

8. The receiving device according to claim 7, wherein the predetermined information is a packet count indicating the data length of the packet data.

9. The receiving device according to claim 6, further comprising: a plurality of packet receiving units provided in parallel for each of a plurality of types of physical layers; and a selector for switching the supply of the Application Specific payload from the plurality of packet receiving units to the Application Specific payload acquisition unit.

10. The receiving device according to claim 6, wherein a serializer acquires the application specific payload from the packet for C-PHY or D-PHY, generates the packet for A-PHY and transmits it to a deserializer, the deserializer acquires the application specific payload from the packet for A-PHY and generates the packet for C-PHY or D-PHY, and the packet receiving unit for C-PHY or D-PHY receives the packet via the corresponding physical layer.

11. A communication system comprising: a transmitting device having an Application Specific payload generating unit that adds an extended packet header different from that for the physical layer to packet data that packs data to be transmitted, and generates an Application Specific payload that is limited as a protection range that must be protected by prohibiting modification on the transmission path; a packet generating unit that adds at least a packet header for a predetermined physical layer to the Application Specific payload, and generates a packet for the physical layer; a packet receiving unit that receives the packet for the physical layer transmitted from the packet generating unit; and an Application Specific payload obtaining unit that obtains the Application Specific payload from the packet.

Images