Semiconductor device

Abstract

The invention relates to a semiconductor device. Provided is a semiconductor device capable of verifying whether or not correct acquisition of image data from a sensor is successful. The semiconductor device includes: a reception interface circuit that receives a plurality of packets each including a plurality of row data, and outputs an image composite signal generated by linking a row synchronization signal with each of the plurality of row data; and a capture circuit provided in a subsequent stage of the receiving interface circuit. The capture circuit includes: a line counter that receives, as an input thereof, a line synchronization signal included in the image composite signal, and counts a number of inputs of the line synchronization signal; and a comparator that compares the count value counted by the row counter with a preset expected value of the row number, and outputs an error signal if the count value and the expected value do not match each other.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The disclosure of Japanese Patent Application No. 2023-194724 filed on November 15, 2023 including the specification, drawings and abstract is incorporated herein by reference in its entirety. Background Art

[0003] The present invention relates to a semiconductor device, and to a semiconductor device that performs image processing, for example.

[0004] The disclosed techniques are listed below.

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2010-86401

[0006] Patent document 1 discloses a microcomputer capable of capturing data in different areas in parallel and transferring the captured data to a storage circuit. The microcomputer includes: a direct RAM interface (DRI) that captures image data in a predetermined area from image data from a camera and transfers the captured image data to a memory block; and a CPU that controls the DRI to transfer corresponding image data of different areas in the image data from the camera to the memory block. Summary of the invention

[0007] For example, an interface circuit based on the MIPI (Mobile Industry Processor Interface) CSI-2 (Camera Serial Interface 2) standard or the like packetizes and transmits / receives row data consisting of image data (such as RAW data generated by a sensor), metadata, etc. The image data included in the received packet is stored in, for example, a memory for each row, and is processed by an ISP (Image Signal Processor), a CPU (Central Processing Unit), or the like.

[0008] In this case, for example, when any abnormality occurs in the path extending from the sensor and the transmission interface circuit to the reception interface circuit, insufficient / excessive row data is formed, and the image data stored in the memory may be more degraded than the image data originally acquired and processed by the sensor as the target of image processing. In this case, the ISP, CPU, etc. perform image processing based on the degraded image data. In particular, in an in-vehicle system such as ADAS (Advanced Driver Assistance System), image recognition based on image data, etc. is performed in order to achieve functional safety. Therefore, the image recognition processing needs to use non-degraded image data.

[0009] The embodiments described below are made in view of such circumstances, and other problems and novel features will become apparent from the description of this specification and the accompanying drawings.

[0010] A semiconductor device according to an embodiment includes: a receiving interface circuit that receives a plurality of packets each including row data and outputs an image composite signal generated according to the linkage of a row synchronization signal with each of the plurality of row data; and a capture circuit provided at a subsequent stage of the receiving interface circuit. The capture circuit includes: a row counter that receives a row synchronization signal included in the image composite signal as its input and counts the number of times the row synchronization signal is input; and a comparator that compares a count value counted by the row counter with a preset expected value of the number of rows, and outputs an error signal when the count value and the expected value do not match each other.

[0011] By using the semiconductor device according to the embodiment, it can be verified whether acquisition of image data from the sensor has been correctly performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a schematic diagram illustrating a configuration example of an image processing system using a semiconductor device according to a first embodiment;

[0013] Figure 2 It is a diagram of Figure 1 A schematic diagram of an example of a configuration of a packet received by a receiving interface circuit in;

[0014] Figure 3 It is a diagram of Figure 1 A timing diagram showing a conceptual configuration example of an image composite signal output by a receiving interface circuit in FIG.

[0015] Figure 4A It's a picture. Figure 1 A circuit block diagram of an example configuration of the illustrated capture circuit;

[0016] Figure 4B It's a picture. Figure 4A A circuit block diagram of an example configuration of the illustrated monitor circuit;

[0017] Figure 5 is a diagram illustrating a semiconductor device according to a second embodiment. Figure 1 A schematic diagram of an example of a configuration of a packet received by the illustrated receiving interface circuit;

[0018] Figure 6 is a diagram illustrating a semiconductor device according to a second embodiment. Figure 1 A timing diagram showing an example of a conceptual configuration of an image composite signal output by the illustrated receiving interface circuit;

[0019] Fig. 7A is a diagram illustrating a semiconductor device according to a second embodiment Figure 1 A circuit block diagram of an example configuration of the illustrated capture circuit;

[0020] Figure 7B It's a picture. Fig. 7A A schematic diagram of an example of contents stored in the illustrated setting register;

[0021] Figure 8 It's a picture. Fig. 7A a timing diagram of a principal example of operation of the illustrated monitor circuit;

[0022] Fig. 9 is a diagram illustrating a semiconductor device according to a third embodiment. Figure 1 A diagram showing an example of a configuration of a packet received by the illustrated reception interface circuit and an example of a configuration of an image composite signal output by the reception interface circuit;

[0023] Fig.10 is a diagram illustrating a semiconductor device according to a third embodiment Figure 1 A circuit block diagram of an example configuration of the illustrated capture circuit;

[0024] Fig.11 It's a picture. Fig.10 A timing diagram of an example of operation of the illustrated row division circuit;

[0025] Fig.12 is a diagram illustrating a semiconductor device according to a fourth embodiment. Figure 1 A diagram showing an example of a configuration of a packet received by the illustrated reception interface circuit and an example of a configuration of an image composite signal output by the reception interface circuit;

[0026] Fig.13 is a diagram illustrating a semiconductor device according to a fourth embodiment Figure 1 A circuit block diagram of a configuration example of the illustrated capture circuit; and

[0027] Fig.14 It's a picture. Fig.13 A timing diagram of a principal operational example of the monitor circuit is illustrated. DETAILED DESCRIPTION

[0028] In the embodiments described below, for convenience, the present invention will be described with multiple sections or embodiments when necessary. However, unless otherwise stated, these sections or embodiments are not independent of each other, and a section or embodiment relates to all or part of other sections or embodiments as its modified examples, details or supplementary explanations. Moreover, in the embodiments described below, when referring to the number of elements (including the number of fragments, value, amount, range, etc.), the number of elements is not limited to a specific number, unless otherwise stated or except in principle the number is obviously limited to a specific number of cases. Greater than or less than the number of specific numbers also applies. Further, in the embodiments described below, it goes without saying that unless otherwise stated or except in principle the components are obviously indispensable, the components (including element steps) are not always indispensable. Similarly, in the embodiments described below, when the shape of the components, their positional relationship, etc. are mentioned, substantially similar and similar shapes, etc. are included therein, unless otherwise stated or except in principle it can be imagined that they are obviously excluded. The above-mentioned numerical value and range are also the same.

[0029] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that in all the drawings used to describe the embodiments, components having the same functions are represented by the same reference symbols, and their repeated descriptions will be omitted. In the following embodiments, the description of the same or similar parts is not repeated in principle unless it is particularly necessary.

[0030] (First embodiment)

[0031] <Overview of Semiconductor Devices>

[0032] Figure 1 1 is a schematic diagram illustrating a configuration example of an image processing system using a semiconductor device according to a first embodiment. The image processing system is, for example, an in-vehicle system such as an ADAS. The image processing system includes a plurality of sensors 11a, 11b, and ..., a semiconductor device 10, and a RAM (random access memory) 12 as an external memory. In this specification, the plurality of sensors 11a, 11b, and ... are collectively referred to as sensors 11. The RAM 12 is, for example, a DRAM (dynamic RAM). The sensor 11 is, for example, an image sensor and a ranging sensor.

[0033] The image sensor includes a CMOS (complementary metal oxide semiconductor) type or CCD (charge coupled device) type imaging element arranged in a matrix pattern. The image sensor generates captured image data based on imaging using the imaging element. Meanwhile, the ranging sensor includes, for example, a radar. The ranging sensor generates distance image data based on distance measurement using a radar. In this specification, the captured image data and the distance image data are collectively referred to as image data IMG.

[0034] The sensor 11 also includes a transmission interface circuit. The transmission interface circuit is, for example, a circuit having various functions based on the MIPI CSI-2 standard. Therefore, the sensor 11 packs line data serving as image data corresponding to one line and transmits a plurality of packets PKT each including the line data.

[0035] The semiconductor device 10 is, for example, an on-vehicle LSI (large scale integration) such as an SoC (system on chip) or a microcontroller composed of a single semiconductor chip. The semiconductor device 10 performs, for example, image processing or image recognition processing on image data included in each of the packets output from the sensor 11. The semiconductor device 10 includes a reception interface circuit 20, a capture circuit 21, an ISP 22, a main processor 23, a RAM 24 used as an internal memory, a memory controller 25, and a system bus 26.

[0036] The system bus 26 connects the capture circuit 21, the ISP 22, the main processor 23, the RAM 24, and the memory controller 25 to each other. The RAM 24 is, for example, an SRAM (static RAM) or others. The memory controller 25 controls access to the RAM 12 used as an external memory. In this specification, each of the RAM 24 and the RAM 12 is simply referred to as MEM unless it is particularly necessary to distinguish. Although not shown, the semiconductor device 10 also includes a nonvolatile memory that stores programs and the like.

[0037] The receiving interface circuit 20 is, for example, a circuit having various functions based on the MIPI CSI-2 standard. The receiving interface circuit 20 receives a plurality of packets PKT from the sensor 11. As described above, each of the plurality of packets PKT includes line data. Although described in detail later, the receiving interface circuit 20 links the horizontal synchronization signal with each of the plurality of line data, and outputs an image composite signal IMCS generated based on the linking.

[0038] The capture circuit 21 is provided at the latter stage of the receiving interface circuit 20, and receives the image composite signal IMCS from the receiving interface circuit 20 as its input. The capture circuit 21 sequentially writes each piece of line data included in the image composite signal IMCS into the memory MEM. More specifically, the capture circuit 21 extracts the image data IMG from each piece of line data. That is, in addition to the image data IMG, each piece of line data may also include various types of additional information. The capture circuit 21 extracts the image data IMG by removing such additional information, and writes the extracted image data IMG into the memory MEM.

[0039] The capture circuit 21 transmits image data IMG based on preset contents (such as image data IMG on a specific channel) to the ISP 22 as needed. Further, the capture circuit 21 includes a monitor circuit 30 for monitoring the image data IMG. Note that the details of the capture circuit 21 will be described later.

[0040] The ISP 22 performs image processing such as demosaic processing, HDR (High Dynamic Range) image generation processing, black level correction processing, and color space conversion processing on the image data IMG stored in the memory MEM or the image data IMG transmitted from the capture circuit 21. The ISP 22 writes the image data after the image processing into the memory MEM.

[0041] In the demosaicing process, for example, the ISP 22 generates image data having an RGB format, a YUV format, or the like by performing a process for interpolating pixel values ​​having missing colors on image data IMG such as RAW data based on, for example, a Bayer layout. In the HDR image generation process, the ISP 22 generates an HDR image by synthesizing images captured at a plurality of exposure amounts (exposures) or the like, respectively.

[0042] The main processor 23 includes a CPU, and is configured by appropriately combining a DSP (digital signal processor), a GPU (graphics processing unit), etc. with the CPU. By executing a program stored in the RAM 24, etc., the main processor 23 controls the processing sequence of the entire semiconductor device 10 while appropriately cooperating with various internal circuits included in the semiconductor device 10. The processing performed by the main processor 23 includes image recognition processing such as object detection processing using a neural network. In the specification, the ISP 22 and the main processor 23 are collectively referred to as a processor PRC.

[0043] <Overview of Receive Interface Circuit>

[0044] Figure 2 It is a diagram of Figure 1 Schematic diagram of a configuration example of a packet PKT received by the receiving interface circuit 20 in FIG. Figure 3 It is a diagram showing Figure 1 1 is a timing diagram showing a conceptual configuration example of an image composite signal IMCS output by the receiving interface circuit 20. Figure 2 As illustrated, in the MIPI CSI-2 standard, a short packet PKT-S and a long packet PKT-L are defined.

[0045] The short packet PKT-S is a packet used to notify, for example, FS (frame start) / FE (frame end) and LS (line start) / LE (line end). FS / FE indicates the start / end of the image data IMG. LS / LE indicates the start / end of each of the line data pieces LD[1] to LD[n] of the "n" line included in the image data IMG.

[0046] Here, the row data LD[1] is data configuring the first row of the image data IMG, and includes a plurality of pixel data PD. Similarly, the row data LD[n] is data configuring the nth row of the image data IMG, and includes a plurality of pixel data PD. The short packet PKT-S includes a value of a virtual channel (VC) and a value of a data type (DT). For example, each sensor 11 in the sensor 11 is identified by a value of a virtual channel (VC), and FS / FE, LS / LE, etc. are identified by a value of a data type (DT).

[0047] On the other hand, the long packet PKT-L is a packet for transmitting each of the row data pieces LD[1] to LD[n] of n rows included in the image data IMG. The long packet PKT-L is formed by adding a packet header PH and a packet footer PF, etc. to the row data LD. Similar to the short packet PKT-S, the packet header PH includes the value of the virtual channel (VC) and the value of the data type (DT). For example, the image format type such as RAW data or RGB data, the number of bits of one pixel, etc. are identified by the value of the data type (DT). As the packet footer PF, for example, a checksum CS, etc. is stored.

[0048] Figure 2 The diagram shows the sequence of packets PKT received by the receiving interface circuit 20. The receiving interface circuit 20 first receives FS composed of short packets PKT-S (step Sfs). Then, the receiving interface circuit 20 sequentially receives LS composed of short packets PKT-S, row data LD[1] of the first row to be loaded into the long packet PKT-L, and LE composed of short packets PKT-S (steps Ss[1], Sd[1], and Se[1]).

[0049] Then, the receiving interface circuit 20 sequentially receives the LS composed of the short packet PKT-S, the row data LD[2] of the second row to be loaded into the long packet PKT-L, and the LE composed of the short packet PKT-S (steps Ss[2], Sd[2], and Se[2]). Similarly, hereinafter, the receiving interface circuit 20 receives the row data to the row data LD[n-1] of the "n-1"th row.

[0050] Then, the receiving interface circuit 20 sequentially receives LS composed of the short packet PKT-S, the row data LD[n] of the nth row to be loaded into the long packet PKT-L, and LE composed of the short packet PKT-S (steps Ss[n], Sd[n], and Se[n]). Finally, the receiving interface circuit 20 receives FE composed of the short packet PKT-S (step Sfe). Note that the image data IMG is, for example, captured image data generated by an image sensor.

[0051] The receiving interface circuit 20 receives this packet PKT and conceptually outputs Figure 3 The illustrated image composite signal IMCS. The image composite signal IMCS includes, for example, a clock signal CK, a frame synchronization signal Vsync, a line synchronization signal Hsync, a data enable signal DEN, and a data signal DAT. The receiving interface circuit 20 asserts the frame synchronization signal Vsync according to FS, and negates the frame synchronization signal Vsync according to FE. Therefore, the receiving interface circuit 20 links the frame synchronization signal Vsync with the entirety of the plurality of line data LD[1] to LD[n].

[0052] The receiving interface circuit 20 asserts the line synchronization signal Hsync according to LS, and negates the line synchronization signal Hsync according to LE. Therefore, the receiving interface circuit 20 links the line synchronization signal Hsync with each of the line data slices LD[1] to LD[n] of the n lines. Note that LS / LE may not be generated depending on the specification. In this case, the receiving interface circuit 20 asserts the line synchronization signal Hsync according to the packet header PH in the long packet PKT-L, and negates the line synchronization signal Hsync according to the packet footer PF.

[0053] In addition to the row data LD[1] to LD[n], the reception interface circuit 20 outputs, for example, a value of a virtual channel (VC) and a value of a data type (DT) added to each piece of row data LD as a data signal DAT. When outputting the row data LD[1] to LD[n], the reception interface circuit 20 asserts the data enable signal DEN, and outputs the row data LD[1] to LD[n] during the assertion period of the data enable signal DEN.

[0054] Note that the transmission interface circuit based on the MIPI-CSI2 standard is also more specifically a serializer, and transmits the line data LD serially by using four channels, etc. On the other hand, the receiving interface circuit 20 is also a deserializer, and converts the line data LD transmitted serially by using four channels, etc. into parallel data. Therefore, the data signal DAT is specifically a parallel signal. The image composite signal IMCS may include various control signals and the like caused by such serial-to-parallel conversion.

[0055] In the above configuration, for example, when any abnormality occurs in the path from the sensor 11 to the receiving interface circuit 20, the number of lines of the line data LD included in the image composite signal IMCS may be excessive / insufficient. If the capture circuit 21 writes such line data LD and pixel data PD as they are to the memory MEM, the image data IMG stored in the memory MEM may be more degraded than the image data expected as the target of image processing. Therefore, there is a concern that the processor PRC performs image processing or image recognition processing based on degraded image data. Therefore, it is useful to use the capture circuit 21 described below.

[0056] <Details of Capture Circuit>

[0057] Figure 4A It's a picture. Figure 1 A circuit block diagram of a configuration example of the illustrated capture circuit 21. Figure 4B It's a picture. Figure 4A A circuit block diagram of an example of a configuration of the illustrated monitor circuit 30a. Figure 4A The illustrated capture circuit 21a includes a monitor circuit 30a, a data extraction circuit 35a, a setting register 36, and an address generator 37. The data extraction circuit 35a receives the image composite signal IMCS as its input and extracts pixel data PD from each of a plurality of line data LD based on a preset rule, thereby extracting image data IMG.

[0058] Specifically, the data extraction circuit 35a refers to the setting register 36 by using the value of the virtual channel (VC) and the value of the data type (DT) included in the image composite signal IMCS. In the setting register 36, the image identifier IMG-ID is pre-linked with the combination of the value of the virtual channel (VC) and the value of the data type (DT). Further, in the setting register 36, the rule of each image identifier IMG-ID is predetermined.

[0059] For example, in Figure 2 and 3 In the illustrated example, the data extraction circuit 35a can obtain pixel data PD by extracting row data LD input during the assertion period of the data enable signal DEN, thereby obtaining image data IMG. Note that in addition to the image data IMG, the row data LD can also include various types of additional information such as padding data. On the other hand, based on the specification, it is predetermined which part includes what type of additional information. Therefore, a rule for excluding such additional information is predetermined by the setting register 36. The data extraction circuit 35a extracts only the image data IMG based on the rule.

[0060] The data extraction circuit 35a asserts the enable signal EN in the period of extracting the image data IMG. The address generator 37 sequentially generates the address signal ADR by performing, for example, a counting operation in the assertion period of the enable signal EN. The address generator 37 acquires a preset start address SADR for each image identifier IMG-ID (in other words, for each image data IMG) from the setting register 36.

[0061] The address generator 37 sequentially generates the address signal ADR while using the acquired start address SADR as an origin. More specifically, the address generator 37 sequentially generates the address signal ADR while using the start address with an offset as an origin (generated by sequentially adding the offset to the start address SADR at each row change). Therefore, the image data IMG extracted by the data extraction circuit 35a is written to the address based on the address signal ADR generated by the address generator 37 in the memory MEM.

[0062] like Figure 4B As shown, the monitor circuit 30a includes an expected value register 41, a row counter 42, and a comparator 43. The expected value register 41 outputs a preset expected value EV regarding the number of rows for each image identifier IMG-ID based on the image identifier IMG-ID. Note that the expected value register 41 may be a part of the setting register 36. The row counter 42 receives the row synchronization signal Hsync included in the image composite signal IMCS as its input, and counts the number of inputs of the row synchronization signal Hsync. Specifically, the row counter 42 counts the number of inputs of the row synchronization signal Hsync within the assertion period of the frame synchronization signal Vsync included in the image composite signal IMCS.

[0063] The comparator 43 compares the count value CV counted by the row counter 42 with the expected value EV output from the expected value register 41. If the count value CV and the expected value EV do not match each other, the comparator 43 outputs an error signal ERR. Figure 2 and 3 In the illustrated example, “n” is preset to an expected value EV, and if the count value CV is not “n”, the comparator 43 outputs an error signal ERR.

[0064] Therefore, it is possible to verify whether the image data IMG has been correctly acquired from the sensor 11. For example, if the error signal ERR is output from the comparator 43, the processor PRC stops the image processing or image recognition processing performed on the image data IMG that is the target of the error signal ERR. Therefore, it is possible to prevent the processor PRC from performing image processing or image recognition processing based on degraded image data, and sufficient functional safety can be achieved particularly in an in-vehicle system or the like.

[0065] As another method, for example, a method of counting the number of pixel data PD included in the image data IMG instead of the line synchronization signal Hsync is also considerable. However, depending on the image processing system, for example, a predetermined number of pixel failures may be allowed in an image sensor. In this case, it is not easy to determine the expected value. From this perspective, it is useful to use a method of counting the line synchronization signal Hsync.

[0066] <Main Effects of the First Embodiment>

[0067] As described above, the method according to the first embodiment includes: a line counter for counting the line synchronization signal output from the receiving interface circuit; and a comparator for comparing its count value with an expected value. Therefore, it is generally possible to verify whether the image data correctly acquired from the sensor can be correctly acquired. In other words, a fault in the sensor itself and a fault on the path from the sensor to the receiving interface circuit can be detected.

[0068] (Second embodiment)

[0069] <Overview of Receive Interface Circuit>

[0070] Figure 5 is a diagram illustrating a semiconductor device according to a second embodiment. Figure 1 The illustrated diagram is a schematic diagram of a configuration example of a packet PKT received by the reception interface circuit 20. Figure 6 is a diagram illustrating a semiconductor device according to a second embodiment. Figure 1 The illustrated timing diagram is a conceptual configuration example of the image composite signal IMCS output by the reception interface circuit 20.

[0071] Figure 5 The configuration example shown is similar to Figure 2 The illustrated configuration example differs in the following points. The first difference is that the reception interface circuit 20 receives the front embedded data FED as the row data LD[0], and receives the rear embedded data RED as the row data LD[n+1]. The front embedded data FED and the rear embedded data RED are each data to be added by the sensor 11, a serializer in the transmission path, etc. As a specific example thereof, metadata including various types of product specific information represented by setting values ​​of an image sensor, etc. is illustrated.

[0072] As a second difference, the receiving interface circuit 20 receives a plurality of (here, two) image data IMG1 and IMG2 in the same frame. Therefore, each of the line data LD[1] to LD[n-1] includes two image data IMG1 and IMG2, specifically, two types of pixel data PD1 and PD2 that respectively configure the two image data IMG1 and IMG2. That is, the line data LD[1] includes line data LD[1] (PD1) composed of pixel data PD1 in the image data IMG1 and line data LD[1] (PD2) composed of pixel data PD2 in the image data IMG2. The two image data IMG1 and IMG2 are data having different exposure amounts (exposure), for example.

[0073] As a third difference, the receiving interface circuit 20 receives the optical black data OB used in the black level correction as the line data LD[n] of the nth row. In this example, the image data IMG1 consists of "n" rows, and the image data IMG2 consists of "n-1" rows. The optical black data OB is stored in a blank area corresponding to the image data IMG2 that does not exist in the line data LD[n] of the nth row.

[0074] The receiving interface circuit 20 receives this packet PKT and conceptually outputs Figure 6 The image composite signal IMCS is shown. Figure 6 The configuration of the image composite signal IMCS shown in the figure is Figure 3 The illustrated configurations are substantially similar. However, due to the above differences in grouping PKT, Figure 6 In the following points Figure 3 different.

[0075] The first difference is that the receiving interface circuit 20 outputs the front embedded data FED as the row data LD[0] (step Sd[0]) in the first assertion cycle of the row synchronization signal Hsync (in step Ss[0], step Se[0]). The receiving interface circuit 20 outputs the rear embedded data RED as the row data LD[n+1] (in step Sd[n+1]) in the last assertion cycle of the row synchronization signal Hsync (in step Ss[n+1], step Se[n+1]).

[0076] The second difference is that the receiving interface circuit 20 outputs two types of pixel data PD1 and PD2 (illustration of LD[n-1] is omitted) configuring two image data IMG1 and IMG2 in each of the line data pieces LD[1] to LD[n-1] of the first to "n-1"th rows, respectively (in steps Sd[1], Sd[2], ...). The third difference is that the receiving interface circuit 20 outputs the pixel data PD1 configuring the image data IMG1 and the optical black data OB as the row data LD[n] of the nth row (in step Sd[n]).

[0077] That is, the receiving interface circuit 20 does not affect the content of the line data LD. Therefore, the receiving interface circuit 20 outputs embedded data (FED and RED), image data IMG, optical black data OB, etc. as the line data LD in the assertion period of the data enable signal DEN without distinguishing these data.

[0078] In recent years, the MIPI CSI-2 standard has been developed with 16 virtual channels (VCs) and 2 16 2 of the data types (DT) 16 For example, each of the sensors 11 can generate four types of image data IMG that differ in maximum exposure (exposure), i.e., are compatible with four exposure channels (EC). Based on these issues, the number of types of image data IMG sent and received by the interface circuit can be at most<VC×EC×DT> =<16×4×65535>.

[0079] In order to effectively transmit various image data IMG within limited resources and limited time period, such as Figure 5 As illustrated, a plurality of image data IMG1 and IMG2 may be multiplexed and transmitted in the same line data LD. In addition to the image data IMG1 and IMG2, the line data LD may further include embedded data (FED and RED), optical black data OB, etc. as additional information.

[0080] In this case, in the method of counting the line synchronization signal Hsync included in the image composite signal IMCS as described in the first embodiment, it is difficult to verify whether the image data IMG is correctly acquired. Figure 6 As can be clearly seen, the additional information is also counted, and therefore, the counting is performed without distinguishing between the image data IMG1 and IMG2. Figure 2 When similar additional information is included in the data, normal verification may also become difficult. Therefore, it is useful to use the capture circuit 21 described below.

[0081] <Details of Capture Circuit>

[0082] Fig. 7A is a diagram illustrating a semiconductor device according to a second embodiment Figure 1 A circuit block diagram of a configuration example of the illustrated capture circuit 21. Figure 7B It's a picture. Fig. 7A The illustrated diagram is a diagram of an example of the contents stored in the setting register 36. Figure 4A The situation shown in the figure is similar. Fig. 7A The illustrated capture circuit 21b includes a data extraction circuit 35b, a setting register 36, and an address generator 37. Figure 4B Unlike the illustrated case, the capture circuit 21b includes a monitor circuit 30b.

[0083] and Figure 4A Similar to the illustrated case, the data extraction circuit 35b extracts the image data IMG from the plurality of line data LD based on a preset rule. More specifically, the data extraction circuit 35b includes, for example, a pixel counter 55, and thus extracts the image data IMG1 and the image data IMG2, that is, the pixel data PD1 and the pixel data PD2 separately. The data extraction circuit 35b asserts the enable signal EN-PD1 during the extraction cycle of the image data IMG1, and asserts the enable signal EN-PD2 during the extraction cycle of the image data IMG2.

[0084] Here, in the setting register 36, as shown in FIG. Figure 7B As shown, "ID1" as an image identifier IMG-ID is linked to a combination of a value of a virtual channel (VC) and a value of a data type (DT). Further, "ID1-1" and "ID1-2" as two sub-image identifiers SUB-ID are linked to "ID1".

[0085] A rule, an expected value EV of the number of rows, and a start address SADR are preset for each of the sub-image identifiers "ID1-1" and "ID1-2". For example, in the rule in the sub-image identifier "ID1-1", an instruction is issued to extract the first pixel PX[1] to the i-th pixel PX[i] on the first row L[1] to the n-th row L[n].

[0086] On the other hand, in the rule in the sub-image identifier "ID1-2", an instruction is issued to extract the "i+1"th pixel PX[i+1] to the jth pixel PX[j] on the first line L[1] to the "n-1"th line L[n-1]. The pixel data extracted while following this rule is the image data to be stored in the memory MEM, that is, the image data to be the target of image processing. The expected value EV represents the number of lines in one frame of the image data to be the target of image processing.

[0087] In the sub-image identifier "ID1-1", the expected value EV1 is set to "n", and the start address SADR1 is set to "#A". On the other hand, in the sub-image identifier "ID1-2", the expected value EV2 is set to "n-1", and the start address SADR2 is set to "#B". The data extraction circuit 35b operates while following the rules, so it can extract Figure 6 The illustrated pixel data PD1 (i.e., image data IMG1) and pixel data PD2 (i.e., image data IMG2) are shown. Note that the data extraction circuit 35b is specifically composed of a combination of various filters, such as a filter for extracting line data LD within a predetermined range, a filter for extracting pixel data PD within a predetermined range, and a filter for excluding a predetermined range.

[0088] The address generator 37 sequentially generates the address signal ADR by performing a count operation in the assertion cycle of the enable signal EN-PD1 while using "#A" as the start address SADR1 set by the setting register 36 as an origin. The address generator 37 sequentially generates the address signal ADR by performing a count operation in the assertion cycle of the enable signal EN-PD2 while using "#B" as the start address SADR2 set by the setting register 36 as an origin.

[0089] On the other hand, the monitor circuit 30b includes a synchronization signal reproducing circuit 51, a line counter 52, and a comparator 53. The synchronization signal reproducing circuit 51 links the line synchronization signal for counting newly generated for each line with the image data IMG extracted by the data extracting circuit 35b. Specifically, the synchronization signal reproducing circuit 51 links the line synchronization signal HsyncC1 and the line synchronization signal HsyncC2 as the line synchronization signal for counting with the image data IMG1 and the image data IMG2 extracted for each line, respectively.

[0090] The line counter 52 receives as its input the line synchronization signal for counting from the synchronization signal reproducing circuit 51, and counts the number of input times of the line synchronization signal for counting. Specifically, the line counter 52 receives as its input the line synchronization signals HsyncC1 and HsyncC2 for counting, and counts each of the number of input times of the line synchronization signals HsyncC1 and HsyncC2 individually as count values ​​CV1 and CV2.

[0091] The comparator 53 compares the count value CV counted by the line counter 52 with the preset expected value EV of the number of lines of the image data IMG when the expected value EV is used, and outputs the error signal ERR if the count value CV and the expected value EV do not match each other. Specifically, the comparator 53 obtains the preset expected values ​​EV1 and EV2 of the image data IMG1 and IMG2 from the setting register 36, respectively. Then, the comparator 53 compares the count value CV1 of the line synchronization signal HsyncC1 with the expected value EV1, and outputs the error signal ERR1 if the count value CV1 and the expected value EV1 do not match each other. Similarly, the comparator 53 compares the count value CV2 of the line synchronization signal HsyncC2 with the expected value EV2, and outputs the error signal ERR2 if the count value CV2 and the expected value EV2 do not match each other.

[0092] Figure 8 It's a picture. Fig. 7A A timing diagram of a main operation example of the illustrated monitor circuit 30b. The monitor circuit 30b receives as its input the frame synchronization signal Vsync and the line synchronization signal Hsync included in the image composite signal IMCS and the enable signals EN-PD1 and EN-PD2 output from the data extraction circuit 35b. As described above, the enable signal EN-PD1 is asserted during the extraction cycle of the pixel data PD1, and the enable signal EN-PD2 is asserted during the extraction cycle of the pixel data PD2.

[0093] The synchronization signal reproduction circuit 51 generates the line synchronization signals HsyncC1 and HsyncC2 for counting based on the enable signals EN-PD1 and EN-PD2 and the line synchronization signal Hsync, for example. In this example, the synchronization signal reproduction circuit 51 generates the line synchronization signal HsyncC1 for counting at the rising edge of the enable signal EN-PD1 and the falling edge of the line synchronization signal Hsync, respectively, to set and reset.

[0094] Similarly, the synchronization signal reproducing circuit 51 generates the line synchronization signal HsyncC2 for counting at the rising edge of the enable signal EN-PD2 and the falling edge of the line synchronization signal Hsync, respectively, to set and reset. Therefore, even if such an operation of temporarily negating the enable signals EN-PD1 and EN-PD2 in a period of several pixels is performed, the line synchronization signals HsyncC1 and HsyncC2 for counting can be generated without causing inconvenience.

[0095] The line counter 52 outputs a count value CV1 by counting the number of inputs of the line synchronization signal HsyncC1 for counting within the assertion period of the frame synchronization signal Vsync. Similarly, the line counter 52 outputs a count value CV2 by counting the number of inputs of the line synchronization signal HsyncC2 for counting within the assertion period of the frame synchronization signal Vsync.

[0096] The enable signals EN-PD1 and EN-PD2 are asserted respectively during the period of extracting the pixel data PD1 and PD2. The extracted pixel data PD1 and PD2 are each image data to be stored in the memory MEM, in other words, image data to be targeted for image processing excluding additional information (such as padding data) that is not necessary for image processing. Therefore, the count value CV1 during the assertion period of the frame synchronization signal Vsync is equal to the number of rows of the image data IMG1 to be stored in the memory MEM, that is, the image data IMG1 to be targeted for image processing. Similarly, the count value CV2 during the assertion period of the frame synchronization signal Vsync is equal to the number of rows of the image data IMG2 to be stored in the memory MEM, that is, the image data IMG2 to be targeted for image processing.

[0097] Therefore, it is found that if the count value CV is equal to the expected value EV, the rule-based image data identified by the image identifier is extracted from the image data acquired by the sensor 11 and stored in the memory MEM. In other words, it is found that if the count value CV is equal to the expected value EV, the image data as the image processing target can be correctly acquired from the image data acquired by the sensor 11. In this example, if the count value CV1 and the count value CV2 at the time point when the frame synchronization signal Vsync is negated are "n" and "n-1", respectively, the comparator 53 does not output the error signals ERR1 and ERR2.

[0098] On the other hand, if line data is lost in the path from the sensor 11 to the receiving interface circuit 20, or if image data to be stored in the memory is not correctly extracted from image data acquired via the receiving interface circuit 20, the count value CV does not match the expected value EV. In this case, the comparator 53 outputs an error signal ERR.

[0099] <Main Effects of the Second Embodiment>

[0100] As described above, when the method according to the second embodiment is used, effects similar to the various effects described in the first embodiment are obtained. In general, it is possible to verify whether the image data is successfully correctly acquired from the sensor. Further, even if image data with additional information, multiplexed image data, etc. have been received, it is possible to verify whether the correct acquisition of each image data is successful. In other words, while flexibly processing various transmission specifications used when the sensor sends image data, it is possible to verify whether the correct reception of each image data is successful. Further, it is possible to verify whether the data has been extracted so that the image data based on various transmission specifications is correctly stored in the memory.

[0101] (Third Embodiment)

[0102] <Overview of Receive Interface Circuit>

[0103] Fig. 9 is a diagram illustrating a semiconductor device according to a third embodiment. Figure 1 The illustrated diagram is a diagram of a configuration example of a packet PKT received by the reception interface circuit 20 and a configuration example of an image composite signal IMCS output from the reception interface circuit 20. Fig. 9 In the illustrated configuration example, a frame includes row data LD for only one row, which is different from Figure 2 The illustrated configuration example is different. The line data LD has, for example, 64k pixel data PD. The image data IMG composed of the pixel data PD is, for example, distance image data generated by a distance measuring sensor. That is, Figure 1 The at least one sensor 11 illustrated is a distance measuring sensor.

[0104] and Figure 3 Similar to the case shown in the figure, the receiving interface circuit 20 receives the packet PKT and outputs the image composite signal IMCS. However, in this case, in the image composite signal IMCS, the line synchronization signal Hsync is asserted only once. Therefore, it is difficult to fully verify whether the correct acquisition of the image data IMG is successful based on the line synchronization signal Hsync. Further, when such a large amount of continuous pixel data PD is stored in the memory MEM as it is, there is a concern that the address management of the memory MEM is complicated and the processing performed by the processor PRC is also complicated. Therefore, it is useful to use the capture circuit 21 described below.

[0105] <Details of Capture Circuit>

[0106] Fig.10 is a diagram illustrating a semiconductor device according to a third embodiment Figure 1 A circuit block diagram of a configuration example of the illustrated capture circuit 21. Fig.11 It's a picture. Fig.10 The illustrated timing diagram is an example of the operation of the row division circuit 60. Figure 4A In addition to the illustrated configuration examples, Fig.10 The illustrated capture circuit 21 c further includes a row division circuit 60 .

[0107] The line division circuit 60 receives the image composite signal IMCS from the receiving interface circuit 20 as its input, and converts Fig. 9 The row data LD of one row shown in the figure is divided into Fig.11 In this example, the row division circuit 60 divides the row data LD having 64k pixel data PD into divided row data each having 1k pixel data PD. In this case, the row division circuit 60 counts the number of pixel data by using the pixel counter 65. Then, due to this row division, the row division circuit 60 outputs 64 rows of divided row data LDd[1] to LDd

[64] .

[0108] Therefore, with Figure 2 Similarly to the case shown in the figure, the line division circuit 60 basically converts the one-dimensional image data IMG into the two-dimensional image data IMG. Then, the line division circuit 60 links the line synchronization signal Hsync with each of the plurality of divided line data LDd[1] to LDd

[64] . Figure 3 Similar to the case illustrated in the figure, the row division circuit 60 generates an image composite signal IMCS-D, which has been divided and includes a frame synchronization signal Vsync, a row synchronization signal Hsync, a data enable signal DEN and a data signal DAT, and outputs the image composite signal IMCS-D to the data extraction circuit 35a.

[0109] Similar to the data extraction circuit 35a, note that the row division circuit 60 more specifically refers to the setting register 36 when using the value of the virtual channel (VC) and the value of the data type (DT). In the setting register 36, information on whether to perform row division and a rule used in row division are preset for the image identifier IMG-ID determined by the value of the virtual channel (VC) and the value of the data type (DT). The row division circuit 60 performs row division based on the rule. When row division is not performed, the row division circuit 60 outputs the input image composite signal IMCS as the image composite signal IMCS-D that has been divided as it is.

[0110] On the other hand, Fig.10 In the embodiment, the monitor circuit 30a has Figure 4B However, unlike Figure 4BUnlike the illustrated case, the monitor circuit 30a receives the frame synchronization signal Vsync and the line synchronization signal Hsync included in the divided image composite signal IMCS-D as its input. Then, the monitor circuit 30a counts the number of inputs of the line synchronization signal Hsync. Fig.11 In the illustrated example, if the count value CV is not 64, the monitor circuit 30 a outputs the error signal ERR.

[0111] <Main Effects of the Third Embodiment>

[0112] As described above, when the method according to the third embodiment is used, effects similar to the various effects described in the first embodiment are obtained. Generally, it is possible to verify whether the image data is correctly acquired from the sensor. Further, even if one-dimensional image data is sent from the sensor, the one-dimensional image data is converted into two-dimensional image data and linked with the line synchronization signal, and therefore, it is possible to verify whether the image data is correctly acquired based on the line synchronization signal.

[0113] (Fourth embodiment)

[0114] <Overview of Receive Interface Circuit>

[0115] Fig.12 is a diagram illustrating a semiconductor device according to a fourth embodiment. Figure 1 The illustrated diagram is a diagram of a configuration example of a packet PKT received by the reception interface circuit 20 and a configuration example of an image composite signal IMCS output from the reception interface circuit 20. Fig.12 In the configuration example shown, Fig. 9 Similar to the illustrated configuration example, a frame includes row data LD for only one row. Fig. 9 Unlike the case in , the line data LD has, for example, two pixel data PD1 and PD2 configuring two image data IMG1 and IMG2. In this case, the number of pixel data PD1 is 64k, and the number of pixel data PD2 is 32k.

[0116] <Details of Capture Circuit>

[0117] Fig.13 is a diagram illustrating a semiconductor device according to a fourth embodiment Figure 1 A circuit block diagram of a configuration example of the illustrated capture circuit 21. Fig.14 It's a picture. Fig.13 A timing diagram of an example of the operation of the monitor circuit 30b is shown. Fig. 7A In addition to the illustrated configuration examples, Fig.13 The capture circuit 21d shown in the figure also includes Fig.10A similar row division circuit 60. The row division circuit 60 is provided at a prior stage of the data extraction circuit 35b.

[0118] The line division circuit 60 receives the image composite signal IMCS as its input and outputs the divided image composite signal IMCS-D to the data extraction circuit 35b. The divided image composite signal IMCS-D has divided line data of "64+32" lines instead of Fig.11 The first half of the 64 lines of divided line data is composed of pixel data PD1, and the second half of the 32 lines of divided line data is composed of pixel data PD2.

[0119] Fig.13 The illustrated monitor circuit 30b includes Fig. 7A The synchronizing signal reproducing circuit 51, the line counter 52 and the comparator 53 are similar to the case shown in the figure, and Fig. 7A The illustrated case operates similarly. However, Fig. 7A Unlike the illustrated case, the synchronization signal reproduction circuit 51 receives as its input the frame synchronization signal Vsync and the line synchronization signal Hsync included in the divided image composite signal IMCS-D.

[0120] exist Fig.14 , monitor circuit 30b receives enable signals EN-PD1 and EN-PD2 as its inputs from data extraction circuit 35b. Enable signal EN-PD1 is a signal that is asserted 64 times in the first half of the frame, and enable signal EN-PD2 is a signal that is asserted 32 times in the second half of the frame.

[0121] The synchronization signal reproducing circuit 51 generates the line synchronization signals HsyncC1 and HsyncC2 for counting based on the enable signals EN-PD1 and EN-PD2 included in the divided image composite signal IMCS-D and the line synchronization signal Hsync. The line counter 52 outputs count values ​​CV1 and CV2 by counting the number of inputs of each of the line synchronization signals HsyncC1 and HsyncC2 for counting within the assertion period of the frame synchronization signal Vsync. In this example, if the count value CV1 and the count value CV2 at the time point when the frame synchronization signal Vsync is negated are "64" and "32", respectively, the error signals ERR1 and ERR2 are not output.

[0122] <Main Effects of the Fourth Embodiment>

[0123] As described above, when the method according to the fourth embodiment is used, effects similar to the various effects described in the second and third embodiments are obtained. Generally, it is possible to verify whether the image data is correctly acquired from the sensor. Further, unlike the case illustrated in the third embodiment, even if multiplexed image data is received, it is possible to verify whether the correct acquisition of each image data is successful.

[0124] In the foregoing, the invention made by the inventors of the present application has been specifically described based on the embodiments. However, it goes without saying that the present invention is not limited to the foregoing embodiments, and various modifications can be made within the scope of the present invention.

Claims

1. A semiconductor device, comprising: a receiving interface circuit that receives a plurality of packets respectively including a plurality of line data and outputs an image composite signal generated by linking a line synchronization signal with each of the plurality of line data; as well as A capture circuit, provided in a subsequent stage of the receiving interface circuit, The capture circuit comprises: a line counter receiving the line synchronization signal included in the image composite signal as an input thereof and counting the number of times the line synchronization signal is input; as well as A comparator compares a count value counted by the row counter with a preset expected value of the number of rows, and outputs an error signal if the count value and the expected value do not match each other.

2. The semiconductor device according to claim 1, The receiving interface circuit is a circuit based on the MIPI (Mobile Industry Processor Interface) CSI-2 (Camera Serial Interface 2) standard.

3. The semiconductor device according to claim 2, The image composite signal further includes a frame synchronization signal, the frame synchronization signal is integrally linked to the plurality of line data, and The row counter counts the number of times the row synchronization signal is input within an assertion period of the frame synchronization signal.

4. The semiconductor device according to claim 1, Wherein each of the plurality of packets is a packet sent by an image sensor that generates image data based on image capture.

5. The semiconductor device according to claim 1, wherein the capture circuit further includes a line division circuit that divides line data of one line into a plurality of divided line data and outputs a divided image composite signal generated by linking the line synchronization signal with each of the plurality of divided line data, and The line counter receives the line synchronization signal included in the divided image composite signal as an input thereof, and counts the number of times the line synchronization signal is input.

6. The semiconductor device according to claim 5, Each of the plurality of packets is a packet sent by a ranging sensor that generates range image data based on a radar.

7. The semiconductor device according to claim 1, further comprising processor, wherein the capture circuit writes the row data into a memory, and The processor reads out the line data written in the memory and performs image processing or image recognition processing.

8. A semiconductor device comprising: a receiving interface circuit that receives a plurality of packets respectively including a plurality of line data and outputs an image composite signal generated by linking a line synchronization signal with each of the plurality of line data; as well as A capture circuit, provided in a subsequent stage of the receiving interface circuit, The capture circuit comprises: a data extraction circuit receiving the image composite signal as its input and extracting image data from the plurality of line data based on a preset rule; a synchronization signal reproducing circuit that links a row synchronization signal for counting with the image data extracted by the data extracting circuit for each row; a row counter receiving the row synchronization signal for counting as an input thereof and counting the number of times the row synchronization signal for counting is input; and A comparator uses a predetermined expected value of the number of lines of the image data to compare a count value counted by the line counter with the expected value, and outputs an error signal if the count value and the expected value do not match each other.

9. The semiconductor device according to claim 8, wherein each of one or more of the plurality of line data comprises first image data and second image data, The data extraction circuit extracts the first image data and the second image data separately, The synchronization signal reproducing circuit links the first row synchronization signal and the second row synchronization signal as the row synchronization signal for counting with the first image data and the second image data, respectively. The row counter receives the first row synchronization signal and the second row synchronization signal as inputs, and counts the number of inputs of the first row synchronization signal and the number of inputs of the second row synchronization signal separately as a first count value and a second count value, respectively, and The comparator uses a first expected value and a second expected value as predetermined expected values ​​of the number of rows of the first image data and the second image data, respectively, to compare the first count value with the first expected value, and output a first error signal if the first count value and the first expected value do not match each other, and compares the second count value with the second expected value, and outputs a second error signal if the second count value and the second expected value do not match each other.

10. The semiconductor device according to claim 8, The receiving interface circuit is a circuit based on the MIPI (Mobile Industry Processor Interface) CSI-2 (Camera Serial Interface 2) standard.

11. The semiconductor device according to claim 10, The image composite signal further includes a value of a virtual channel and a value of a data type based on the MIPI CSI-2 standard. For each combination of the value of the virtual channel and the value of the data type, the capture circuit saves in advance the rule used in the data extraction circuit and the expected value used in the comparator, The data extraction circuit determines the rule based on the value of the virtual channel and the value of the data type included in the image composite signal, and The comparator determines the expected value based on a value of the virtual channel and a value of the data type included in the image composite signal.

12. The semiconductor device according to claim 8, The image composite signal further includes a frame synchronization signal, the frame synchronization signal is integrally linked to the plurality of line data, and The row counter counts the number of times the row synchronization signal is input for counting within an assertion period of the frame synchronization signal.

13. The semiconductor device according to claim 8, Wherein each of the plurality of packets is a packet sent by an image sensor that generates image data based on image capture.

14. The semiconductor device according to claim 8, wherein the capture circuit further includes a line division circuit, which is provided at a front stage of the data extraction circuit, receives the image composite signal as an input of the line division circuit, divides line data of one line among the plurality of line data into a plurality of divided line data, and outputs a divided image composite signal generated by linking the line synchronization signal with each of the plurality of divided line data, and The data extraction circuit receives the divided image composite signal as an input thereof, and extracts image data from the plurality of divided line data based on a preset rule.

15. The semiconductor device according to claim 14, Each of the plurality of packets is a packet sent by a ranging sensor that generates range image data based on a radar.

16. The semiconductor device according to claim 8, further comprising processor, wherein the capture circuit writes the image data extracted by the data extraction circuit into a memory, and The processor reads out the image data written in the memory and performs image processing or image recognition processing.

Images