Image sensor and sensor system

Abstract

This image sensor comprises a calculation unit that calculates a hash value of a parameter read from a non-volatile memory provided outside the image sensor at startup.

Description

Image Sensor and Sensor System

[0001] The present disclosure relates to an image sensor and a sensor system.

[0002] It is known that parameters are used for operation settings and the like of an image sensor (also referred to as a solid-state imaging device).

[0003] Japanese Patent Application Laid-Open No. 2021-125716 International Publication No. WO 2018 / 066348 International Publication No. WO 2018 / 180575

[0004] The parameters are written, for example, in a non-volatile memory provided outside the image sensor. There is a possibility that the parameters written in the non-volatile memory may be tampered with.

[0005] One aspect of the present disclosure enables tamper checking of parameters.

[0006] An image sensor according to one aspect of the present disclosure includes a calculation unit that calculates a hash value of a parameter read from a non-volatile memory provided outside the image sensor at startup.

[0007] A sensor system according to one aspect of the present disclosure includes an image sensor, a first non-volatile memory in which parameters for operation setting of the image sensor are written, and a host device that performs data communication with the image sensor, wherein the image sensor calculates a hash value of the parameter read from the first non-volatile memory at startup, and the host device compares the hash value calculated by the image sensor with a expected value prepared in advance.

[0008] FIG. 1 is a diagram showing an example of a schematic configuration of a sensor system according to a first embodiment. FIG. 2 is a diagram showing an example of a schematic configuration of a non-volatile memory. FIG. 3 is a flowchart showing an example of processing (method) executed in the sensor system. FIG. 4 is a diagram showing a modified example. FIG. 5 is a diagram showing an example of a schematic configuration of a sensor system according to a second embodiment. FIG. 6 is a diagram showing an example of a schematic configuration of an OTP memory. FIG. 7 is a flowchart showing an example of processing (method) executed in the sensor system. FIG. 8 is a block diagram showing an example of a schematic configuration of a vehicle control system. FIG. 9 is an explanatory diagram showing an example of installation positions of an outside-vehicle information detection unit and an imaging unit.

[0009] Embodiments of this disclosure will be described in detail below with reference to the drawings. In each of the following embodiments, the same elements will be denoted by the same reference numerals to avoid redundant descriptions.

[0010] This disclosure will be described in the following order of items: 1. First Embodiment 2. Second Embodiment 3. Conclusion 4. Application Examples to Mobile Devices

[0011] 1. Figure 1 of the first embodiment shows an example of the schematic configuration of the sensor system 1 according to the first embodiment. The sensor system 1 is used by being mounted on a moving object such as a vehicle. One of the purposes of the sensor system 1 is imaging, and it is used, for example, to acquire image data of the area around a vehicle.

[0012] Figure 1 illustrates several components of the sensor system 1, indicated by reference numerals. Several data flows are also schematically shown with arrows. Note that the data may be interpreted as information, and they may be reinterpreted as appropriate within the bounds of consistency.

[0013] The sensor system 1 includes a module 3 and a host device 7. A lens 2 for guiding (for example, focusing) light from the subject to the module 3, more specifically to the imaging unit 51 of the image sensor 5 described later, is also shown with reference numerals. This lens 2 may also be included in the sensor system 1 as a component of the sensor system 1.

[0014] Figure 1 illustrates some of the components of module 3, denoted by reference numerals. In this example, module 3 includes a non-volatile memory 4, an image sensor 5, and an IF unit 6. These are modularized, for example, by mounting them on a substrate, such that at least the non-volatile memory 4 and the image sensor 5 are electrically connected to each other, and the non-volatile memory 4 and the IF unit 6 are electrically connected to each other.

[0015] The non-volatile memory 4 is located outside the image sensor 5. Various types of data used by module 3 are written to the non-volatile memory 4. Various known non-volatile memories may be used, one example being flash memory. The non-volatile memory 4 will be explained with reference to Figure 2.

[0016] Figure 2 shows an example of the schematic configuration of the non-volatile memory 4. The non-volatile memory 4 includes multiple storage areas, in this example, two storage areas. The first storage area is referred to as storage area 41 and is shown in the figure. The second storage area is referred to as storage area 42 and is shown in the figure.

[0017] Memory area 41 is an area reserved for the image sensor 5 and can also be called a reserved area. Various data used by the image sensor 5 is written to memory area 41. This data includes parameters for setting the operation of the image sensor 5, and these are called parameters p. Memory area 42 is an area that the user can use at will and can also be called a user area.

[0018] Returning to Figure 1, the parameter p written to the non-volatile memory 4 is also shown. This parameter p is used to set the operation of the image sensor 5 mounted on module 3 together with the non-volatile memory 4. The parameter p may be individually adjusted for each image sensor 5, thereby ensuring optimal operation of the image sensor 5 according to the application, etc., and absorbing individual differences in the image sensor 5.

[0019] Parameter p may be written to the non-volatile memory 4 at any time, and may also be overwritten (updated) at any time. Examples of timing include when adjusting the image sensor 5, when adjusting module 3, when adjusting sensor system 1, etc. The adjustment may be performed before shipment or after shipment.

[0020] The image sensor 5 is a sensor that detects light from a subject in order to capture an image of the subject. The image sensor 5 may be a solid-state imaging device such as a CMOS image sensor, and is composed of a semiconductor substrate such as silicon. In addition to the imaging function, the image sensor 5 may also have a distance measuring function.

[0021] Several components of the image sensor 5 are shown with reference numerals. In this example, the image sensor 5 includes an imaging unit 51, a calculation unit 52, and a read-only area 53.

[0022] The imaging unit 51 detects light from a subject guided to the imaging unit 51 via the lens 2, and generates and processes electrical signals indicating the detection results. Various known configurations may be employed. For example, the imaging unit 51 includes a plurality of pixels (pixel array) arranged in two dimensions. Each pixel may include a photoelectric conversion unit such as a PD (photodiode), and a transistor, floating diffusion, etc., for controlling the exposure of the photoelectric conversion unit and generating and extracting a voltage signal corresponding to the charge generated in the photoelectric conversion unit. The imaging unit 51 may also include a drive circuit for driving the transistors of each pixel, a signal processing circuit for generating image data of the subject based on signals from each pixel, etc.

[0023] The operation settings of the image sensor 5 using the parameter p described above may include the operation settings of the imaging unit 51. For example, the operation of the transistor, drive circuit, signal processing circuit, etc., can be set.

[0024] The image sensor 5 reads parameter p from the non-volatile memory 4 when it starts up. Parameter p is loaded into registers, memory, etc., within the image sensor 5, thereby completing the operation settings of the image sensor 5, for example, the operation settings of the imaging unit 51.

[0025] The calculation unit 52 calculates the hash value of parameter p. This value is referred to as the hash value v1. As mentioned earlier, parameter p is read from the non-volatile memory 4 when the image sensor 5 is started. The calculation unit 52 calculates the hash value v1 of parameter p read from the non-volatile memory 4 when the image sensor 5 is started.

[0026] Each time the image sensor 5 is activated, the hash value v1 may be calculated automatically. In this case, the choice of whether or not to perform the calculation of the hash value v1 by the calculation unit 52 (Enable / Disable) cannot be determined by external operation or settings of the image sensor 5.

[0027] Various hash functions may be used to calculate the hash value v1, one example being SHA-256. The calculation process by the calculation unit 52 may be implemented by hardware design, by software design, or by both hardware and software design.

[0028] The calculation of the hash value v1 by the calculation unit 52 may include a salting process. In this case, the calculation unit 52 adds arbitrary data to the parameter p and calculates the hash value v1 using that hash value. The arbitrary data may be data specific to the image sensor 5, one example being the ID of the image sensor 5 (sensor ID). Even with the same parameter p, the expected value v2 can be changed for each image sensor 5. This has advantages such as making it harder to analyze.

[0029] In one embodiment, error correction (e.g., ECC (Error-Correcting Code)) may be performed on the parameter p read from the non-volatile memory 4. This can improve the reliability of the hash calculation. In that case, the calculation unit 52 calculates the hash value v1 of the error-corrected parameter p. The main component of the error correction process may be the calculation unit 52 or other elements.

[0030] The read-only area 53 stores the hash value v1 calculated by the calculation unit 52. The read-only area 53 is an area where data can be read from an external device of the image sensor 5, such as the host device 7, but data cannot be written to it. This prevents the hash value v1 from being changed from outside the image sensor 5, thereby increasing the reliability of the hash value v1, for example.

[0031] The read-only area 53 may be accessible from other elements within the image sensor 5 for both reading and writing data. For example, the hash value v1 calculated by the calculation unit 52 may be written to the read-only area 53 by the calculation unit 52.

[0032] The IF unit 6 provides an interface for data communication with an external device of module 3, in this example, the host device 7. Various interfaces that can be used for data communication may be used. Examples of interfaces include the I2C (Inter-Integrated Circuit) interface, MIPI (Mobile Industry Processor Interface), and GPIO (General Purpose Input / Output) interface.

[0033] The host device 7 is located outside the image sensor 5, or more specifically, outside module 3 in this example. The host device 7 communicates with the image sensor 5 and thereby utilizes the image sensor 5. For example, when the sensor system 1 is mounted on a vehicle, the ECU (Electronic Control Unit) for controlling various electronic devices mounted on the vehicle can become the host device 7. One example of an ECU's use is providing a driver assistance system (e.g., ADAS).

[0034] Figure 1 shows several components of the host device 7, denoted by reference numerals. In this example, the host device 7 includes an IF unit 8, a memory 9, and a processor 10.

[0035] The IF unit 8 provides an interface for data communication with the image sensor 5 of module 3 via the IF unit 6 of module 3. Various interfaces may be used, and specific examples may be the same as those described earlier for the IF unit 6. For example, when the IF unit 6 of module 3 and the IF unit 8 of the host device 7 perform serial communication, one of the IF unit 6 and the IF unit 8 may function as a serializer and the other as a deserializer.

[0036] Memory 9 is located outside the processor 10. Various types of data used by the host device 7 are written to memory 9. Memory 9 includes non-volatile memory 91 and volatile memory 92. Various known types of memory may be used. An example of non-volatile memory 91 is flash memory. An example of volatile memory 92 is DRAM (Dynamic Random Access Memory).

[0037] Furthermore, the sensor system 1 has two non-volatile memories: non-volatile memory 4 and non-volatile memory 91. Non-volatile memory 4 and non-volatile memory 91 can also be called the first non-volatile memory and the second non-volatile memory.

[0038] Figure 1 shows the expected value v2 as data written to the non-volatile memory 91. The expected value v2 is a value that is compared with the hash value v1 and is prepared in advance and written to the non-volatile memory 91. The expected value v2 is the same as the hash value v1 of the parameter p written to the non-volatile memory 4 of module 3, assuming that the parameter p has not been tampered with.

[0039] In the example shown in Figure 1, the non-volatile memory 91 includes a secure area 910. The expected value v2 is written to this secure area 910. The secure area 910 is a secure area, and for example, data cannot be read or written (accessible) from devices outside the host device 7. On the other hand, data can be read and written to the secure area 910 from internal elements of the host device 7, such as the processor 10. Various data security technologies may be used. Note that the secure area 910 on which the expected value v2 is written may be located outside the host device 7 (for example, on a server device, etc.).

[0040] The processor 10 is a main control unit in the host device 7, and executes various types of processing necessary for operating, for example, a computer as the host device 7. An example of the processor 10 is a CPU (Central Processing Unit). Unless otherwise specifically described, the operation of the host device 7 shall be performed under processing by the processor 10.

[0041] Note that although not shown in FIG. 1, the processor 10 may communicate with other devices besides the module 3. For example, when the sensor system 1 is mounted on a vehicle for use, the processor 10 is connected to an in-vehicle network and communicates with various other devices mounted on the vehicle.

[0042] In FIG. 1, as one of the functional blocks of the processor 10, a comparison unit 11 is illustrated with a reference numeral. The comparison unit 11 compares the hash value v1 with the expected value v2. The comparison result indicates, for example, that the hash value v1 and the expected value v2 match, or that the hash value v1 and the expected value v2 do not match.

[0043] As described above, the hash value v1 is stored in the read-only area 53 of the image sensor 5 of the module 3. In this example, data indicating the hash value v1 is transmitted from the image sensor 5 to the comparison unit 11 of the processor 10 via the IF unit 6 of the module 3 and the IF unit 8 of the host device 7.

[0044] In one embodiment, the hash value v1 stored in the read-only area 53 may be read using protected data communication. The protected data communication may be, for example, data communication for which communication authentication has been performed, and an example thereof is authenticated I2C communication. Communication reliability can be improved.

[0045] If the parameter p written in the non-volatile memory 4 has not been tampered with, the hash value v1 and the expected value v2 will match. Conversely, if the parameter p written in the non-volatile memory 4 has been tampered with, the hash value v1 and the expected value v2 will not match. Tampering of the parameter p can be checked (the presence or absence of tampering can be detected) based on the comparison result between the hash value v1 and the expected value v2.

[0046] Note that, regarding the possibility of tampering, for example, if an SPI interface connected to the nonvolatile memory 4 is provided in the module 3, a conceivable case is that the parameter p in the nonvolatile memory 4 is intentionally tampered with from the outside via the SPI interface. Another conceivable case is that the parameter p is tampered with from the IF unit 6 by an entity impersonating the host device 7.

[0047] The comparison performed by the comparison unit 11 may be performed, for example, at the timing after the image sensor 5 is activated and the hash value v1 is calculated, and before the host device 7 starts using the image sensor 5. If the parameter p has not been tampered with, the host device 7 may start using the image sensor 5. Appropriate operation of the image sensor 5 based on an appropriate parameter p can thus be ensured.

[0048] If the parameter p written to the nonvolatile memory 4 has been tampered with, the host device 7 may, for example, without starting use of the image sensor 5, or concurrently with starting use, perform various types of operation (error operation) to notify of the tampering. The error operation may be, for example, an operation for notifying a user that the parameter p has been tampered with, or for notifying a management apparatus of the sensor system 1 (e.g., an external server apparatus, etc.) of the tampering.

[0049] An example of operation of the image sensor 5 after the start of use will now be described. The processor 10 of the host device 7 transmits various requests to the image sensor 5. A signal indicating this request is denoted as signal Sig1 and illustrated in the drawings. The signal Sig1 is a signal transmitted from the processor 10 of the host device 7 to the image sensor 5 of the module 3, and in this example, is transmitted via the IF unit 8 of the host device 7 and the IF unit 6 of the module 3.

[0050] In the image sensor 5, the imaging unit 51 detects light from the subject and generates image data of the subject in response to the signal Sig1. The image sensor 5 transmits the generated image data to the processor 10 of the host device 7. This signal indicating the image data is referred to as signal Sig2 and is shown in the diagram. Signal Sig2 is transmitted from the image sensor 5 of module 3 to the processor 10 of the host device 7, and in this example, it is transmitted via the IF unit 6 of module 3 and the IF unit 8 of the host device 7.

[0051] The processor 10 of the host device 7 processes the image data from the image sensor 5 of module 3. Various image processing may be performed, one example of which is the driver assistance mentioned earlier.

[0052] According to the sensor system 1 of the first embodiment described above, the host device 7 can check for tampering with parameter p simply by comparing the hash value v1 automatically calculated within the image sensor 5 with the expected value v2. The computational load of the hash value v1 using the hash function is light, and the processing burden is small even in the case of software implementation. Even in the case of hardware implementation, the circuit size can be small. The impact on the startup time of the image sensor 5 is also small.

[0053] The algorithm for calculating the hash value v1, including the input data sequence (the order of input for parameter p), is entirely contained within the image sensor 5, so the user doesn't need to worry about it at all. In other words, the calculation of the hash value v1 is completed within the image sensor 5, and there is the advantage that its algorithm is difficult to analyze from the outside.

[0054] The expected value v2 is the value written to the non-volatile memory 91 on the host device 7, more specifically, the secure area 910. If the parameter p written to the non-volatile memory 4 is updated, the expected value v2 can be easily updated accordingly.

[0055] This section will discuss comparisons with other detection methods. One possible method involves the host device 7 reading and verifying the parameter p from the non-volatile memory 4. However, due to limitations in communication bandwidth such as I2C, it may be difficult to read all the data. Another method involves security keys. For example, the image sensor 5 could store the expected value of an algorithm (HMAC, CMAC, etc.) using a key, and perform MAC calculation and expected value comparison in the image sensor 5. However, this would increase the circuit size, and require consideration of key management such as the division of responsibility between vendors and users, management methods, writing locations, and handling on the manufacturing line, resulting in a high implementation burden (cost). For example, these problems are addressed by the sensor system 1 according to the first embodiment described above.

[0056] Figure 3 is a flowchart illustrating an example of the processing (method) performed in sensor system 1. Explanations of content that overlaps with previous explanations will be omitted as appropriate.

[0057] In step S1, the hash value v1 of parameter p is calculated. When the image sensor 5 is started, parameter p is read from the non-volatile memory 4. The calculation unit 52 of the image sensor 5 calculates the hash value v1 of that parameter p.

[0058] In step S2, the hash value v1 is stored in the read-only area 53. For example, the calculation unit 52 of the image sensor 5 writes the hash value v1 calculated in step S1 to the read-only area 53.

[0059] In step S3, the hash value v1 is read using the protected data communication. For example, I2C communication authentication is performed between the IF unit 6 of module 3 and the IF unit 8 of host device 7. The processor 10 of host device 7 reads the hash value v1 stored in the read-only area 53 of image sensor 5 using the authenticated I2C communication.

[0060] In step S4, the hash value v1 and the expected value v2 are compared. The comparison unit 11 of the processor 10 of the host device 7 compares the hash value v1 read in the previous step S3 with the expected value v2 written to the secure area 910 of the non-volatile memory 91 of the host device 7. As mentioned earlier, the comparison result can be used to check for tampering with parameter p.

[0061] <Different example>

[0062] Figure 4 shows a modified example. Compared to the configuration shown in Figure 1, the sensor system 1 differs in that module 3 does not include the IF unit 6, and the host device 7 does not include the IF unit 8. The image sensor 5 of module 3 and the processor 10 of the host device 7 are configured to communicate directly. The absence of the IF unit 6 and IF unit 8 simplifies the configuration.

[0063] 2. Second Embodiment In the first embodiment described above, the case in which the host device 7 compares the hash value v1 and the expected value v2 was explained as an example. In the second embodiment described below, the image sensor 5 compares the hash value v1 and the expected value v2.

[0064] Figure 5 shows an example of the schematic configuration of the sensor system 1 according to the second embodiment. The image sensor 5 differs from the configuration of Figure 1 described earlier, in that it does not include a read-only area 53, but includes an OTP memory 54 and a comparison unit 55.

[0065] The OTP memory 54 is an OTP (One Time Programmable) memory that has an area where data can be written only once. Various types of OTP memory may be used, one example being an eFuse (electronic fuse) memory.

[0066] The expected value v2 is written to the OTP memory 54. The comparison unit 55 compares the hash value v1 calculated by the calculation unit 52 with the expected value v2 written to the OTP memory 54. The data showing the comparison result may be transmitted to a host device 7 located outside the image sensor 5, or more specifically, outside the module 3 in this example.

[0067] The comparison process performed by the comparison unit 55 may be implemented by hardware design, by software design, or by both hardware and software design.

[0068] The host device 7 differs from the configuration shown in Figure 1, as described earlier, in that the expected value v2 is not written to the non-volatile memory 91, and the processor 10 does not include the comparison unit 11. In the host device 7, neither the calculation of the hash value v1 nor the comparison of the hash value v1 and the expected value v2 is necessary. These processes are completed within the image sensor 5 when the image sensor 5 is started. The host device 7 can check for tampering with the parameter p simply by referring to the comparison results sent from the image sensor 5.

[0069] In one embodiment, the OTP memory 54 may be configured to be able to write multiple expected values. This will be explained with reference to Figure 6.

[0070] Figure 6 shows an example of the schematic configuration of the OTP memory 54. The OTP memory 54 includes multiple storage areas 540. In this example, N storage areas 540 are exemplified. N is an integer greater than or equal to 2, and specific examples include 8, 16, etc. In Figure 6, the 1st and Nth storage areas 540 are exemplified as storage area 540-1 and storage area 540-N.

[0071] Each memory area 540 has a capacity to write one expected value v2. An expected value v2 can be written to a single memory area 540 only once. An expected value v2 is written to at least one of the multiple memory areas 540.

[0072] For example, if a parameter p written to the non-volatile memory 4 is overwritten (updated) for a legitimate reason, not tampering, the hash value of the updated parameter p is written as the latest expected value v2 to one of the unused memory areas 540 among the multiple memory areas 540. Although there are multiple expected values ​​v2 in the OTP memory 54, in this case the latest expected value v2 is used for comparison with the hash value v1. That is, the comparison unit 55 compares the latest expected value v2 among the multiple expected values ​​v2 written to the OTP memory 54 with the hash value v1.

[0073] Figure 7 is a flowchart illustrating an example of the processing (method) performed in sensor system 1. Explanations of content that overlaps with previous explanations will be omitted as appropriate.

[0074] In step S11, the hash value v1 of parameter p is calculated. The calculation unit 52 of the image sensor 5 calculates the hash value v1 of parameter p read from the non-volatile memory 4.

[0075] In step S12, the hash value v1 and the expected value v2 are compared. The comparison unit 55 of the image sensor 5 compares the hash value v1 calculated in the previous step S11 with the expected value v2 written to the OTP memory 54. As described above, the comparison result can be used to check for tampering with parameter p.

[0076] 3. The technologies described above can be identified, for example, as follows: One of the disclosed technologies is an image sensor 5. As explained with reference to Figures 1 to 7, the image sensor 5 includes a calculation unit 52 that calculates a hash value v1 of a parameter p read from a non-volatile memory 4 located outside the image sensor 5 at startup. For example, the hash value v1 may be compared with a pre-prepared expected value v2. The expected value v2 may be the same value as the hash value of the parameter p if the parameter p written to the non-volatile memory 4 has not been tampered with. This makes it possible to check for tampering with the parameter p.

[0077] As explained with reference to Figure 1, the image sensor 5 includes a read-only area 53 where the hash value v1 calculated by the calculation unit 52 is stored. The read-only area 53 may be an area where data can be read from an external device (e.g., a host device 7) but not written to. This prevents the hash value v1 from being altered from outside the image sensor 5. By preventing the hash value v1 from being altered from outside the image sensor 5, the reliability of the hash value v1 can be improved, for example.

[0078] As explained with reference to Figure 1, the calculation unit 52 calculates the hash value v1 by adding arbitrary data to the parameter p, and the arbitrary data may include data specific to the image sensor 5. Even with the same parameter p, the expected value v2 can be changed for each image sensor 5. This has advantages such as making it more difficult to analyze.

[0079] As explained with reference to Figure 1, the calculation unit 52 may calculate the hash value v1 of the error-corrected parameter p. This can improve the reliability of the calculation.

[0080] As explained with reference to Figure 1, the hash value v1 may be compared with a pre-prepared expected value v2 by a host device 7 located outside the image sensor 5. This enables the host device 7 to check for tampering with parameter p. For example, the hash value v1 may be read by the host device 7 using protected data communication (e.g., authenticated I2C communication). This can improve the reliability of the communication and, consequently, the reliability of checking for tampering with parameter p.

[0081] As explained with reference to Figure 5, the image sensor 5 may include an OTP memory 54 on which a pre-prepared expected value v2 is written, and a comparison unit 55 that compares the hash value v1 calculated by the calculation unit 52 with the expected value v2 written to the OTP memory 54 (e.g., eFuse memory). The comparison process can be completed within the image sensor 5. The OTP memory 54 may include a plurality of storage areas 540, each on which an expected value v2 can be written. The latest expected value v2 can be written to the OTP memory 54 in accordance with the update of the parameter p.

[0082] The sensor system 1, described with reference to Figures 1 to 4, is also one of the disclosed technologies. The sensor system 1 comprises an image sensor 5, a non-volatile memory 4 (first non-volatile memory) on which parameters p for setting the operation of the image sensor 5 are written, and a host device 7 that communicates data with the image sensor 5. When the image sensor 5 starts up, it calculates a hash value v1 of the parameters p read from the non-volatile memory 4. The host device 7 compares the hash value v1 calculated by the image sensor 5 with a pre-prepared expected value v2. As described above, this sensor system 1 also makes it possible to check for tampering with the parameters p.

[0083] As explained with reference to Figure 1, the host device 7 may include a non-volatile memory 91 (second non-volatile memory) on which the expected value v2 is written. The non-volatile memory 91 includes a secure area 910, and the expected value v2 may be written to the secure area 910. This improves the reliability of the comparison between the hash value v1 and the expected value v2, and consequently, the reliability of the parameter p tampering check.

[0084] 4. Examples of Applications to Mobile Devices The technology disclosed herein (the technology) can be applied to a variety of products. For example, the technology disclosed herein may be implemented as a device mounted on any type of mobile device, such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, and robots.

[0085] Figure 8 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0086] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 8, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0087] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0088] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0089] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0090] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0091] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0092] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0093] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0094] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0095] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying the vehicle occupants or those outside the vehicle of information. In the example in Figure 8, the output devices are exemplified as an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0096] Figure 9 shows an example of the installation position of the imaging unit 12031.

[0097] In Figure 9, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0098] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0099] Figure 9 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0100] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0101] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0102] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0103] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0104] The above describes an example of a vehicle control system to which the technology relating to this disclosure may be applied. The technology relating to this disclosure can be applied to, for example, the external information detection unit 12030 among the configurations described above. For example, the sensor system 1 and image sensor 5 described earlier can be applied to the external information detection unit 12030 and imaging unit 12031 described above. This makes it possible to check for tampering with parameter p, thereby increasing the likelihood of ensuring proper operation, for example.

[0105] The effects described in this disclosure are merely illustrative and not limited to those disclosed. Other effects may also occur.

[0106] While embodiments of this disclosure have been described above, the technical scope of this disclosure is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of this disclosure. Furthermore, components from different embodiments and modifications may be combined as appropriate.

[0107] Furthermore, this technology can also take the following configurations: (1) An image sensor comprising: a calculation unit that calculates a hash value of a parameter read from a non-volatile memory provided outside the image sensor when the image sensor is started up. (2) The image sensor according to (1), wherein the hash value is compared with a pre-prepared expected value. (3) The image sensor according to (2), wherein the expected value is the same as the hash value of the parameter if the parameter written to the non-volatile memory has not been tampered with. (4) The image sensor according to any one of (1) to (3), comprising: a read-only area where the hash value calculated by the calculation unit is stored, wherein the read-only area is an area from which data can be read but not written by a device outside the image sensor. (5) The image sensor according to any one of (1) to (4), wherein the calculation unit calculates the hash value by adding arbitrary data to the parameter, and the arbitrary data includes data specific to the image sensor. (6) The image sensor according to any one of (1) to (5), wherein the calculation unit calculates the hash value of the parameter after error correction. (7) The image sensor according to any one of (1) to (6), wherein the hash value is compared with a pre-prepared expected value by a host device provided outside the image sensor. (8) The image sensor according to (7), wherein the hash value is read by the host device using protected data communication. (9) The image sensor according to (8), wherein the data communication includes authenticated I2C communication. (10) The image sensor according to any one of (1) to (6), comprising: an OTP (One Time Programmable) memory on which a pre-prepared expected value is written; and a comparison unit that compares the hash value calculated by the calculation unit with the expected value written in the OTP memory. (11) The image sensor according to (10), wherein the OTP memory includes a plurality of storage areas, each on which the expected value can be written.(12) The image sensor according to (10) or (11), wherein the OTP memory includes an eFuse memory. (13) A sensor system comprising: an image sensor; a first non-volatile memory on which parameters for setting the operation of the image sensor are written; and a host device that communicates data with the image sensor, wherein the image sensor calculates a hash value of the parameters read from the first non-volatile memory at startup, and the host device compares the hash value calculated by the image sensor with a pre-prepared expected value. (14) The sensor system according to (13), wherein the expected value is the same as the hash value of the parameters written to the first non-volatile memory if the parameters have not been tampered with. (15) The sensor system according to (13) or (14), wherein the host device includes a second non-volatile memory on which the expected value is written. (16) The sensor system according to (15), wherein the second non-volatile memory includes a secure area, and the expected value is written to the secure area. (17) The sensor system according to any one of (13) to (16), wherein the host device reads the hash value from the image sensor using protected data communication. (18) The sensor system according to (17), wherein the data communication includes authenticated I2C communication.

[0108] 1 Sensor system 2 Lens 3 Module 4 Non-volatile memory 41 Storage area 42 Storage area 5 Image sensor 51 Imaging unit 52 Calculation unit 53 Read-only area 54 OTP memory 540 Storage area 55 Comparison unit 6 IF unit 7 Host device 8 IF unit 9 Memory 91 Non-volatile memory 910 Secure area 92 Volatile memory 10 Processor p Parameter v1 Hash value v2 Expected value

Claims

1. An image sensor comprising a calculation unit that calculates a hash value of parameters read from a non-volatile memory provided outside the image sensor when the image sensor is started up.

2. The image sensor according to claim 1, wherein the hash value is compared with a pre-prepared expected value.

3. The image sensor according to claim 2, wherein the expected value is the same as the hash value of the parameter if the parameter written to the non-volatile memory has not been tampered with.

4. The image sensor according to claim 1, comprising a read-only area in which the hash value calculated by the calculation unit is stored, wherein the read-only area is an area in which data can be read from but not written to by an external device of the image sensor.

5. The image sensor according to claim 1, wherein the calculation unit calculates the hash value by adding arbitrary data to the parameters, and the arbitrary data includes data specific to the image sensor.

6. The image sensor according to claim 1, wherein the calculation unit calculates the hash value of the parameter after error correction.

7. The image sensor according to claim 1, wherein the hash value is compared with a pre-prepared expected value by a host device provided outside the image sensor.

8. The image sensor according to claim 7, wherein the hash value is read by the host device using protected data communication.

9. The image sensor according to claim 8, wherein the data communication includes authenticated I2C communication.

10. An image sensor according to claim 1, comprising: an OTP (One Time Programmable) memory on which a pre-prepared expected value is written; and a comparison unit that compares the hash value calculated by the calculation unit with the expected value written in the OTP memory.

11. The image sensor according to claim 10, wherein the OTP memory includes a plurality of storage areas, each capable of writing the expected value.

12. The image sensor according to claim 10, wherein the OTP memory includes an eFuse memory.

13. A sensor system comprising: an image sensor; a first non-volatile memory on which parameters for setting the operation of the image sensor are written; and a host device that communicates data with the image sensor, wherein the image sensor calculates a hash value of the parameters read from the first non-volatile memory at startup, and the host device compares the hash value calculated by the image sensor with a pre-prepared expected value.

14. The sensor system according to claim 13, wherein the expected value is the same as the hash value of the parameter if the parameter written to the first non-volatile memory has not been tampered with.

15. The sensor system according to claim 13, wherein the host device includes a second non-volatile memory on which the expected value is written.

16. The sensor system according to claim 15, wherein the second non-volatile memory includes a secure area, and the expected value is written to the secure area.

17. The sensor system according to claim 13, wherein the host device reads the hash value from the image sensor using protected data communication.

18. The sensor system according to claim 17, wherein the data communication includes authenticated I2C communication.

Images