Data processing device, imaging system, and ranging system

Abstract

The present disclosure pertains to a data processing device, an imaging system, and a distance measurement system that make it possible to transmit data more suitably. In the present invention, a security processing unit performs security processing on each group of designated data items, among a plurality of types of data items of one frame obtained from a sensor chip, and a transmission unit performs framing of data acquired through the security processing by the security processing unit and transmits the data to a host device. Further, each of the groups is set in accordance with a data unit to be read from a memory, in the host device, for temporarily storing the data transmitted from the transmission unit. The present technology is applicable to an imaging system and a distance measurement system, for example.

Description

[Technical Field]

[0001] The present disclosure relates to a data processing device, an imaging system, and a ranging system, and more particularly to a data processing device, an imaging system, and a ranging system that are capable of more suitably transmitting data. [Background technology]

[0002] Conventionally, data transmission standards have been proposed that allow different types of data, such as image data and image plane phase difference data, to be transmitted from an imaging element to a host device. For example, in a data transmission standard that transmits multiple types of data in one frame from the frame start to the frame end, each type of data is distinguished by an ID (Identification).

[0003] For example, Patent Document 1 discloses a technique for distinguishing components that have acquired data from each other using a component ID associated with the imaging component in a method for communicating between imaging components of an X-ray imaging system. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2020-533924 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if the entire data set from the frame start to the frame end is treated as a single processing range for security processing, there is a concern that the data may be difficult to handle in the subsequent processing stage to which the data is transmitted. Therefore, a more suitable data transmission method is needed, taking into account the handling of data in the subsequent processing stage.

[0006] The present disclosure has been made in light of such circumstances, and aims to enable more suitable data transmission. [Means for solving the problem]

[0007] A data processing device according to a first aspect of the present disclosure includes a security processing unit that performs security processing on each group of specified data from among multiple types of data for one frame obtained from a sensor chip, and a transmission unit that frames the data obtained by the security processing by the security processing unit and transmits it to a host device.

[0008] In a first aspect of the present disclosure, security processing is performed on each group of specified data from multiple types of data for one frame obtained from a sensor chip, and the data obtained by the security processing is framed and transmitted to a host device.

[0009] An imaging system according to a second aspect of the present disclosure includes an image sensor chip capable of acquiring image data or image plane phase difference data, a security processing unit that performs security processing on each group consisting of specified data from among multiple types of data for one frame obtained from the image sensor chip, and a transmission unit that frames the data acquired by the security processing by the security processing unit and transmits it to a host device, and the security processing unit performs security processing on each of the groups including the image data and the groups including the image plane phase difference data.

[0010] In a second aspect of the present disclosure, a security process is performed on each group of designated data among multiple types of data for one frame obtained from an image sensor chip capable of acquiring image data or image plane phase difference data, and the data acquired by the security process is framed and transmitted to a host device. Then, the security process is performed on each group including image data and each group including image plane phase difference data.

[0011] A ranging system according to a third aspect of the present disclosure includes an iTOF sensor chip used to measure the distance to a subject from the phase shift in the reflected light of pulsed light output toward the subject; a security processing unit that performs security processing on each group consisting of specified data from among multiple types of data for one frame obtained from the iTOF sensor chip; and a transmission unit that converts the data obtained by the security processing by the security processing unit into frames and transmits them to a host device, and the security processing unit performs security processing on each of the groups output from the iTOF sensor chip, each of which includes brightness data with different phases.

[0012] In a third aspect of the present disclosure, a security process is performed on each group of designated data among multiple types of data for one frame obtained from an iTOF sensor chip used to measure the distance to a subject from the phase shift in the reflected light of pulsed light output toward the subject, and the data obtained by the security process is framed and transmitted to a host device. Then, the security process is performed on each group of brightness data output from the iTOF sensor chip, each of which has a different phase. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an example of the configuration of an imaging system according to a first embodiment to which the present technology is applied. [Figure 2] FIG. 10 is a diagram illustrating an example of a frame structure in which specified data in one frame is set as a security group. [Figure 3] FIG. 10 is a diagram illustrating an example of a frame structure in which all data in one frame is set as a security group. [Figure 4] FIG. 10 is a diagram illustrating an identification method in SLVS-EC. [Figure 5] FIG. 1 is a diagram illustrating an identification method in MIPI. [Figure 6] FIG. 10 is a block diagram showing an example configuration of a ranging system according to a second embodiment to which the present technology is applied. [Figure 7] FIG. 10 is a diagram illustrating an example of a frame structure in which specified data in one frame is set as a security group. [Figure 8] FIG. 10 is a diagram illustrating an example of a frame structure in which all data in one frame is set as a security group. [Figure 9] FIG. 10 is a diagram illustrating an example of a frame structure applied to a dual gain system. [Figure 10] FIG. 10 is a diagram showing a first modified example of a frame structure applied to a dual gain system. [Figure 11] FIG. 10 is a diagram illustrating a second modified example of a frame structure applied to a dual gain system. [Figure 12] FIG. 10 is a diagram illustrating an example of a frame structure applied to a digital overlap method. [Figure 13] FIG. 1 is a block diagram illustrating an example of the configuration of an imaging device. [Figure 14] FIG. 1 is a diagram illustrating an example of use of an image sensor. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

[0015] <Example of imaging system configuration> FIG. 1 is a block diagram showing an example of the configuration of an imaging system according to a first embodiment to which the present technology is applied.

[0016] 1 is configured by connecting an imaging element 21 made of a CIS (CMOS (Complementary Metal Oxide Semiconductor) Image Sensor) and a host device 22 made up of an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), etc. via a predetermined communication interface. The imaging system 11 also includes a focus driver 23 for driving a focus lens of an optical system that forms an image of a subject on the imaging surface of the imaging element 21.

[0017] For example, it is assumed that data transmission standards such as SLVS-EC (Scalable Low Voltage Signaling with Embedded Clock), MIPI (Mobile Industry Processor Interface Alliance), and subLVDS (Low Voltage Differential Signaling) are adopted for the communication interface used in the imaging system 11. Of course, the data transmission standards are not limited to these, and various other data transmission standards may also be adopted.

[0018] The imaging element 21 has a stacked structure in which an image sensor chip 31 and a signal processing chip 32 are stacked.

[0019] The image sensor chip 31 has a plurality of pixels arranged on its imaging surface, and each pixel outputs a pixel signal corresponding to the amount of light received. For example, R pixels that receive light transmitted through a red filter, G pixels that receive light transmitted through a green filter, and B pixels that receive light transmitted through a blue filter are arranged in a Bayer array on the imaging surface of the image sensor chip 31. Furthermore, on the imaging surface of the image sensor chip 31, image plane phase difference pixels for detecting phase differences on the imaging surface are arranged in place of some of the G pixels. Therefore, image data and image plane phase difference data can be obtained from the pixel signals output from the image sensor chip 31.

[0020] The signal processing chip 32 is configured to include an image data processing unit 41, an image plane phase difference data processing unit 42, a security processing unit 43, a link layer interface 44, a physical layer interface 45, a processing range designation unit 46, and a communication unit 47. For example, pixel signals output from the R pixels, G pixels, and B pixels of the image sensor chip 31 are supplied to the image data processing unit 41, and pixel signals output from the image plane phase difference pixels of the image sensor chip 31 are supplied to the image plane phase difference data processing unit 42.

[0021] The image data processing unit 41 performs image data processing to generate RAW data from pixel signals output from R pixels, G pixels, and B pixels, and supplies the RAW data generated by the image data processing to the security processing unit 43. For example, the image data processing unit 41 can generate RAW data by removing image plane phase difference pixel information or by interpolating image plane phase difference pixel information. Furthermore, in a monitoring mode, the image data processing unit 41 can separate image plane phase difference pixel information and then generate sum data by adding pixel signals of multiple same-color pixels as RAW data. Furthermore, the image data processing unit 41 generates embedded data that includes additional information other than the image (for example, values ​​such as gain, white balance, and exposure time), and supplies the embedded data to the security processing unit 43.

[0022] The image plane phase difference data processing unit 42 performs image plane phase difference data processing to generate image plane phase difference data indicating the distance to the subject from the pixel signals output from the image plane phase difference pixels, and supplies the image plane phase difference data generated by the image plane phase difference data processing to the security processing unit 43.

[0023] The security processing unit 43 performs security processing required to ensure security for each data within a predetermined processing range on the embedded data and RAW data supplied from the image data processing unit 41 and the image plane phase difference data supplied from the image plane phase difference data processing unit 42. For example, the security processing unit 43 can classify one frame's worth of embedded data, RAW data, and image plane phase difference data within a processing range that conforms to specifications, register settings, etc., into security groups and perform security processing on each security group. For example, the data that falls into a security group is specified according to the number of lines, the type of data, and the data unit read from the memory 53 of the host device 22.

[0024] Here, the security processing involves either a process of encrypting or decrypting security group data, or a process of adding security information such as a CRC (Cyclic Redundancy Check) or MAC (Message Authentication Code) derived from the security group data to ensure data integrity or functional safety. Alternatively, the security processing may involve a combination of these processes. Note that the CRC added in the security processing is a CRC that can be retained even after high-speed interface communication, separate from the CRC or ECC (Error Correction Code) for the communication path.

[0025] 2A, the security processing unit 43 can encrypt embedded data and RAW data as a security group and perform security processing to add first security information determined from the embedded data and RAW data. Furthermore, the security processing unit 43 can encrypt image plane phase difference data as a security group and perform security processing to add second security information determined from the image plane phase difference data.

[0026] The link layer interface 44 performs link layer processing for communication with the host device 22 and transmits data in accordance with the standard of the communication interface (e.g., SLVS-EC or MIPI) used in the imaging system 11. For example, the link layer interface 44 performs processing to store data for each packet that has been security processed for each security group in the security processing unit 43 in a payload and packetize the data by adding a packet header PH and a packet footer PF. The link layer interface 44 also performs processing to frame one frame of data that has been security processed for each security group in the security processing unit 43 by adding a frame start FS, which is a signal indicating the start of a frame, and a frame end FE, which is a signal indicating the end of a frame.

[0027] The physical layer interface 45 performs processing related to the physical layer in communication with the host device 22 and transmits data in accordance with any of A-PHY, C-PHY, and D-PHY that comply with the MIPI standard. Note that the physical layer interface 45 may also employ other standards such as SLVS and subLVDS.

[0028] The processing range designation unit 46 designates data for each security group that will be the processing range when the security processing unit 43 performs security processing, in accordance with processing range setting information supplied from the outside via the communication unit 47. Note that the data designated as a security group by the processing range designation unit 46 will be described later with reference to FIGS. 2 and 3.

[0029] The communication unit 47 performs communication required for external control of the image sensor 21. For example, when the communication unit 47 acquires processing range setting information through communication with the outside, the communication unit 47 supplies the processing range setting information to the processing range designation unit 46.

[0030] The host device 22 is configured to include a physical layer interface 51, a link layer interface 52, a memory 53, an output image processing unit 54, and an autofocus processing unit 55. The output image processing unit 54 includes a security processing unit 61, and the autofocus processing unit 55 includes a security processing unit 62.

[0031] Like the physical layer interface 45, the physical layer interface 51 performs processing related to the physical layer in communication with the image sensor 21 in accordance with any of A-PHY, C-PHY, and D-PHY that comply with the MIPI standard.

[0032] Similar to the link layer interface 44 , the link layer interface 52 performs processing related to the link layer in communication with the image sensor 21 in accordance with the standard of the communication interface used in the image capturing system 11 .

[0033] The memory 53 temporarily stores the embedded data, RAW data, image plane phase difference data, and security information transmitted from the image sensor 21.

[0034] The output image processing unit 54 performs output image processing to read the embedded data and RAW data from the memory 53 and generate an output image to be output to a display unit (not shown) at a subsequent stage. At this time, in the output image processing unit 54, for example, as shown in A of Fig. 2 (described later), if the embedded data and RAW data are encrypted as a security group and first security information required from the security group is added, the security processing unit 61 performs processing to decrypt the embedded data and RAW data as a security group and can perform processing related to data integrity verification and functional safety using the first security information.

[0035] The autofocus processing unit 55 reads the image plane phase difference data from the memory 53 and performs autofocus processing to control the focus driver 23 in accordance with the distance to the subject based on the image plane phase difference data. At this time, in the output image processing unit 54, for example, as shown in A of Fig. 2 described later, if the image plane phase difference data is encrypted as a security group and second security information determined from the security group is added, the security processing unit 62 performs processing to decrypt the image plane phase difference data as a security group and can perform processing related to data integrity verification and functional safety using the second security information.

[0036] The imaging system 11 is configured as described above, and data that has been security processed for each security group in the security processing unit 43 is transmitted from the imaging element 21 to the host device 22. Therefore, in the host device 22, the output image processing unit 54 can collectively read from the memory 53 the embedded data and RAW data that have been security processed as one security group, and the autofocus processing unit 55 can read from the memory 53 the image plane phase difference data that has been security processed as one security group.

[0037] That is, in the imaging system 11, by setting a security group according to the data unit read from the memory 53, data can be easily handled in the host device 22. For example, since the paths for RAW data and image plane phase difference data are different in the host device 22, by encrypting each data and generating security information such as CRC or MAC for each data, data can be easily handled in the output image processing unit 54 and the autofocus processing unit 55.

[0038] Furthermore, even if security processing is performed on each line of data, it is conceivable that the output image processing unit 54 and the autofocus processing unit 55 in the host device 22 can read only the necessary data. However, in this case, there are conceivable disadvantages, such as an increased frequency of bandwidth-limited key exchange and the need for additional functionality to handle missing lines within a frame.

[0039] In contrast to this, by performing security processing for each group consisting of specified data among a plurality of types of data, as in the imaging system 11, data can be transmitted more suitably.

[0040] FIG. 2 is a diagram illustrating a frame structure when data is transmitted from the imaging element 21 to the host device 22 in the imaging system 11. As shown in FIG.

[0041] 2A, one frame is configured by arranging data in the following order between the frame start and frame end: embedded data, RAW data, first security information, image plane phase difference data, and second security data. That is, in the first frame structure, the processing range is specified so that the embedded data and RAW data are grouped into a security group, and the processing range is also specified so that the image plane phase difference data is grouped into a security group. Then, in the security processing, the first security information is obtained from the embedded data and RAW data and placed next to the RAW data, and the second security information is obtained from the image plane phase difference data and placed next to the image plane phase difference data.

[0042] 2B, one frame is configured by arranging the following data between the frame start and frame end in this order: embedded data, RAW data, image plane phase difference data, first security information, and second security data. That is, in the second frame structure, the processing range is specified so that the embedded data and RAW data are grouped into a security group, and the processing range is also specified so that the image plane phase difference data is grouped into a security group. Then, in the security processing, the first security information is obtained from the embedded data and RAW data and placed next to the image plane phase difference data, and the second security information is obtained from the image plane phase difference data and placed next to the first security information.

[0043] 2C, one frame is configured by arranging data in the following order between the frame start and frame end: front embedded data, RAW data, image plane phase difference data, rear embedded data, first security information, and second security data. That is, in the third frame structure, the processing range is specified so that the front embedded data, RAW data, and rear embedded data are grouped into a security group, and the processing range is also specified so that the image plane phase difference data is grouped into a security group. Then, in the security processing, the first security information is obtained from the front embedded data, RAW data, and rear embedded data and placed after the rear embedded data, and the second security information is obtained from the image plane phase difference data and placed after the first security information.

[0044] In this way, a processing range to be set as a security group is set in the imaging system 11. Furthermore, in the imaging system 11, the processing range to be set as a security group can be switched.

[0045] For example, as shown in Fig. 3, in the imaging system 11, all of the embedded data, RAW data, and image plane phase difference data of one frame can be designated as a security group, and security information determined from the security group can be added. In the imaging system 11, the processing range designation unit 46 can switch between designating designated data of one frame as a security group as shown in Fig. 2, and designating the entire data of one frame as a security group as shown in Fig. 3.

[0046] In the example shown in FIG. 2, the embedded data and the RAW data are collectively designated as a security group, but the embedded data and the RAW data may be designated individually as security groups so that security processing is performed on each of them.

[0047] Furthermore, even if the signal processing chip 32 can extract metadata (e.g., information for motion detection, feature points, object position information, automatic exposure, automatic white balance, etc.) from the RAW data, security groups can be similarly set for the metadata as needed.

[0048] Furthermore, the security information may be output immediately after each piece of target data in accordance with bandwidth processing, or may be output all at once during a V blank period after the transmission of the target data.

[0049] <How to identify security groups> An identification method for identifying a security group in the image capture system 11 will be described with reference to FIGS.

[0050] For example, in the imaging system 11, security groups can be identified by a data ID added to each line of transmitted data. For example, in SLVS-EC, the data ID in the packet header can be used, and in MIPI, the data type in the packet header can be used. Also, in systems where data IDs are not standardized, such as SLVS (subLVDS), data indicating line information (Line Info) may be used at the beginning of line data, and this data can be used.

[0051] FIG. 4 is a diagram for explaining a method for identifying a security group when SLVS-EC is adopted as the communication interface used in the imaging system 11. In FIG.

[0052] 4A, the specifications are set so that embedded data and data with a data ID of 2 (RAW data in the illustrated example) are grouped into a security group. Also, in the illustrated example, the specifications are set so that first security information obtained from the embedded data and RAW data is output with a data ID of 3.

[0053] In the second case shown in B of Fig. 4, the specifications are set so that data with the same data ID are grouped into a security group. In the example shown, the embedded data and RAW data, which are data with a data ID of 2, are grouped into a security group, and the specifications are set so that the first security information obtained from the embedded data and RAW data is output with a data ID of 3.

[0054] In this way, in the first and second cases, the receiving host device 22 can recognize data that is part of a security group by checking whether it is embedded data and by checking the data ID.

[0055] In the third case shown in Figure 4C, the specifications are set up so that the target for MAC generation and the security group are notified in the reserved area, as in the past. For example, a bit in the reserved area is used to recognize that a MAC is to be generated. Also, when a new line is output as information indicating the security group, a new ID number can be assigned to output that information. In this case, either the conventional ID area or the reserved area can be used.

[0056] In this way, SLVS-EC allows security groups to be specified using embedded data, data IDs, and reserved areas. Note that which data is to be included in a security group may be determined by the product in which the imaging system 11 is installed, or may be set using a register or the like.

[0057] FIG. 5 is a diagram for explaining a method for identifying a security group when MIPI is adopted as the communication interface used in the imaging system 11. In FIG.

[0058] 5A, the specifications are set so that embedded data whose data type is EBD and data whose data type is RAWxx (RAW data in the illustrated example) are grouped into a security group. Also, in the illustrated example, the specifications are set so that the first security information obtained from the embedded data and RAW data is output as a data type of User Def1.

[0059] In the second case shown in B of Fig. 5, the specifications are set so that data of the same data type are grouped into a security group. In the example shown, the specifications are set so that embedded data and RAW data with a data type of User Def1 are grouped into a security group, and the first security information obtained from the embedded data and RAW data is output with a data type of User Def2.

[0060] In the third case shown in C of Fig. 5, the specification is set so that the target for MAC generation and the security group are notified by the virtual channel ID. Also, in the third case, since the security group is identified by the virtual channel ID, the line that outputs security information can be unified by the data type value (User Def1 in the example shown).

[0061] Note that the identification method described with reference to Figures 4 and 5 is just one example, and other identification methods may be used, such as embedding identification information in the payload data rather than the packet header to identify the security group to which security processing is applied.

[0062] <Example of distance measurement system configuration> Fig. 6 is a block diagram showing an example of the configuration of a ranging system according to a second embodiment to which the present technology is applied. In the ranging system 11A shown in Fig. 6, components common to those in the imaging system 11 of Fig. 1 are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0063] Similar to the imaging system 11, the ranging system 11A is configured by connecting an iTOF device 71 and a host device 22A via a communication interface that employs a data transmission standard such as SLVS-EC, MIPI, or subLVDS.

[0064] The iTOF device 71 has a stacked structure in which an iTOF sensor chip 72 and a signal processing chip 32A are stacked.

[0065] The iTOF sensor chip 72 is based on the iTOF (indirect time of flight) method, which measures the distance to a subject from the phase shift of reflected light of pulsed light output toward the subject, and outputs pulsed light with different phases, for example, 0°, 90°, 180°, and 270°, and acquires the amount of light reflected by the subject from each pulsed light.The iTOF sensor chip 72 then outputs, for each pixel, brightness data indicating the four light amounts acquired for each phase.

[0066] The signal processing chip 32A is configured similarly to the signal processing chip 32 in FIG. 1 in that it includes a security processing unit 43, a link layer interface 44, a physical layer interface 45, a processing range designation unit 46, and a communication unit 47.

[0067] That is, in the signal processing chip 32A, the luminance data for each of the four phases (0°, 90°, 180°, and 270°) output from the iTOF sensor chip 72 is supplied to the security processing unit 43, and security processing is performed on the luminance data. For example, when the processing range designation unit 46 designates a processing range such that luminance data of different phases are grouped into security groups, the security processing unit 43 performs security processing on the luminance data for each phase.

[0068] 7, the front embedded data, luminance data with a phase of 0°, and rear embedded data are encrypted as a security group, and the first security information obtained from the security group is placed after the rear embedded data. Similarly, the front embedded data, luminance data with a phase of 90°, and rear embedded data are encrypted as a security group, and the second security information obtained from the security group is placed after the rear embedded data.

[0069] Furthermore, the front embedded data, luminance data with a phase of 180°, and rear embedded data are encrypted as a security group, and third security information derived from the security group is placed next to the rear embedded data. Finally, the front embedded data, luminance data with a phase of 270°, and rear embedded data are encrypted as a security group, and fourth security information derived from the security group is placed next to the rear embedded data.

[0070] The data that has been subjected to security processing for each security group in the security processing unit 43 is then transmitted to the host device 22A via the link layer interface 44 and the physical layer interface 45.

[0071] 1 in that it is configured with a physical layer interface 51, a link layer interface 52, and a memory 53. In the host device 22A, data transmitted from the iTOF device 71 is temporarily stored in the memory 53 via the physical layer interface 51 and the link layer interface 52.

[0072] Furthermore, the host device 22A is configured to include security processing units 81-1 to 81-4 and a depth map creation processing unit .

[0073] The security processing units 81-1 to 81-4 read the luminance data for each phase stored in the memory 53, perform a decryption process for each security group, and perform a process related to functional safety using the corresponding security information. Then, the security processing units 81-1 to 81-4 each supply the luminance data for each line to the depth map creation processing unit 82.

[0074] For example, the security processing unit 81-1 reads the front embedded data, the luminance data with a phase of 0°, and the rear embedded data from the memory 53, and performs a process of decrypting them as a security group, while also performing a process related to functional safety using the first security information. The security processing unit 81-2 reads the front embedded data, the luminance data with a phase of 90°, and the rear embedded data from the memory 53, and performs a process of decrypting them as a security group, while also performing a process related to functional safety using the second security information.

[0075] Similarly, the security processing unit 81-3 reads the front embedded data, the 180° phase luminance data, and the rear embedded data from the memory 53, and decrypts them as a security group, while also performing processing related to functional safety using the third security information. Furthermore, the security processing unit 81-4 reads the front embedded data, the 270° phase luminance data, and the rear embedded data from the memory 53, and decrypts them as a security group, while also performing processing related to functional safety using the fourth security information.

[0076] The depth map creation processing unit 82 performs a depth map creation process to create a depth map, in which the distance to the subject is mapped line by line, using the luminance data of phase 0° supplied from the security processing unit 81-1, the luminance data of phase 90° supplied from the security processing unit 81-2, the luminance data of phase 180° supplied from the security processing unit 81-3, and the luminance data of phase 270° supplied from the security processing unit 81-4. For example, the depth map creation processing unit 82 creates the first line of the depth map using the same line, i.e., the first line of luminance data of phase 0°, the first line of luminance data of phase 90°, the first line of luminance data of phase 180°, and the first line of luminance data of phase 270°. The depth map creation processing unit 82 then outputs the depth map created by the depth map creation process to a subsequent processing block (not shown).

[0077] The ranging system 11A is configured as described above, and data that has been security processed for each security group in the security processing unit 43 is transmitted from the iTOF device 71 to the host device 22A. Therefore, in the host device 22A, the security processing units 81-1 to 81-4 can read out the luminance data for each phase that has been security processed for each security group from the memory 53, and can supply the luminance data for the same line to the depth map creation processing unit 82. This allows the ranging system 11A to easily handle data in the host device 22A.

[0078] For example, if security processing is performed on the luminance data for each of the four phases collectively, the luminance data will be read from the first line of the entire data, making it difficult to read the same line of each luminance data. In contrast, the distance measuring system 11A can transmit data more appropriately so that it is easy to read the same line of each luminance data.

[0079] In other words, the ranging system 11A is configured so that the iTOF device 71 can output brightness data in sequence from the beginning of a frame, and the host device 22A can read brightness data from the memory 53 in the order required for the depth map creation processing unit 82 to create a depth map.

[0080] Furthermore, for example, when data is output at high speed, such as in the case of the iTOF device 71, the host device 22A may combine multiple pieces of data into a frame, taking into consideration that it cannot receive frames that are too fast, or that it is desired to handle the data as a single block. In this case, the iTOF device 71 can combine luminance data of four phases and output it as one frame, and security processing can be performed in the security processing unit 43, which is a higher layer, without considering, for example, the positions at which the frame start and frame end are added by the link layer interface 44.

[0081] Furthermore, in the distance measuring system 11A, similarly to the imaging system 11, the processing range to be set as a security group can be switched.

[0082] For example, the ranging system 11A can switch between designating the luminance data for each phase of one frame as a security group as shown in Fig. 7 and designating all of the luminance data of one frame as a security group as shown in Fig. 8. In the frame structure shown in Fig. 8, the pre-embedded data of the luminance data with a phase of 0° to the post-embedded data of the luminance data with a phase of 270° is set as a security group, and security information obtained from that security group is placed after the post-embedded data of the luminance data with a phase of 270°.

[0083] <Dual gain and digital overlap frame structures> The frame structures of the dual gain system and the digital overlap system will be described with reference to FIGS.

[0084] FIG. 9 is a diagram illustrating a frame structure in which the present technology is applied to a dual gain system.

[0085] As shown in the figure, in the dual-gain method, different types of data, such as high-gain RAW data and low-gain RAW data, are output on one line. The embedded data and high-gain RAW data are grouped together as a security group, and first security information derived from the security group is placed next to the high-gain RAW data. The embedded data and low-gain RAW data are grouped together as a security group, and first security information derived from the security group is placed next to the low-gain RAW data. In other words, security processing is applied to both the high-gain RAW data and the low-gain RAW data.

[0086] In the frame structure shown in FIG. 9, a frame start and a frame end are arranged in each of the high-gain RAW data and the low-gain RAW data.

[0087] In contrast to this, as shown in FIG. 10, the frame start of the low-gain RAW data and the frame end of the high-gain RAW data may be omitted.

[0088] In this way, when the present technology is applied to the dual-gain method, high-gain RAW data and low-gain RAW data can be set as security groups. Similarly to the case described with reference to FIGS. 4 and 5, high-gain RAW data and low-gain RAW data can be distinguished from each other by using information contained in the packet header, such as a data ID, a data type, and a virtual channel ID. Based on the information contained in the packet header, the host device 22 can determine how to handle the data, for example, select data to be subjected to security processing. Similarly to the third case described with reference to FIGS. 4 and 5, security information can be output by adding the same data ID and a security-related information flag.

[0089] Instead of using the information contained in the packet header, line information (Line Info) may be added to the payload data as shown in Fig. 11. In this case, similar to the first and second cases described with reference to Figs. 4 and 5, it is necessary to set in advance specifications for identifying the security group that is the processing range for security processing.

[0090] FIG. 12 is a diagram illustrating a frame structure when the present technology is applied to the digital overlap method.

[0091] As shown in the figure, in the digital overlap method, different types of data, such as RAW data with a long exposure time and RAW data with a short exposure time, are output in one line. Note that in the digital overlap method, the timing at which the output of RAW data with a short exposure time starts is later than the timing at which the output of RAW data with a long exposure time starts.

[0092] Then, the embedded data and the long exposure time RAW data are grouped into a security group, and the first security information obtained from the security group is placed next to the long exposure time RAW data.Furthermore, the embedded data and the short exposure time RAW data are grouped into a security group, and the first security information obtained from the security group is placed next to the short exposure time RAW data.In other words, security processing is performed on each of the long exposure time RAW data and the short exposure time RAW data.

[0093] In this digital overlap method, similarly to the above-mentioned dual gain method, it is possible to identify security groups.

[0094] <Example of electronic device configuration> The imaging element 21 and iTOF device 71 described above can be applied to various electronic devices, such as imaging systems such as digital still cameras and digital video cameras, mobile phones with imaging functions, or other devices with imaging functions.

[0095] FIG. 13 is a block diagram showing an example of the configuration of an imaging device mounted on an electronic device.

[0096] As shown in FIG. 13, the imaging device 101 includes an optical system 102, an imaging element 103, a signal processing circuit 104, a monitor 105, and a memory 106, and is capable of capturing still images and moving images.

[0097] The optical system 102 is configured to have one or more lenses, and guides image light (incident light) from a subject to the image sensor 103, forming an image on the light receiving surface (sensor section) of the image sensor 103.

[0098] The imaging element 103 may be the imaging element 21 or the iTOF device 71 described above. Electrons are accumulated in the imaging element 103 for a certain period of time in accordance with an image formed on the light receiving surface via the optical system 102. A signal corresponding to the electrons accumulated in the imaging element 103 is then supplied to a signal processing circuit 104.

[0099] The signal processing circuit 104 performs various types of signal processing on the pixel signals output from the image sensor 103. The image (image data) obtained by the signal processing performed by the signal processing circuit 104 is supplied to a monitor 105 to be displayed, or supplied to a memory 106 to be stored (recorded).

[0100] In the imaging device 101 configured in this manner, by applying the imaging element 21 and the iTOF device 71 described above, for example, data can be transmitted more suitably.

[0101] <Examples of using image sensors> FIG. 14 is a diagram showing an example of using the image sensor (imaging element or iTOF device) described above.

[0102] The image sensor described above can be used in various cases for sensing light such as visible light, infrared light, ultraviolet light, and X-rays, for example, as follows.

[0103] ·Digital cameras, mobile devices with camera functions, and other devices that take images for viewing purposes - Devices used for traffic purposes, such as in-vehicle sensors that take pictures of the front, rear, surroundings, and interior of a vehicle for safe driving such as automatic stopping, and for recognizing the driver's condition, surveillance cameras that monitor moving vehicles and roads, and distance measuring sensors that measure distances between vehicles. A device used in home appliances such as TVs, refrigerators, and air conditioners to capture user gestures and operate the appliances according to those gestures. -Medical and healthcare equipment, such as endoscopes and devices that take blood vessel images using infrared light - Security devices such as surveillance cameras for crime prevention and cameras for person authentication Cosmetic devices such as skin measuring devices that take pictures of the skin and microscopes that take pictures of the scalp Sports equipment such as action cameras and wearable cameras for sports purposes Agricultural equipment such as cameras for monitoring the condition of fields and crops

[0104] <Configuration combination example> The present technology can also be configured as follows. (1) a security processing unit that performs security processing on each group of designated data among multiple types of data for one frame obtained from the sensor chip; a transmitting unit that frames data acquired through the security processing by the security processing unit and transmits the framed data to a host device; A data processing device comprising: (2) The group is set in accordance with a data unit read from a memory that temporarily stores data transmitted from the transmission unit in the host device. The data processing device according to (1) above. (3) The security processing unit performs a process of obtaining and adding security information including at least one of a CRC (Cyclic Redundancy Check) and a MAC (Message Authentication Code) from the data for each group. The data processing device according to (1) or (2) above. (4) The security processing unit performs a process of encrypting the data for each group. A data processing device according to any one of (1) to (3) above. (5) the sensor chip is an image sensor chip capable of acquiring image data or image plane phase difference data, The security processing unit performs security processing for each of the groups including the image data and the groups including the image plane phase difference data. A data processing device according to any one of (1) to (4) above. (6) In the host device, the image data is read from a memory that temporarily stores the data transmitted from the transmission unit, and a process for generating an output image is performed, and the image plane phase difference data is read and an autofocus process is performed. The data processing device according to (5) above. (7) the sensor chip is an iTOF sensor chip used to measure a distance to a subject from a phase shift in reflected light of pulsed light output toward the subject, The security processing unit performs security processing for each of the groups including brightness data with different phases output from the iTOF sensor chip. A data processing device according to any one of (1) to (4) above. (8) The host device reads out the luminance data of each phase line by line from a memory that temporarily stores the data transmitted from the transmission unit, and generates a depth map. The data processing device according to (7) above. (9) The security processing unit performs security processing on each of the high-gain image data and the low-gain image data output from the sensor chip. A data processing device according to any one of (1) to (4) above. (10) The security processing unit performs security processing on each of the image data with long exposure time and the image data with short exposure time output from the sensor chip. A data processing device according to any one of (1) to (4) above. (11) a designation unit for designating data for each group to be subjected to security processing by the security processing unit; The data processing device according to any one of (1) to (10) above, further comprising: (12) The designation unit can switch between designating designated data in one frame as the group and designating all data in one frame as the group. The data processing device according to (11) above. (13) an image sensor chip capable of acquiring image data or image plane phase difference data; a security processing unit that performs security processing on each group of designated data among a plurality of types of data for one frame obtained from the image sensor chip; a transmitting unit that frames data acquired through the security processing by the security processing unit and transmits the framed data to a host device; Equipped with The security processing unit performs security processing for each of the groups including the image data and the groups including the image plane phase difference data. Imaging system. (14) an iTOF sensor chip used to measure a distance to a subject from a phase shift in a reflected light of a pulsed light output toward the subject; a security processing unit that performs security processing on each group of designated data among a plurality of types of data for one frame obtained from the iTOF sensor chip; a transmitting unit that frames data acquired through the security processing by the security processing unit and transmits the framed data to a host device; Equipped with The security processing unit performs security processing for each of the groups including brightness data with different phases output from the iTOF sensor chip. Ranging system.

[0105] It should be noted that the present embodiment is not limited to the above-described embodiment, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, the effects described in this specification are merely examples and are not intended to be limiting, and other effects may also be obtained. [Explanation of symbols]

[0106] 11 imaging system, 11A ranging system, 21 imaging element, 22 host device, 23 focus drive unit, 31 image sensor chip, 32 signal processing chip, 41 image data processing unit, 42 image plane phase difference data processing unit, 43 security processing unit, 44 link layer interface, 45 physical layer interface, 46 processing range designation unit, 47 communication unit, 51 physical layer interface, 52 link layer interface, 53 memory, 54 output image processing unit, 55 autofocus processing unit, 61 and 62 security processing unit, 71 iTOF device, 72 iTOF sensor chip, 81-1 to 81-4 security processing unit, 82 depth map creation processing unit

Claims

1. a security processing unit that performs security processing on each group of designated data among a plurality of types of data for one frame obtained from the sensor chip; a transmitting unit that frames data acquired through the security processing by the security processing unit and transmits the framed data to a host device; A data processing device comprising:

2. The group is set in accordance with a data unit read from a memory that temporarily stores data transmitted from the transmission unit in the host device.

2. The data processing device according to claim 1.

3. The security processing unit performs a process of obtaining and adding security information including at least one of a CRC (Cyclic Redundancy Check) and a MAC (Message Authentication Code) from the data for each group.

2. The data processing device according to claim 1.

4. The security processing unit performs a process of encrypting the data for each group.

2. The data processing device according to claim 1.

5. the sensor chip is an image sensor chip capable of acquiring image data or image plane phase difference data, The security processing unit performs security processing for each of the groups including the image data and the groups including the image plane phase difference data.

2. The data processing device according to claim 1.

6. In the host device, the image data is read from a memory that temporarily stores the data transmitted from the transmission unit, and a process for generating an output image is performed, and the image plane phase difference data is read and an autofocus process is performed.

6. A data processing device according to claim 5.

7. the sensor chip is an iTOF sensor chip used to measure a distance to a subject from a phase shift in reflected light of pulsed light output toward the subject, The security processing unit performs security processing for each of the groups including brightness data with different phases output from the iTOF sensor chip.

2. The data processing device according to claim 1.

8. The host device reads out the luminance data of each phase line by line from a memory that temporarily stores the data transmitted from the transmission unit, and generates a depth map.

8. A data processing device according to claim 7.

9. The security processing unit performs security processing on each of the high-gain image data and the low-gain image data output from the sensor chip.

2. The data processing device according to claim 1.

10. The security processing unit performs security processing on each of the image data with long exposure time and the image data with short exposure time output from the sensor chip.

2. The data processing device according to claim 1.

11. a designation unit for designating data for each group to be subjected to security processing by the security processing unit; The data processing apparatus of claim 1 further comprising:

12. The designation unit can switch between designating designated data in one frame as the group and designating all data in one frame as the group.

12. A data processing apparatus according to claim 11.

13. an image sensor chip capable of acquiring image data or image plane phase difference data; a security processing unit that performs security processing on each group of designated data among a plurality of types of data for one frame obtained from the image sensor chip; a transmitting unit that frames data acquired through the security processing by the security processing unit and transmits the framed data to a host device; Equipped with The security processing unit performs security processing for each of the groups including the image data and the groups including the image plane phase difference data. Imaging system.

14. an iTOF sensor chip used to measure a distance to a subject from a phase shift in a reflected light of pulsed light output toward the subject; a security processing unit that performs security processing on each group of designated data among a plurality of types of data for one frame obtained from the iTOF sensor chip; a transmitting unit that frames data acquired through the security processing by the security processing unit and transmits the framed data to a host device; Equipped with The security processing unit performs security processing for each of the groups including brightness data with different phases output from the iTOF sensor chip. Ranging system.

Images