Photoelectric conversion device, photoelectric conversion system

Abstract

To make it difficult for heat generated in a machine learning unit to propagate to a pixel array unit to reduce a deterioration in image quality.SOLUTION: A photoelectric conversion device according to one embodiment comprises: a first substrate that has a pixel array unit in which a plurality of photoelectric conversion units are arranged in a two-dimensional array shape in plan view, and a first wiring pattern; and a second substrate that has a machine learning unit performing processing on signals obtained from electric charges generated in the photoelectric conversion units, and a second wiring pattern, and is laminated on the first substrate. The first wiring pattern of the first substrate and the second wiring pattern of the second substrate are joined to each other to form a metal junction unit. A heat radiation unit including the metal junction unit connected with the machine learning unit is arranged at a position overlapping the machine learning unit. The pixel array unit and the machine learning unit do not overlap each other.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a photoelectric conversion device and a photoelectric conversion system. [Background technology]

[0002] Patent Document 1 discloses a photoelectric conversion device with a stacked structure having a machine learning unit within the photoelectric conversion device for the purpose of performing advanced processing within the photoelectric conversion device. Patent Document 1 discloses that an electromagnetic shield is disposed between a pixel array unit and the machine learning unit, which are disposed on different substrates, to prevent noise generated in the machine learning unit from entering the pixel array unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-072410 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the machine learning unit that executes the machine learning process, heat generation during processing is a concern because a large amount of data is processed at high speed. In Patent Document 1, the pixel array unit, the machine learning unit, and the electromagnetic shield are arranged so as to overlap each other. If each component is arranged as in Patent Document 1, heat generated in the machine learning unit may propagate to the pixel array unit, causing degradation of image quality.

[0005] Therefore, an object of the present invention is to reduce degradation in image quality by making it more difficult for heat generated in the machine learning section to propagate to the pixel array section. [Means for solving the problem]

[0006] A photoelectric conversion device according to one embodiment includes a pixel array section in which a plurality of photoelectric conversion units are arranged in a two-dimensional array in a planar view, a first substrate having a first wiring pattern, a machine learning unit that processes signals obtained from charges generated in the photoelectric conversion units, and a second substrate having a second wiring pattern and laminated on the first substrate, the first wiring pattern of the first substrate being joined to the second wiring pattern of the second substrate to form a metal junction, and the metal junction is connected to the machine learning unit at a position overlapping the machine learning unit. A plurality of the metal joints are provided. a heat dissipation unit is disposed at a position where the pixel array unit and the machine learning unit do not overlap with each other; The machine learning unit has a first circuit that operates at a first operating speed and a second circuit that operates at a second operating speed faster than the first operating speed, the heat dissipation unit includes a first region and a second region that is between the first region and the pixel array unit in a planar view, the first region and the first circuit are in an overlapping position in a planar view, the second region and the second circuit are in an overlapping position in a planar view, and a number of the plurality of metal junctions in the second region is greater than a number of the plurality of metal junctions in the first region. . According to another embodiment, a photoelectric conversion device includes a pixel array unit in which a plurality of photoelectric conversion units are arranged in a two-dimensional array in a plan view, a first substrate having a first wiring pattern, a machine learning unit that processes a signal obtained from the charge generated by the photoelectric conversion unit; and a second substrate that has a second wiring pattern and is laminated on the first substrate; a first wiring pattern of the first substrate and a second wiring pattern of the second substrate are joined to form a metal joint; a heat dissipation unit having a plurality of metal junctions including the metal junction connected to the machine learning unit is disposed at a position overlapping the machine learning unit; the pixel array unit and the machine learning unit are arranged at positions where they do not overlap, the machine learning unit has a first circuit that operates at a first operating speed and a second circuit that operates at a second operating speed that is faster than the first operating speed; the heat dissipation section includes a first region and a second region located between the first region and the pixel array section in a plan view; the first region and the first circuit are positioned to overlap in a plan view, and the second region and the second circuit are positioned to overlap in a plan view, The arrangement density of the plurality of metal junctions in the second region is higher than the arrangement density of the plurality of metal junctions in the first region. . Effect of the Invention

[0007] By making it difficult for heat generated in the machine learning section to propagate to the pixel array section, a photoelectric conversion device capable of reducing degradation in image quality can be provided. [Brief description of the drawings]

[0008] [Figure 1] 1 is a plan view and a perspective view of a photoelectric conversion device according to a first embodiment; [Diagram 2] FIG. 1 is a block diagram showing an overall configuration of a photoelectric conversion device according to a first embodiment. [Diagram 3] 1 is a schematic cross-sectional view of a photoelectric conversion device according to a first embodiment. [Figure 4] 1 is a plan view of a photoelectric conversion device according to a second embodiment. [Diagram 5] Schematic cross-sectional view of a photoelectric conversion device according to a second embodiment. [Figure 6]1 is a plan view of a photoelectric conversion device according to a third embodiment. [Figure 7] 1 is a plan view of a photoelectric conversion device according to a fourth embodiment. [Figure 8] 11 is a plan view of a photoelectric conversion device according to a fifth embodiment. [Figure 9] 1 is a schematic cross-sectional view of a photoelectric conversion device according to a fifth embodiment. [Figure 10] 13 is a plan view of a photoelectric conversion device according to a sixth embodiment. [Figure 11] 13 is a schematic cross-sectional view of a photoelectric conversion device according to a sixth embodiment. [Figure 12] Functional block diagram of a photoelectric conversion system according to a seventh embodiment [Figure 13] Functional block diagram of a distance sensor according to embodiment 8 [Figure 14] Functional block diagram of endoscopic surgery according to embodiment 9 [Figure 15] FIG. 13 is a diagram of a photoelectric conversion system and a moving object according to the tenth embodiment. [Figure 16] Schematic diagram of a photoelectric conversion system according to an eleventh embodiment. [Figure 17] Functional block diagram of a photoelectric conversion system according to a twelfth embodiment DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The following embodiments are intended to embody the technical ideas of the present invention, and are not intended to limit the present invention. The sizes and positional relationships of the components shown in the drawings may be exaggerated to clarify the description. In the following description, the same components may be designated by the same reference numerals, and the description may be omitted.

[0010] In this specification, "planar view" refers to a view from a direction perpendicular to the light incident surface of the first substrate 2 described below. Additionally, a cross section refers to a surface in a direction perpendicular to the light incident surface of the first substrate 2. Note that, when the light incident surface of the first substrate 2 is a rough surface when viewed microscopically, the planar view is defined based on the light incident surface of the first substrate 2 when viewed macroscopically.

[0011] In this specification, the depth direction is the direction from the light incident surface (first surface) of first substrate 2 toward the surface (second surface) on which circuit board 21 is disposed.

[0012] (Embodiment 1) A first embodiment will be described. FIG. 1 shows a photoelectric conversion device of the first embodiment, FIG. 1(c) is a perspective view of the photoelectric conversion device, and FIG. 1(a) is a plan view of the first substrate 2 when the photoelectric conversion device is viewed from the light incident side. FIG. 1(b) is a plan view of the second substrate 5 when the photoelectric conversion device of FIG. 1(c) is viewed from the light incident side. The light incident side of the second substrate 5 is a view from the surface side of the second substrate 5 that is bonded to the first substrate 2. As shown in FIG. 1(c), the photoelectric conversion device of this embodiment has a structure in which the first substrate 2 and the second substrate 5 are laminated and bonded. A wiring structure is disposed between the first substrate 2 and the second substrate 5, and this wiring structure includes a plurality of wiring layers. In FIG. 1, the configuration disposed on the first substrate 2 is given the subscript A, and the configuration disposed on the second substrate 5 is given the subscript B, and A and B are arranged so as to overlap when the first substrate 2 and the second substrate 5 are bonded together. For example, the pixel array section 1A and the pixel circuit section 1B overlap in a plan view and are electrically connected via a wiring layer.

[0013] As shown in Figure 1(a), the first substrate 2 is provided with a pixel array section 1A in which a plurality of photoelectric conversion units are arranged in a two-dimensional array in a planar view, a heat dissipation section 3 arranged around the pixel array section 1A in a planar view, and pads 4 arranged on the outer periphery of the first substrate 2.

[0014] 1(b), the second substrate 5 is provided with a pixel circuit unit 1B, a readout circuit unit 6, a machine learning unit 7, and an output interface unit 8. The machine learning unit 7 is disposed at a position overlapping with the heat dissipation unit 3 disposed on the first substrate 2 in a plan view.

[0015] The machine learning unit 7 processes signals obtained from charges generated in the photoelectric conversion unit. The machine learning unit 7 functions as a processing unit that executes various processes using a trained model created by machine learning using, for example, a deep neural network (DNN). This trained model (neural network calculation model) may be designed based on parameters generated by inputting an input signal corresponding to the output of the pixel array unit 1A and training data associated with a label for the input signal into a predetermined machine learning model. The predetermined machine learning model may also be a training model using a multi-layer neural network (also called a multi-layer neural network model).

[0016] The machine learning unit 7 executes a process of multiplying the dictionary coefficients of the learned model by the image data obtained from the pixel array unit 1A by executing a calculation process based on the learned model included in the machine learning unit 7. The result obtained by such a calculation process (calculation result) can be output to the output interface unit 8. The calculation result may include image data obtained by executing the calculation process using the learned model and various information (metadata) obtained from the image data.

[0017] The machine learning unit 7 can learn the learning model by changing the weighting of various parameters in the learning model using the learning data. Also, the machine learning unit 7 can prepare a plurality of learning models and change the learning model to be used depending on the contents of the calculation process. Furthermore, the machine learning unit 7 can obtain a learned learning model from an external device. In this way, the machine learning unit 7 can execute the above calculation process.

[0018] The pads 4 include an input pad IP for inputting a power supply voltage, a signal of a pixel circuit transistor, etc. from an external circuit, and an output pad OP for outputting a signal to an external circuit. The pads 4 may be any of pad electrodes electrically connected to an external circuit and arranged on a wiring layer, pad electrodes connected to a through electrode penetrating from one surface of a semiconductor substrate to the other surface facing the semiconductor substrate, and pad electrodes arranged on a semiconductor layer of each substrate. As shown in FIG. 1(a) and FIG. 1(c), the pads 4 are arranged along the four sides constituting the outer periphery of the first substrate in FIG. 1, but the arrangement of the pads 4 is not limited to this configuration. For example, the pads 4 may be arranged along two opposing sides, or along one of the four sides.

[0019] In Fig. 1, the output pads OP and the input pads IP are arranged alternately, but this is not limited to the above. As will be described in detail later, it is preferable to arrange the pads 4 near the machine learning unit 7 because heat generated in the machine learning unit 7 can be dissipated via the output pads OP or the input pads IP. By arranging the output pads OP and the input pads IP alternately, the heat dissipation unit 3 can be provided around the entire periphery of the pixel array unit 1A regardless of whether heat is dissipated from the output pads OP or the input pads IP, and the area of ​​the heat dissipation unit 3 can be increased. Therefore, uneven heat dissipation can be reduced.

[0020] The output pads OP and input pads IP do not need to be alternated, and may be arranged at a predetermined interval. In addition, the pads 4 closer to the machine learning unit 7 may be output pads OP, and the pads 4 farther from the machine learning unit 7 may be input pads IP.

[0021] The number of output pads OP and the number of input pads IP may be the same or different. The number of output pads OP and the number of input pads IP can be compared, for example, with a plurality of pads 4 arranged along the area where the machine learning unit 7 is arranged. Even if the number of input pads IP is greater than the number of output pads OP, a certain amount of heat dissipation effect can be obtained.

[0022] FIG. 2 is a diagram showing the overall configuration of the photoelectric conversion device in the first embodiment. As shown in the figure, the photoelectric conversion device is composed of a pixel array unit 1, a readout circuit unit 6, a machine learning unit 7, and an output interface unit 8. The pixel array unit 1 includes a plurality of light receiving pixels 9 in the horizontal and vertical directions. The readout circuit unit 6 includes a vertical scanning circuit 12 and a readout circuit 13. The vertical scanning circuit 12 supplies a control pulse for controlling the driving of a transistor included in the pixel to a pixel driving signal line 10. The light receiving pixels 9 of the pixel array unit 1 are operated by the control pulse supplied from the vertical scanning circuit 12. Each light receiving pixel 9 includes a photoelectric conversion unit such as a photodiode. The light receiving pixel 9 converts incident light from the outside into an electrical signal by photoelectric conversion, and outputs a pixel signal according to the amount of incident light based on the obtained electrical signal. A common pixel driving signal line 10 is connected to the light receiving pixels 9 in one row. In addition, a common vertical output line 11 is connected to the light receiving pixels 9 in one column. The pixel signals of the light-receiving pixels 9 output to the vertical output lines 11 of each column are input to a readout circuit 13 arranged in each column.

[0023] The readout circuit 13 amplifies the pixel signals, converts them into analog signals, and outputs data that is output from the readout circuit 13. The machine learning unit 7 executes the above-mentioned processing of the machine learning unit 7 on the output data of the readout circuit 13. The data after the machine learning processing is output via the output interface unit 8 and the output pad.

[0024] Fig. 3 is a schematic cross-sectional view taken along line XX' in Fig. 1. It shows the pixel array section 1A, heat dissipation section 3, and pads 4 of the first substrate 2, and the structure of the corresponding second substrate 5. The first substrate 2 and the second substrate 5 each have a structure in which a semiconductor layer 301 and a semiconductor layer 315 are stacked with a plurality of wiring layers sandwiched between insulating films.

[0025] A photoelectric conversion section is disposed in the semiconductor layer 301 disposed on the light incident surface side of the first substrate 2. In addition, a microlens 307 for condensing light is disposed on the light incident surface side of the pixel array section 1A of the first substrate 2. The semiconductor layer 301 has a transistor region 308 in which a source and a drain of a transistor are disposed. The interlayer insulating film 302 includes a gate electrode 310 of the transistor, a wiring layer 312, and a contact plug 311 connecting the transistor region 308 and the wiring layer 312. A plurality of interlayer insulating films 303, 304, and 305 are laminated between the interlayer insulating film 302 and a bonding surface 306 of the first substrate 2 and the second substrate 5.

[0026] The interlayer insulating film 303 includes a via plug 313 that connects the wiring layer 314 and the wiring layer 312, and the interlayer insulating film 304 has a similar configuration. In addition to the wiring layer and the via plug, the interlayer insulating film 305 has a wiring pattern 321 that configures a metal junction MB. The metal junction MB is connected to the wiring patterns of the adjacent wiring layers, and can dissipate heat generated in the machine learning unit 7. In FIG. 3, the wiring pattern 321 and the wiring pattern on the upper layer are connected via a via plug, but the wiring patterns may be connected to each other without via a via plug. The metal junction MB can be configured by joining the wiring pattern 321 formed in the same layer as the wiring layer included in the interlayer insulating film 305 and the wiring pattern 322 formed in the same layer as the wiring layer included in the interlayer insulating film 319.

[0027] The second substrate 5 has a layered structure similar to that of the first substrate 2. The semiconductor layer 315 includes a transistor region 320 including a source and a drain. The transistors included in the transistor region 320 are transistors that configure the machine learning unit 7.

[0028] The interlayer insulating film 316 includes a gate, a wiring layer, and a contact plug, similar to the interlayer insulating film 302. A plurality of interlayer insulating films 317, 318, and 319 are laminated between the interlayer insulating film 316 and the bonding surface 306. The interlayer insulating films 317 and 318 include a wiring layer and a via plug, similar to the interlayer insulating films 303 and 304, and the interlayer insulating film 319 has a wiring pattern 322 that constitutes a wiring layer, a via plug, and a metal junction MB, similar to the interlayer insulating film 305. The formation layer and configuration of the wiring pattern 322 are similar to those of the wiring pattern 321.

[0029] It is preferable that multiple metal junctions MB are connected to the same wiring pattern, as shown in Fig. 3. This makes it possible to secure the area of ​​the wiring pattern to which the multiple metal junctions MB are connected, thereby increasing the number of heat dissipation paths.

[0030] It is preferable that the wiring pattern to which the multiple metal junctions MB are connected constitutes an electrode of the pad 4. In other words, it is preferable that the wiring pattern to which the multiple metal junctions MB are connected extends to the opening of the pad 4, and that the wiring pattern and an external circuit are connected via a bonding wire or the like.

[0031] At a position overlapping with the opening of pad 4 in plan view, it is preferable that the wiring pattern constituting the electrode of pad 4 and the wiring pattern included in interlayer insulating film 318 are connected via metal junction MB2. This makes it possible to increase the volume of wiring such as the wiring pattern connected to metal junction MB1 connected to machine learning unit 7, thereby making it possible to increase the heat capacity.

[0032] As shown in FIG. 3, in this embodiment, a heat dissipation unit 3 including a metal junction MB connected to a wiring pattern arranged on an adjacent interlayer insulating film is arranged at a position overlapping the machine learning unit 7. The pixel array unit 1A and the machine learning unit 7 are arranged at positions where they do not overlap in a planar view. When the first substrate 2 and the second substrate 5 are joined via the metal junction MB, it is assumed that the machine learning unit 7 and the pixel array unit 1A are arranged so as to overlap in a planar view. In this case, heat generated in the machine learning unit 7 is transferred to the pixel array unit 1A, and noise such as dark current noise occurs in the imaging pixel, which may deteriorate the image quality. On the other hand, according to this embodiment, the pixel array unit 1A and the machine learning unit 7 are arranged at positions where they do not overlap in a planar view, so that the heat generated in the machine learning unit 7 is less likely to propagate to the pixel array unit 1A. Therefore, deterioration of image quality can be suppressed.

[0033] In the first substrate 2, the wiring pattern 314-1 in the heat dissipation section 3 is preferably also connected to the semiconductor layer 301 through a via plug or a wiring layer. This can increase the number of heat dissipation paths for the heat generated in the machine learning section 7. In the heat dissipation section 3, it is preferable that a plurality of metal junctions MB1 are connected to the wiring pattern 314-1. On the other hand, if the heat generated in the machine learning section 7 is dissipated near the photoelectric conversion section, this may lead to a decrease in image quality. Therefore, when connecting to the semiconductor layer 301, it is preferable to connect to the semiconductor layer at a position closer to the pad 4 than the pixel array section 1A. Also, in the heat dissipation section 3, it is preferable that the number of contact plugs connected to the semiconductor layer 301 is smaller than the number of metal junctions MB1.

[0034] A transistor region 309 may be disposed in the semiconductor layer 301, and the transistor region 309 may be connected to the wiring pattern 314-1. The transistor region 309 may have the same configuration as the transistor disposed in the pixel array section 1A. As shown in FIG. 3, a capacitance section is disposed in the transistor region 309 of the heat dissipation section 3. The capacitance section is, for example, a bypass capacitor. The bypass capacitor includes a substrate bias section, a semiconductor region, and a gate electrode. FIG. 3 shows the gate electrode and the semiconductor region of the bypass capacitor. The capacitance section may be a coupling capacitance or the like provided between signal lines.

[0035] As described above, according to this embodiment, it is possible to make it difficult for heat generated in the machine learning section 7 to propagate to the pixel array section, thereby making it possible to reduce degradation in image quality.

[0036] (Embodiment 2) A photoelectric conversion device according to the second embodiment will be described with reference to Figs. 9 and 10. Fig. 4(a) is a schematic plan view of the first substrate 2 as viewed from the light incident surface side, and Fig. 4(b) is a schematic plan view of the second substrate 5 as viewed from the light incident surface side. The photoelectric conversion device according to this embodiment differs from the first embodiment in the arrangement of the heat dissipation section 3 and the output interface section 8. This embodiment also differs from the first embodiment in that the wiring pattern 314-1 is not connected to the semiconductor layer 301. Other than these points and the matters described below, this embodiment is substantially the same as the first embodiment, and therefore the same reference numerals are used for the same configurations as the first embodiment, and the description thereof may be omitted.

[0037] As shown in Fig. 4(a), the pixel array section 1A of the first substrate 2 has a plurality of pixels arranged in a matrix so as to have a longitudinal direction and a lateral direction. In this embodiment, the heat dissipation section 3 is arranged along the longitudinal side (long side) that constitutes the outer edge of the pixel array section 1A. In Fig. 4, the heat dissipation section 3 is arranged along the two long sides of the pixel array section 1A.

[0038] In this embodiment, the output interface unit 8 is disposed on the first substrate 2. In this case, the output interface unit 8 is not disposed on the second substrate 5. As shown in Fig. 4, the output interface unit 8 may be disposed on the first substrate 2 due to restrictions on the layout area of ​​the readout circuit unit 6 and the machine learning unit 7. The output interface unit 8 may be disposed at each of the four corners of the sides constituting the outer edge of the chip on the first substrate 2.

[0039] 5 is a schematic cross-sectional view taken along line X-X' in FIG. 4. The pixel array section 1A, heat dissipation section 3, pads 4, and the structure of the second substrate 5 corresponding to these are shown. As described above, in the heat dissipation section 3 of the first embodiment, the wiring pattern 314-1 connected to the metal junction MB1 connected to the machine learning section 7 is not connected to the semiconductor layer 301. In other words, the metal junction MB1 is connected only to the wiring patterns and via plugs of each wiring layer. The wiring patterns may form a MIM (Metal Insulator Metal) capacitance.

[0040] As in the first embodiment, this embodiment also makes it possible to prevent the heat generated in the machine learning section 7 from being easily transmitted to the pixel array section, thereby reducing degradation in image quality.

[0041] (Embodiment 3) A photoelectric conversion device according to embodiment 3 will be described with reference to Fig. 6. Fig. 6(a) is a schematic plan view of the first substrate 2 as viewed from the light incident surface side, and Fig. 6(b) is a schematic plan view of the second substrate 5 as viewed from the light incident surface side. The photoelectric conversion device according to this embodiment differs from embodiment 2 in the arrangement of the heat dissipation unit 3, readout circuit unit 6, and machine learning unit 7. Other than these points and the matters described below, the photoelectric conversion device according to this embodiment is substantially similar to embodiment 2. Therefore, the same reference numerals are used for the same configurations as embodiment 2, and the description thereof may be omitted.

[0042] As shown in Fig. 6(a), the heat dissipation section 3 is disposed along each of the four sides of the pixel array section 1A of the first substrate 2. Also, as shown in Fig. 6(b), the readout circuit section 6 and the machine learning section 7 are disposed on the second substrate 5 along the four sides of the pixel circuit section 1B. The heat dissipation section 3 disposed on the first substrate 2 is disposed at a position overlapping the machine learning section 7 in a planar manner. The cross-sectional structure from when heat generated in the machine learning section 7 is dissipated by the pad 4 via the heat dissipation section 3 is the same as in the second embodiment, and therefore will not be described.

[0043] Moreover, by arranging the heat dissipation section 3 along the four sides of the pixel array section 1A, it is possible to increase the number of heat dissipation paths.

[0044] (Embodiment 4) A photoelectric conversion device according to embodiment 4 will be described with reference to FIG. 7. FIG. 7(a) is a schematic plan view of the first substrate 2 as viewed from the light incident surface side, and FIG. 7(b) is a schematic plan view of the second substrate 5 as viewed from the light incident surface side. The photoelectric conversion device according to this embodiment differs from embodiment 1 in the arrangement of the heat dissipation unit 3, readout circuit unit 6, and machine learning unit 7 arranged on the first substrate 2 and the second substrate 5. Other than this point and the matters described below, the photoelectric conversion device according to this embodiment is substantially similar to embodiment 1. Therefore, the same reference numerals are used for the same configurations as embodiment 1, and the description thereof may be omitted.

[0045] 7(a), on the first substrate 2, the heat dissipation section 3 is arranged along two short sides of the pixel array section 1A, and the readout circuit section 6 is arranged along two long sides of the pixel array section 1A. The readout circuit section 6 is also arranged separately on the first substrate 2 and the second substrate 5. For example, the AD conversion section of the readout circuit section 6 can be arranged on the second substrate 5, and the output section can be arranged on the first substrate 2.

[0046] 7(b), the heat dissipation unit 3 is disposed at a position that overlaps in plan view with the machine learning unit 7 disposed on the second substrate 5. The cross-sectional structure from when heat generated by the machine learning unit 7 is dissipated by the pad 4 via the heat dissipation unit 3 is the same as in the first embodiment, and therefore will not be described.

[0047] In the present embodiment, as in the second embodiment, it is possible to make it difficult for heat generated in the machine learning section 7 to propagate to the pixel array section, thereby making it possible to reduce degradation in image quality.

[0048] (Embodiment 5) A photoelectric conversion device according to embodiment 5 will be described with reference to Figs. 8 and 9. Fig. 8 is a schematic plan view of the first substrate 2 as viewed from the light incident surface side. Fig. 9 is a schematic cross-sectional view taken along line X-X' in Fig. 8. This embodiment differs from embodiment 1 in the density of the heat dissipation structure included in the heat dissipation section 3 disposed on the first substrate 2. Figs. 8 and 9 show an example in which the density of the metal junction MB1 is different. Other than this point and the matters described below, this embodiment is substantially the same as embodiment 1, and therefore the same reference numerals may be used to designate the same configuration as embodiment 1, and the description thereof may be omitted.

[0049] Depending on the configuration of the machine learning unit 7, due to the constraints of the wiring density, it may not be possible to uniformly arrange the heat dissipation structures such as the metal junctions MB1. In this case, it is preferable to increase the heat dissipation of the heat dissipation structure at a location close to the heat source in the machine learning unit 7. In this embodiment, in the heat dissipation unit 3, the heat dissipation in the heat dissipation unit is changed between the first region 3b and the second region 3a located between the first region 3b and the pixel array unit 1A. For example, in the machine learning unit 7, when the region close to the pixel array unit 1A generates more heat than the region farther away, the number of the metal junctions MB1 in the second region 3b is made greater than the number of the metal junctions MB1 in the first region 3a, and the heat dissipation in the second region 3b is made higher than the heat dissipation in the first region 3a. In this way, the heat dissipation characteristics of the heat dissipation unit 3 may be changed according to the amount of heat generated by the machine learning unit 7.

[0050] Furthermore, in order to ensure stable operation of the circuit, it is better for the temperature of the board to be uniform. Therefore, the heat dissipation characteristics of the heat dissipation structure may be changed so that the temperature distribution is uniform between the area overlapping the heat dissipation section 3 and the area not overlapping the heat dissipation section 3. For example, the number of metal joints MB1 in the area where the amount of heat generated is large may be greater than the number of metal joints MB1 in the area where the amount of heat generated is small.

[0051] In the present embodiment, similarly to the first embodiment, it is possible to make it difficult for heat generated in the machine learning unit 7 to propagate to the pixel array unit, thereby reducing degradation in image quality. Furthermore, according to the present embodiment, it is possible to change the heat dissipation performance according to the amount of heat generated by the machine learning unit 7, thereby reducing unevenness in the temperature distribution on the second substrate 5.

[0052] (Embodiment 6) A photoelectric conversion device according to the sixth embodiment will be described with reference to Figs. 10 and 11. Fig. 10(a) is a schematic plan view of the first substrate 2 as viewed from the light incident surface side, and Fig. 10(b) is a schematic plan view of the second substrate 5 as viewed from the light incident surface side. Fig. 11 is a schematic cross-sectional view taken along the line X-X' in Fig. 10. This embodiment differs from the first embodiment in that a plurality of heat dissipation units 3a and 3b having different structures are arranged, and a plurality of machine learning units 7a and 7b having different operating speeds are arranged on the second substrate 5. Other than this point and the matters described below, the present embodiment is substantially the same as the first embodiment, and therefore the same reference numerals may be used to designate the same configuration as the first embodiment, and the description thereof may be omitted.

[0053] As shown in Fig. 10(a), the first substrate 2 has heat dissipation units 3a and 3b arranged in positions surrounding the pixel array unit 1A. As shown in Fig. 10(b), the machine learning unit 7 of the second substrate 5 includes a first circuit 7b having a first operating speed and a second circuit 7a having an operating speed faster than the first circuit 7b. On the second substrate 5, the first circuit 7b of the machine learning unit 7 is arranged in a position overlapping with the first region 3b of the heat dissipation unit 3, and the second circuit 7a is arranged in a position overlapping with the second region 3a of the heat dissipation unit 3.

[0054] The first region 3b has a higher heat dissipation property of the heat dissipation structure than the second region 3a. As an example, in Fig. 11, the number of metal bonding parts MB1 in the first region 3b is made larger than the number of metal bonding parts MB1 in the second region 3a. In other words, the density of the metal bonding parts MB1 in the first region 3b is made higher than the density of the metal bonding parts MB1 in the second region 3a.

[0055] Since the first circuit 7b of the machine learning unit 7 operates at a higher speed than the second circuit 7a, it is expected that the effect of heat generation will be greater than that of the second circuit 7a. Therefore, the first circuit 7b is arranged in a position away from the area where the second circuit 7a is arranged with respect to the pixel array unit 1A, and the heat dissipation effect is improved by increasing the arrangement density of the heat dissipation structure arranged in the heat dissipation unit 3b that overlaps with the first circuit 7b. This makes it easier to suppress the heat generated in the first circuit 7b and second circuit 7a of the machine learning unit 7 from propagating to the pixel array unit 1A.

[0056] In this embodiment, as in embodiment 1, it is possible to make it difficult for heat generated in the machine learning unit 7 to propagate to the pixel array unit, thereby reducing image quality degradation. Furthermore, according to this embodiment, it is possible to change the heat dissipation properties according to the amount of heat generated by the machine learning unit 7, thereby making it possible to obtain a more significant effect of reducing image quality degradation.

[0057] (Embodiment 7) A photoelectric conversion system 11200 according to the seventh embodiment will be described with reference to FIG. 12. FIG. 12 is a block diagram showing the configuration of the photoelectric conversion system 11200 of this embodiment. The photoelectric conversion system 11200 of this embodiment includes a photoelectric conversion device 11204. Here, any of the photoelectric conversion devices described in the above-mentioned embodiments can be applied to the photoelectric conversion device 11204. The photoelectric conversion system 11200 can be used as, for example, an imaging system. Specific examples of the imaging system include a digital still camera, a digital camcorder, a surveillance camera, and a network camera. FIG. 12 shows an example of a digital still camera as the photoelectric conversion system 11200.

[0058] 12 includes a photoelectric conversion device 11204 and a lens 11202 that forms an optical image of a subject on the photoelectric conversion device 11204. The photoelectric conversion system 11200 also includes an aperture 11203 that varies the amount of light passing through the lens 11202, and a barrier 11201 that protects the lens 11202. The lens 11202 and the aperture 11203 form an optical system that focuses light on the photoelectric conversion device 11204.

[0059] The photoelectric conversion system 11200 has a signal processing unit 11205 that processes an output signal output from the photoelectric conversion device 11204. The signal processing unit 11205 performs signal processing operations such as performing various corrections and compression on an input signal as necessary and outputting the signal. The photoelectric conversion system 11200 further has a buffer memory unit 11206 for temporarily storing image data, and an external interface unit (external I / F unit) 11209 for communicating with an external computer or the like. The photoelectric conversion system 11200 further has a recording medium 11211 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 11210 for recording or reading out data on the recording medium 11211. The recording medium 11211 may be built in the photoelectric conversion system 11200 or may be removable. In addition, communication between the recording medium control I / F unit 11210 and the recording medium 11211 and communication from the external I / F unit 11209 may be performed wirelessly.

[0060] Furthermore, the photoelectric conversion system 11200 has an overall control / calculation unit 11208 that performs various calculations and controls the entire digital still camera, and a timing generation unit 11207 that outputs various timing signals to the photoelectric conversion device 11204 and the signal processing unit 11205. Here, timing signals and the like may be input from the outside, and the photoelectric conversion system 11200 only needs to have at least the photoelectric conversion device 11204 and the signal processing unit 11205 that processes the output signal output from the photoelectric conversion device 11204. The overall control / calculation unit 11208 and the timing generation unit 11207 may be configured to implement some or all of the control functions of the photoelectric conversion device 11204.

[0061] The photoelectric conversion device 11204 outputs the image signal to the signal processing unit 11205. The signal processing unit 11205 performs a predetermined signal processing on the image signal output from the photoelectric conversion device 11204 and outputs image data. The signal processing unit 11205 generates an image using the image signal. The signal processing unit 11205 may perform distance measurement calculation on the signal output from the photoelectric conversion device 11204. The signal processing unit 11205 and the timing generating unit 11207 may be mounted on the photoelectric conversion device. That is, the signal processing unit 11205 and the timing generating unit 11207 may be provided on a substrate on which pixels are arranged, or may be provided on a different substrate. By configuring an imaging system using the photoelectric conversion device of each of the above-mentioned embodiments, an imaging system capable of acquiring a better quality image can be realized.

[0062] (Embodiment 8) Fig. 13 is a block diagram showing a configuration example of a photoelectric conversion system using the photoelectric conversion device described in any one of Embodiments 1 to 6. Fig. 13 shows a configuration example of a range image sensor, which is an electronic device, as an example of the photoelectric conversion system.

[0063] 13, the distance image sensor 12401 is configured to include an optical system 12407, a photoelectric conversion device 12408, an image processing circuit 12404, a monitor 12405, and a memory 12406. The distance image sensor 12401 can obtain a distance image according to the distance to the subject by receiving light (modulated light or pulsed light) that is projected from a light source device 12409 toward the subject and reflected by the surface of the subject.

[0064] The optical system 12407 is configured to have one or more lenses, and guides image light (incident light) from a subject to a photoelectric conversion device 12408 , forming an image on the light receiving surface (sensor portion) of the photoelectric conversion device 12408 .

[0065] As the photoelectric conversion device 12408 , the photoelectric conversion device of each of the above-mentioned embodiments is applied, and a distance signal indicating a distance determined from a light receiving signal output from the photoelectric conversion device 12408 is supplied to the image processing circuit 12404 .

[0066] The image processing circuit 12404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 12408. The distance image (image data) obtained by this image processing is then supplied to a monitor 12405 for display, or supplied to a memory 406 for storage (recording).

[0067] In the range image sensor 12401 configured in this way, by applying the above-mentioned photoelectric conversion device, it is possible to obtain, for example, a more accurate range image as the pixel characteristics improve.

[0068] (Embodiment 9) The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgery system.

[0069] FIG. 14 is a diagram showing an example of a schematic configuration of an endoscopic surgery system to which the technology according to the present disclosure (the present technology) can be applied.

[0070] 14 shows a state in which an operator (doctor) 13131 is performing surgery on a patient 13132 on a patient bed 13133 using an endoscopic surgery system 13003. As shown in the figure, the endoscopic surgery system 13003 is composed of an endoscope 13100, a surgical tool 13110, and a cart 13134 on which various devices for endoscopic surgery are mounted.

[0071] The endoscope 13100 is composed of a lens barrel 13101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 13132, and a camera head 13102 connected to the base end of the lens barrel 13101. In the illustrated example, the endoscope 13100 is configured as a so-called rigid scope having a rigid lens barrel 13101, but the endoscope 13100 may be configured as a so-called flexible scope having a flexible lens barrel.

[0072] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 13101. A light source device 13203 is connected to the endoscope 13100, and light generated by the light source device 13203 is guided to the tip of the lens barrel by a light guide extending inside the lens barrel 13101. This light is irradiated via the objective lens toward an observation target in the body cavity of the patient 13132. The endoscope 13100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0073] An optical system and a photoelectric conversion device are provided inside the camera head 13102, and reflected light (observation light) from an observation target is collected by the optical system onto the photoelectric conversion device. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. The photoelectric conversion device described in each of the above-mentioned embodiments can be used as the photoelectric conversion device. The image signal is transmitted to a camera control unit (CCU) 13135 as RAW data.

[0074] The CCU 13135 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 13100 and the display device 13136. Furthermore, the CCU 13135 receives an image signal from the camera head 13102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.

[0075] Under the control of the CCU 13135, the display device 13136 displays an image based on an image signal that has been subjected to image processing by the CCU 13135.

[0076] The light source device 13203 is composed of a light source such as an LED (Light Emitting Diode), and supplies the endoscope 13100 with irradiation light when photographing an operation site or the like.

[0077] The input device 13137 is an input interface for the endoscopic surgery system 13003. A user can input various information and instructions to the endoscopic surgery system 13003 via the input device 13137.

[0078] The treatment tool control device 13138 controls the driving of the energy treatment tool 13112 for cauterizing tissue, incising, sealing blood vessels, or the like.

[0079] The light source device 13203 that supplies irradiation light to the endoscope 13100 when photographing the surgical site can be composed of a white light source composed of, for example, an LED, a laser light source, or a combination of these. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 13203. In this case, it is also possible to capture images corresponding to each of the RGB colors in a time-division manner by irradiating the observation target with laser light from each of the RGB laser light sources in a time-division manner and controlling the driving of the image sensor of the camera head 13102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0080] The light source device 13203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the driving of the image sensor of the camera head 13102 in synchronization with the timing of the change in the light intensity to acquire images in a time-division manner and synthesizing the images, it is possible to generate an image with a high dynamic range that is free of so-called blackout and whiteout.

[0081] The light source device 13203 may be configured to supply light in a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, the wavelength dependency of light absorption in body tissue is utilized. Specifically, a predetermined tissue such as blood vessels on the mucous membrane surface is photographed with high contrast by irradiating light in a narrower band than the irradiation light (i.e., white light) in normal observation. Alternatively, the special light observation may be a fluorescent observation in which an image is obtained by fluorescence generated by irradiating excitation light. In the fluorescent observation, it is possible to irradiate excitation light to the body tissue and observe the fluorescence from the body tissue, or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 13203 may be configured to supply narrow band light and / or excitation light corresponding to such special light observation.

[0082] (Embodiment 10) The photoelectric conversion system and the moving body of this embodiment will be described with reference to Fig. 15. Fig. 15 is a schematic diagram showing a configuration example of the photoelectric conversion system and the moving body according to this embodiment. In this embodiment, an example of an in-vehicle camera is shown as the photoelectric conversion system.

[0083] FIG. 15 shows an example of a vehicle system and a photoelectric conversion system mounted thereon for capturing images. The photoelectric conversion system 14301 includes a photoelectric conversion device 14302, an image preprocessing unit 14315, an integrated circuit 14303, and an optical system 14314. The optical system 14314 forms an optical image of a subject on the photoelectric conversion device 14302. The photoelectric conversion device 14302 converts the optical image of the subject formed by the optical system 14314 into an electric signal. The photoelectric conversion device 14302 is any of the photoelectric conversion devices according to the above-mentioned embodiments. The image preprocessing unit 14315 performs a predetermined signal processing on the signal output from the photoelectric conversion device 14302. The function of the image preprocessing unit 14315 may be incorporated in the photoelectric conversion device 14302. The photoelectric conversion system 14301 is provided with at least two sets of an optical system 14314 , a photoelectric conversion device 14302 , and an image pre-processing unit 14315 , and the output from each set of the image pre-processing unit 14315 is input to the integrated circuit 14303 .

[0084] The integrated circuit 14303 is an integrated circuit for use in an imaging system, and includes an image processing unit 14304 including a memory 14305, an optical distance measuring unit 14306, a distance measuring calculation unit 14307, an object recognition unit 14308, and an abnormality detection unit 14309. The image processing unit 14304 performs image processing such as development processing and defect correction on the output signal of the image pre-processing unit 14315. The memory 14305 stores the primary storage of the captured image and the defective positions of the captured pixels. The optical distance measuring unit 14306 performs focusing and distance measurement of the subject. The distance measuring calculation unit 14307 calculates distance measurement information from multiple image data acquired by multiple photoelectric conversion devices 14302. The object recognition unit 14308 recognizes subjects such as cars, roads, signs, and people. When the abnormality detection unit 14309 detects an abnormality in the photoelectric conversion device 14302, it issues an abnormality report to the main control unit 14313.

[0085] The integrated circuit 14303 may be realized by dedicated hardware, a software module, or a combination of these. It may also be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or a combination of these.

[0086] The main control unit 14313 supervises and controls the operations of the photoelectric conversion system 14301, the vehicle sensor 14310, the control unit 14320, etc. It is also possible to adopt a method in which the main control unit 14313 is not provided, and the photoelectric conversion system 14301, the vehicle sensor 14310, and the control unit 14320 each have a communication interface and each transmits and receives a control signal via a communication network (for example, CAN standard).

[0087] The integrated circuit 14303 has a function of receiving a control signal from the main control unit 14313 or transmitting a control signal or a set value to the photoelectric conversion device 14302 by its own control unit.

[0088] The photoelectric conversion system 14301 is connected to the vehicle sensor 14310, and can detect the vehicle's driving state such as vehicle speed, yaw rate, and steering angle, as well as the state of the environment outside the vehicle and other vehicles and obstacles. The vehicle sensor 14310 is also a distance information acquisition means for acquiring distance information to an object. The photoelectric conversion system 14301 is also connected to the driving assistance control unit 1311 that performs various driving assistance such as automatic steering, automatic cruising, and collision prevention functions. In particular, regarding the collision determination function, it determines whether or not a collision with another vehicle or obstacle has occurred based on the detection results of the photoelectric conversion system 14301 and the vehicle sensor 14310. As a result, avoidance control is performed when a collision is estimated, and a safety device is activated in the event of a collision.

[0089] The photoelectric conversion system 14301 is also connected to an alarm device 14312 that issues an alarm to the driver based on the result of the determination by the collision determination unit. For example, if the collision determination unit determines that there is a high possibility of a collision, the main control unit 14313 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 14312 warns the user by sounding an alarm, displaying alarm information on a display screen of a car navigation system or a meter panel, vibrating a seat belt or steering wheel, etc.

[0090] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are photographed by the photoelectric conversion system 14301. Fig. 15(b) shows an example of the arrangement of the photoelectric conversion system 14301 when the photoelectric conversion system 14301 photographs the area in front of the vehicle.

[0091] The two photoelectric conversion devices 14302 are disposed in front of the vehicle 14300. Specifically, the center line of the vehicle 14300 with respect to its forward / backward direction or outer shape (for example, vehicle width) is regarded as an axis of symmetry, and the two photoelectric conversion devices 1302 are disposed in line symmetry with respect to the axis of symmetry. This form is preferable for obtaining distance information between the vehicle 14300 and a subject to be photographed and for judging the possibility of a collision. In addition, the photoelectric conversion device 14302 is preferably disposed so as not to obstruct the driver's field of vision when the driver visually checks the situation outside the vehicle 14300 from the driver's seat. The alarm device 14312 is preferably disposed so as to be easily within the driver's field of vision.

[0092] In addition, in this embodiment, the control to prevent collision with other vehicles has been described, but the control can also be applied to automatic driving control to follow other vehicles and automatic driving control to prevent the vehicle from going out of the lane. Furthermore, the photoelectric conversion system 14301 can be applied not only to vehicles such as the vehicle itself, but also to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the photoelectric conversion system 14301 can be applied not only to moving bodies but also to devices that widely use object recognition, such as intelligent transport systems (ITS).

[0093] The photoelectric conversion device of the present invention may further be configured to be capable of acquiring various types of information such as distance information.

[0094] (Embodiment 11) FIG. 16 is a diagram illustrating glasses 16600 (smart glasses) as an example of the photoelectric conversion system of the eleventh embodiment. The glasses 16600 have a photoelectric conversion device 16602. The photoelectric conversion device 16602 is the photoelectric conversion device described in each of the above embodiments. A display device including a light emitting device such as an OLED or LED may be provided on the back side of the lens 16601. The photoelectric conversion device 16602 may be one or more. A combination of multiple types of photoelectric conversion devices may be used. The arrangement position of the photoelectric conversion device 16602 is not limited to that shown in FIG. 16(a).

[0095] The glasses 16600 further include a control device 16603. The control device 16603 functions as a power source that supplies power to the photoelectric conversion device 16602 and the display device. The control device 16603 also controls the operations of the photoelectric conversion device 16602 and the display device. The lens 16601 is formed with an optical system for collecting light on the photoelectric conversion device 16602.

[0096] FIG. 16(b) illustrates glasses 16610 (smart glasses) according to another example of the photoelectric conversion system of this embodiment. The glasses 16610 have a control device 16612, and the control device 16612 is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 16602 and a display device. The lens 16611 is formed with an optical system for projecting light emitted from the photoelectric conversion device in the control device 16612 and the display device, and an image is projected onto the lens 16611. The control device 16612 functions as a power source that supplies power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device may have a line of sight detection unit that detects the line of sight of the wearer. Infrared light may be used to detect the line of sight. The infrared light emission unit emits infrared light toward the eyeball of a user gazing at a display image. An image of the eyeball is obtained by detecting the reflected light of the emitted infrared light from the eyeball with an imaging unit having a light receiving element. By providing a reduction unit that reduces the amount of light from the infrared light emitting unit to the display unit in a plan view, degradation of image quality is reduced.

[0097] The gaze of the user with respect to the displayed image is detected from an image of the eyeball obtained by capturing infrared light. Any known method can be applied to gaze detection using the image of the eyeball. As an example, a gaze detection method based on a Purkinje image formed by reflection of irradiated light on the cornea can be used.

[0098] More specifically, a gaze detection process based on the pupil-corneal reflex method is performed. Using the pupil-corneal reflex method, a gaze vector that indicates the direction (rotation angle) of the eyeball is calculated based on the pupil image and the Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.

[0099] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the display image of the display device based on information on the user's line of sight from the photoelectric conversion device.

[0100] Specifically, the display device determines a first field of view area to which the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. In the display area of ​​the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.

[0101] The display area may have a first display area and a second display area different from the first display area, and a high priority area may be determined from the first display area and the second display area based on line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. The resolution of the high priority area may be controlled to be higher than the resolution of areas other than the high priority area. In other words, the resolution of an area with a relatively low priority may be lowered.

[0102] In addition, AI may be used to determine the first field of view area and the area with high priority. The AI ​​may be a model configured to estimate the angle of the line of sight and the distance to an object at the end of the line of sight from the image of the eyeball, using the image of the eyeball and the direction in which the eyeball in the image was actually looking as teacher data. The AI ​​program may be included in the display device, the photoelectric conversion device, or an external device. If included in the external device, it is transmitted to the display device via communication.

[0103] When display control is performed based on visual recognition detection, the present invention is preferably applicable to smart glasses further including a photoelectric conversion device for capturing an image of the outside world. The smart glasses can display captured outside information in real time.

[0104] (Embodiment 12) A photoelectric conversion system according to a twelfth embodiment will be described with reference to Fig. 17. The photoelectric conversion system according to this embodiment (hereinafter referred to as "system") can be used, for example, as a pathological diagnosis system in which a doctor or the like observes cells or tissues collected from a patient to diagnose a lesion, or a diagnostic support system that supports the diagnosis. The system according to this embodiment may diagnose a lesion or support the diagnosis based on an acquired image.

[0105] 17, the system of this embodiment includes one or more pathology systems 15510. It may further include an analysis unit 15530 and a medical information system 15540.

[0106] Each of the one or more pathology systems 15510 is a system used mainly by pathologists, and is installed in, for example, a laboratory or a hospital. Each pathology system 15510 may be installed in a different hospital, and is connected to the analysis unit 15530 and the medical information system 15540 via various networks such as a wide area network or a local area network.

[0107] Each pathology system 15510 includes a microscope 15511, a server 15512, and a display device 15513.

[0108] The microscope 15511 has the function of an optical microscope, captures an object to be observed placed on a glass slide, and obtains a pathological image, which is a digital image. The object to be observed is, for example, tissue or cells collected from a patient, and may be a piece of flesh from an organ, saliva, blood, etc.

[0109] The server 15512 stores and saves the pathological images acquired by the microscope 15511 in a storage unit (not shown). When the server 15512 receives a viewing request, it can search for the pathological images stored in a memory or the like and display the searched pathological images on the display device 15513. The server 15512 and the display device 15513 may be connected via a device that controls the display.

[0110] Here, when the observation object is a solid object such as a piece of flesh of an organ, the observation object may be, for example, a stained thin section. The thin section may be prepared, for example, by slicing a block piece cut out from a specimen such as an organ. In addition, when slicing, the block piece may be fixed with paraffin or the like.

[0111] The microscope 15511 may include a low-resolution imaging unit for capturing images at low resolution and a high-resolution imaging unit for capturing images at high resolution. The low-resolution imaging unit and the high-resolution imaging unit may be different optical systems or may be the same optical system. If they are the same optical system, the microscope 15511 may change the resolution depending on the object to be captured.

[0112] The observation object is accommodated in a glass slide or the like and placed on a stage located within the angle of view of the microscope 15511. The microscope 15511 first acquires an entire image within the angle of view using a low-resolution imaging unit, and identifies the area of ​​the observation object from the acquired entire image. Next, the microscope 15511 divides the area in which the observation object exists into a plurality of divided areas of a predetermined size, and acquires high-resolution images of each divided area by sequentially imaging each divided area using a high-resolution imaging unit. When switching between the target divided areas, the stage may be moved, the imaging optical system may be moved, or both may be moved. In addition, each divided area may overlap with an adjacent divided area in order to prevent the occurrence of an unimaging area due to unintended slippage of the glass slide. Furthermore, the entire image may include identification information for associating the entire image with the patient. This identification information may be, for example, a character string or a QR code (registered trademark), etc.

[0113] The high-resolution image acquired by the microscope 15511 is input to the server 15512. The server 15512 can divide each high-resolution image into partial images of smaller size. When the partial images are generated in this way, the server 15512 executes a synthesis process for all the partial images, in which a predetermined number of adjacent partial images are synthesized to generate one image. This synthesis process can be repeated until one partial image is finally generated. By this process, a group of partial images in a pyramid structure is generated, each layer being composed of one or more partial images. In this pyramid structure, the number of pixels of a partial image in a certain layer is the same as that of a partial image in a layer other than this layer, but the resolutions are different. For example, when a total of four partial images (2×2) are synthesized to generate one partial image in the upper layer, the resolution of the partial image in the upper layer is 1 / 2 times the resolution of the partial image in the lower layer used in the synthesis.

[0114] By constructing such a pyramid-structured group of partial images, it becomes possible to switch the level of detail of the observed object displayed on the display device depending on the layer to which the tile image to be displayed belongs. For example, when a partial image in the lowest layer is used, a narrow area of ​​the observed object is displayed in detail, and as a partial image in a higher layer is used, a wide area of ​​the observed object is displayed coarser.

[0115] The generated pyramid-structured partial image group can be stored in, for example, a memory etc. Then, when the server 15512 receives a request for acquiring a partial image including identification information from another device (for example, the analysis unit 15530), it transmits the partial image corresponding to the identification information to the other device.

[0116] In addition, the partial image, which is a pathology image, may be generated for each imaging condition such as focal length and staining condition. When a partial image is generated for each imaging condition, other pathology images corresponding to imaging conditions different from the specific imaging condition and of the same region as the specific pathology image may be displayed side by side together with the specific pathology image. The specific imaging condition may be specified by the viewer. Furthermore, when a plurality of imaging conditions are specified by the viewer, pathology images of the same region corresponding to each imaging condition may be displayed side by side.

[0117] Furthermore, the server 15512 may store the pyramid-structured partial image group in a storage device other than the server 15512, for example, a cloud server. Furthermore, a part or all of the above-described partial image generation process may be executed by a cloud server or the like. By using partial images in this way, the user can get the feeling that he or she is observing the observation object while changing the observation magnification. In other words, by controlling the display, it can function like a virtual microscope. The virtual observation magnification here actually corresponds to the resolution.

[0118] The medical information system 15540 is a so-called electronic medical record system, and stores information related to diagnosis, such as information for identifying a patient, information about the patient's disease, test information and image information used in the diagnosis, diagnosis results, and prescription drugs. For example, a pathological image obtained by imaging an observation object of a certain patient may be temporarily stored via the server 15512 and then displayed on the display device 15514. A pathologist using the pathological system 15510 makes a pathological diagnosis based on the pathological image displayed on the display device 15513. The results of the pathological diagnosis made by the pathologist are stored in the medical information system 15540.

[0119] The analysis unit 15530 may perform an analysis on the pathology image. A learning model created by machine learning may be used for this analysis. The analysis unit 15530 may derive a classification result of a specific region, a tissue identification result, or the like, as the analysis result. Furthermore, the analysis unit 15530 may derive an identification result such as cell information, number, position, brightness information, or the like, and scoring information therefor. The information obtained by the analysis unit 15530 may be displayed on the display device 15513 of the pathology system 15510 as diagnosis support information.

[0120] The analysis unit 15530 may be a server system including one or more servers (including a cloud server). The analysis unit 15530 may be incorporated in, for example, the server 15512 in the pathology system 15510. That is, various analyses of the pathology image may be performed in the pathology system 15510.

[0121] The photoelectric conversion device described in the above embodiment can be suitably applied to, for example, the microscope 15511 among the above-described configurations. Specifically, it can be applied to the low-resolution imaging section and / or the high-resolution imaging section in the microscope 15511. This allows the low-resolution imaging section and / or the high-resolution imaging section to be miniaturized, and thus the microscope 15511 to be miniaturized. This makes it easier to transport the microscope 15511, which makes it easier to introduce the system, reconfigure the system, and the like. Furthermore, by applying the photoelectric conversion device described in the above embodiment, part or all of the processing from acquisition of the pathological image to analysis of the pathological image can be performed on the fly within the microscope 15511, which allows for more rapid and accurate output of diagnostic support information.

[0122] The configuration described above can be applied not only to the diagnosis support system but also to biological microscopes in general, such as confocal microscopes, fluorescence microscopes, and video microscopes. Here, the observation object may be a biological sample such as cultured cells, fertilized eggs, and sperm, a biological material such as a cell sheet and three-dimensional cell tissue, or a living organism such as a zebrafish or mouse. In addition, the observation object is not limited to a glass slide, and may be observed in a state stored in a well plate, a petri dish, or the like.

[0123] Furthermore, a moving image may be generated from still images of an observation object acquired using a microscope. For example, a moving image may be generated from still images captured continuously for a predetermined period of time, or an image sequence may be generated from still images captured at a predetermined interval. In this way, by generating a moving image from still images, it becomes possible to analyze the dynamic characteristics of an observation object, such as the pulsation, elongation, migration, etc. of cancer cells, nerve cells, cardiac muscle tissue, sperm, etc., and the division process of cultured cells and fertilized eggs, using machine learning.

[0124] <Other embodiments> Although each embodiment has been described above, the present invention is not limited to these embodiments, and various changes and modifications are possible. In addition, each embodiment is mutually applicable. That is, a part of one embodiment can be replaced with a part of the other embodiment, and a part of one embodiment can be added to a part of the other embodiment. Also, a part of an embodiment can be deleted. [Explanation of symbols]

[0125] 1A Pixel array section 1B Pixel circuit section 2 First board 3 Heat dissipation part 4 Pad 5 Second board 7 Machine Learning Department

Claims

1. a pixel array section in which a plurality of photoelectric conversion units are arranged in a two-dimensional array in a plan view; and a first substrate having a first wiring pattern; a machine learning unit that processes a signal obtained from the charge generated by the photoelectric conversion unit; and a second substrate that has a second wiring pattern and is laminated on the first substrate; a first wiring pattern of the first substrate and a second wiring pattern of the second substrate are joined to form a metal joint; a heat dissipation unit having a plurality of metal junctions including the metal junction connected to the machine learning unit is disposed at a position overlapping the machine learning unit; the pixel array unit and the machine learning unit are arranged at positions where they do not overlap, the machine learning unit has a first circuit that operates at a first operating speed and a second circuit that operates at a second operating speed that is faster than the first operating speed; the heat dissipation section includes a first region and a second region located between the first region and the pixel array section in a plan view; the first region and the first circuit are positioned to overlap in a plan view, and the second region and the second circuit are positioned to overlap in a plan view, A photoelectric conversion device, characterized in that the number of the plurality of metal junctions in the second region is greater than the number of the plurality of metal junctions in the first region.

2. A pixel array section in which a plurality of photoelectric conversion units are arranged in a two-dimensional array in a planar view; a first substrate having a first wiring pattern; a machine learning unit that processes a signal obtained from the charge generated by the photoelectric conversion unit; and a second substrate that has a second wiring pattern and is laminated on the first substrate; a first wiring pattern of the first substrate and a second wiring pattern of the second substrate are joined to form a metal joint; a heat dissipation unit having a plurality of metal junctions including the metal junction connected to the machine learning unit is disposed at a position overlapping the machine learning unit; the pixel array unit and the machine learning unit are arranged at positions where they do not overlap, the machine learning unit has a first circuit that operates at a first operating speed and a second circuit that operates at a second operating speed that is faster than the first operating speed; the heat dissipation section includes a first region and a second region located between the first region and the pixel array section in a plan view; the first region and the first circuit are positioned to overlap in a plan view, and the second region and the second circuit are positioned to overlap in a plan view, A photoelectric conversion device, characterized in that an arrangement density of the plurality of metal junctions in the second region is higher than an arrangement density of the plurality of metal junctions in the first region.

3. 3. The photoelectric conversion device according to claim 1, wherein the heat dissipation section is disposed around the pixel array section in a plan view.

4. The photoelectric conversion device according to claim 3 , wherein the heat dissipation section is disposed so as to surround the entire periphery of the pixel array section in a plan view.

5. the pixel array portion has a longitudinal direction and a lateral direction in a plan view, The photoelectric conversion device according to claim 3 , wherein the heat dissipation portion is disposed along the longitudinal direction.

6. The heat dissipation portion includes a first portion and a second portion, The photoelectric conversion device according to claim 3 , wherein the pixel array portion is sandwiched between the first portion and the second portion in a plan view.

7. The photoelectric conversion device according to any one of claims 1 to 6, characterized in that the number of metal junctions in the area overlapping the machine learning section is greater than the number of metal junctions in the area not overlapping the machine learning section.

8. The photoelectric conversion device according to any one of claims 1 to 7, characterized in that a plurality of the metal junctions in a region overlapping the machine learning section in a planar view are connected to a single wiring pattern.

9. The photoelectric conversion device according to claim 1 , and a signal processing unit that processes a signal output from the photoelectric conversion device.

10. The photoelectric conversion device according to claim 1 , a distance information acquisition means for acquiring distance information to an object from distance measurement information based on a signal from the photoelectric conversion device, A moving body further comprising a control means for controlling the moving body based on the distance information.

Images