electronic devices

Abstract

[Problem] To provide an electronic device that achieves image capture with a wide viewing angle in a compact housing. [Solution] The electronic device according to this disclosure includes: a display unit (2) configured to be deformable; and at least one first imaging unit (3) arranged on the side of the display unit (2) opposite to the display screen and configured to image incident light passing through the display unit. The display unit (2) may be foldable. At least a portion of the display unit (2) may be flexible. Furthermore, the optical system of the first imaging unit (3) for image capture may be switched according to the shape of the display unit (2).

Description

Technical Field

[0001] This disclosure relates to electronic devices. Background Technology

[0002] Typically, cameras are housed within the casing of electronic devices such as smartphones, tablets, game consoles, or personal computers (PCs). Today, the use of cameras mounted on electronic devices extends beyond video calls or selfies to include image capture of scenes, portraits, and high-speed objects. Consequently, there has been a growing demand in recent years for improved performance of cameras mounted on electronic devices.

[0003] Reference List

[0004] Patent documents

[0005] Patent document 1: Japanese Patent Application Publication No. 2019-506651.

[0006] Patent document 2: Japanese Patent Application Publication No. 2019-8202. Summary of the Invention

[0007] The problem to be solved by the present invention

[0008] One of the performance requirements of a camera is the size of the field of view it can capture. When using a camera with a wide field of view, it is possible to capture powerful images or images that include entire objects of large size. Therefore, there is a huge potential demand for increasing the field of view of cameras mounted on electronic devices.

[0009] However, when mounting a camera with a fisheye lens to increase the angle of view, the following problems arise: a protrusion is formed in the electronic device, or the device's housing becomes larger. Furthermore, when a camera with a large angle of view is mounted within the electronic device's housing, there is a possibility that adjacent components may be included in the captured image.

[0010] Therefore, this disclosure provides an electronic device that enables image capture with a wide viewing angle in a compact housing.

[0011] Solution to the problem

[0012] An electronic device according to one aspect of the present disclosure may include: a display unit configured to be deformable; and at least one first imaging unit disposed on the opposite side of the display surface of the display unit and configured to image incident light transmitted through the display unit.

[0013] The display unit can be foldable.

[0014] The display unit may include a first region, a second region, and a third region, and the portion of the display unit between the first region and the second region, as well as the portion of the display unit between the second region and the third region, may be foldable.

[0015] The first imaging unit can be arranged on the opposite side of the display surface in the second region.

[0016] In the display unit, the first imaging unit may be arranged on the opposite side of the display surface in the first region and on the opposite side of the display surface in the third region.

[0017] At least a portion of the display unit can be flexible.

[0018] The first imaging unit can be positioned at a location corresponding to the inner periphery of the convex surface formed when the display unit is bent.

[0019] When the display unit is bent, at least a portion of the components located on the light incident direction side of the first imaging unit can be moved out of the field of view of the first imaging unit.

[0020] It may further include a second imaging unit, which is arranged on a surface on the side opposite to the display unit.

[0021] In the first imaging unit, the optical system for image capture can be switched according to the shape of the display unit.

[0022] It may further include processing circuitry configured to output a first image based on the captured image of the first imaging unit to a display unit.

[0023] The processing circuit can be configured to determine the position and extent of the display unit for displaying each part of the first image based on the viewpoint in the captured image.

[0024] The display unit may include a touch panel, and the processing circuitry may be configured to detect at least one object included in the captured image.

[0025] The processing circuit can be configured to detect that an area of ​​the object displayed on the display unit is touched, and output a second image obtained by cropping a portion of the viewing angle of the first imaging unit including the area to at least one part of the display unit.

[0026] The processing circuit can be configured to detect that an area of ​​an object displayed on the display unit is touched, and to begin recording a first video image obtained by cropping a portion of the viewpoint of a first imaging unit that includes that area.

[0027] The processing circuitry can be configured to record a second video image with a larger angle of view than the first video image, in parallel with the recording of the first video image.

[0028] An electronic device according to one aspect of the present disclosure may include: a display unit configured to be deformable; and an imaging unit configured to image incident light transmitted through the display unit, wherein the imaging unit may be configured to perform imaging at a first viewing angle when the display unit is in a first shape, and to perform imaging at a second viewing angle wider than the first viewing angle when the display unit is in a second shape.

[0029] The first shape can be in an unfolded state, and the second shape can be in a folded state.

[0030] An electronic device according to one aspect of the present disclosure may include: a display unit configured to be deformable; and a third imaging unit and a fourth imaging unit, respectively configured to image incident light transmitted through the display unit; wherein the third imaging unit may be configured to perform imaging from a third perspective, and the fourth imaging unit may be configured to perform imaging from a fourth perspective, wherein when the display unit is in a first shape, the third and fourth perspectives may partially overlap each other, and when the display unit is in a second shape, the third and fourth perspectives may not overlap each other.

[0031] It may further include processing circuitry configured to synthesize an image obtained by a third imaging unit and an image obtained by a fourth imaging unit when the display unit is in the second shape. Attached Figure Description

[0032] Figure 1 This is an illustration showing an example of narrowing the field of view of image capture by using adjacent components.

[0033] Figure 2 This is a diagram illustrating an example of the effective field of view range in a captured image.

[0034] Figure 3 This is a perspective view showing an example of an electronic device in an unfolded state according to the present disclosure.

[0035] Figure 4 This is a perspective view showing an example of a folded state of an electronic device according to this disclosure.

[0036] Figure 5 This is a cross-sectional view showing an example of an electronic device in an unfolded state according to the present disclosure.

[0037] Figure 6 This is a cross-sectional view showing an example of a folded state of an electronic device according to the present disclosure.

[0038] Figure 7 This is a schematic cross-sectional view of an electronic device according to the present disclosure.

[0039] Figure 8 This is a diagram illustrating an example of the cross-sectional structure of an imaging unit according to the present disclosure.

[0040] Figure 9 This is a block diagram illustrating an example of the internal configuration of an electronic device according to this disclosure.

[0041] Figure 10 This is a perspective view showing an example of an electronic device according to Modification 1 in its state before bending.

[0042] Figure 11 This is a perspective view showing an example of the bent state of an electronic device according to Modification 1.

[0043] Figure 12 This is a cross-sectional view showing an example of the structure of the electronic device and fixture according to Modification 1.

[0044] Figure 13 It is a schematic cross-sectional view of the electronic device based on Modification 1.

[0045] Figure 14 This is a cross-sectional view showing an example of a camera module capable of switching the optical system to be used based on the bending state of the electronic device.

[0046] Figure 15 This is a cross-sectional view showing an example of a camera module capable of switching the optical system to be used based on the bending state of the electronic device.

[0047] Figure 16 This is a cross-sectional view showing an example of a flexible electronic device and camera module.

[0048] Figure 17 This is a cross-sectional view showing an example of a flexible electronic device and camera module.

[0049] Figure 18 This is a perspective view showing an example of the deployed state of an electronic device according to Modification 2.

[0050] Figure 19 This is a cross-sectional view showing an example of the folded state of an electronic device according to Modification 2.

[0051] Figure 20 This is a diagram illustrating an example of the viewing angle that an electronic device according to Modification 3 can capture in an image.

[0052] Figure 21 This is an illustration showing an example of an image displayed on an electronic device according to the present disclosure.

[0053] Figure 22 This is a diagram illustrating an example of an object selection operation in an electronic device according to the present disclosure.

[0054] Figure 23 This is a diagram illustrating an example of screen display and image capture operations for the selected object.

[0055] Figure 24 This is a cross-sectional view showing an example of an electronic device according to Modification 4.

[0056] Figure 25 This is a plan view of an electronic device according to the present disclosure being used in a capsule endoscope.

[0057] Figure 26 This is a rear view of an electronic device according to this disclosure applied to a digital SLR camera.

[0058] Figure 27 This is a diagram illustrating an example of applying electronic devices to an HMD.

[0059] Figure 28 This is a diagram showing the current HMD. Detailed Implementation

[0060] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that components having substantially the same functional configuration are indicated by the same reference numerals in this specification and the drawings, and redundant explanations are omitted.

[0061] In electronic devices such as smartphones, tablets, game consoles, personal computers (PCs), or display devices, cameras (front-facing cameras) can be mounted on the frame (bezel) of the display unit, enabling video calls or selfies. However, to increase screen size or display images at higher resolutions, electronic devices with narrower bezels and bezel-less devices have been developed. For this purpose, it has been proposed that the camera be mounted behind the display panel, rather than on the bezel, when viewed from the front. In this case, the camera captures images using light transmitted through the display panel. Such cameras are also known as under-display cameras.

[0062] Figure 1 The cross-sectional view illustrates an example of an electronic device 300, which includes an imaging unit 8 that performs image capture using light passing through a display panel 4A. In the electronic device 300, the display panel 4A, support member 78, transistor layer 77, and wiring layer 76 are stacked from top to bottom. Note that in the following description, the positive z-axis direction in the figure is referred to as "up," and the negative z-axis direction is referred to as "down." The display panel 4A may be protected by a protective layer 7. The protective layer 7 is formed, for example, of a glass material. However, other materials may be used as the protective layer 7, as long as visible light can pass through it.

[0063] like Figure 1 As shown, on the lower side of the display panel 4A, there is a portion where the transistor layer 77 and wiring layer 76 are not formed. A camera module 3 is mounted in this portion on the lower side of the display panel 4A where the transistor layer 77 and wiring layer 76 are not formed. In the camera module 3, a fisheye lens 80, a lens 75, a lens 74, and an imaging unit 8 are arranged from top to bottom. The fisheye lens 80, lens 75, and lens 74 form an optical system on the imaging unit 8 that converges and transmits light through the display panel 4A. The imaging unit 8 is a component that performs imaging through photoelectric conversion. The imaging unit 8 may, for example, include a complementary metal-oxide-semiconductor (CMOS) sensor, a charge-coupled device (CCD) sensor, an organic photoelectric conversion film, or a combination thereof.

[0064] Because of the fisheye lens 80, camera module 3 itself can be... Figure 1 Image capture is performed by a viewing angle V1. However, viewing angle V1 includes a portion of the transistor layer 77 and wiring layer 76 adjacent to the camera module 3. Therefore, the viewing angle that the camera module 3 mounted on the electronic device 300 can actually perform image capture on is a viewing angle V2, which is narrower than viewing angle V1. The imaging unit (camera module) can actually perform image capture on a viewing angle (such as...) Figure 1 The perspective V2 in the image is called the effective perspective.

[0065] Figure 2 Image 72 illustrates an example of the relationship between viewpoints V1 and V2 in an image captured by imaging unit 8. There is a possibility that the transistor layer 77 and wiring layer 76 adjacent to camera module 3 are included in region 73 between viewpoints V1 and V2. Therefore, a portion within region 73 of image 72 can be discarded by circuitry connected in the subsequent stages of imaging unit 8. This process of discarding portions of the image captured by imaging unit 8 other than the effective viewpoint corresponds to cropping (also known as trimming) of the effective viewpoint. In this case, the portion within region 73 of image 72 is provided to the user of electronic device 300 as the captured image.

[0066] When as in Figure 2 In the example where the image is cropped, the user can only obtain a captured image with a limited field of view, regardless of the performance of the camera module 3 (optical system and imaging unit 8). Furthermore, it is inevitable that the number of pixels included in the captured image will be reduced. Therefore, this disclosure describes an electronic device that uses a flexible display as the display panel and achieves image capture with a wide field of view within a compact housing. That is, the electronic device according to this disclosure has both image display and image capture functions.

[0067] Figure 3 and Figure 4 This is a perspective view showing an example of an electronic device according to this disclosure. Figure 3An example of the shape of the electronic device 1 in its unfolded state is shown. A display unit 2 is mounted on the upper surface (first surface) of the electronic device 1. The display unit 2 includes a flexible display. For example, a combination of polyimide and organic light-emitting diodes (OLEDs) can be used as a flexible display. However, the materials and methods used for flexible displays are not limited. For example, other flexible materials, such as flexible glass sheets, can be used instead of polyimide. Furthermore, other display methods, such as liquid crystals or microLED arrays, can be used instead of OLEDs.

[0068] The display unit 2 is divided into a first region 21, a second region 22 and a third region 23. Figure 3 The first region 21, the second region 22, and the third region 23 in the plan view all have a generally rectangular shape. However, the shape and area of ​​the display unit of the electronic device according to this disclosure are not limited. The electronic device 1 is foldable, so that when it is moved from... Figure 3 When viewed from above, a mountain-shaped fold is formed between the first region 21 and the second region 22 of the display unit 2. Similarly, in the folded state, the electronic device 1 is foldable, so that when viewed from above... Figure 3 When viewed from above, a mountain-shaped fold is formed between the second region 22 and the third region 23 of the display unit 2. That is, the electronic device 1 is configured to fold into a shape like a book cover or a long wallet.

[0069] Figure 4 An example of the shape of electronic device 1 in its folded state is shown. (Reference) Figure 4 The electronic device 1 is folded so that the display surface of the display unit 2 faces outward from the housing. That is, in Figure 4 In this case, display unit 2 folds outward. To illustrate this metaphorically using a bound book as an example, when electronic device 1 is folded, the display surface of display unit 2 is arranged in the areas corresponding to the front cover, spine, and back cover. That is, in... Figure 4 In the display unit 2, the third region 23 is arranged on the first surface (corresponding to the front cover) on the upper side of the electronic device 1, the first region 21 of the display unit 2 is arranged on the second surface (corresponding to the rear cover) on the opposite side of the first surface of the electronic device 1, and the second region 22 is arranged on the third surface (corresponding to the ridge) of the electronic device 1 perpendicular to the first surface and the second surface.

[0070] That is, in the electronic device according to the present disclosure, when the display unit is folded, the first area and the second area of ​​the display unit can face opposite directions.

[0071] Furthermore, when viewed from the front, the camera module 3 is mounted behind the second region 22 of the display unit 2. That is, the camera module 3 (first imaging unit) is arranged on the opposite side of the display surface in the second region 22. Therefore, the camera module 3 of the electronic device 1 performs image capture using light transmitted through the second region 22 of the display unit 2. Figure 4 In the example case, camera module 3 can perform image capture from a perspective that includes the direction of the third surface.

[0072] Electronic device 1 may include a locking mechanism (not shown) with a fixed shape. By using the locking mechanism, electronic device 1 can be fixed in an unfolded or folded state. In this case, the detailed configuration of the locking mechanism is not particularly limited. By using the locking mechanism, it is possible to prevent the shape of electronic device 1 from changing at times unintentional by the user.

[0073] In this way, the display unit of the electronic device according to the present disclosure may include a first region, a second region, and a third region. The portion between the first region and the second region of the display unit, and the portion between the second region and the third region of the display unit, may be foldable.

[0074] Here, an electronic device having a generally flat shape in both its unfolded and folded states has been described as an example. However, the shape of the electronic device shown here is merely an example. Therefore, the shape of the electronic device according to this disclosure in at least one of its unfolded and folded states may differ from this shape.

[0075] Figure 5 yes Figure 3 A cross-sectional view of electronic device 1 taken along line AA. However, Figure 6 It shows Figure 5 The cross-sectional view is in a folded state. Figure 5 and Figure 6 Both show an enlarged portion of the electronic device 1 with camera module 3 mounted on it. Camera module 3 is mounted on support member 15. Support member 15 may include, for example, […]. Figure 1 At least one of transistor layer 77 and wiring layer 76 is present. A display panel 4 is arranged above the camera module 3. The display panel 4 corresponds to the aforementioned flexible display. However, a support member 16 is arranged in the portion of the display panel 4 below which the camera module 3 is not located. The display panel 4 is protected by a protective layer 7. The protective layer 7 can be, for example, a light-transmitting and flexible material, such as polyimide or a flexible glass sheet. The protective layer 7 and the display panel 4 correspond to the aforementioned display unit 2.

[0076] As support member 15, a flexible substrate comprising, for example, plastic film, polyimide, or PET can be used. As support member 16, a material comprising plastic film, polyimide, or PET can be used. At least a portion of support members 15 and 16 above the folding mechanism 10 may be formed of a material that is elastic and flexible. However, the type of material used as support members 15 and 16 is not limited.

[0077] A folding mechanism 10 is provided on the side of the support member 15 opposite to the side where the camera module 3 is arranged. The folding mechanism 10 realizes the mountain-shaped folding deformation between the first region 21 and the second region 22 of the display unit 2, and the mountain-shaped folding deformation between the second region 22 and the third region 23 of the display unit 2, and restores the original shape from these deformations. Figure 5 and Figure 6 The folding mechanism 10 is implemented by assembling a plurality of gears 12 in a space 11 provided in the housing of the electronic device 1. The plurality of gears 12 are formed, for example, from resin or metal. However, the material of the plurality of gears 12 is not limited.

[0078] A sensor 14 is arranged above the gear 12. The sensor 14 is a sensor capable of detecting deformation in at least any part of the electronic device 1. For example, in Figure 5 and Figure 6 In the example case, sensor 14 detects the presence of a fold via folding mechanism 10. A strain gauge can be used as sensor 14, for example. However, other types of sensors can be used as sensor 14. Sensor 14 only needs to be mounted at a location where deformation of the electronic device 1 (display unit 2) can be detected. Therefore, sensor 14 can be arranged in a different position than... Figure 5 and Figure 6 The example is located at the following position.

[0079] Note that a protective member may be arranged above the gear 12 to prevent damage to the sensor 14 and the support member 15 from the gear 12. In this case, the protective member can be formed using a material with a higher hardness than the gear 12. However, the material of the protective member is not limited. Furthermore, a plate-like member 13 is arranged on the opposite side of the protective layer 7 of the electronic device 1 and the display panel 4. The plate-like member 13 supports other components of the electronic device 1 and maintains the generally plate-like structure of the housing.

[0080] Notice, Figure 5 and Figure 6 The folding mechanism 10 shown is merely an example. Therefore, electronic devices according to this disclosure may include folding mechanisms with structures different from this. For example, the folding mechanism may be implemented using a rotatable component such as a hinge.

[0081] In the display panel 4, at least a portion of the upper side of the camera module 3 and the portion nearby thereto are transmissive to electromagnetic waves in the wavelength band used by the camera module 3 for image capture. For example, when the imaging unit 8 in the camera module 3 performs image capture using visible light, the aforementioned portion of the display panel 4 is transmissive to the visible light band. Furthermore, when the imaging unit 8 in the camera module 3 performs image capture using infrared light, the aforementioned portion of the display panel 4 is transmissive to the infrared light band.

[0082] refer to Figure 6 Corresponding to the folded state of electronic device 1, translucent components (display panel 4 and protective layer 7) are arranged above and to the left and right of camera module 3. Therefore, with Figure 1 Unlike the electronic device 300, the camera module 3 (imaging unit 8) does not narrow its field of view for capturing images by including adjacent components. Therefore, when using the electronic device 1 in its folded state, image capture can be performed with a wider field of view. Figure 6 In this state, the user can perform image capture while rotating the electronic device 1 around the z-axis. This image capture mode of the electronic device 1 is called the wide-angle image capture mode. By using the wide-angle image capture mode, the user can capture a wide-angle image relative to the z-axis from 180 degrees to 360 degrees. Note that when using the wide-angle image capture mode, the rotation operation of the electronic device 1 can be performed manually by the user or by a rotating machine (not shown), such as a motor.

[0083] exist Figure 5 and Figure 6 In the example, the shape of the camera module 3 remains almost unchanged regardless of the deformation of the display panel 4 and the protective layer 7 (display unit 2). Therefore, in the electronic device according to this disclosure, a camera module (imaging unit) that maintains its original shape regardless of the deformation of the display unit can be installed. Thus, a conventional camera module (imaging unit) without significant elasticity and flexibility can be used.

[0084] Note that, in order to suppress the stress applied to the camera module 3, the bottom of the camera module 3 can be supported by a plate-like structure. Furthermore, structures can be provided to support parts other than the upper surface of the camera module 3. For example, these structures can be made of metal, but the material of the structure is not limited. By providing a structure to suppress the stress applied to the camera module 3, damage to the camera module 3 during deformation of the electronic device 1 can be prevented.

[0085] An electronic device according to this disclosure may include: a display unit configured to be deformable; and an imaging unit configured to image incident light transmitted through the display unit. In this configuration, the imaging unit may be configured to perform imaging at a first viewing angle when the display unit is in a first shape, and to perform imaging at a second viewing angle wider than the first viewing angle when the display unit is in a second shape. For example, the first shape may be an unfolded state, and the second shape a folded state. However, at least one of the first and second shapes may be in a different state. For example, the first shape may be an unbent state, and the second shape may be a bent state.

[0086] Figure 7 This is a cross-sectional view of the electronic device 1, including the display unit 2 and the camera module 3. In the following text, reference will be made to... Figure 7 An example describing the configuration of electronic device 1.

[0087] Figure 7 The electronic device 1 includes a display unit 2 and a camera module 3 (imaging unit). The camera module 3 is mounted on the side opposite to the display surface of the display unit 2. That is, when viewed from above... Figure 7 When the display unit 2 is in use, the camera module 3 is arranged behind the display unit 2. Therefore, the camera module 3 of the electronic device 1 performs image capture using light transmitted through the display unit 2.

[0088] like Figure 7 As shown, the display unit 2 can be constructed by sequentially stacking a display panel 4, a circular polarizing plate 5, a touch panel 6, and a protective layer 7. The display panel 4 is, for example, a flexible and elastic plate-like structure that displays images via electrical methods (flexible display). The display panel 4 can be, for example, an organic light-emitting diode (OLED), a liquid crystal panel, or a micro-LED. However, the method of displaying the panel 4 is not limited.

[0089] Typically, display panels 4, such as OLEDs or liquid crystals, have multiple layers. For example, a portion of the display panel 4 includes components that reduce light transmittance, such as a color filter layer. Therefore, as... Figure 1 As shown, depending on the position of the camera module 3, the through-hole can be set in a component that prevents light transmission into the display panel 4. Light that has passed through the through-hole can be incident on the camera module 3 without passing through the corresponding component. Therefore, the degradation of image quality of the image captured by the camera module 3 can be suppressed.

[0090] The circular polarizer 5 is installed, for example, to reduce glare or improve visibility. The touch panel 6 is a plate-like structure with a built-in touch sensor. Examples of touch sensors include capacitive touch sensors or resistive film touch sensors. However, any type of touch sensor can be used. Note that in the electronic device according to this disclosure, a display panel with integrated touch panel functionality can be used. A protective layer 7 is provided to protect the display panel 4 from external influences.

[0091] Note that the display unit 2 may include a fingerprint sensor 6A. As the fingerprint sensor 6A, for example, an optical fingerprint sensor or an ultrasonic fingerprint sensor can be used. However, the method of fingerprint sensor application is not limited. For example, the fingerprint sensor 6A may be mounted on at least any layer of the display unit 2. Furthermore, the fingerprint sensor 6A may be mounted in the camera module 3.

[0092] Camera module 3 includes, for example, an imaging unit 8 and an optical system 9. The optical system 9 is arranged between display unit 2 and imaging unit 8. The optical system 9 focuses light transmitted through display unit 2 onto imaging unit 8. The optical system 9 may include multiple lenses. To obtain a wide viewing angle, the optical system 9 may include a fisheye lens.

[0093] Figure 8 An example of the detailed cross-sectional structure of the imaging unit is shown. Similar to the above, the positive z-axis direction is referred to as "up," and the negative z-axis direction is referred to as "down." Figure 7 In the imaging unit 8, a plurality of photoelectric conversion units 108a are formed in the substrate 111. Examples of photoelectric conversion units 108a include complementary metal-oxide-semiconductor (CMOS) sensors or charge-coupled device (CCD) sensors that include photodiodes. Furthermore, other types of sensors, such as organic photoelectric conversion films, can be used as photoelectric conversion units 108a.

[0094] Then, an interlayer insulating film 113 is formed on the surface 111b side (lower side) of the substrate 111. A plurality of wiring layers 112 are arranged inside the interlayer insulating film 113. At least one of a contact electrode and a through electrode (not shown) can be provided between the photoelectric conversion unit 108a and the wiring layer 112. Similarly, at least one of the contact electrode and the through electrode can also be provided between the wiring layers 112.

[0095] However, a planarization layer 114 is formed on the second surface 111a side (upper side) of the substrate 111. An underlying insulating layer 116 is formed on the planarization layer 114. Furthermore, a light-shielding layer 115 may be formed on a portion of the planarization layer 114. The light-shielding layer 115 is disposed at or near the boundary of a pixel. At least a portion of the surface of the light-shielding layer 115 may contact the underlying insulating layer 116. Furthermore, an insulating layer 117 is formed on the underlying insulating layer 116. A polarizing element may be formed within the insulating layer 117. Examples of polarizing elements include a wire grid polarizing element having both linear and spatial structures. However, the structure and arrangement orientation of the polarizing element are not particularly limited.

[0096] Protective layers 118 and 119 are formed on an insulating layer 117 comprising multiple polarization elements. Furthermore, a planarization layer 120 is formed on the protective layer 119. A color filter layer 121 is disposed on the planarization layer 120. The color filter layer 121 selectively transmits light of a portion of the wavelength band, enabling a photoelectric conversion unit disposed below to detect light within a predetermined wavelength band.

[0097] Then, an on-plate lens 122 is arranged on the color filter layer 121. Figure 8 In the cross-sectional structure, the on-chip lens 122 is arranged on multiple polarizing elements. However, multiple polarizing elements can be arranged on the on-chip lens 122. In this way, the polarization can be changed. Figure 8 The stacking order of the layers in the cross-sectional structure.

[0098] In the imaging unit 8, a single photoelectric conversion unit 108a can be formed for each pixel. Each photoelectric conversion unit 108a performs photoelectric conversion on light incident via the display unit 2 and outputs any color signal. That is, the imaging unit 8 can be described as a collection of multiple pixels that perform photoelectric conversion and output color signals. Examples of color signals include red, green, and blue signals. However, the color signal output from the pixels of the imaging unit 8 can also be a color other than the three primary colors of light. For example, the pixels of the imaging unit 8 can output a color signal that is at least one of cyan, magenta, and yellow, which are complementary colors of the three primary colors of light. In addition, the pixels of the imaging unit 8 can also output a color signal corresponding to an intermediate color of the above-mentioned colors, or they can output a white signal.

[0099] Figure 9 This is a block diagram illustrating the internal configuration of an electronic device according to the present disclosure. Figure 9The electronic equipment includes a display unit 2, an optical system 9, an imaging unit 8, a sensor 14, an A / D converter 31, a clamping unit 32, a color output unit 33, a defect correction unit 34, a linear matrix unit 35, a gamma correction unit 36, a luminance and chrominance signal generation unit 37, an output unit 38, a storage unit 41, and a flash lamp 42. For example, the A / D converter 31, clamping unit 32, color output unit 33, defect correction unit 34, linear matrix unit 35, gamma correction unit 36, luminance and chrominance signal generation unit 37, and output unit 38 can be mounted on the processing circuit 200.

[0100] For example, the optical system 9 includes one or more lenses 9a and an infrared (IR) cutoff filter 9b. However, the IR cutoff filter 9b can be omitted. As described above, the imaging unit 8 includes multiple pixels that perform photoelectric conversion and output color signals. The sensor 14 is a sensor capable of detecting shape changes (deformations) of the electronic device 1. The sensor 14 only needs to be able to detect shape changes in at least any part of the electronic device 1. For example, the sensor 14 is configured to determine whether the electronic device 1 is in an unfolded state. Figure 3 and Figure 5 (State) or is it in a folded state? Figure 4 and Figure 6 (The state). However, the type of deformation detected by sensor 14 may be different.

[0101] For example, when sensor 14 detects that the shape of electronic device 1 has changed from an unfolded state to a folded state, processing circuit 200 can activate (enable) imaging unit 8. Furthermore, when sensor 14 detects that the shape of electronic device 1 has changed from an unfolded state to a folded state, processing circuit 200 can initiate image capture operation of imaging unit 8. Control of imaging unit 8, including image capture operation, can be performed by the hardware of imaging unit 8 or processing circuit 200, or by software operating on processing circuit 200. Furthermore, control of imaging unit 8, including image capture operation, can be achieved through a combination of the former and the latter.

[0102] The color signals output from each pixel are input to the A / D converter 31. The A / D converter 31 generates digitized digital pixel data based on the color signals of multiple pixels.

[0103] Clamping unit 32 performs processing to define the black level. For example, clamping unit 32 subtracts black level data from digital pixel data. Data output from clamping unit 32 is input to color output unit 33. For example, color output unit 33 classifies digital pixel data into individual color information. For example, defect correction unit 34 performs correction processing such as removing noise components and signal levels. Examples of noise components include flash components or diffracted light components. For example, defect correction unit 34 can interpolate the data of polarized pixels by using digital pixel data of surrounding unpolarized pixels. However, the content of the correction processing performed by defect correction unit 34 is not limited.

[0104] Linear matrix unit 35 performs matrix operations on color information (e.g., RGB). Therefore, the color reproduction of the image can be improved. Linear matrix unit 35 is also referred to as a color matrix unit. For example, suppose an imaging unit is used that includes at least one of cyan, yellow, and magenta pixels. In this case, color information of at least one of cyan, yellow, and magenta is input to linear matrix unit 35. Linear matrix unit 35 can perform matrix operations to convert color information of at least one of cyan, yellow, and magenta into red / green / blue (RGB) format color information.

[0105] The gamma correction unit 36 ​​performs gamma correction on the color information output from the linear matrix unit 35. For example, the gamma correction unit 36 ​​improves the visibility of the captured image on the display unit 2 by performing gamma correction according to the display characteristics of the display unit 2. The luminance and chrominance signal generation unit 37 generates luminance and chrominance signals based on the output data of the gamma correction unit 36. The luminance and chrominance signals are signals used for display on the display unit 2. The output unit 38 transmits image data to the display unit 2, etc.

[0106] Based on the output signal from sensor 14, the content of image processing performed on the captured image can be changed. For example, brightness correction can be performed on an image captured when the electronic device 1 is in a folded state, with content different from that captured when the electronic device 1 is in an unfolded state. There is a possibility that the angle of incidence of light may change depending on the deformed state of the display unit 2 located above the imaging unit 8. Furthermore, there is a possibility that the amount of incident light on the imaging unit 8 may change. In particular, if the tilt incidence characteristics of the imaging unit 8 change, brightness correction (shadow correction) different for each image height can be performed.

[0107] The processing circuit 200 can perform the processing of cropping the region corresponding to the effective viewpoint from the captured image (as described above). Figure 2In this case, the content of the processing to be performed can be changed according to the output signal from sensor 14. For example, if the imaging unit 8 may include adjacent components when capturing an image while the electronic device 1 is in the unfolded state, the processing circuit 200 can crop the area corresponding to the effective viewing angle from the captured image and discard the area in the captured image that includes adjacent components. Furthermore, when the electronic device 1 is in the folded state, if the imaging unit 8 does not include adjacent components when capturing an image, the processing circuit 200 can skip the processing of cropping a portion of the captured image.

[0108] Furthermore, when image capture is performed while the electronic device 1 is rotating around any axis in its folded state, the processing circuit 200 can execute the function of a wide-angle image capture mode that generates an image. Although the case where the rotation axis of the electronic device 1 is the z-axis was described above as an example, the direction of the rotation axis is not limited. The function of the wide-angle image capture mode can be implemented by hardware circuitry or by a program executed on the processing circuit 200. Furthermore, the function of the wide-angle image capture mode can be implemented through combinations thereof.

[0109] Note that the processing circuit 200 can perform video image recording processing, but is not limited to image capture processing. Figure 9 At least a portion of the signal processing of the defect correction unit 34, linear matrix unit 35, gamma correction unit 36, or luminance / chrominance signal generation unit 37 can be performed by logic circuitry in the imaging sensor including the imaging unit 8. Furthermore, at least a portion of this signal processing can be performed by signal processing circuitry in the electronic device 1. Additionally, the processing circuitry 200 can perform other types of processing, such as exposure adjustment processing and edge enhancement processing.

[0110] Storage unit 41 is a memory or storage device capable of storing data of images captured by imaging unit 8, data of video images captured by imaging unit 8, or programs. Examples of memory include volatile memory such as SRAM and DRAM, and non-volatile memory such as NAND flash memory and NOR flash memory. Examples of storage devices include hard disks or SSDs. However, the type of memory or storage device used as storage unit 41 is not limited. For example, linear matrix unit 35 can synthesize or correct images using multiple images stored in storage unit 41. However, components other than linear matrix unit 35 can be used to synthesize or correct images.

[0111] The flash lamp 42 is a light source that illuminates the object in conjunction with the imaging operation of the imaging unit 8. For example, a white LED can be used as the flash lamp 42. However, the type of light source used as the flash lamp 42 is not limited. The storage unit 41 and the flash lamp 42 are any components. Therefore, at least one of these components can be omitted.

[0112] For example, the electronic device disclosed herein includes: a display unit configured to be deformable; and at least one first imaging unit disposed on the opposite side of the display surface of the display unit and configured to image incident light transmitted through the display unit. Here, the camera module 3 described above is an example of the first imaging unit.

[0113] In the above description, examples of (foldable) electronic devices including foldable display units have been described. However, the electronic device according to this disclosure can be configured to be deformable in a mode other than this. As shown in the following modification 1, the electronic device according to this disclosure can be configured to deform into a convex curved shape.

[0114] Figure 10 and Figure 11 Both perspective views show examples of electronic devices according to Modification 1. (As shown in the original text) Figure 10 and Figure 11 As shown, similar to the aforementioned electronic device 1, electronic device 1A includes a display unit 2, a camera module 3, and a sensor 14. The display unit 2 is arranged on any surface of electronic device 1A (on...). Figure 10 In the example, on the upper surface. Similar to the display unit 2 of electronic device 1, display unit 2 includes a flexible display, such as a flexible OLED. Furthermore, display unit 2 may include, for example, a protective layer formed of polyimide or a glass plate. As described above, in the electronic device according to this disclosure, at least a portion of the display unit may be flexible.

[0115] Furthermore, when viewed from the front, the camera module 3 is positioned behind the display unit 2. Therefore, the camera module 3 performs image capture using light transmitted through the display unit 2. Figure 11 As shown, electronic device 1A can be bent so that its cross-section has an inverted U-shape (convex upwards). That is, in the electronic device according to this disclosure, the display unit can be bent into a convex curved surface shape. Sensor 14 is mounted in the housing of electronic device 1A and detects the deformation of electronic device 1A. Sensor 14 is, for example, a strain gauge. However, sensor 14 can be other types of sensors. Sensor 14 is configured to be able to determine whether electronic device 1A is in an unbent state ( Figure 10 (In the state of) or is electronic device 1A in a bent state ( Figure 11 (The state in the middle).

[0116] According to this disclosure, the first imaging unit (e.g., a camera module) of the electronic device can be arranged at a position corresponding to the inner peripheral side of the convex surface formed when the display unit is bent. This is achieved by changing the state of the electronic device 1A from... Figure 10 The state in the middle changes to Figure 11The state (bent state) captured by the camera module 3 (imaging unit 8) can be magnified in terms of the field of view. However, in the state of bending... Figure 11 The structure of electronic device 1A in the bent state exists to generate to Figure 10 The possibility of restoring the state in this situation. In this case, in order to keep the electronic device 1A in a state of... Figure 11 In this state, the user needs to hold both ends of the electronic device 1A inwards with their fingers. This can be a limitation on the use of the electronic device 1A if the user expects to continuously capture images from a magnified perspective.

[0117] Therefore, as in Figure 12 In the example, electronic device 1A can be used by mounting (attaching) it to clamp 17. In the case where electronic device 1A is operated by a rechargeable battery installed inside, clamp 17 can be a battery charger (a bracket with battery charging function). Alternatively, clamp 17 can also be a structure that does not function as an electronic device. By using clamp 17, electronic device 1A can be held in a bent state regardless of the aforementioned restoring force. Therefore, a user can capture images using electronic device 1A mounted on clamp 17 in a stationary state or capture images of a desired scene while carrying electronic device 1A mounted on clamp 17. Note that... Figure 12 The shape of the clamp 17 shown is merely an example. Therefore, clamps with the same shape as... Figure 12 The examples show clamps of different shapes. By using, for example... Figure 12 The fixture shown can perform image capture in a stable environment such as a tripod by using electronic device 1A.

[0118] There is a possibility that a component may be arranged above or near the camera module 3 of the electronic device 1A, which, by being included, narrows the viewing angle of the camera module 3 (imaging unit 8) or reduces the transmittance of light in a desired wavelength band. Examples of the former component include... Figure 1 The transistor layer 77 and wiring layer 76 are included. Furthermore, an example of the latter component includes a circular polarizing film. Therefore, in order to widen the viewing angle of the image captured by the camera module 3 (imaging unit 8) or improve the image quality of the image captured by the camera module 3 (imaging unit 8), as in... Figure 12 In the example, when the electronic device 1A is in a bent state, at least a portion of the component located on the side of the light incident direction from the camera module 3 (imaging unit 8) can be moved out of the field of view of the camera module 3 (imaging unit 8). For example, if the component above the camera module 3 is a circular polarizing film 18, as in Figure 12In one example, a cut may be formed in a portion of the circular polarizing film 18 above the camera module 3, and the circular polarizing film 18 may be separated when the electronic device 1A is in a bent state. That is, in the electronic device according to the present disclosure, when the display unit is bent, at least a portion of the components located on the light incident direction side of the first imaging unit (e.g., the camera module) may be moved out of the viewing angle of the first imaging unit.

[0119] Electronic device 1A is compatible with wide-angle image capture mode. When using wide-angle image capture mode, the user can... Figure 11 or Figure 12 While rotating the electronic device 1A around the z-axis, image capture is performed. Therefore, the user can obtain a wide-angle image from 180 degrees to 360 degrees relative to the z-axis.

[0120] When the camera module 3 performs image capture using light passing through the display unit 2, such as Figure 13 As shown in the example, the through-hole is set in a component with relatively low light transmittance in the display unit 2, and can improve the image quality of the image captured by the camera module 3.

[0121] Figure 13 This is a schematic cross-sectional view of the electronic device 1A, including the display unit 2 and the camera module 3. Figure 13 The display unit 2 in the electronic device 1 has multiple through holes 2a in the portion overlapping with the camera module 3 in the thickness direction. These through holes 2a are disposed in multiple layers with low transmittance in the display unit 2, such as touch sensors, circular polarizers 5, and display panels 4. The diameter of the through holes 2a increases from the top to the bottom. That is, the through holes 2a have a tapered shape. As described above, the display unit may have multiple tapered through holes in the portion overlapping with the first imaging unit in the thickness direction.

[0122] Because the multiple through holes 2a have a tapered shape, even when Figure 13 In the case where the display unit 2 is deformed to protrude upwards (for example, in Figure 11 and Figure 12 (Due to deformation), the bottom of through hole 2a is not closed. Therefore, by adopting Figure 13 The configuration shown improves the image quality of images captured when the electronic device 1A is in a bent state.

[0123] Figure 14 and Figure 15 The cross-sectional view shows an example of a camera module that can switch the optical system to be used based on the bending state of the electronic device.

[0124] Camera module 3A includes an imaging unit 8, a stage 25, a first lens system 26, a second lens system 27, a sliding mechanism 24, and a guide wire 19. The stage 25 is a plate-like structure formed of a material capable of transmitting electromagnetic waves (e.g., visible light) in the bands detected by the imaging unit 8. The first lens system 26 and the second lens system 27 are positioned on the upper side of the stage 25. The first lens system 26 and the second lens system 27 can be fixed to the stage 25.

[0125] The first lens system 26 is in a non-bent state when the electronic device 1A is ( Figure 10 The lens system is used when the electronic device 1A is in a bent state. However, the second lens system 27 is used when the electronic device 1A is in a bent state. Figure 11 The lens system used in the state of (the image unit 8). The first lens system 26 and the second lens system 27 are configured to focus the incident light onto the imaging unit 8.

[0126] The platform 25 can move horizontally (x-axis) via the sliding mechanism 24 located below. Figure 14 and Figure 15 The sliding mechanism 24 is formed by multiple rollers. However, sliding mechanisms with different structures or methods can be used. The platform 25 is connected to the frame of the housing of the electronic device 1A via wire 19. In the unbent state of the electronic device 1A ( Figure 10 In the current state, the conductor 19 is relaxed, causing the stage 25 to be located on the right side (positive x-axis direction), and the first lens system 26 is positioned above the imaging unit 8. Figure 14 However, in the bent state of electronic device 1A ( Figure 11 In the current state, the distance between the camera module 3A and the fixing point of the wire 19 in the frame of the housing of the fixed electronic device 1A increases. At this time, because the wire 19 is pulled through the frame of the housing of the electronic device 1A, the stage 25 moves to the left (negative x-axis direction), and the second lens system 27 is arranged above the imaging unit 8. Figure 15 ).

[0127] For example, metal wire, nylon wire, fluorocarbon wire, or PE wire can be used as conductor 19. However, the material of conductor 19 is not limited. Furthermore, strip materials, chains, etc., can be used instead of wire materials as conductor 19.

[0128] By using Figure 14 and Figure 15The camera module shown in the diagram can capture images using different lens systems when the electronic device (display unit) is deformed and when it is not deformed. In this way, the first imaging unit of the electronic device according to this disclosure can be configured such that the optical system for image capture can be switched according to the shape of the display unit. Note that the optical system for image capture in the imaging unit can be switched using a lens system that is compatible with the shape of the display unit. Figure 14 and Figure 15 The different configurations shown switch according to the shape of the electronic device (display unit). However, the electronic device according to this disclosure may not be configured to always switch the optical system for image capture in the imaging unit according to the shape of the electronic device (display unit).

[0129] In the above Figure 5 and Figure 6 An example of an electronic device equipped with a camera module (imaging unit) has already been described, which maintains its original shape regardless of deformation of the display unit (lacking significant elasticity and flexibility). However, as described below, a camera module (imaging unit) capable of changing shape according to deformation of the display unit can be used.

[0130] Figure 16 This is a cross-sectional view showing an example of a flexible camera module. Figure 16 The camera module 3B shown includes a plurality of imaging units 8a. The plurality of imaging units 8a are mounted on a support member 66. In the support member 66, at least the connection units 67 between the imaging units 8a are formed of a material with elasticity and flexibility. The support member 66 may include at least one of a wiring layer and a transistor layer (not shown). The plurality of imaging units 8a are connected to the processing circuitry 200 in a subsequent stage via at least one of the wiring layer and the transistor layer.

[0131] Additionally, a lens 64 is mounted above each imaging unit 8a. The lens 64 is supported above the imaging unit 8a, for example, by a support unit 65. The lens 64 converges the incident light onto the corresponding imaging unit 8a. Alternatively, instead of the lens 64, an optical system comprising multiple lenses may be mounted above the imaging unit 8a. The camera module 3B is fixed to the housing of the electronic device via a plate-like member 15a. The plate-like member 15a is formed of a deformable material (a material with elasticity and flexibility) that can deform according to the deformation of the electronic device (display unit 2).

[0132] Figure 17 The shape of the camera module 3B is shown when the electronic device (display unit 2) is in a bent state. For example... Figure 17As shown, according to the deformation of the electronic device (display unit 2), the connecting unit 67 of the support member 66 extends, and each imaging unit 8a is oriented in a different direction. By synthesizing the images captured by each imaging unit 8a in the processing circuit 200, an image with a wider viewing angle can be obtained compared to an image captured by a single imaging unit. As described above, in the electronic device according to the present disclosure, a second image with a wider viewing angle than the first image can be generated by synthesizing a first image captured by multiple imaging units. The electronic device according to the present disclosure may include multiple first imaging units connected by connecting units having elasticity and flexibility.

[0133] exist Figures 3 to 6 In the illustrated electronic device 1, the camera module 3 is arranged on the opposite side of the second region 22 of the display unit 2. However, in the electronic device according to this disclosure, the camera module (imaging unit) may be arranged in a different region.

[0134] Figure 18 The perspective view shows an example of the deployed state of the electronic device according to Modification 2. Furthermore, Figure 19 The cross-sectional view shows an example of the folded state of the electronic device according to Modification 2. Figure 18 and Figure 19 The illustrated electronic device 1B is a foldable electronic device similar to the described electronic device 1. However, electronic device 1B differs from electronic device 1 in the number and arrangement of its camera modules. In electronic device 1B, camera module 3A is arranged on the opposite side of the first region 21 of the display unit 2. Furthermore, in electronic device 1B, camera module 3B is arranged on the opposite side of the third region 23 of the display unit 2. That is, camera module 3A uses light passing through the first region 21 of the display unit 2 to perform image capture. Furthermore, camera module 3B uses light passing through the third region 23 of the display unit 2 to perform image capture.

[0135] In this way, in an electronic device according to the present disclosure, a first imaging unit (e.g., a camera module) may be arranged on each of the opposite sides of the display surface in the first region and the opposite sides of the display surface in the third region.

[0136] like Figure 19 As shown, when the electronic device 1B is in a folded state, camera modules 3A and 3B face opposite directions. In a subsequent stage, the images captured by camera module 3A and camera module 3B can be combined into a single image by processing circuitry 200. Therefore, an image with a wider field of view can be obtained than when using only one camera module.

[0137] Furthermore, the electronic device 1B is compatible with wide-angle image capture mode. When using wide-angle image capture mode, the user... Figure 19Image capture is performed while the electronic device 1B rotates around the z-axis. Therefore, the user can capture wide-angle images from 180 degrees to 360 degrees relative to the z-axis.

[0138] In this manner, the electronic device according to this disclosure may include: a display unit configured to be deformable; and a third imaging unit and a fourth imaging unit, respectively configured to image incident light transmitted through the display unit. For example, the camera modules 3A and 3B described above correspond to the third imaging unit and the fourth imaging unit. Here, the third imaging unit is configured to perform imaging from a third viewpoint. Furthermore, the fourth imaging unit is configured to perform imaging from a fourth viewpoint. When the display unit is in a first shape, the third and fourth viewpoints may partially overlap each other. Furthermore, when the display unit is in a second shape, the third and fourth viewpoints may not overlap each other.

[0139] Furthermore, the electronic device according to this disclosure may further include processing circuitry configured to synthesize an image obtained by a third imaging unit and an image obtained by a fourth imaging unit when the display unit is in a second shape.

[0140] Note that in Figure 18 and Figure 19 Previously, a foldable electronic device including multiple camera modules that capture images using light transmitted through a display unit has been described. However, the electronic device can be deformed in a different way. For example, multiple camera modules that capture images using light transmitted through a display unit can be mounted on a flexible electronic device.

[0141] Figure 20 This is a diagram illustrating an example of the viewing angle from which an electronic device, according to modification 3, can capture an image. Figure 20 The electronic device 1C is configured to be flexible into a convex curved shape similar to that of the electronic device 1A described above. The electronic device 1C includes camera modules 3A and 3B arranged at different positions on opposite sides of the display surface of the display unit 2. For example... Figure 20 As shown, when the electronic device 1C is in a bent state, camera module 3A and camera module 3B face different directions. In a subsequent stage, the image captured by camera module 3A and the image captured by camera module 3B can be combined into a single image by processing circuit 200. Therefore, when using electronic device 1C, an image with a wider field of view can be captured compared to using a single camera module.

[0142] Figure 21 Examples of images displayed in an electronic device according to this disclosure are shown. Figure 21 The electronic device 1 is in a folded state. Then, the electronic device 1 is mounted on a flat surface such that the second area 22 of the display unit 2 faces upward.

[0143] The output unit 38 of the processing circuit 200 can determine the content to be displayed in various areas of the display unit 2 of the electronic device 1. Examples of content include images, graphics, text, moving images, or combinations thereof. For example, in the case of an image captured by the camera module 3 (imaging unit 8), the output unit 38 of the processing circuit 200 can display the image or content generated based on that image on the display unit 2. Content generated based on the captured image can be, for example, an image with at least one of the corrected shape, size, color, and brightness, a combination of multiple images, or a graphic or moving image generated based on the image. That is, the electronic device according to this disclosure may further include processing circuitry configured to output a first image based on the captured image of the first imaging unit to the display unit.

[0144] Furthermore, when an image is captured by camera module 3 (imaging unit 8), the output unit 38 of processing circuit 200 can determine the position and extent of display unit 2 for displaying each part of the image based on the viewing angle in the image. That is, the processing circuit in this example is configured to determine the position and extent of display unit 2 for displaying each part of the first image based on the viewing angle in the captured image. For example, in Figure 21 In the example, object 28 (in the example, an object shaped like a rabbit) is positioned in the first area 21 of the display unit 2 facing the electronic device 1. Furthermore, object 29 (in the example, an object shaped like a penguin) is positioned in the third area 23 of the display unit 2 facing the electronic device 1.

[0145] In this case, the output unit 38 of the processing circuit 200 can display the portion of the object 29 shown in the image captured by the camera module 3 (imaging unit 8) on the third region 23 of the display unit 2. Similarly, the output unit 38 of the processing circuit 200 can display the portion of the object 28 shown in the image captured by the camera module 3 (imaging unit 8) on the first region 21 of the display unit 2. As described above, the output unit 38 of the processing circuit 200 can correct the shape, size, color, and brightness of the corresponding portion of the image and perform display on the region of the display unit 2. Furthermore, when a part of the object is outside the viewing angle or the image is unclear, the processing circuit 200 can supplement at least a part of the object when the object is displayed in at least one region of the display unit 2. Furthermore, when a missing part of the object occurs in the image, the processing circuit 200 can magnify to display the object so that the missing part and its vicinity are not displayed on the display unit 2.

[0146] By execution Figure 21The processing shown presents the surrounding scene or objects in a clear and easily viewable manner on the display unit 2 of the electronic device 1. Therefore, the user can use the electronic device 1 like a mirror. Note that, depending on the passage of time, the electronic device 1 can change the objects displayed in each area of ​​the display unit 2 or the content of the processing performed on the image. Therefore, when the electronic device 1 rotates around the z-axis or performs image capture while the electronic device 1 is moving, the content displayed in each area of ​​the display unit 2 of the electronic device 1 can be updated in real time.

[0147] Note that it is not limited to executable. Figure 21 The foldable electronic device 1 shown is a type of flexible electronic device 1A. For example, a bendable electronic device 1A... Figure 11 and Figure 12 Executable Figure 21 The processing.

[0148] Figure 22 An example of an object selection operation in an electronic device according to this disclosure is shown. Figure 22 The electronic device 1 is also in a folded state, and is mounted on a flat surface, so that the second area 22 of the display unit 2 faces upward. Figure 22 In the third region 23 of the display unit 2 of the electronic device 1, a portion of the viewpoint of the image captured by the camera module 3 (imaging unit 8) is displayed. Specifically, in the third region 23 of the display unit 2, a portion including the range facing the third region 23 of the display unit 2 in the viewpoint from which the image can be captured by the camera module 3 (imaging unit 8) is displayed.

[0149] exist Figure 22 In the third region 23 of the display unit 2, a door, floor, wall, and a child's face, which are objects of the camera module 3 (imaging unit 8), are displayed. As described above, the electronic device 1 can display multiple objects within a wide viewing angle on the display unit 2. The electronic device according to this disclosure can perform selective operation on at least one of the multiple objects captured by the imaging unit. For example, the processing circuit 200 can be configured to perform image recognition to detect objects in the captured image or recorded video image. In this case, the detection of objects in the captured image or recorded video image can be face recognition. Generally, as the viewing angle of the image that can be captured by the camera module 3 (imaging unit 8) increases, the number of detectable objects becomes greater.

[0150] The display unit of the electronic device according to this disclosure may include a touch panel. Furthermore, the processing circuitry may be configured to detect at least one object included in an image captured by the imaging unit.

[0151] For example, in Figure 22In this state, assume that electronic device 1 records a first video image using camera module 3 (imaging unit 8). At this time, the first video image recorded by electronic device 1 can be a video image with a viewing angle of 180 degrees to 360 degrees according to an equidistant cylindrical projection. However, the type and viewing angle of the first video image are not limited. Figure 22 In this case, the user taps on the area displaying the child's face among multiple objects displayed on the display unit 2. At this time, the processing circuit 200 detects that the user has selected the child's face as the object to be recorded. Then, the processing circuit 200 begins recording a second video image (selection of the recording object) obtained by cropping a portion of the viewpoint of the camera module 3 (imaging unit 8) that includes the area 70 containing the child's face. The processing is similar when performing continuous image capture of multiple images instead of video image recording.

[0152] In this way, electronic device 1 can record two video images in parallel. Note that recording of the second video image can begin at a time when electronic device 1 is not recording the first video image. Furthermore, processing circuitry 200 can be configured to track and record objects selected by the user. In this case, as... Figure 23 As shown, depending on the change in the relative orientation of the object, the electronic device 1 can perform image capture or recording from different perspectives in different directions over time. Therefore, for example, images of a child playing and moving can be captured or recorded. Furthermore, images of various moving objects or animals can be captured or recorded in a tracking manner. Generally, as the camera module 3 (imaging unit 8) can capture images with a wider field of view, the period for tracking the object becomes longer.

[0153] In this manner, the processing circuitry of the electronic device according to this disclosure can be configured to detect a touch on an area of ​​a displayed object on a display unit and begin recording a first video image obtained by cropping a portion of the viewing angle of a first imaging unit including that area. Furthermore, the processing circuitry can be configured to record a second video image having a larger viewing angle than the first video image in parallel with the recording of the first video image.

[0154] In the above description, an example of the operation of selectively capturing images of specific objects within the field of view of the camera module 3 (imaging unit 8) has been described. However, the electronic device according to this disclosure can selectively display specific objects within the field of view of the camera module 3 (imaging unit 8) on a display unit.

[0155] For example, as in Figure 22In the example, when a user taps on an area displaying a child's face among multiple objects displayed on display unit 2, processing circuit 200 detects that the user has selected the child's face as the object to be displayed. Then, processing circuit 200 displays an image on at least any portion of display unit 2 (the selected display object) obtained by cropping a portion of the view from camera module 3 (imaging unit 8) including the area 70 containing the child's face. The user can perform the above-described selection of the display object during the timing of recording a first video image using camera module 3 (imaging unit 8) on electronic device 1. Therefore, during the period when electronic device 1 records a wide-angle first video image, an image obtained by cropping a portion of the view from camera module 3 (imaging unit 8) can be displayed on at least any portion of display unit 2.

[0156] As described above, the processing circuit of the electronic device according to the present disclosure can be configured to detect a touch on an area of ​​a displayed object on a display unit, and output a second image obtained by cropping a portion of the viewing angle of a first imaging unit including the area to at least one portion of the display unit.

[0157] like Figure 23 As shown, similarly, when performing a selection operation on a display object, the electronic device 1 can capture images from different perspectives over time, depending on the change in the relative orientation of the object, and display the captured images in at least one area of ​​the display unit 2. Therefore, for example, the state of a child moving while playing can be continuously displayed on the display unit 2. Furthermore, images obtained by tracking various moving objects or animals can be displayed on the display unit 2. In this way, the user can continuously observe and view the state of objects displayed on the display unit 2. For example, as... Figure 23 As shown, users can enjoy the changes in the expressions of the children as subjects.

[0158] Note that the electronic device according to this disclosure can perform the above-described processes of selecting the recording object and selecting the display object in parallel. In this case, depending on the change in the relative orientation of the object, the electronic device 1 performs recording from different perspectives in different directions over time, and further displays the real-time recorded video image in at least a portion of the display unit 2.

[0159] exist Figure 22 and Figure 23 The examples have already described the cases of selecting a recording object and selecting a display object using a foldable type electronic device 1. However, electronic devices of other types besides foldable can be used. For example, a curved type electronic device 1A can be used. Figure 11 and Figure 12 This is used to perform at least one of the selection of the record object and the selection of the display object.

[0160] Figure 24 This is a cross-sectional view showing an example of an electronic device according to Modification 4. Figure 24 The electronic device 1D corresponds to the camera module 79, which is installed in conjunction with... Figures 10 to 13 The electronic device 1A shown is located on the surface opposite to the display unit 2. The imaging unit in the camera module 79 is connected, for example, to the aforementioned processing circuit 200. Since the camera module 79 is located on the surface opposite to the display unit 2, it is also referred to as a rear camera. As described above, the electronic device according to this disclosure may further include a second imaging unit arranged on the surface opposite to the display unit.

[0161] By using the camera module 79 of the electronic device 1A, an image of an object can be captured in a direction opposite to the display surface of the display unit 2, which is difficult to capture by the camera module 3. The processing circuit 200 can use the image captured by the camera module 79 to supplement defective parts or parts of degraded image quality in the image captured by the camera module 3. Furthermore, the processing circuit 200 can synthesize the image captured by the camera module 79 and the image captured by the camera module 3 to generate a single image.

[0162] As described above, by using the electronic device according to this disclosure, image capture with a wide field of view can be achieved in a compact housing. Therefore, powerful images or images including entire objects of large size can be captured without using expensive and specialized equipment, thus meeting market demands. Furthermore, in the electronic device according to this disclosure, a camera with a fisheye lens or a camera with a wide-angle lens can be mounted without increasing the size of the housing or forming protrusions within the housing.

[0163] Furthermore, in the electronic device according to this disclosure, since the imaging unit is mounted on the opposite side of the display surface of the display unit, the width of the bezel can be narrowed or a borderless width can be achieved. Therefore, a large-screen display unit can be used in the electronic device, and the user can enjoy powerful images with high resolution. In the electronic device according to this disclosure, the viewing angle of the image captured by the imaging unit can be expanded by deforming the display unit. Therefore, even if the imaging unit is mounted inside the housing, adjacent components move with the deformation of the display unit and / or the housing, making it possible to prevent adjacent components from being included during image capture.

[0164] Furthermore, in the electronic device according to this disclosure, images of multiple objects can be captured simultaneously using a wide viewing angle, and the image of a specific object can be displayed on a display unit. In this way, various experiences can be provided to the user by using the electronic device according to this disclosure.

[0165] Various alternatives can be conceived as specific candidates for electronic devices with the above configuration. For example, Figure 25 This is a plan view of the application of the electronic device according to this disclosure to a capsule endoscope 50. Figure 25 The capsule endoscope 50 includes, for example: a housing 51 with hemispherical ends and a cylindrical central portion; a camera (miniature camera) 52 for capturing images inside the body cavity; a memory 53 for recording image data captured by the camera 52; and a wireless transmitter 55 for transmitting the recorded image data to the outside via an antenna 54 after the capsule endoscope 50 is discharged outside the body of the subject.

[0166] Furthermore, a central processing unit (CPU) 56 and a coil (magnetic force / current conversion coil) 57 are disposed within the housing 51. The CPU 56 controls image capture by the camera 52 and data accumulation operations in the memory 53, and controls the wireless transmitter 55 to transmit data from the memory 53 to a data receiving device (not shown) outside the housing 51. The coil 57 provides power to the camera 52, the memory 53, the wireless transmitter 55, the antenna 54, and the light source 52b, which will be described later.

[0167] Additionally, housing 51 is provided with a magnetic (reed) switch 58, which is used to detect when capsule endoscope 50 is set as a data receiving device. CPU 56 supplies power from coil 57 to wireless transmitter 55 when reed switch 58 detects the setting of the data receiving device and data transmission becomes possible.

[0168] Camera 52 includes, for example, an imaging element 52a having an objective lens optical system 9 for capturing images within the body cavity; and a plurality of light sources 52b illuminating the body cavity. Specifically, camera 52 includes, for example, a complementary metal-oxide-semiconductor (CMOS) sensor including a light-emitting diode (LED), a charge-coupled device (CCD), etc., as light sources 52b.

[0169] The display unit 2 in the electronic device according to this disclosure includes, for example, Figure 25 The concept of the light source 52b in the image. Figure 25 The capsule endoscope 50 includes, for example, two light sources 52b, but these light sources 52b can be composed of a display panel 4 with multiple light source units or an LED module with multiple LEDs. In this case, by configuring the imaging unit 8 of the camera 52 below the display panel 4 or the LED module, the restrictions on the layout configuration of the camera 52 are reduced, and a capsule endoscope 50 with a smaller size can be achieved.

[0170] also, Figure 26This is a rear view of a digital SLR camera 60 in which the electronic device according to the present disclosure is applied. The digital SLR camera 60 and the compact camera include a display unit 2 that displays a preview image on the rear side opposite to the lens. By arranging the camera module 3 on the side opposite to the display surface of the display unit 2, an image of the face of the person capturing the image can be displayed on the display screen of the display unit 2. In the electronic device according to the present disclosure, since the camera module 3 can be arranged in the area overlapping with the display unit 2, it is unnecessary to place the camera module 3 in the frame portion of the display unit 2, and the size of the display unit 2 can be increased as much as possible.

[0171] Figure 27 This is a plan view illustrating an example of applying the electronic device according to this disclosure to a head-mounted display (HMD) 61. Figure 27 The HMD 61 is used for Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and Alternate Reality (SR), etc. For example... Figure 28 As shown, in the current HMD, camera 62 is mounted on the outer surface, and the wearer of the HMD can visually recognize the surrounding images. However, there is a problem that people around cannot recognize the wearer's eye or facial expressions.

[0172] Therefore, in Figure 27 In this design, the display surface of display unit 2 is disposed on the outer surface of HMD 61, and camera module 3 is disposed on the opposite side of the display surface of display unit 2. Therefore, the facial expressions of the wearer captured by the image captured by camera module 3 can be displayed on the display surface of display unit 2, and people around the wearer can monitor the wearer's facial expressions and eye movements in real time.

[0173] exist Figure 27 In this case, since the camera module 3 is located on the rear surface of the display unit 2, there are no restrictions on the installation position of the camera module 3, and the design flexibility of the HMD 61 can be increased. Furthermore, since the camera can be positioned optimally, problems such as misalignment of the wearer's line of sight when displayed on the surface can be prevented.

[0174] In this way, the electronic device according to this disclosure can be used for various purposes and the use value of the device can be increased.

[0175] Note that this technology can have the following configurations. (1)

[0177] An electronic device, comprising:

[0178] The display unit is configured to be deformable; and

[0179] At least one first imaging unit is arranged on the opposite side of the display surface of the display unit and configured to perform photoelectric conversion on incident light transmitted through the display unit. (2)

[0181] According to the electronic device in (1), where,

[0182] The display unit is foldable. (3)

[0184] According to the electronic device in (2), where,

[0185] The display unit includes a first area, a second area, and a third area; and

[0186] The portion between the first and second regions of the display unit, and the portion between the second and third regions of the display unit, can be bent. (4)

[0188] According to the electronic device in (3), where,

[0189] The first imaging unit is arranged on the opposite side of the display surface in the second region. (5)

[0191] According to the electronic device in (3), where,

[0192] In the display unit, the first imaging unit is respectively arranged on the opposite side of the display surface in the first region and on the opposite side of the display surface in the third region. (6)

[0194] According to the electronic device in (1), where,

[0195] At least a portion of the display unit is flexible. (7)

[0197] According to the electronic equipment in (6), where,

[0198] The first imaging unit is positioned at a location corresponding to the inner periphery of the convex surface formed when the display unit is bent. (8)

[0200] According to the electronic device of (6) or (7), wherein,

[0201] When the display unit is bent, at least a portion of the components located on the light incident direction side of the first imaging unit are moved out of the field of view of the first imaging unit. (9)

[0203] The electronic device according to any one of (1) to (8) further includes:

[0204] The second imaging unit is arranged on the surface opposite to the display unit. (10)

[0206] An electronic device according to any one of (1) to (9), wherein,

[0207] In the first imaging unit, the optical system for image capture can be switched according to the shape of the display unit. (11)

[0209] The electronic device according to any one of (1) to (10) further includes:

[0210] The processing circuit is configured to output a first image based on the captured image of the first imaging unit to the display unit. (12)

[0212] According to the electronic device of (11), wherein,

[0213] The processing circuit is configured to determine the position and extent of the display unit for displaying each part of the first image based on the viewpoint in the captured image. (13)

[0215] According to the electronic device of (11) or (12), wherein,

[0216] The display unit includes a touch panel; and

[0217] The processing circuitry is configured to detect at least one object included in the captured image. (14)

[0219] According to the electronic device in (13), where,

[0220] The processing circuit is configured to detect that an area of ​​the display object on the display unit is touched, and to output a second image obtained by cropping a portion of the viewing angle of the first imaging unit including the area to at least one part of the display unit. (15)

[0222] According to the electronic device in (13), where,

[0223] The processing circuit is configured to detect that an area of ​​the displayed object on the display unit is touched, and to begin recording a first video image obtained by cropping a portion of the viewpoint of the first imaging unit that includes that area. (16)

[0225] According to the electronic device of (15), in which,

[0226] The processing circuitry is configured to record a second video image with a wider field of view than the first video image, in parallel with the recording of the first video image. (17)

[0228] An electronic device, comprising:

[0229] The display unit is configured to be deformable; and

[0230] An imaging unit is configured to image incident light transmitted through the display unit; wherein,

[0231] The imaging unit is configured to perform imaging at a first viewing angle when the display unit is in a first shape, and to perform imaging at a second viewing angle that is wider than the first viewing angle when the display unit is in a second shape. (18)

[0233] According to the electronic device in (17), where,

[0234] The first shape is in an unfolded state, and the second shape is in a folded state. (19)

[0236] An electronic device, comprising:

[0237] The display unit is configured to be deformable; and

[0238] The third and fourth imaging units are respectively configured to image the incident light transmitted through the display unit; wherein...

[0239] The third imaging unit is configured to perform imaging from a third-person perspective;

[0240] The fourth imaging unit is configured to perform imaging from a fourth perspective;

[0241] With the display unit in its first shape, the third and fourth perspectives partially overlap; and

[0242] When the display unit is in the second shape, the third and fourth perspectives do not overlap. (20)

[0244] According to the electronic device of (19), it further includes:

[0245] The processing circuit is configured to synthesize the image obtained by the third imaging unit and the image obtained by the fourth imaging unit when the display unit is in the second shape. (twenty one)

[0247] According to any one of (3) to (5), the electronic device, wherein,

[0248] When the display unit is folded, the first area and the second area of ​​the display unit face opposite directions. (twenty two)

[0250] According to the electronic device in (6), where,

[0251] The display unit can be bent into a convex curved shape. (twenty three)

[0253] An electronic device according to any one of (1) to (16), wherein,

[0254] The display unit has multiple tapered through holes in the portion that overlaps with the first imaging unit in the thickness direction. (twenty four)

[0256] The electronic device according to any one of (1) to (16) further includes:

[0257] Multiple first imaging units are connected by flexible and elastic connection units.

[0258] This disclosure is not limited to the embodiments described above, but includes various modifications that can be conceived by those skilled in the art, and the effects of this disclosure are not limited to the foregoing. That is, various additions, modifications, and partial deletions can be made without departing from the conceptual idea and spirit of this disclosure as defined in the claims and their equivalents.

[0259] Reference tag list

[0260] 1. Electronic devices, 1A, 1B

[0261] 2 Display Units

[0262] 3, 3A, 3B, 79 camera modules

[0263] 4 Display Panel

[0264] 7. Protective layer

[0265] 8 imaging units

[0266] 10 Folding Mechanism

[0267] 14 Sensors

[0268] 17 Fixtures

[0269] 21 First District

[0270] 22 Second Region

[0271] 23 Third Region

[0272] 41 storage units

[0273] 200 Processing Circuit.

Claims

1. An electronic device, comprising: The display unit is configured to be foldable; as well as At least one first imaging unit is arranged on opposite sides of the display surface of the display unit, wherein, At least one of the first imaging units is configured to capture an image of light passing through the display unit; Sensors are configured to detect the folded and unfolded states of the display unit; and The processing circuitry is configured to perform cropping of a region from the captured image based on detection results from the sensor, wherein, The region corresponds to the effective field of view of at least one of the first imaging units.

2. The electronic device according to claim 1, wherein, The display unit includes a first region, a second region, and a third region; and The portion of the display unit between the first region and the second region, and the portion of the display unit between the second region and the third region, can be folded.

3. The electronic device according to claim 2, wherein, The first imaging unit is arranged on the opposite side of the display surface in the second region.

4. The electronic device according to claim 2, wherein, In the display unit, the first imaging unit is respectively arranged on the opposite side of the display surface in the first region and on the opposite side of the display surface in the third region.

5. The electronic device according to claim 1, wherein, At least a portion of the display unit is flexible.

6. The electronic device according to claim 5, wherein, The first imaging unit is positioned at a location corresponding to the inner peripheral side of the convex surface formed when the display unit is bent.

7. The electronic device according to claim 5, wherein, When the display unit is bent, at least a portion of the components located on the light incident direction side of the first imaging unit are moved out of the field of view of the first imaging unit.

8. The electronic device according to claim 1, further comprising: The second imaging unit is arranged on the surface on the side opposite to the display unit.

9. The electronic device according to claim 1, wherein, In the first imaging unit, the optical system for image capture can switch according to the shape of the display unit.

10. The electronic device according to claim 1, wherein, The processing circuit is also configured to output a first image based on the captured image of the first imaging unit to the display unit.

11. The electronic device according to claim 10, wherein, The processing circuit is also configured to determine the position and extent of the display unit for displaying each portion of the first image based on the viewpoint in the captured image.

12. The electronic device according to claim 10, wherein, The display unit includes a touch panel; and The processing circuitry is also configured to detect at least one object included in the captured image.

13. The electronic device according to claim 12, wherein, The processing circuit is further configured to detect that an area of ​​the displayed object on the display unit is touched, and to output a second image obtained by cropping a portion of the viewpoint of the first imaging unit including the area to at least one portion of the display unit.

14. The electronic device according to claim 12, wherein, The processing circuit is also configured to detect that an area of ​​the displayed object on the display unit is touched, and to begin recording a first video image obtained by cropping a portion of the viewpoint of the first imaging unit including the area.

15. The electronic device according to claim 14, wherein, The processing circuit is also configured to record a second video image with a wider field of view than the first video image, in parallel with the recording of the first video image.

16. An electronic device comprising: The display unit is configured to be foldable; A sensor is configured to detect a first shape and a second shape of the display unit; The imaging unit is configured as follows: Capture an image of the light passing through the display unit. Imaging is performed from a first perspective when the display unit is in the first shape, and Imaging is performed at a second viewing angle that is wider than the first viewing angle when the display unit is in the second shape; as well as The processing circuitry is configured to perform cropping of a region from the captured image based on detection results from the sensor, wherein, The region corresponds to the effective field of view of the imaging unit.

17. The electronic device according to claim 16, wherein, The first shape is in an unfolded state, and the second shape is in a folded state.

18. An electronic device comprising: The display unit is configured to be foldable; as well as The first imaging unit and the second imaging unit are respectively configured to capture images of light passing through the display unit; wherein... The first imaging unit is configured to perform imaging from a first viewpoint; The second imaging unit is configured to perform imaging from a second perspective; When the display unit is in the first shape, the first viewing angle and the second viewing angle partially overlap each other; The first shape is in an unfolded state; When the display unit is in the second shape, the first viewing angle and the second viewing angle do not overlap; and The second shape is in a folded state.

19. The electronic device of claim 18, further comprising: The processing circuit is configured to synthesize the image obtained by the first imaging unit and the image obtained by the second imaging unit when the display unit is in the second shape.

Images