Electronic device including multiple cameras

Abstract

An electronic device is provided. The electronic device includes a display panel and a first camera and a second camera disposed below the display panel. The display panel includes an emission layer including a plurality of pixels and a patterning layer disposed between the emission layer and the first camera or the second camera. The patterning layer includes a first patterning portion disposed on a front surface of a first lens of the first camera and a second patterning portion disposed on a front surface of a second lens of the second camera and having a different shape from the first patterning portion. Furthermore, various other embodiments that can be contemplated by the specification can be allowed.

Description

Technical Field

[0001] This disclosure relates to multiple cameras and electronic devices including such cameras. Background Technology

[0002] Smartphones or electronic devices such as tablet PCs may include cameras (or camera modules or camera devices) and can take photos or videos.

[0003] Currently, with the expansion of the effective area of ​​displays, full-frontal displays are emerging that realize a large portion of one surface (e.g., the front surface) of an electronic device as the effective area of ​​the display. In this case, a camera module in the first surface (e.g., the front surface) can be arranged below the display panel (under display camera, UDC). The electronic device can acquire image data using light passing through the display panel and the lens of the camera module.

[0004] The above information is presented as background information only to aid in understanding this disclosure. No determination or assertion is made regarding whether any of the above can be applied to this disclosure as prior art. Summary of the Invention

[0005] [Technical Issues]

[0006] When electronic devices include a down-display camera (UDC), image quality degradation can occur due to the characteristics of the pattern in the display panel. For example, diffraction or scattering of light caused by the pattern in the display panel may cause attenuation of components in certain frequency bands, which can lead to reduced resolution. Alternatively, when photographing a light source, light flares as light bursts may be lost. Electronic devices can compensate for the image quality of the down-display camera (UDC) using software methods such as signal processing or deep learning. In this case, when the signal attenuation is high, it may be difficult to recover the lost frequency components, and the degradation in image quality may not be fully compensated.

[0007] [Technical Solution]

[0008] The aspects of this disclosure at least address the aforementioned problems and / or disadvantages, and provide at least the following advantages. Therefore, one aspect of this disclosure is to provide an electronic device for acquiring and / or storing images of objects using patterns on display panels, each corresponding to one of the cameras included in the electronic device. Furthermore, another aspect of this disclosure is to provide a method for processing (e.g., synthesizing or combining) images in an electronic device (e.g., an image processor), and an electronic device for performing this method.

[0009] According to one aspect of this disclosure, an electronic device is provided. The electronic device includes a display panel and a first camera module and a second camera module disposed below the display panel. The display panel includes an emitting layer and a patterned layer. The emitting layer includes a plurality of pixels, and the patterned layer is disposed between the emitting layer and the first camera module or the second camera module. The patterned layer includes a first patterned portion disposed on the front surface of a first lens of the first camera module and a second patterned portion disposed on the front surface of a second lens of the second camera module and having a different shape from the first patterned portion.

[0010] According to another aspect of this disclosure, an electronic device is provided. The electronic device includes a display panel and a plurality of camera modules disposed beneath the display panel. The display panel includes an emitting layer and a patterning layer. The emitting layer includes a plurality of pixels, and the patterning layer is disposed between the emitting layer and the plurality of camera modules. The patterning layer includes different patterned portions on the front surfaces of lenses included in the plurality of camera modules.

[0011] According to another aspect of this disclosure, an image processing method performed in an electronic device is provided. The image processing method includes: acquiring first image data via a first camera module of the electronic device, and acquiring second image data via a second camera module of the electronic device; converting the first image data by reflecting a first characteristic of a first lens of the first camera module, and converting the second image data by reflecting a second characteristic of a second lens of the second camera module; and generating an image by combining the converted first image data and the converted second image data. The first camera module and the second camera module are arranged below a display panel of the electronic device, and different patterns are formed on the front surfaces of the first lens and the second lens.

[0012] [Beneficial Effects]

[0013] Electronic devices according to various embodiments of the present disclosure can compensate for the degradation of image quality of the display camera by using different patterns of a patterned layer arranged on the front surface of the lenses of multiple camera modules.

[0014] Furthermore, electronic devices according to various embodiments of this disclosure can compensate for image quality degradation caused by diffraction or scattering of light passing through openings in the display panel.

[0015] Furthermore, electronic devices according to various embodiments of this disclosure can display light burst effects of light sources without distortion by using multiple camera modules. Attached Figure Description

[0016] Figure 1 This is a block diagram of an electronic device in a network environment according to embodiments of the present disclosure;

[0017] Figure 2This is a block diagram illustrating a camera module according to an embodiment of the present disclosure;

[0018] Figure 3 An electronic device according to an embodiment of the present disclosure is illustrated;

[0019] Figure 4 This is a cross-sectional view illustrating a display panel and a camera module according to an embodiment of the present disclosure;

[0020] Figure 5 The illustration shows a pixel structure and a first type pattern arranged adjacent to a first camera module according to an embodiment of the present disclosure;

[0021] Figure 6 The illustration shows a pixel structure and a second type pattern arranged adjacent to a second camera module according to an embodiment of the present disclosure;

[0022] Figure 7 The illustration shows a configuration of an image processor according to an embodiment of the present disclosure;

[0023] Figure 8 The illustration shows a variation in the modulation transfer function (MTF) characteristics of a display camera according to an embodiment of the present disclosure;

[0024] Figure 9 The illustration shows compensation for image quality degradation through image combination according to embodiments of the present disclosure; and

[0025] Figure 10 The illustration shows the light burst effect of a light source according to an embodiment of the present disclosure. Detailed Implementation

[0026] The following description, provided with reference to the accompanying drawings, is intended to aid in a full understanding of the various embodiments of this disclosure as defined by the claims and their equivalents. It includes various specific details to aid understanding, but these are to be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of this disclosure. Furthermore, for clarity and brevity, descriptions of well-known functions and structures may be omitted.

[0027] The terms and words used in the following description and claims are not limited to their documentary meaning, but are used by the inventors only to enable a clear and consistent understanding of this disclosure. Therefore, it will be apparent to those skilled in the art that the following description of various embodiments of this disclosure is provided for illustrative purposes only and is not intended to limit the purpose of this disclosure as defined by the appended claims and their equivalents.

[0028] It should be understood that the singular forms “a,” “an,” and “the” include the plural they refer to, unless the context clearly specifies otherwise. Thus, for example, the reference to “component surface” includes one or more such surfaces.

[0029] Figure 1 This is a block diagram of an electronic device in a network environment according to embodiments of the present disclosure.

[0030] Electronic devices according to the various embodiments disclosed in this disclosure can be of various types. Electronic devices may include at least one of the following: portable communication devices (e.g., smartphones, computer devices (e.g., personal digital assistants (PDAs)), tablet PCs, laptop PCs, desktop PCs, workstations, or servers), portable multimedia devices (e.g., e-book readers or Moving Image Experts Group (MPEG-1 or MPEG-2) Audio Layer 3 (MP3) players), portable medical devices (e.g., heart rate, blood glucose, blood pressure, or body temperature measuring devices), cameras, or wearable devices. Wearable devices may include at least one of the following: accessory-type devices (e.g., watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted wearable devices (HMDs)), fabric or clothing integrated devices (e.g., electronic clothing), body attachment devices (e.g., skin pads or tattoos), or bio-implantable circuitry. In some embodiments, the electronic device may include at least one of the following: a television, a digital video disc (DVD) player, an audio device, an audio accessory device (e.g., a speaker, headphones, or headset), a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a game console, an electronic dictionary, an electronic key, a portable camera, or an electronic photo frame.

[0031] In another embodiment, the electronic device may include at least one of the following: a navigation device, a Global Navigation Satellite System (GNSS), an Event Data Recorder (EDR) (e.g., a black box for a vehicle / ship / aircraft), an automotive infotainment device (e.g., a vehicle head-up display), an industrial or household robot, a drone, an ATM, a point-of-sale (POS) instrument, a measuring instrument (e.g., a water, electricity, or gas measuring device), or an Internet of Things (IoT) device (e.g., a light bulb, sprinkler system, fire alarm, temperature regulator, or streetlight). The electronic device according to embodiments of this disclosure is not limited to the devices described above. Furthermore, for example, such as in a smartphone equipped with measurements of an individual's biometric information (e.g., heart rate or blood glucose), the electronic device may have a combination of the functions of multiple devices. In this disclosure, the term "user" may refer to a person using the electronic device or a device using the electronic device (e.g., an artificial intelligence electronic device).

[0032] refer to Figure 1 In network environment 100, electronic device 101 can communicate with electronic device 102 via a first network 198 (e.g., a short-range wireless communication network), or with electronic device 104 or server 108 via a second network 199 (e.g., a long-range wireless communication network). According to an embodiment, electronic device 101 can communicate with electronic device 104 via server 108. According to an embodiment, electronic device 101 may include a processor 120, memory 130, input device 150, sound output device 155, display device 160, audio module 170, sensor module 176, interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module (SIM) 196, or antenna module 197. In some embodiments, at least one component (e.g., display device 160 or camera module 180) may be omitted from electronic device 101, or one or more other components may be added to electronic device 101. In some embodiments, some components may be implemented as a single integrated circuit. For example, the sensor module 176 (e.g., a fingerprint sensor, an iris sensor, or an illuminance sensor) can be implemented as an embedded component in the display device 160 (e.g., a display).

[0033] Processor 120 can execute, for example, software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of electronic device 101 coupled to processor 120, and can perform various data processing or calculations. According to one embodiment, as at least part of data processing or calculation, processor 120 can load commands or data received from another component (e.g., sensor module 176 or communication module 190) into volatile memory 132, process the commands or data stored in volatile memory 132, and store the resulting data in non-volatile memory 134. According to one embodiment, processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or application processor (AP)) and an auxiliary processor 123 (e.g., a graphics processing unit (GPU), image signal processor (ISP), sensor hub processor, or communication processor (CP)), the auxiliary processor 123 operating independently of or in conjunction with the main processor 121. Additionally or alternatively, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or dedicated to a specific function. The auxiliary processor 123 may be implemented independently of the main processor 121 or as part of the main processor 121.

[0034] When the main processor 121 is inactive (e.g., in sleep) state, the auxiliary processor 123 may replace the main processor 121, or when the main processor 121 is active (e.g., executing an application), the auxiliary processor 123 may work with the main processor 121 to control at least some functions or states associated with at least one component of the electronic device 101 (e.g., display device 160, sensor module 176, or communication module 190). According to embodiments, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 180 or communication module 190) functionally associated with the auxiliary processor 123.

[0035] Memory 130 may store various data used by at least one component of electronic device 101 (e.g., processor 120 or sensor module 176). The various data may include, for example, input or output data of software (e.g., program 140) and associated commands. Memory 130 may include volatile memory 132 or non-volatile memory 134.

[0036] Program 140 may be stored as software in memory 130 and may include, for example, an operating system (OS) 142, middleware 144, or application 146.

[0037] Input device 150 can receive commands or data from outside electronic device 101 (e.g., a user) that will be used by other components of electronic device 101 (e.g., processor 120). Input device 150 may include, for example, a microphone, mouse, keyboard, or digital pen (e.g., stylus).

[0038] The sound output device 155 can output sound signals to the outside of the electronic device 101. The sound output device 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as playing multimedia or playing records, while the receiver can be used for incoming calls. According to embodiments, the receiver can be implemented independently of the speaker or as part of the speaker.

[0039] Display device 160 can visually provide information to the outside of electronic device 101 (e.g., to a user). Display device 160 may include, for example, a display, a holographic device, or a projector, and control circuitry that controls a corresponding one of the display, holographic device, and projector. According to an embodiment, display device 160 may include touch circuitry adapted to detect a touch, or sensor circuitry (e.g., a pressure sensor) adapted to measure the intensity of the force caused by a touch.

[0040] Audio module 170 can convert sound into electrical signals and vice versa. According to an embodiment, audio module 170 can obtain sound via input device 150, or output sound via sound output device 155 or via headphones of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly coupled to electronic device 101.

[0041] Sensor module 176 can detect the operating state of electronic device 101 (e.g., power or temperature) or the environmental state outside electronic device 101 (e.g., user state), and then generate an electrical signal or data value corresponding to the detected state. According to embodiments, sensor module 176 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0042] Interface 177 may support one or more specified protocols for coupling electronic device 101 directly (e.g., wired) or wirelessly to external electronic device (e.g., electronic device 102). According to embodiments, interface 177 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0043] Connection terminal 178 may include a connector via which electronic device 101 can be physically connected to an external electronic device (e.g., electronic device 102). According to embodiments, connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0044] The tactile module 179 can convert electrical signals into mechanical stimuli (e.g., vibration or motion) or electrical stimuli, which a user can identify through their tactile or kinesthetic senses. According to embodiments, the tactile module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0045] Camera module 180 can capture still or moving images. According to an embodiment, camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0046] The power management module 188 can manage the power supplied to the electronic device 101. According to one embodiment, the power management module 188 can be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0047] Battery 189 can supply power to at least one component of electronic device 101. According to embodiments, battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0048] Communication module 190 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 101 and external electronic devices (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. Communication module 190 may include one or more communication processors that operate independently of processor 120 (e.g., application processor (AP)) and support direct (e.g., wired) or wireless communication. According to embodiments, communication module 190 may include wireless communication module 192 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 194 (e.g., local area network (LAN) communication module or power line communication (PLC) module). A corresponding one of these communication modules can communicate via a first network 198 (e.g., such as Bluetooth). TM The communication module 192 communicates with external electronic devices via a short-range communication network such as Wi-Fi Direct or Infrared Data Association (IrDA) or a second network 199 (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN)) . These various types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components that are separate from each other (e.g., multiple chips). The wireless communication module 192 can use subscriber information (e.g., International Mobile Subscriber Identity (IMSI)) stored in the subscriber identification module 196 to identify and authenticate electronic devices 101 in communication networks such as the first network 198 or the second network 199.

[0049] Antenna module 197 can transmit signals or power to or from the outside of electronic device 101 (e.g., external electronic device). According to an embodiment, antenna module 197 may include an antenna comprising a radiating element made of conductive material or conductive patterns formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, antenna module 197 may include multiple antennas. In this case, for example, communication module 190 (e.g., wireless communication module 192) may select at least one antenna suitable for a communication scheme used in a communication network such as a first network 198 or a second network 199 from among the multiple antennas. Signals or power can then be transmitted or received between communication module 190 and external electronic device via the selected at least one antenna. According to an embodiment, another component besides the radiating element (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of antenna module 197.

[0050] At least some of the aforementioned components can be coupled to each other and transmit signals (e.g., commands or data) therebetween via peripheral communication schemes (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)).

[0051] According to an embodiment, commands or data can be sent or received between electronic device 101 and external electronic device 104 via server 108 coupled to a second network 199. Each of electronic devices 102 and 104 can be a device of the same or different type as electronic device 101. According to an embodiment, all or some operations to be performed at electronic device 101 can be performed at one or more external electronic devices 102, 104, or 108. For example, if electronic device 101 is required to perform a function or service automatically or in response to a request from a user or another device, instead of performing that function or service, or in addition to performing that function or service, electronic device 101 can request one or more external electronic devices to perform at least a portion of that function or service. The one or more external electronic devices receiving the request can perform at least a portion of the requested function or service, or additional functions or services associated with the request, and transmit the result of the performance to electronic device 101. Electronic device 101 can provide a result, with or without further processing of the result, as at least part of a response to the request. For this purpose, cloud computing, distributed computing, or client-server computing technologies can be used, for example.

[0052] The electronic device according to various embodiments can be one of a variety of types of electronic devices. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer equipment, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. According to embodiments of this disclosure, the electronic device is not limited to those described above.

[0053] Figure 2 This is a block diagram illustrating a camera module according to an embodiment of the present disclosure.

[0054] The reference is to the block diagram 200. Figure 2 Camera module 180 may include lens assembly 210, flash 220, image sensor 230, image stabilizer 240, memory 250 (e.g., buffer memory), or image signal processor 260. Lens assembly 210 may collect light emitted or reflected from an object from which an image is to be captured. Lens assembly 210 may include one or more lenses. According to embodiments, camera module 180 may include multiple lens assemblies 210. In this case, camera module 180 may form, for example, a dual-camera, a 360-degree camera, or a spherical camera. Some of the multiple lens assemblies 210 may have the same lens properties (e.g., angle of view, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties different from another lens assembly. Lens assembly 210 may include, for example, a wide-angle lens or a telephoto lens.

[0055] Flash 220 can emit light to enhance light reflected from an object. According to embodiments, flash 220 may include one or more light-emitting diodes (LEDs) (e.g., red-green-blue (RGB) LEDs, white LEDs, infrared (IR) LEDs, or ultraviolet (UV) LEDs) or xenon lamps. Image sensor 230 can obtain an image corresponding to an object by converting light emitted or reflected from the object and transmitted via lens assembly 210 into an electrical signal. According to embodiments, image sensor 230 may include one selected from image sensors with different properties, such as an RGB sensor, a black-and-white (BW) sensor, an IR sensor, or a UV sensor, multiple image sensors having the same properties, or multiple image sensors with different properties. Each image sensor included in image sensor 230 may be implemented using, for example, a charge-coupled device (CCD) sensor or a complementary metal-oxide-semiconductor (CMOS) sensor.

[0056] Image stabilizer 240 can move image sensor 230 or at least one lens included in lens assembly 210 in a specific direction, or control the operating properties of image sensor 230 (e.g., adjust readout timing) in response to movement of camera module 180 or electronics 101 including camera module 180. This allows compensation for at least some of the negative effects (e.g., image blur) caused by movement on the image being captured. According to embodiments, image stabilizer 240 can use a gyroscope sensor (not shown) or accelerometer sensor (not shown) disposed inside or outside camera module 180 to sense such movement of camera module 180 or electronics 101. According to embodiments, image stabilizer 240 can be implemented as, for example, an optical image stabilizer. Memory 250 can at least temporarily store at least a portion of the images acquired via image sensor 230 for subsequent image processing tasks. For example, if image capture is delayed due to shutter lag or multiple images are captured rapidly, the acquired original image (e.g., Bayer pattern image, high-resolution image) can be stored in memory 250, and its corresponding copy image (e.g., low-resolution image) can be previewed via display device 160. Subsequently, if specified conditions are met (e.g., by user input or system command), at least a portion of the original image stored in memory 250 can be obtained and processed, for example, by image signal processor 260. According to embodiments, memory 250 may be configured as at least a portion of memory 130, or as a separate memory operating independently of memory 130.

[0057] Image signal processor 260 can perform one or more image processing operations on images acquired via image sensor 230 or stored in memory 250. One or more image processing operations may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image compositing, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, image signal processor 260 can perform control (e.g., exposure time control or readout timing control) on at least one of the components included in camera module 180 (e.g., image sensor 230). Images processed by image signal processor 260 can be stored back in memory 250 for further processing, or they can be provided to external components outside camera module 180 (e.g., memory 130, display device 160, electronics 102, electronics 104, or server 108). According to embodiments, image signal processor 260 can be configured as at least a part of processor 120, or as a separate processor operating independently of processor 120. If the image signal processor 260 is configured as a processor separate from the processor 120, at least one image processed by the image signal processor 260 can be displayed as is via the display device 160 through the processor 120, or displayed after further processing.

[0058] According to an embodiment, the electronic device 101 may include a plurality of camera modules 180 with different attributes or functions. In this case, at least one of the plurality of camera modules 180 may form, for example, a wide-angle camera, and at least another of the plurality of camera modules 180 may form a telephoto camera. Similarly, at least one of the plurality of camera modules 180 may form, for example, a front-facing camera, and at least another of the plurality of camera modules 180 may form a rear-facing camera. Furthermore, the plurality of camera modules 180 may include at least one of a wide-angle camera, a telephoto camera, a color camera, a monochrome camera, or an infrared (IR) camera (e.g., a time-of-flight (TOF) camera, a structured light camera). According to an embodiment, the IR camera may serve as a sensor module (e.g., Figure 1 The sensor module 176) operates as at least a part of the sensor module. For example, a TOF camera can be used as a sensor module for detecting the distance to an object (e.g., Figure 1 At least a portion of the sensor module 176) is used to operate.

[0059] Figure 3 An electronic device according to an embodiment of the present disclosure is illustrated.

[0060] refer to Figure 3 Electronic device 301 may include a main body 310 and a display 320 (e.g., Figure 1 The display device 160), the first camera module 330 (e.g., Figure 1 or Figure 2 The camera module 180) and the second camera module 340 (e.g., Figure 1 or Figure 2 Camera module 180). Although Figure 3 An example of an electronic device 301 including two camera modules is illustrated, but this disclosure is not limited thereto.

[0061] According to various embodiments, the body (or housing) 310 may include various components required for the operation of the electronic device 301. For example, the body 310 may include various components such as circuit boards (e.g., printed circuit boards (PCBs), flexible PCBs, or rigid-flex PCBs), processors (e.g., ... Figure 1 The processor 120), memory (e.g., Figure 1 The memory 130) or communication module (e.g., Figure 1 (Communication module 190).

[0062] According to various embodiments, the display 320 may be arranged on a first surface (e.g., the front surface) of the body 310, and the first camera module 330 and the second camera module 340 may be arranged to face the first surface. For example, the first camera module 330 and the second camera module 340 may not be visually exposed and may be under-display cameras (UDCs).

[0063] although Figure 3 An example is illustrated where the first camera module 330 and the second camera module 340 are arranged facing a first surface of the body 310 (e.g., the front surface, or the surface on which the display 320 is primarily arranged), but this disclosure is not limited thereto. For example, when the display 320 extends to a second surface of the body 310 (e.g., the rear surface), the first camera module 330 and the second camera module 340 may be arranged facing the second surface of the body 310 (e.g., the rear surface).

[0064] The display 320 can display various content, such as text or images. The display 320 can be implemented using multiple layers. For example, the display 320 can have a structure in which window layers, touchscreen panels, display panels, and / or protective layers are stacked sequentially (see reference). Figure 4 ).

[0065] Display 320 allows external light to pass through at least a portion of the area where the first camera module 330 and the second camera module 340 are arranged. For example, display 320 allows external light to pass through the spaces between pixels included in display 320. The first camera module 330 and the second camera module 340 can use the external light incident through display 320 to capture images.

[0066] The first camera module (or first camera device) 330 and the second camera module (or second camera device) 340 can be arranged facing the first surface of the subject 310. The first camera module 330 and the second camera module 340 can be arranged on the rear surface (opposite surface to the surface displaying the image) of the display panel included in the display 320, and can be hidden so that they are not visible to the user from the outside.

[0067] According to an embodiment, the first camera module 330 and the second camera module 340 may be devices of the same type. For example, the first camera module 330 and the second camera module 340 may be devices having substantially the same modulation transfer function (MTF) characteristics. According to another embodiment, the first camera module 330 and the second camera module 340 may be devices of different types. For example, the first camera module 330 may be a camera including a wide-angle lens, while the second camera module 340 may be a camera including a telephoto lens.

[0068] According to an embodiment, the first camera module 330 and the second camera module 340 can be arranged in an area where at least a portion of the layers included in the display 320 have been removed. For example, layers that external light cannot penetrate (e.g., shielding layers) can be removed, and the lenses of the first camera module 330 and the second camera module 340 (e.g., lenses) can be placed within the display 320. Figure 4 Lenses 331 and 341 are arranged in the area where the layer has been removed.

[0069] According to various embodiments, metal layers with different patterns (hereinafter referred to as patterned layers) can be arranged in the first lens of the first camera module 330 (e.g., Figure 4 The first lens 331) and the second lens of the second camera module 340 (e.g., Figure 4 On the front surface of lens 341. For example, a patterned layer (e.g., Figure 4 The patterning layer 410 can be a layer constituting the display 320, and can be arranged between the pixels of the display panel and the lens of the camera module (see reference). Figure 5 and Figure 6 ).

[0070] Figure 4 This is a cross-sectional view illustrating a display panel and a camera module according to an embodiment of the present disclosure. Figure 4 The camera module can be Figure 3 One of the first camera module 330 or the second camera module 340. Figure 4 This is illustrative, and this disclosure is not limited thereto.

[0071] Reference Figure 4 Display (e.g., Figure 3The display (320) may include a window layer 401, a display panel 403, and a protective layer (or, shielding layer or back cover) 480.

[0072] According to various embodiments, the window layer (e.g., ultra-thin glass (UTG)) 401 may comprise a polymer. In this case, the window layer 401 may comprise polyethylene terephthalate (PET) or polyimide (PI). In various embodiments, the window layer 401 may comprise multiple window layers.

[0073] Despite Figure 4 Not shown in the figure, the display 320 may also include a touch panel (touch sensor) between the window layer 401 and the display panel 403.

[0074] According to various embodiments, electronic device 301 may further include components (e.g., a digitizer) disposed below display 320 and capable of recognizing handwritten input. For example, electronic device 301 may include coil components disposed on a dielectric substrate to detect resonant frequencies of the electromagnetic induction type applied from an electronic pen. According to embodiments, display 320 may include control circuitry (not shown). For example, control circuitry (not shown) may include display driver ICs (DDI) and / or touch display driver ICs (TDDI) disposed in a chip-on-board (COP) or chip-on-film (COF) type configuration.

[0075] According to various embodiments, electronic device 301 may include a plurality of displays (e.g., a first display and a second display), and at least one of the plurality of displays may include flexible features. For example, the first display (e.g., display 320) may include an external touch AMOLED (OCTA) display, while the second display (e.g., a flexible display) may include a shatterproof (UB) type active-matrix organic light-emitting diode (OLED) display.

[0076] The display panel 403 may include a substrate layer 405, a patterning layer 410, a wiring layer 420, an emissive layer (or organic layer) 430, and an encapsulation layer (or protective layer) 440. Although Figure 4 Not shown in the figure, but the display panel 403 may also include a polarizer (e.g., a polarizing film), an adhesive layer, and a touch panel. For example, the adhesive layer may be arranged between the layers as an adhesive component (e.g., optically clear adhesive (OCA) or pressure-sensitive adhesive (PSA)).

[0077] According to various embodiments, a base layer 405 may be formed beneath a patterned layer 410 (e.g., in a direction toward camera module 330 or 340). For example, the patterned layer 410, wiring layer 420, and emission layer 430 may be stacked on the base layer 405. According to various embodiments, the base layer 405 may include a transparent insulating substrate (e.g., a substrate). For example, the base layer 405 may be implemented using a glass substrate, a quartz substrate, or a transparent resin substrate. For example, the transparent resin substrate may comprise a polyimide-based resin, an acrylic-based resin, a polyacrylate-based resin, a polycarbonate-based resin, a polyether-based resin, a sulfonic acid-based resin, and / or a polyethylene terephthalate-based resin.

[0078] A patterned layer (or bottom metal layer (BML)) 410 may be patterned in an area where at least a portion of the protective layer (or shielding layer or back cover) 480 is removed for mounting the camera module 330 or 340. The patterned layer 410 may include a blocking portion (or blocking area) 415 and an opening (or open area) 416. The blocking portion 415 may be an area at least partially corresponding to a pixel 431 of the emitting layer (organic layer) 430, while the opening 416 may be an area at least partially corresponding to a panel opening in the pixel 431 of the emitting layer (organic layer) 430. According to an embodiment, the patterned layer 410 may be formed of a metallic material and may be formed beneath the wiring layer 420 by deposition and / or patterning. The patterned layer 410 may protect the pixels 431 of the emitting layer (organic layer) 430 and may block light generated from the pixels 431. According to various embodiments, the patterned layer 410 may include a specified pattern (black matrix) for reducing the diffraction of light incident on the camera module 330 or 340, or an opaque metal layer (e.g., a buffer layer) including the specified pattern.

[0079] According to various embodiments, external light passing through opening 416 can be incident on lens 331 or 341. For example, depending on the shape or size of opening 416, the light may be diffracted or scattered, and image data distortion or image quality degradation may occur depending on the characteristics of the point spread function (PSF). The processor (or image processor) in electronic device 301 can generate an image that compensates for the image quality degradation by comparing or combining image data obtained by the first camera module 330 and the second camera module 340 (see reference). Figure 5 (and the following attached figures).

[0080] Wiring layer 420 and emitting layer 430 may have the form of depositing light-emitting elements (e.g., organic electroluminescent (EL)) on a thin-film transistor (TFT) substrate. According to various embodiments, emitting layer 430 may include pixels 431, each pixel being composed of multiple sub-pixels (e.g., red, green, and blue sub-pixels). According to various embodiments, display panel 403 may include active areas (e.g., display areas) and inactive areas (e.g., non-display areas). For example, the active area may be an area corresponding to an area where multiple pixels 431 are arranged, while the inactive area may be an area arranged around the active area and corresponding to a border area of ​​display panel 403.

[0081] According to various embodiments, the wiring layer 420 may include TFT elements, metal wiring, insulating films, etc., for driving the respective pixels 431 in the effective area. According to various embodiments, the wiring layer 420 may include liquid crystal polymer or low-temperature polycrystalline silicon (LTPS) glass, and the plurality of pixels 431 may include thin-film transistors (TFTs) formed on the LTPS glass.

[0082] According to various embodiments, the emitting layer 430 may include a light-emitting element (e.g., an organic electroluminescent (EL)). When holes and electrons are injected from the cathode and anode into the organic EL, the organic EL can generate light.

[0083] According to various embodiments, when viewing the display panel 403 from a first surface (e.g., the front surface), at least one component of the display panel 403 and the electronic device 301 (e.g., lens 331 or 341 or sensor module (e.g., ...) Figure 1 In the sensor module 176) at least partially overlapping areas, multiple pixels 431 may not be arranged, or multiple pixels 431 may be arranged with a lower placement density than in non-overlapping areas.

[0084] According to various embodiments, the encapsulation layer 440 (e.g., a thin-film encapsulation (TFE)) may include alternating organic and inorganic layers laminated on the emitting layer 430 to protect the light-emitting element from oxygen or moisture. For example, the encapsulation layer 440 may be a pixel protection layer for protecting a plurality of pixels 431. For example, the encapsulation layer 440 may include encapsulation glass.

[0085] According to various embodiments, a protective layer (or shielding layer) 480 may support and protect the display panel 403. The protective layer 480 may block light or electromagnetic waves introduced from the display panel 403 from entering the electronic device 301. The protective layer 480 may include a black film and a metal (e.g., copper) plate. For example, the protective layer 480 disposed beneath the display panel 403 may provide a dark background to ensure the visibility of the display panel 403 and may be formed of shock-absorbing members (e.g., shock-absorbing pads) for cushioning. For example, the protective layer 480 may include an opaque metal layer (e.g., a black layer including an uneven pattern) to remove air bubbles that may form between the display panel 403 and objects attached to its underside, and to block light generated from the display panel 403 or incident from the outside, and / or a shock-absorbing pad layer (e.g., a sponge layer) for absorbing impacts.

[0086] According to various embodiments, the protective layer 480 may include heat-radiating components (e.g., graphite sheets) and / or conductive components (e.g., metal plates) for heat radiation. For example, conductive components may help strengthen electronic device 301 and may be used to shield ambient noise and disperse heat released from surrounding heat-radiating components.

[0087] According to various embodiments, at least a portion of the protective layer 480 may be open, and the lens 331 or 341 may be disposed in the open area. The pattern of the patterned layer 410 may be formed in the area where at least a portion of the protective layer 480 has been removed.

[0088] Figure 5 The illustration shows a pixel structure and a first type pattern arranged adjacent to a first camera module according to an embodiment of the present disclosure.

[0089] Reference Figure 5 The emitting layer 430 may include pixel regions 435 in which pixels are arranged and panel openings 436. Pixel regions 435 may be regions in which pixels, implemented using useful light-emitting elements (e.g., organic electroluminescence (EL)), are arranged. Panel openings 436 may be empty spaces between pixel regions 435. Panel openings 436 may have various shapes. For example, panel openings 436 may be cross-shaped polygons. In another example, panel openings 436 may have a quadrilateral shape. Panel openings 436 allow light incident from the outside to pass through, and the light passing through panel openings 436 can reach the first lens 331 and can be used to capture images.

[0090] The first portion (hereinafter referred to as the first patterned portion) (or first patterned area) 510 of the patterned layer 410 may include a first blocking portion 515 and a first type of opening 516. The first blocking portion 515 may be a region corresponding to the pixel region 435, and the first type of opening 516 may be a region corresponding to the panel opening 436.

[0091] The first blocking portion 515 prevents light generated from the pixel region 435 from entering the first lens 331. The first type of opening 516 allows light passing through the panel opening 436 to pass through.

[0092] According to one embodiment, the first blocking portion 515 may have substantially the same shape and size as the pixel region 435. For example, the first blocking portion 515 and the pixel region 435 may have a quadrilateral shape of substantially the same size. According to another embodiment, the first blocking portion 515 may have substantially the same shape as the pixel region 435, but may have a different size. For example, the first blocking portion 515 and the pixel region 435 may have a quadrilateral shape, and the first blocking portion 515 may have a larger size than the pixel region 435.

[0093] According to one embodiment, the first type of opening 516 may have substantially the same shape and size as the panel opening 436. For example, the first type of opening 516 and the panel opening 436 may be cross-shaped polygons with substantially the same size. According to another embodiment, the first type of opening 516 may have substantially the same shape as the panel opening 436, but may have a different size. For example, the first type of opening 516 and the panel opening 436 may be cross-shaped polygons, and the first type of opening 516 may have a smaller size than the panel opening 436.

[0094] The first blocking portion 515 and the first type of opening 516 can be arranged on the front surface of the first lens 331. Light passing through the panel opening 436 and the first type of opening 516 can reach the first lens 331 and can be used to capture an image. For example, image distortion or image quality degradation may occur when external light is diffracted or scattered during its passage through the panel opening 436 and the first type of opening 516. The processor (or image processor) in the electronic device 301 can transmit light through the first camera module (e.g., Figure 3 The first image data obtained by the first camera module 330 and the data obtained by the second camera module (e.g., Figure 3 The second image data obtained by the second camera module 340 is compared, and the image quality of the image can be compensated.

[0095] Figure 6The illustration shows a pixel structure and a second type pattern arranged adjacent to a second camera module according to an embodiment of the present disclosure.

[0096] Reference Figure 6 The emitting layer 430 may include pixel regions 435 in which pixels are arranged and panel openings 436. Pixel regions 435 may be regions in which pixels, implemented using useful light-emitting elements (e.g., organic electroluminescence (EL)), are arranged. Panel openings 436 may be empty spaces between pixel regions 435. Panel openings 436 may have various shapes. For example, panel openings 436 may be cross-shaped polygons. In another example, panel openings 436 may have a quadrilateral shape. Panel openings 436 allow light incident from the outside to pass through, and the light passing through panel openings 436 can reach the second lens 341 and can be used to capture images.

[0097] The second portion (hereinafter referred to as the second patterned portion) (or second patterned area) 610 of the patterned layer 410 may include a second blocking portion 615 and a second type of opening 616. The second blocking portion 615 may be a region corresponding to the pixel region 435, and the second type of opening 616 may be a region corresponding to the panel opening 436.

[0098] The second blocking portion 615 prevents light generated from the pixel area 435 from entering the second lens 341. The second type of opening 616 allows light passing through the panel opening 436 to pass through.

[0099] According to one embodiment, the second blocking portion 615 may have a different shape than the pixel region 435, but may have substantially the same size as the pixel region 435. For example, when the pixel region 435 has a quadrilateral shape, the second blocking portion 615 may have a rhombus shape, which has substantially the same size as the pixel region 435. According to another embodiment, the second blocking portion 615 may have a different shape and size than the pixel region 435. For example, when the pixel region 435 has a quadrilateral shape, the second blocking portion 615 may have a rhombus shape, which is larger than the pixel region 435.

[0100] According to an embodiment, the second blocking portion 615 disposed on the front surface of the second lens 341 can have such a shape that the portion disposed on the front surface of the second lens 341 is... Figure 5 The first blocking portion 515 on the front surface of the first lens 331 is rotated by a specified angle. For example, when the first blocking portion 515 has a square shape, the second blocking portion 615 can have a rhombus shape, wherein the first blocking portion 515 is rotated by 45 degrees.

[0101] According to an embodiment, the second type of opening 616 may have a different shape than the panel opening 436. The second type of opening 616 and the panel opening 436 may have a common main area through which light can pass, and may have different boundaries. For example, when the panel opening 436 is a cross-shaped polygon, the second type of opening 616 may have an octagonal shape.

[0102] The second blocking portion 615 and the second type of opening 616 can be arranged on the front surface of the second lens 341. Light passing through the panel opening 436 and the second type of opening 616 can reach the second lens 341 and can be used to capture images. For example, image distortion or image quality degradation may occur when external light is diffracted or scattered during its passage through the panel opening 436 and the second type of opening 616. The processor (or image processor) in the electronic device 301 can transmit light through the second camera module (e.g., Figure 3 The second image data obtained by the second camera module 340) and the second image data obtained by the first camera module (e.g., Figure 3 The first image data obtained by the first camera module 330 is compared, and the image quality of the image can be compensated.

[0103] When the first camera module 330 and the second camera module 340 are devices with essentially the same characteristics (e.g., MTF characteristics), image compensation can be performed through simple operation. When the first camera module 330 and the second camera module 340 are devices with different characteristics, additional processing depending on the characteristics of the camera modules may be required.

[0104] Figure 7 The illustration shows the configuration of an image processor according to an embodiment of the present disclosure. Figure 7 This is illustrative, and the disclosure is not limited thereto. The image processor 710 may be operated by a processor in the electronic device 301, or may be operated under the control of a processor. Alternatively, the image processor 710 may be a computing element implemented separately from a processor.

[0105] Reference Figure 7 The first camera module 330 may include a first lens 331 and a first image sensor 332. The first camera module 330 may collect light passing through the first patterned portion 510 through the first lens 331. The first image sensor 332 may convert the light collected through the first lens 331 into electrical signals.

[0106] The second camera module 340 may include a second lens 341 and a second image sensor 342. The second camera module 340 may collect light passing through the second patterned portion 610 through the second lens 341. The second image sensor 342 may convert the light collected through the second lens 341 into electrical signals.

[0107] According to various embodiments, the image processor 710 may include a first converter 711, a second converter 712, a combiner 730, and a compensator 740. The components are divided according to their function, and this disclosure is not limited thereto.

[0108] The first converter 711 can convert first image data collected by the first image sensor 332. The second converter 712 can convert second image data collected by the second image sensor 342. For example, the first converter 711 and the second converter 712 can perform a conversion (e.g., plane correction) such that the first image data and the second image data are projected onto a common plane.

[0109] Combiner 730 can combine the converted first image data and the converted second image data into a single image. According to an embodiment, combiner 730 can combine (e.g., fuse) the first image data and the second image data into a single image by reflecting the MTF characteristics of the first camera module 330 and the second camera module 340.

[0110] The compensator 740 can compensate for (e.g., recover) attenuated signal components by reflecting the characteristics of the first camera module 330 and the second camera module 340. The first patterning portion 510 can alter the MTF characteristics of the first camera module 330, while the second patterning portion 610 can alter the MTF characteristics of the second camera module 340. The attenuation of specific signal components, depending on the changes in the MTF characteristics of lenses 331 or 341, may be a major cause of image quality degradation in the display camera.

[0111] For example, when the first camera module 330 and the second camera module 340 are devices with substantially the same characteristics, the combiner 730 can combine the first image data and the second image data using a simple addition method. In another example, when the first camera module 330 and the second camera module 340 are devices with different characteristics, the combiner 730 can combine the first image data and the second image data by reflecting the MTF characteristics of the first camera module 330 and the second camera module 340.

[0112] Figure 8 The illustration shows a variation in the MTF characteristics of a display camera according to an embodiment of the present disclosure. Figure 8 In the middle, the patterned portion 850 can be the first patterned portion (e.g., Figure 5 The first patterned part 510).

[0113] refer to Figure 8The patterned portion 850 may include a blocking portion 855 and an opening 856. The MTF characteristics of the display camera may vary due to the patterned portion 850 (e.g., variation 801). The MTF may be a curve obtained by quantifying the variation in resolution based on the distance from the center point of the image. Resolution can mean the ability to represent different objects, making them optically distinguishable from each other, and can be associated with contrast, sharpness, or agility. For example, a camera module (e.g., Figure 4 The camera module 330 or 340 can obtain a lens (e.g., included in the camera module 330 or 340) that is compatible with the lens included in the camera module 330 or 340. Figure 4 The MTF curve is essentially the same as the limiting diffraction curve of lens 331 or 341. For example, the MTF curve can mean an expression of camera performance with a specified value or greater modulation pass (e.g., contrast input / output ratio) regarding contrast in the low frequency band.

[0114] The lens's unique MTF characteristics (characteristics lacking patterned portion 850) can have the following form, where the MTF amplitude gradually decreases with increasing frequency, as shown in the first pattern 810. The first pattern 810 can have a near-linear form without individual ripple. For example, in camera modules 330 or 340 arranged on a display panel (e.g., Figure 4 In the case of the rear surface (the surface opposite to the surface displaying the image) of the display panel 403, depending on the patterning layer of the display panel 430 (e.g., Figure 4 Patterned layer 410) or emission layer (e.g., Figure 4 The structure of the emitting layer 430 may cause diffraction or scattering of various frequencies in the external light entering the camera module 330 or 340. For example, when diffraction or scattering occurs, the MTF of each frequency (e.g., spatial frequency) of the light may fluctuate in a curved form (e.g., ripple or sine wave form) rather than a linear form (e.g., the phenomenon of MTF generally decreasing), and this phenomenon may cause a degradation in the image quality of the camera module 330 or 340.

[0115] For example, when the patterned portion 850 is arranged on the front surface of the lens, certain frequency components may be attenuated depending on the length of the patterned portion 850 in the vertical / horizontal directions A1 and A2 or the length of the patterned portion 850 in the diagonal directions B1 and B2. This can lead to a deterioration in image quality. For example, the MTF characteristics of the display camera may vary depending on the shape (e.g., size or shape) of the patterned portion 850.

[0116] The lengths of opening 856 in the vertical / horizontal directions A1 and A2 and the lengths of opening 856 in the diagonal directions B1 and B2 can be different from each other.

[0117] Typically, the line spread function (LSF) or point spread function (PSF) of a lens can bring an image to the diffraction limit known as the Airy disk. The location where the dark pattern appears on the image is represented by Equation 1 below.

[0118] [Equation 1]

[0119]

[0120] z: Location where the dark pattern forms

[0121] d: Diameter of the aperture

[0122] λ: wavelength

[0123] As shown in Equation 1, the diffraction limit may be inversely proportional to the aperture diameter. In the case of a lower display camera, the diffraction characteristics can be determined by the lengths in the vertical / horizontal directions A1 and A2 or the lengths in the diagonal directions B1 and B2.

[0124] For example, when the patterned portion 850 is arranged on the front surface of the lens, the MTF characteristics in the vertical / horizontal directions A1 and A2 can have a rippled form, where, as shown in the second pattern 811, attenuation occurs in a first frequency range of 0 to 0.1, a third frequency range of 0.2 to 0.3, and a fifth frequency range of 0.4 to 0.5. The MTF characteristics in the diagonal directions B1 and B2 can also have a rippled form, where, as shown in the third pattern 812, attenuation occurs in a second frequency range of 0.1 to 0.2 and a fourth frequency range of 0.3 to 0.4. Rippled MTF characteristics can lead to image distortion and / or degraded image quality. For example, the processor can remove the ripple through a separate compensation operation (see...). Figure 9 ).

[0125] According to various embodiments, when the patterned portions 850 have the same shape and the length of the patterned portions 850 varies, the MTF characteristics can vary accordingly. For example, when the length of the patterned portions 850 in the vertical / horizontal directions A1 and A2 or the length of the patterned portions 850 in the diagonal directions B1 and B2 varies, the forms of the second pattern 811 and the third pattern 812 can vary accordingly.

[0126] Figure 9 The illustration shows compensation for image quality degradation through image combination according to an embodiment of the present disclosure.

[0127] Reference Figure 9 In the first graphic 901, the first lens (e.g., Figure 5 The unique MTF characteristics of the first shot 331 (e.g., lack of Figure 5The characteristics of the first patterned portion 510 can have the following form, where, as shown in the first MTF pattern 910, the amplitude of the MTF gradually decreases with increasing frequency. The first MTF pattern 910 can have a near-linear form without individual ripples.

[0128] When the first patterned portion 510 is arranged on the front surface of the first lens 331, certain frequency components may be attenuated depending on whether it is a vertical / horizontal direction as a first direction or a diagonal direction as a second direction. As a result, image quality may be degraded.

[0129] For example, when the first patterned portion 510 is disposed on the front surface of the first lens 331, the MTF pattern 1-1 911 in the horizontal / vertical direction and the MTF pattern 1-2 912 in the diagonal direction may have a rippled form. The first ripple in the horizontal / vertical direction and the second ripple in the diagonal direction may have opposite characteristics. For example, when the first ripple in the horizontal / vertical direction increases, the second ripple in the diagonal direction may decrease, and when the first ripple in the horizontal / vertical direction decreases, the second ripple in the diagonal direction may increase.

[0130] In the second figure 902, the second lens (e.g., Figure 6 The unique MTF characteristics of the second lens 341 (e.g., lack of Figure 6 The characteristics of the second patterned portion 610 can have the following form, where, as shown in the second MTF pattern 920, the amplitude of the MTF gradually decreases with increasing frequency. The second MTF pattern 920 can have a near-linear form without individual ripples.

[0131] According to an embodiment, when the first camera module 330 and the second camera module 340 are devices with substantially the same characteristics, the unique MTF characteristics of the first lens 331 can be substantially the same as the unique MTF characteristics of the second lens 341.

[0132] When the second patterned portion 610 is disposed on the front surface of the second lens 341, certain frequency components may be attenuated depending on whether the direction is vertical / horizontal (as the first direction) or diagonal (as the second direction). This may result in a decrease in image quality.

[0133] For example, when the second patterned portion 610 is disposed on the front surface of the second lens 341, the MTF pattern 2-1 921 in the horizontal / vertical direction and the MTF pattern 2-2 922 in the diagonal direction may have a rippled form. The third ripple in the horizontal / vertical direction and the fourth ripple in the diagonal direction may have opposite characteristics. For example, when the third ripple in the horizontal / vertical direction increases, the fourth ripple in the diagonal direction may decrease, and when the third ripple in the horizontal / vertical direction decreases, the fourth ripple in the diagonal direction may increase.

[0134] According to various embodiments, the MTF pattern 1-1 911 of the first camera module 330 in the horizontal / vertical direction may be the same as or similar to the MTF pattern 2-2 922 of the second camera module 340 in the diagonal direction. The MTF pattern 1-2 912 of the first camera module 330 in the diagonal direction may be the same as or similar to the MTF pattern 2-1921 of the second camera module 340 in the horizontal / vertical direction.

[0135] According to various embodiments, the processor of electronic device 301 (e.g., Figure 7 The image processor 710 can combine first image data from the first camera module 330 and second image data from the second camera module 340. For example, MTF pattern 3-1931 can be a value obtained by combining or averaging the MTF pattern 1-1911 of the first camera module 330 in the horizontal / vertical direction and the MTF pattern 2-2922 of the second camera module 340 in the diagonal direction. MTF pattern 3-2932 can be a value obtained by combining or averaging the MTF pattern 1-2912 of the first camera module 330 in the diagonal direction and the MTF pattern 2-1922 of the second camera module 340 in the vertical / horizontal direction.

[0136] In the third pattern 903, the third MTF pattern 940 of the combined image data can have a form where ripple is canceled, and can have a form where the amplitude of the MTF gradually decreases with increasing frequency. The third MTF pattern 940 of the combined image data can be similar to the second MTF pattern 920 or the first MTF pattern 910.

[0137] processor (or, Figure 7The image processor 710 can use the combined image data without modification, or it can compensate the combined image data in the form of a third MTF pattern 940 that is closer to the first MTF pattern 910 or the second MTF pattern 920. In this case, the processor can process data with low distortion for each frequency. Therefore, when processing the signal used for recovery (e.g., boost-up), mismatches for each frequency can be robust and compensation can be easily performed.

[0138] According to various embodiments, in the case of multiple cameras, the blocking portion constitutes a pattern arranged in front of the lens (e.g., Figure 5 The first blocking part 515 and Figure 6 The second blocking section 615 can rotate at different angles, causing the phases of the ripples to intersect each other. The rotation angle of each blocking section can be calculated using Equation 2 below.

[0139] [Equation 2]

[0140]

[0141] Angle between BML patterns

[0142] n: Number of cameras

[0143] For example, when two cameras are set up, the angle between the patterns placed in front of the lenses can be approximately 45 degrees, while when four cameras are set up, the angle between the patterns placed in front of the lenses can be approximately 22.5 degrees.

[0144] Figure 10 The illustration shows the light burst effect of a light source according to an embodiment of the present disclosure.

[0145] Reference Figure 10 When shooting with essentially the same light source, in the first image 1010 captured by the first camera module 330, the flare in the up / down direction (first direction) can be relatively long, while the flare in the diagonal direction (second direction) can be relatively short. In the second image 1020 captured by the second camera module 340, the flare in the up / down direction (first direction) can be relatively short, while the flare in the diagonal direction (second direction) can be relatively long.

[0146] The processor can combine the first image 1010 and the second image 1020 to generate a third image 1030, which includes light spots uniformly arranged in the first and second directions. This is achieved by using patterned portions with different patterns (e.g., Figure 5 The first patterned part 510 and Figure 6 The second patterning part (610) can eliminate distortion or reduce the degree of distortion and reduce the computational complexity of the processor.

[0147] The embodiments of this disclosure disclosed in this specification and accompanying drawings are provided to readily explain the technical content according to the embodiments of this disclosure and to aid in understanding the embodiments of this disclosure, and are not intended to limit the scope of this disclosure. Therefore, all modifications and embodiments derived from the scope of this disclosure, other than those disclosed herein, should be construed as being included within the scope of this disclosure.

[0148] Electronic devices according to various embodiments (e.g., Figure 1 Electronic devices 101 or Figure 3 The electronic device 301 may include a display panel (e.g., Figure 3 The display panel 403) and the arrangement on the display panel (e.g., Figure 3 The first camera module (e.g., below the display panel 403) Figure 1 and Figure 2 Camera module 180 or Figure 3 The first camera module 330) and the second camera module (e.g., Figure 1 and Figure 2 Camera module 180 or Figure 3 The second camera module 340). Display panel (e.g., Figure 3 The display panel 403 may include an emission layer containing multiple pixels (e.g., Figure 4 The emission layer 430) and the arrangement in the emission layer (e.g., Figure 4 The emission layer 430) and the first camera module (e.g., Figure 1 and 2 Camera module 180 or Figure 3 The first camera module 330) or the second camera module (e.g., Figure 1 and 2 Camera module 180 or Figure 3 Patterned layers between the second camera module 340 and (e.g., Figure 4 Patterned layer 410). Patterned layer (e.g., Figure 4 The patterned layer 410 may include a first camera module (e.g., Figure 1 and 2 Camera module 180 or Figure 3 The first lens of the first camera module 330 (e.g., Figure 2 Lens assembly 210 or Figure 5 The first patterned portion (e.g., on the front surface of the first lens 331) Figure 5 The first patterned portion 510) and the second patterned portion (e.g., Figure 6 The second patterned portion 610), the second patterned portion (e.g., Figure 6 The second patterned portion 610) is arranged in the second camera module (e.g., Figure 1 and Figure 2 Camera module 180 or Figure 3 The second lens of the second camera module 340 (e.g., Figure 2 Lens assembly 210 or Figure 6 On the front surface of the second lens 341), and having a patterned portion (e.g., Figure 5 The first patterned part 510) has different shapes.

[0149] According to various embodiments, the first patterned portion (e.g., Figure 5 The first patterned portion 510 may include a first blocking portion (e.g., ...) arranged in a region corresponding to a plurality of pixels. Figure 5 The first blocking portion 515) and the first opening (e.g.,) arranged in the region corresponding to the space between multiple pixels. Figure 5 The first type of opening 516 in the middle). The second patterned part (e.g., Figure 6 The second patterned portion 610 may include a second blocking portion (e.g., in a region corresponding to a plurality of pixels) arranged in a region corresponding to a plurality of pixels. Figure 6 The second blocking portion 615) and the second opening (e.g.,) arranged in the region corresponding to the space between multiple pixels. Figure 6 The second type of opening 616), and the first blocking portion (e.g., Figure 5 The first blocking portion 515) and the second blocking portion (e.g., Figure 6 The second blocking part 615) can have different shapes.

[0150] According to various embodiments, the second blocking portion (e.g., Figure 6 The second blocking portion 615) may have a first blocking portion (e.g., Figure 5 The first blocking portion 515) is a shape that rotates in a specified direction.

[0151] According to various embodiments, the first blocking portion (e.g., Figure 5 The first blocking portion 515) can have a square shape, while the second blocking portion (e.g., Figure 6 The second blocking portion 615) can have a rhomboid shape.

[0152] According to various embodiments, the second blocking portion (e.g., Figure 6 The second blocking portion 615) may have the same characteristics as the first blocking portion (e.g., Figure 5 The first blocking part (515) has the same area.

[0153] According to various embodiments, the first blocking portion (e.g., Figure 5 The first blocking portion 515) and the second blocking portion (e.g., Figure 6 The area of ​​the second blocking portion 615 can be greater than or equal to that in the emission layer (e.g., Figure 4 The area of ​​the region in the emission layer 430 where multiple pixels are arranged.

[0154] According to various embodiments, the first opening in the first direction (e.g., Figure 5 The length of the first type of opening 516 in the middle can be equal to the length of the second opening in the second direction (e.g., Figure 6 The length of the second type of opening (616) in the middle. The first direction can form a 45-degree angle with the second direction.

[0155] According to various embodiments, the display panel (e.g., Figure 3 The display panel 403 may also include a patterned layer (e.g., Figure 4 The protective layer (e.g., patterned layer 410) below the patterned layer 410 Figure 4 Protective layer 480), protective layer (e.g., Figure 4 The protective layer 480 may include an opening region in at least a portion thereof, and the first camera module (e.g., ...) may be arranged at least partially in the opening region. Figure 1 and Figure 2 Camera module 180 or Figure 3 The first lens of the first camera module 330 (e.g., Figure 2 Lens assembly 210 or Figure 5 The first lens 331) and the second camera module (e.g., Figure 1 and Figure 2 Camera module 180 or Figure 3 The second lens of the second camera module 340 (e.g., Figure 2 Lens assembly 210 or Figure 6 The second shot (341).

[0156] According to various embodiments, the protective layer (e.g., Figure 4 The protective layer 480 may include at least one layer that blocks light incident from the outside.

[0157] According to various implementation schemes, the first patterned portion (e.g., Figure 5 The first patterned portion 510) and the second patterned portion (e.g., Figure 6 The second patterned portion can be formed on the emission layer (e.g., by deposition or patterning) Figure 4 On the rear surface of the emission layer 430.

[0158] According to various embodiments, the first camera module (e.g., Figure 1 and 2 Camera module 180 or Figure 3 The first camera module 330 can have a connection with the second camera module (e.g., Figure 1 and 2 Camera module 180 or Figure 3 It has the same characteristics as the second camera module 340.

[0159] According to various embodiments, electronic devices (e.g., Figure 1 Electronic devices 101 or Figure 3 The electronic device 301 may further include a memory (e.g., Figure 1 The memory 130) and the processor (e.g., Figure 1 Processor 120 or Figure 7 Image processor 710), processor (e.g., Figure 1 Processor 120 or Figure 7 The image processor 710 can be combined with the first camera module (e.g., Figure 1 and Figure 2 Camera module 180 or Figure 3 The first image data obtained by the first camera module 330 and the second image data obtained by the second camera module (e.g., Figure 1 and Figure 2 Camera module 180 or Figure 3 The second image data obtained by the second camera module 340 is used to generate a combined image, and the combined image can be stored in a memory (e.g., ...). Figure 1 In the memory 130).

[0160] According to various embodiments, the processor (e.g., Figure 1 Processor 120 or Figure 7 The image processor 710 can generate a combined image by converting first image data and second image data so that the first image data and second image data are projected onto a common plane.

[0161] According to various embodiments, the processor (e.g., Figure 1 Processor 120 or Figure 7 The image processor 710 can be based on the first camera module (e.g., Figure 1 and 2 Camera module 180 or Figure 3 The first camera module 330) or the second camera module (e.g., Figure 1 and 2 Camera module 180 or Figure 3 The characteristics of the second camera module 340 are used to compensate for the combined images.

[0162] According to various embodiments, the first patterned portion (e.g., Figure 5 The first patterned portion 510) and the second patterned portion (e.g., Figure 6 The second patterned part 610 can be implemented with different materials or different densities.

[0163] Electronic devices according to various embodiments (e.g., Figure 1 Electronic devices 101 or Figure 3 The electronic device 301 may include a display panel (e.g., Figure 3 The display panel 403) and the arrangement on the display panel (e.g., Figure 3 Multiple camera modules (e.g., below the display panel 403) Figure 1 and Figure 2 Camera module 180 or Figure 3 The first camera module 330 and the second camera module 340). Display panel (e.g., Figure 3 The display panel 403 may include an emission layer containing multiple pixels (e.g., Figure 4 The emission layer 430) and the arrangement in the emission layer (e.g., Figure 4 The emission layer 430) and multiple camera modules (e.g., Figure 1 and Figure 2 Camera module 180 or Figure 3 A patterned layer (e.g., between the first camera module 330 and the second camera module 340) Figure 4 The patterned layer 410), and the patterned layer (e.g., Figure 4 The patterned layer 410 can be included in multiple camera modules (e.g., Figure 1 and Figure 2 Camera module 180 or Figure 3 The lenses included in the first camera module 330 and the second camera module 340 (e.g., Figure 2 Lens assembly 210, or Figure 5 First shot 331 and Figure 6 Different patterned portions on the front surface of the second lens 341 (e.g., Figure 5 The first patterned part 510 and Figure 6 The second patterned part 610).

[0164] According to various embodiments, each patterned portion (e.g., Figure 5 The first patterned part 510 and Figure 6 The second patterned portion 610 may include blocking portions (e.g., in regions corresponding to multiple pixels) arranged in a region corresponding to multiple pixels. Figure 5 The first blocking part 515 or Figure 6The second blocking portion 615) and the openings arranged in the region corresponding to the space between multiple pixels (e.g., Figure 5 The first type of opening 516 or Figure 6 The second type of opening 616 in the middle).

[0165] According to various embodiments, the patterned portion (e.g., Figure 5 The first patterned part 510 and Figure 6 The blocking portion of the second patterned portion 610) (e.g., Figure 5 The first blocking part 515 or Figure 6 The second blocking portion 615) can have the same shape and can rotate sequentially in a specified direction.

[0166] Image processing methods according to various embodiments can be used in electronic devices (e.g., Figure 1 Electronic devices 101 or Figure 3 Performed in electronic device 301), and may include: via electronic device (e.g., Figure 1 Electronic devices 101 or Figure 3 The first camera module of the electronic device 301 (e.g., Figure 1 and 2 Camera module 180 or Figure 3 The first camera module 330) acquires the first image data and transmits it through an electronic device (e.g., Figure 1 Electronic devices 101 or Figure 3 The second camera module of the electronic device 301 (e.g., Figure 1 and 2 Camera module 180 or Figure 3 The second camera module 340) obtains second image data, which is reflected by the first camera module (e.g., Figure 1 and 2 Camera module 180 or Figure 3 The first lens of the first camera module 330 (e.g., Figure 2 Lens assembly 210 or Figure 5 The first image data is converted by the first characteristics of the first lens 331 and reflected by the second camera module (e.g., Figure 1 and Figure 2 Camera module 180 or Figure 3 The second lens of the second camera module 340 (e.g., Figure 2 Lens assembly 210 or Figure 6 The second feature of the second lens 341) is used to convert the second image data, and an image is generated by combining the converted first image data and the converted second image data. The first camera module (e.g., Figure 1 and 2Camera module 180 or Figure 3 The first camera module 330) and the second camera module (e.g., Figure 1 and 2 Camera module 180 or Figure 3 The second camera module 340 can be arranged in electronic devices (e.g., Figure 1 Electronic devices 101 or Figure 3 The display panel of the electronic device 301 (e.g., Figure 3 Below the display panel 403), and different patterns can be formed on the first lens (e.g., Figure 2 Lens assembly 210 or Figure 5 The front surface of the first lens 331) and the second lens (e.g., Figure 2 Lens assembly 210 or Figure 6 On the front surface of the second lens 341.

[0167] It should be understood that the various embodiments of this disclosure and the terminology used therein are not intended to limit the technical features set forth herein to the particular embodiments, but rather to include various variations, equivalents, or substitutions of the corresponding embodiments. Regarding the description of the drawings, similar reference numerals may be used to refer to similar or related elements. It should be understood that the singular form of a noun corresponding to an item may include one or more of that thing unless the relevant context clearly indicates otherwise. As used herein, each of phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” may include any one or all possible combinations of the items listed together in the corresponding phrase. As used herein, terms such as “first” and “second,” or “first” and “second”, may be used simply to distinguish one component from another and do not limit the components in other respects (e.g., importance or order). It should be understood that if an element (e.g., the first element) is referred to as being “coupled,” “coupled to,” “connected,” or “connected to” another element (e.g., the second element), whether or not the terms “operationally” or “communically” are used, it means that the element can be coupled to the other element directly (e.g., wired), wirelessly, or via a third element.

[0168] As used herein, the term "module" can include units implemented in hardware, software, or firmware, and is used interchangeably with other terms such as "logic," "logic block," "part," or "circuit." A module can be a single integrated component adapted to perform one or more functions, or its smallest unit or part. For example, according to one embodiment, the module can be implemented as an application-specific integrated circuit (ASIC).

[0169] The various embodiments described herein can be implemented as software (e.g., program 140) comprising one or more instructions stored in a machine-readable storage medium (e.g., internal memory 136 or external memory 138). For example, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) can invoke at least one of the one or more instructions stored in the storage medium and execute the instructions under the control of the processor, with or without one or more other components. This allows the machine to be operated to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. The term "non-transitory" simply means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

[0170] According to embodiments, methods according to various embodiments of this disclosure may be included in and provided therein in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)) or via an app store (e.g., the Play Store). TM The computer program product may be distributed online (e.g., downloaded or uploaded) or directly between two user devices (e.g., smartphones). If distributed online, at least a portion of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium, such as the memory of a manufacturer's server, an app store's server, or a relay server.

[0171] According to various embodiments, each of the above-described components (e.g., a module or program) may include a single entity or multiple entities. According to various embodiments, one or more of the above-described components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform one or more functions of each of the multiple components in the same or similar manner as performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by a module, program, or other component may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more operations may be performed in a different order or omitted, or one or more other operations may be added.

Claims

1. An electronic device, comprising: Display panel; and The first and second cameras are positioned below the display panel. The display panel includes an emitting layer and a patterning layer. The emitting layer includes a plurality of pixels, and the patterning layer is disposed between the emitting layer and the first camera and between the emitting layer and the second camera. The patterning layer includes: A first patterned portion disposed on the front surface of the first lens of the first camera, and A second patterned portion is disposed on the front surface of the second lens of the second camera, the second patterned portion having a different shape from the first patterned portion. The first patterned portion includes: A first blocking portion arranged in the region corresponding to the plurality of pixels, and A first opening is arranged in a region corresponding to the space between the plurality of pixels. The second patterned portion includes: A second blocking portion arranged in the region corresponding to the plurality of pixels, and A second opening is arranged in a region corresponding to the space between the plurality of pixels. Wherein, the first blocking portion has the same shape as each of the plurality of pixels, and The second blocking portion has the shape in which the first blocking portion rotates in a specified direction.

2. The electronic device according to claim 1, The first blocking portion has a square shape, and The second blocking portion has a rhomboid shape.

3. The electronic device according to claim 1, wherein, The second blocking portion has the same area as the first blocking portion.

4. The electronic device according to claim 1, wherein, The areas of the first blocking portion and the second blocking portion are greater than or equal to the area of ​​the region in the emission layer where the plurality of pixels are arranged.

5. The electronic device according to claim 1, wherein, The length of the first opening in the first direction is equal to the length of the second opening in the second direction.

6. The electronic device according to claim 5, wherein, The first direction and the second direction form a 45-degree angle.

7. The electronic device according to claim 1, The display panel further includes a protective layer disposed beneath the patterned layer. The protective layer includes an opening region in at least a portion thereof, and The first lens of the first camera and the second lens of the second camera are at least partially arranged in the opening area.

8. The electronic device according to claim 7, wherein, The protective layer includes at least one layer configured to block light incident from the outside.

9. The electronic device according to claim 1, wherein, The first patterned portion and the second patterned portion are formed on the back surface of the emission layer by deposition or patterning.

10. The electronic device of claim 1, further comprising: Memory and processor The processor generates a combined image by combining first image data obtained from the first camera and second image data obtained from the second camera. The combined image is stored in the memory.

11. The electronic device according to claim 10, wherein, The processor generates the combined image by transforming the first image data and the second image data so that the first image data and the second image data are projected onto a common plane.

12. The electronic device according to claim 11, wherein, The processor compensates for the combined image based on the characteristics of the first camera or the second camera.

13. An image processing method performed in an electronic device, the image processing method comprising: First image data is obtained through the first camera of the electronic device, and second image data is obtained through the second camera of the electronic device; The first image data is converted by reflecting a first characteristic of the first lens of the first camera, and the second image data is converted by reflecting a second characteristic of the second lens of the second camera; and An image is generated by combining the transformed first image data and the transformed second image data. The first camera and the second camera are positioned below the display panel of the electronic device, and Different patterns are formed on the front surfaces of the first lens and the second lens, wherein the patterns include: A first patterned portion disposed on the front surface of the first lens of the first camera, and A second patterned portion is disposed on the front surface of the second lens of the second camera, the second patterned portion having a different shape from the first patterned portion. The first patterned portion includes: A first blocking portion arranged in an area corresponding to a plurality of pixels of the display panel, and A first opening is arranged in a region corresponding to the space between the plurality of pixels. The second patterned portion includes: A second blocking portion arranged in the region corresponding to the plurality of pixels, and A second opening is arranged in a region corresponding to the space between the plurality of pixels. Wherein, the first blocking portion has the same shape as each of the plurality of pixels, and The second blocking portion has the shape in which the first blocking portion rotates in a specified direction.

Images