Image data acquisition device

Abstract

An image data acquisition apparatus of the present disclosure includes a camera module and a pattern mounting portion. A camera module generates image data for image restoration training. The pattern mounting portion may selectively position the dummy pattern on the camera module.

Description

[0001] Cross - reference to related applications

[0002] This application claims priority to Korean Patent Application No. 10 - 2023 - 0171444, filed with the Korean Intellectual Property Office on November 30, 2023, the entire contents of which are incorporated herein by reference. Technical field

[0003] The present disclosure relates to an image data acquisition device and a system including the image data acquisition device. Background art

[0004] Electronic devices (e.g., mobile devices) are released in various sizes according to functions and user preferences and may include a large - screen touch display to ensure wide visibility and convenience of operations. The electronic device may include at least one camera module (e.g., an image sensor). For example, the electronic device may include at least one under - display camera (UDC) disposed under (e.g., below) the display. A general electronic device includes a display area and a camera area, and since the display may not be driven in the camera area, a partial area of the display cannot display an image. On the other hand, since the display can also be driven in the camera area (e.g., the UDC area), a display device applying UDC can display an image in the entire display area. Summary of the invention

[0005] The technical problem to be solved is to provide an image data acquisition device that generates image data for further improved image restoration training.

[0006] According to an embodiment of the present disclosure, the image data acquisition device of the present disclosure may include a camera module and a pattern mounting part. The camera module may generate image data. The pattern mounting part may selectively position a dummy pattern on the camera module.

[0007] In an embodiment, the generated image data may include first image data and second image data. The first image data may be generated when the dummy pattern is in a first position to allow external light to pass through the dummy pattern and enter the camera module. The second image data may be generated when the dummy pattern is in a second position to allow external light to enter the camera module without passing through the dummy pattern.

[0008] In an embodiment, the image data acquisition device may further include a housing in which the camera module may be positioned. The pattern mounting part may be positioned on the housing.

[0009] In an embodiment, the pattern mounting part may include a dummy pattern corresponding to a display pattern provided on an under - display camera and a body mechanically coupled to the dummy pattern.

[0010] In an embodiment, a dummy pattern may be provided on a dummy pattern carrier portion. The main body may include a guide member to move the dummy pattern carrier portion. The dummy pattern carrier portion may engage with the guide member to move to one of a first position and a second position.

[0011] In an embodiment, the guide member may include a groove formed as a straight line on an inner surface of a cavity formed in the main body. The dummy pattern carrier portion may include a protrusion that engages with the groove.

[0012] In an embodiment, the guide member may include a protrusion formed as a straight line on an inner surface of a cavity formed in the main body. The dummy pattern carrier portion may include a groove that engages with the protrusion.

[0013] In an embodiment, a dummy pattern may be provided on a dummy pattern carrier portion. The pattern mounting portion may further include a hinge. The hinge may be coupled to the main body to move the dummy pattern carrier portion. The dummy pattern carrier portion may move to one of a first position and a second position by opening or closing around the hinge.

[0014] In an embodiment, a dummy pattern may be provided on a dummy pattern carrier portion. The pattern mounting portion may further include a fixing pin. The fixing pin may be coupled to the main body to move the dummy pattern carrier portion. The dummy pattern carrier portion may move from the first position to the second position by rotating clockwise around the fixing pin.

[0015] In an embodiment, a dummy pattern may be provided on a dummy pattern carrier portion. The pattern mounting portion may further include a fixing pin. The fixing pin may be coupled to the main body to move the dummy pattern carrier portion. The dummy pattern carrier portion may move from the first position to the second position by rotating counterclockwise around the fixing pin.

[0016] In an embodiment, the pattern mounting portion may further include an actuator to move the dummy pattern to the first position or the second position.

[0017] According to an embodiment of the present disclosure, a system may include: an image data acquisition device configured to generate first image data and second image data; and an image restoration module configured to receive the first image data and the second image data and perform restoration training. The image data acquisition device may include: a camera module configured to generate the first image data and the second image data; and a pattern mounting portion configured to selectively position a dummy pattern on the camera module.

[0018] In an embodiment, the first image data may be generated when the dummy pattern is in a first position where external light passes through the dummy pattern to enter the camera module, and the second image may be generated when the dummy pattern is in a second position where external light can enter the camera module without passing through the dummy pattern.

[0019] In an embodiment, the image data acquisition device may further include a housing in which the camera module may be positioned. The pattern mounting portion may be positioned on the housing.

[0020] In an embodiment, the pattern mounting portion may include a dummy pattern corresponding to a display pattern provided on an under-display camera and a body mechanically coupled to the dummy pattern.

[0021] In an embodiment, the dummy pattern may be provided on a dummy pattern carrier portion. The body may include a guide to move the dummy pattern carrier portion. The dummy pattern carrier portion may engage with the guide to move to one of a first position and a second position.

[0022] In an embodiment, the guide may include a groove formed as a straight line on an inner surface of a cavity formed in the body. The dummy pattern carrier portion may include a protrusion that engages with the groove.

[0023] In an embodiment, the guide may include a protrusion formed as a straight line on an inner surface of a cavity formed in the body. The dummy pattern carrier portion may include a groove that engages with the protrusion.

[0024] In an embodiment, the dummy pattern may be provided on a dummy pattern carrier portion. The pattern mounting portion may further include a hinge. The hinge may be coupled to the body to allow movement of the dummy pattern carrier portion. The dummy pattern carrier portion may move to one of a first position and a second position by opening or closing around the hinge.

[0025] In an embodiment, the dummy pattern may be provided on a dummy pattern carrier portion. The pattern mounting portion may further include a fixing pin. The fixing pin may be coupled to the body to allow movement of the dummy pattern carrier portion. The dummy pattern carrier portion may move from the first position to the second position by rotating counterclockwise around the fixing pin.

[0026] According to the image data acquisition device according to the present disclosure, image data for further improving image restoration training may be generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other aspects of the present disclosure will become more apparent by describing embodiments of the present disclosure in further detail with reference to the accompanying drawings, in which:

[0028] Figure 1 is a schematic view showing an embodiment of a mobile device equipped with an under-display camera;

[0029] Figure 2 is a schematic view showing Figure 1 the display pattern on the under-display camera of;

[0030] Figure 3 is a schematic diagram showing an embodiment of a mobile device equipped with a camera module for reference image capture;

[0031] Figure 4 is a schematic diagram showing an image data acquisition device according to an example;

[0032] Figure 5 is a schematic block diagram showing a system for training an image restoration module using first image data and second image data;

[0033] Figure 6 is a schematic block diagram showing a method of using a trained image restoration module to repair degraded image data;

[0034] Figure 7 is shown according to Figure 4 a schematic diagram of the difference between the first image data and the second image data acquired by the image data acquisition device;

[0035] Figure 8A is shown by Figure 7 a schematic diagram of the first image data generated by the first camera module;

[0036] Figure 8B is shown by Figure 7 a schematic diagram of the second image data generated by the second camera module;

[0037] Figure 9 is a schematic diagram showing an image data acquisition device according to an embodiment of the present disclosure;

[0038] Figure 10A and Figure 10B is shown Figure 9 a schematic cross-sectional view of the operation of the image data acquisition device;

[0039] Figure 11A and Figure 11B is a schematic diagram showing an embodiment of a pattern mounting portion included in the image data acquisition device;

[0040] Figure 12A and Figure 12B are respectively Figure 11A and Figure 11B schematic cross-sectional views of the pattern mounting portion shown in;

[0041] Figure 13A and Figure 13B is a schematic diagram showing an embodiment of a pattern mounting portion included in the image data acquisition device;

[0042] Figure 14A and Figure 14B are respectivelyFigure 13A and Figure 13B a schematic cross-sectional view of the pattern mounting portion shown in;

[0043] Figure 15A and Figure 15B is a schematic diagram showing an embodiment of a pattern mounting portion included in an image data acquisition device;

[0044] Figure 16 is a schematic block diagram showing a system for training an image restoration module using an image data acquisition device according to an embodiment of the present disclosure; and

[0045] Figure 17 is a schematic block diagram showing a method of restoring degraded image data using a trained image restoration module. DETAILED DESCRIPTION

[0046] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of various embodiments or implementations of the present invention. As used herein, "embodiment" and "implementation" are interchangeable words that are non-limiting examples of the devices or methods disclosed herein. However, it will be apparent that the various embodiments may be practiced without these specific details or in the presence of one or more equivalent arrangements. Here, the various embodiments need not be exclusive or limit the present disclosure. For example, the specific shapes, configurations, and characteristics of an embodiment may be used or implemented in another embodiment.

[0047] Unless otherwise specified, the embodiments shown should be understood to provide features of the present invention. Thus, unless otherwise specified, the features, components, modules, layers, films, panels, regions, and / or aspects, etc. (hereinafter individually or collectively referred to as "elements") of the various embodiments may be otherwise combined, separated, interchanged, and / or rearranged without departing from the inventive concept.

[0048] The use of cross-hatching and / or shading in the drawings is generally provided to clarify the boundaries between adjacent elements. Thus, unless specified, the presence or absence of cross-hatching or shading does not convey or imply any preference or requirement for a particular material, material property, dimension, ratio, commonality between the elements shown, and / or any other characteristic, attribute, property, etc. of the elements. Additionally, in the drawings, for clarity and / or description purposes, the dimensions and relative dimensions of the elements may be exaggerated. When an embodiment may be implemented differently, a particular process order may be performed differently from the order described. For example, two consecutively described processes may be performed substantially simultaneously or in an order opposite to the order described. Additionally, the same reference numerals and / or reference symbols denote the same elements.

[0049] When an element such as a layer is referred to as being “on,” “connected to,” or “coupled to” another element or layer, it can be directly on, directly connected to, or directly coupled to the other element or layer, or intervening elements or layers may be present. However, when an element or layer is referred to as being “directly on,” “directly connected to,” or “directly coupled to” another element or layer, no intervening elements or layers are present. For this reason, the term “connected” can refer to physical, electrical, and / or fluid connection with or without intervening elements. In addition, the X-axis, Y-axis, and Z-axis are not limited to the three axes such as the x-axis, y-axis, and z-axis of a rectangular coordinate system and can be interpreted in a broader sense. For example, the X-axis, Y-axis, and Z-axis can be perpendicular to each other, or can represent different directions that are not perpendicular to each other.

[0050] For purposes of this disclosure, “at least one of A and B” can be interpreted as only A, only B, or any combination of A and B. In addition, “at least one of X, Y, and Z” and “at least one selected from the group consisting of X, Y, and Z” can be interpreted as only X, only Y, only Z, or any combination of two or more of X, Y, and Z. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items.

[0051] Although the terms “first,” “second,” etc. may be used herein to describe various types of elements, these elements should not be limited by these terms. These terms are used to distinguish one element from another. Thus, a first element discussed below can be termed a second element without departing from the teachings of this disclosure.

[0052] For descriptive purposes, spatial relative terms such as “below,” “beneath,” “under,” “lower,” “above,” “upper,” “on top of,” “higher,” “side” (e.g., as in “sidewall”), etc. may be used herein and are thus used to describe the relationship of one element to another(s) as illustrated in the figures. In addition to the orientation depicted in the figures, the spatial relative terms are intended to encompass different orientations of the device in use, operation, and / or manufacture. For example, if the device in the figures is turned over, an element described as “below” or “beneath” another element or feature will then be oriented “above” the other element or feature. Thus, the term “below” can encompass both an orientation above and below. In addition, the device may be otherwise oriented (e.g., rotated 90 degrees or at other orientations), and thus, the spatial relative descriptors used herein should be interpreted accordingly.

[0053] The terms used herein are for the purpose of describing particular embodiments and are not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Additionally, when used in this specification, the terms "comprises", "comprising", "includes" and / or "including" specify the presence of the stated features, integers, steps, operations, elements, components and / or groups thereof, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should also be noted that, as used herein, the terms "substantially", "about" and other similar terms are used as approximate terms and not as terms of degree, and thus are used to account for inherent variations in measured, calculated and / or provided values that would be recognized by one of ordinary skill in the art. It should also be noted that, as used herein, the terms "substantially", "about" and other similar terms are used as approximate terms and not as terms of degree, and thus are used to account for inherent variations in measured, calculated and / or provided values that would be recognized by one of ordinary skill in the art.

[0054] Various embodiments are described herein with reference to cross-sectional views and / or exploded views that are schematic illustrations of embodiments and / or intermediate structures. Accordingly, variations in the shapes of the figures due to, for example, manufacturing techniques and / or tolerances are to be expected. Thus, the embodiments disclosed herein should not necessarily be construed as limited to the particular shapes shown of regions, but should include, for example, deviations in shapes due to manufacturing. In this manner, the regions shown in the figures may be schematic in nature, and the shapes of these regions may not reflect the actual shape of regions of the device, and thus are not necessarily intended to be limiting.

[0055] As is customary in the art, for functional blocks, units, and / or modules, some embodiments are described and illustrated in the drawings. Those skilled in the art will understand that these blocks, units, and / or modules are physically implemented by electrical (or optical) circuits such as logic circuits, discrete components, microprocessors, hardwired circuits, memory elements, wiring connectors, etc., which can be formed using semiconductor-based manufacturing techniques or other manufacturing techniques. In the case where the blocks, units, and / or modules are implemented by a microprocessor or other similar hardware, they can be programmed and controlled using software (e.g., microcode) to perform the various functions discussed herein, and can be selectively driven by firmware and / or software. It is also contemplated that each block, unit, and / or module can be implemented by dedicated hardware, or can be implemented as a combination of dedicated hardware for performing some functions and a processor (e.g., one or more programmed microprocessors and associated circuits) for performing other functions. Additionally, without departing from the scope of the inventive concept, each block, unit, and / or module of some embodiments can be physically separated into two or more interacting and discrete blocks, units, and / or modules. Furthermore, without departing from the scope of the inventive concept, the blocks, units, and / or modules of some embodiments can be physically combined into more complex blocks, units, and / or modules.

[0056] Unless otherwise defined or implied herein, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will also be understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning consistent with their meaning in the relevant art and the context of this disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0057] Figure 1 is a schematic diagram showing an embodiment of a mobile device equipped with an under-display camera.

[0058] Reference Figure 1 , the mobile device 10 may include a housing 11, a display panel 12 coupled to the housing 11, and a camera module 15. As an embodiment, the camera module 15 may be disposed under the display panel 12 or inside the display panel 12. For example, in the case where the camera module 15 is disposed by an under-display camera (UDC) method, the area of the display panel 12 corresponding to the upper portion of the camera module 15 may display an image.

[0059] In an under-display camera, since light (e.g., external light) needs to enter the lens of the camera through the display panel, the transmittance of the light entering the camera module may be reduced. The opaque regions having a pattern arrangement on the display panel may further partially reduce the transmittance of the entering light.

[0060] Figure 2 is a schematic diagram showing Figure 1 a display pattern on an under-display camera.

[0061] Referring to Figure 2 , a mobile device 10 including a camera module 15 provided in the form of an under-display camera is shown. As described above, the mobile device 10 may include a housing 11, a display panel 12 coupled to the housing 11, and the camera module 15. The display panel 12 may include a pattern area 17 over the camera module 15. In an embodiment, when the camera module 15 is not operating, the display panel 12 may display an image on the pattern area 17, and when the camera module 15 is operating, the display panel 12 may display an image on the pattern area 17.

[0062] To this end, the pattern area 17 may also include a pattern for displaying an image in the same or similar manner as the remaining area of the display panel 12. As an example, the pattern area 17 over the camera module 15 may include a regular pattern or a predetermined or selectable pattern as shown in (a) to (f) of Figure 2 . However, Figure 2 the patterns of (a) to (f) of

[0063] Figure 2 are exemplary, and the pattern formed in the pattern area 17 in the display panel 12 may include a regular pattern or an irregular pattern. Figure 2 Each of the patterns shown in (a) to (f) of

[0064] may include an opaque area, and thus a diffraction phenomenon caused by the pattern may occur when light passes through the pattern area 17. Figure 2 The opaque areas included in the patterns shown in (a) to (f) of

[0064] may reduce the amount of light incident on the camera module 15. Accordingly, the image data captured by the camera module 15 provided by the under-display method may exhibit degraded image quality.

[0064] An image restoration module may be used to repair image data with degraded image quality. In an embodiment, the image restoration module may use a deep learning algorithm to restore image data with degraded image quality such that the image data with degraded image quality approaches the original captured object. Deep learning may be a field of machine learning and may be a method of learning data through successive layers of an artificial neural network. In the case of implementing the image restoration module using deep learning technology, a sufficiently large dataset is required. As an example, for deep learning technology for repairing image data, image data pairs in which image data with degraded image quality and image data with good quality that does not degrade are paired may be required. Since it is possible to pass through Figure 1 and Figure 2The camera module 15 of the mobile device 10 shown in [Fig. 0] obtains image data with degraded image quality, so an additional device for obtaining image data with good quality may be required.

[0065] Figure 3 is a schematic diagram showing an embodiment of a mobile device equipped with a camera module for reference image capture.

[0066] Reference Figure 3 Fig. 1, the mobile device 20 may include a housing 21, a first camera module 23, a second camera module 25, and a third camera module 27. The first camera module 23, the second camera module 25, and the third camera module 27 may have different viewing angles and resolutions. However, this is an example, and the mobile device may include only one camera module. Although not shown in Figure 3 Fig. 1, the mobile device 20 may include a display panel. The display panel may be mounted on a surface opposite to the surface where the first camera module 23, the second camera module 25, and the third camera module 27 may be provided. The first camera module 23, the second camera module 25, and the third camera module 27 may be configured as ordinary cameras other than the under-display camera. Using one of the first camera module 23, the second camera module 25, and the third camera module 27, image data with good quality that is not degraded by the display pattern can be obtained. As an example, in the case where the second camera module 25 among the first camera module 23, the second camera module 25, and the third camera module 27 has the smallest viewing angle difference compared to the camera module 15 configured as the under-display camera, the second camera module 25 can be used to obtain image data with good quality for constructing an image data pair.

[0067] Figure 4 is a schematic diagram showing an image data acquisition device according to an example.

[0068] Reference Figure 4 Fig. 2, the image data acquisition device may include a plurality of mobile devices 10 and mobile devices 20. The camera module 15 of the mobile device 10 may be an under-display camera, and the mobile device 10 may obtain degraded image data through the camera module 15. The camera module 25 of the mobile device 20 may be configured as an ordinary camera other than the under-display camera, and may obtain image data with good quality that is not degraded. In this specification, image data with good quality that is not degraded is referred to as first image data, and image data degraded by the display pattern is referred to as second image data.

[0069] In the example, the image data acquisition device may include a fixing part 30 for coupling and fixing the mobile device 10 and the mobile device 20. The fixing part 30 may couple the mobile device 10 and the mobile device 20 to each other while the camera modules 15 of the respective mobile devices 10 and 20 and the camera module 25 acquire image data.

[0070] As an example, in order to reduce the viewing angle difference between the first image data and the second image data, the fixing part 30 may fix the mobile device 10 and the mobile device 20 such that the camera module 25 of the mobile device 20 and the camera module 15 of the mobile device 10 can be positioned as close as possible.

[0071] Figure 5 is a schematic block diagram showing a system for training an image restoration module using first image data and second image data.

[0072] Reference Figure 5 , the first camera module 25 may acquire first image data 29 and transmit the first image data 29 to the image restoration module 40. The first camera module 25 may be configured as a normal camera other than an under-display camera, and may acquire the first image data 29 through a capturing operation. The first image data 29 may be image data with good quality that does not deteriorate.

[0073] Reference Figures 1 to 5 , the second camera module 15 may be configured as an under-display camera, and may acquire second image data 19. The second image data 19 may be image data deteriorated by the pattern area 17 of the display panel 12. At this time, each of the plurality of first image data 29 may form an image data pair with a corresponding second image data 19 among the plurality of second image data 19. The first image data 29 and the second image data 19 may be transmitted to the image restoration module 40 in the form of an image data pair. In the example, the image restoration module 40 may be electrically connected to the mobile device 10 and the mobile device 20 through a communication cable. When the first camera module 25 and the second camera module 15 acquire the first image data 29 and the second image data 19 respectively, the first image data 29 and the second image data 19 may be transmitted to the image restoration module 40 in real time.

[0074] In the example, the first image data 29 and the second image data 19 acquired by the first camera module 25 and the second camera module 15 may be respectively stored in the storage media included in each of the mobile device 20 and the mobile device 10. Thereafter, the first image data 29 and the second image data 19 stored in the storage media may be transmitted to the image restoration module 40.

[0075] The image restoration module 40 may perform restoration training based on the received first image data 29 and second image data 19. As described above, the image restoration module 40 may perform restoration training through deep learning using a plurality of image data pairs.

[0076] Figure 6 is a schematic block diagram illustrating a method of repairing degraded image data using a trained image restoration module.

[0077] refer to Figure 6 , the second image data 19 generated by the capture of the second camera module 15 can be transmitted to the trained image restoration module 40'. The image restoration module 40' can be in the same manner as in the reference Figure 5 The described state is trained using a large amount of image data, and the third image data 35 can be generated by repairing the received second image data 19. The third image data 35 can have the same or comparable image quality as the first image data generated by the first camera module 25.

[0078] Figure 7 It shows that according to Figure 4 A schematic diagram of a difference between first image data and second image data acquired by an image data acquisition device. Figure 8A It is shown by Figure 7 Schematic diagram of first image data generated by a first camera module. Figure 8B It is shown by Figure 7 Schematic diagram of the second image data generated by the second camera module. Figure 7 , Figure 8A and Figure 8B The present disclosure is described.

[0079] refer to Figure 7 , shows a capture area 41 and a capture area 42 of image data generated when the first camera module 25 and the second camera module 15 capture the object OBJ. Figure 4 The first camera module 25 and the second camera module 15 shown in the figure may be camera modules with different specifications. For example, the viewing angles or resolutions of the first camera module 25 and the second camera module 15 may be different from each other. Since each of the first camera module 25 and the second camera module 15 does not capture the object OBJ at the same position, there is a parallax between the first camera module 25 and the second camera module 15, which means that the field of view FOV1 of the first camera module 25 and the field of view FOV2 of the second camera module 15 may be different from each other. Therefore, the capture area 41 of the image data generated by the first camera module 25 and the capture area 42 of the image data generated by the second camera module 15 may become different from each other.

[0080] Specifically,Figure 8A As shown, the image data generated by the first camera module 25 may include image data for the capture region 41. Each capture region 41 may have a width W1 and a height H1. In the capture region 41, the object OBJ may have a horizontal position x1 and a vertical position y1.

[0081] As Figure 8B As shown, the image data generated by the second camera module 15 may include image data for the capture region 42. Each capture region 42 may have a width W2 and a height H2. In the capture region 42, the object OBJ may have a horizontal position x2 and a vertical position y2. Since the second camera module 15 may be configured as an under-screen camera, the image data generated by the second camera module 15 may have shadows.

[0082] As described above, since the viewing angles of the first camera module 25 and the second camera module 15 may be different, the width W1 and height H1 of the capture region 41 of the image data generated by the first camera module 25 may be different from the width W2 and height H2 of the capture region 42 of the image data generated by the second camera module 15, respectively. Since the viewing angles and positions of the first camera module 25 and the second camera module 15 may be different, the horizontal position x1 and vertical position y1 of the object OBJ in the capture region 41 of the image data generated by the first camera module 25 may be different from the horizontal position x2 and vertical position y2 of the object OBJ in the capture region 42 of the image data generated by the second camera module 15.

[0083] In the image restoration training using a deep learning algorithm, when the position and size of the object OBJ in the first image data are similar to those in the second image data, the result of the restoration training may be better. When the size and position of the object OBJ are the same in the first image data and the second image data, the most ideal training result can be obtained, and the image restoration operation based on it can also be successfully performed.

[0084] However, when the captured first image data and second image data as shown in Figure 8A and Figure 8B are used for restoration training, since the capture regions of each image data may be different and the positions of the object OBJ in each capture region may be different, the result of the restoration training may be insufficient. Therefore, it may be necessary for the size and position of the object OBJ to match in the first image data and the second image data. However, as described above, when the specifications of the first camera module 25 and the second camera module 15 are different, since the sizes of the capture regions may be different, even if the image data can be adjusted to the same resolution, the sizes of the object OBJ in the capture regions of each image data may be different.

[0085] Even if the first camera module 25 and the second camera module 15 have the same specifications, parallax may occur due to the possible difference in the positions of the first camera module 25 and the second camera module 15. The position of the object OBJ in the capture area of each image data may be different.

[0086] According to the image data acquisition device according to an embodiment of the present disclosure, the image data acquisition device includes means for selectively positioning a dummy pattern identical or similar to the display pattern on the camera module. Therefore, the camera module can generate the first image data and the second image data without changing the position and the viewing angle. Accordingly, the result of the restoration training for image restoration can be improved, and thus, the quality of the restored image data generated by the image restoration module can also be improved.

[0087] Figure 9 is a schematic diagram showing an image data acquisition device according to an embodiment of the present disclosure. Figure 10A and Figure 10B may be a schematic cross-sectional view showing the operation of the image data acquisition device Figure 9 . In particular, Figure 10A and Figure 10B is a schematic cross-sectional view of the image data acquisition device taken along the line A-A' of Figure 9 . Hereinafter, Figure 9 , Figure 10A and Figure 10B may be referred to together to describe the image data acquisition device according to an embodiment of the present disclosure.

[0088] Referring to Figure 9 , the image data acquisition device 100 includes a housing 101, a camera module 105, and a pattern mounting portion 140. The pattern mounting portion 140 may include a dummy pattern 151 and a main body 150.

[0089] The housing 101 and the camera module 105 may be included in an existing commercial mobile device. At this time, the camera module 105 may be a normal camera other than an under-display camera. For example, the housing 101 may be a housing of a normal mobile phone or a tablet PC. In an embodiment, the camera module 105 may be positioned in the housing 101. Although not shown in Figure 9 , the camera module 105 may be a camera provided on the rear surface of the mobile phone other than the front surface where the display panel may be formed. The image data acquisition device 100 may include a mobile device.

[0090] However, this may be an example, and the housing 101 and the camera module 105 may not be included in the mobile device. The housing 101 may only accommodate the camera module 105 and circuits related to the camera module 105. The image data acquisition device 100 according to the present disclosure may be implemented with only a camera module without a mobile device.

[0091] In an embodiment, the pattern mounting portion 140 may be positioned on the housing 101. The body 150 of the pattern mounting portion 140 may be fixed to the housing 101. The body 150 may be mechanically coupled to the dummy pattern 151. According to an embodiment of the present disclosure, the position of the dummy pattern 151 may not be fixed, and the dummy pattern 151 may be coupled to the body 150 so as to be in at least two different positions. The dummy pattern 151 may include a pattern that is the same as or similar to a display pattern formed on the under-screen camera. In an embodiment, the dummy pattern 151 may correspond to a display pattern formed (or provided) on the under-screen camera. However, the pattern formed on the dummy pattern 151 may not constitute a display panel.

[0092] Figure 10A When the dummy pattern 151 is in the first position, Figure 9 A schematic cross-sectional view of the image data acquisition device taken along line AA'. Figure 10A , when the dummy pattern 151 is in the first position, light (e.g., external light) may pass through the dummy pattern 151 and enter the camera module 105. Image data generated by the camera module 105 when the dummy pattern 151 is in the first position may correspond to degraded image data generated by a camera module configured as an under-screen camera.

[0093] Figure 10B When the dummy pattern 151 is in the second position, Figure 9 According to an embodiment of the present disclosure, the second position of the dummy pattern 151 may be any position other than the upper portion of the camera module 105. Figure 10B In the embodiment, the dummy pattern 151 of the second position can be omitted, and reference can be made to Figures 11A to 15B A specific position of a dummy pattern according to a detailed embodiment of the present disclosure is described.

[0094] refer to Figure 10B , when the dummy pattern 151 is in the second position, light may enter the camera module 105 without passing through the dummy pattern 151. Image data generated by the camera module 105 when the dummy pattern 151 is in the second position may correspond to image data with good quality generated by a camera module configured as a general camera other than the under-screen camera.

[0095] The image data acquisition device 100 according to an embodiment of the present disclosure includes a pattern mounting portion 140 configured to selectively position a dummy pattern 151 on the camera module 105. Accordingly, the camera module 105 can generate both image data degraded by the dummy pattern and image data of good quality that is not degraded without changing the position and viewing angle. Accordingly, the result of restoration training for image restoration can be improved, and thus, the quality of the restored image data generated by the image restoration module can also be improved.

[0096] Figure 11A and Figure 11B is a schematic diagram showing an embodiment of a pattern mounting portion included in the image data acquisition device.

[0097] Referring together Figure 11A and Figure 11B , the pattern mounting portion 200 includes a main body 201 and a dummy pattern carrier portion 203. A dummy pattern 151 identical or similar to the display pattern may be formed (or provided) in the dummy pattern carrier portion 203. The main body 201 may be formed with a guide 205 for moving the dummy pattern carrier portion 203. The guide 205 may be a groove formed in a linear shape in an orbital shape on the inner surface of a cavity formed in the main body 201 in a rectangular shape. A protrusion may be formed on the side surface of the dummy pattern carrier portion 203 and may engage with the groove of the guide 205.

[0098] Conversely, the guide 205 may be a protrusion formed in a linear shape in an orbital shape on the inner surface of a rectangular-shaped cavity formed in the main body 201. A groove may be formed on the side surface of the dummy pattern carrier portion 203 and may thus engage with the protrusion of the guide 205. In either case, the dummy pattern carrier portion 203 may move left and right along the track. The dummy pattern carrier portion 203 may have the shape of a sliding door. Figure 11A Shows the pattern mounting portion 200 in a case where the dummy pattern carrier portion 203 can move to the right so that the dummy pattern 151 can be in the first position. Figure 11B Shows the pattern mounting portion 200 in a case where the dummy pattern carrier portion 203 can move to the left so that the dummy pattern 151 can be in the second position.

[0099] Figure 12A and Figure 12B are respectively Figure 11A and Figure 11B Schematic cross-sectional views of the pattern mounting portion shown in. Figure 12A is a schematic cross-sectional view of the pattern mounting portion taken along line B-B' of Figure 11A in a case where the dummy pattern 151 is in the first position. Figure 12Bis a schematic cross-sectional view of a pattern mounting portion taken along line B-B' of Figure 11B when the dummy pattern 151 is in the second position. Figure 11B

[0100] Referring Figure 11A and Figure 12A , when the dummy pattern 151 is in the first position, the dummy pattern 151 can be positioned on the camera module 105. Accordingly, light (e.g., external light) can pass through the dummy pattern 151 and enter the camera module 105. The image data generated by the camera module 105 when the dummy pattern 151 is in the first position can correspond to the degraded image data generated by the camera module configured as an under-display camera.

[0101] Referring Figure 11B and Figure 12B , when the dummy pattern 151 is in the second position, light (e.g., external light) can enter the camera module 105 without passing through the dummy pattern 151. The image data generated by the camera module 105 when the dummy pattern 151 is in the second position can correspond to the image data with good quality generated by the camera module configured as a normal camera other than the under-display camera.

[0102] In an embodiment, a person can adjust (e.g., directly adjust) the position of the dummy pattern carrier portion 203 by hand. In an embodiment, the pattern mounting portion 200 can electrically adjust the position of the dummy pattern carrier portion 203 by an electrically controllable device such as an actuator.

[0103] Figure 13A and Figure 13B are schematic views showing embodiments of a pattern mounting portion included in an image data acquisition device.

[0104] Referring together Figure 13A and Figure 13B , the pattern mounting portion 300 includes a main body 301, a dummy pattern carrier portion 303, and a hinge 305. As described above with reference to Figure 11A and Figure 11B , a dummy pattern 151 identical or similar to the display pattern can be formed in the dummy pattern carrier portion 303. The main body 301 can be physically coupled to the dummy pattern carrier portion 303 by the hinge 305. The dummy pattern carrier portion 303 can rotate about the hinge 305. The dummy pattern carrier portion 303 can have a hinge door configuration.

[0105] Figure 13A shows the pattern mounting portion 300 in a state where the dummy pattern carrier portion 303 can be closed such that the dummy pattern 151 can be in the first position. Figure 13B ​The pattern mounting portion 300 is shown in a state where the dummy pattern carrier portion 303 can be opened so that the dummy pattern 151 can be in the second position.

[0106] Figure 14A And Figure 14B Are respectively Figure 13A And Figure 13B Schematic cross-sectional views of the pattern mounting portion shown in. Figure 14A Is a schematic cross-sectional view of the pattern mounting portion taken along the line C-C' of Figure 13A In the case where the dummy pattern 151 is in the first position. Figure 14B Is a schematic cross-sectional view of the pattern mounting portion taken along the line C-C' of Figure 13B In the case where the dummy pattern 151 is in the second position.

[0107] Refer to Figure 13A And Figure 14A , in the case where the dummy pattern 151 is in the first position, the dummy pattern 151 can be positioned on the camera module 105. Accordingly, light can pass through the dummy pattern 151 and enter the camera module 105. The image data generated by the camera module 105 when the dummy pattern 151 is in the first position can correspond to the degraded image data generated by the camera module configured as an under-display camera.

[0108] Refer to Figure 13B And Figure 14B , in the case where the dummy pattern 151 is in the second position, light can enter the camera module 105 without passing through the dummy pattern 151. The image data generated by the camera module 105 when the dummy pattern 151 is in the second position can correspond to the good-quality image data generated by the camera module configured as a normal camera other than the under-display camera.

[0109] In an embodiment, a person can adjust (e.g., directly adjust) the position of the dummy pattern carrier portion 303 by hand. In an embodiment, the pattern mounting portion 300 can electrically adjust the position of the dummy pattern carrier portion 303 by an electrically controllable device such as an actuator.

[0110] Figure 15A And Figure 15B Are schematic views showing embodiments of the pattern mounting portion included in the image data acquisition device. Referring together to Figure 15A And Figure 15B , the pattern mounting portion 400 includes a main body 401, a dummy pattern carrier portion 403, and a fixing pin 407. As described above, a dummy pattern 151 identical or similar to the display pattern can be formed in the dummy pattern carrier portion 403. The main body 401 can be physically coupled to the dummy pattern carrier portion 403 by the fixing pin 407. AsFigure 15B As described, the dummy pattern carrier portion 403 can rotate clockwise about the fixed pin 407 to move from the first position to the second position. However, this can be an example, and the dummy pattern carrier portion 403 can rotate counterclockwise about the fixed pin 407 to move from the first position to the second position.

[0111] has been referred to Figures 11A to 15B Embodiments of a pattern mounting portion included in an image data acquisition device according to embodiments of the present disclosure have been described. However, the present disclosure may not be limited to Figures 11A to 15B the embodiments shown therein, and may include any means for selectively positioning a dummy pattern at a first position on the camera module or at a second position other than the upper portion of the camera module by manipulation. As described above, the pattern mounting portion included in the image data acquisition device according to embodiments of the present disclosure can be operated by a human hand, but can also be electrically operated by being coupled to a device such as an actuator.

[0112] Figure 16 is a schematic block diagram showing a system for training an image restoration module using an image data acquisition device according to embodiments of the present disclosure.

[0113] Referring to Figure 16 , the camera module 105 included in the image data acquisition device can acquire both the first image data 451 and the second image data 452, and transmit both the first image data 451 and the second image data 452 to the image restoration module 500 as an image data pair. The image restoration module 500 can perform restoration training based on the received first image data 451 and second image data 452. As described above, the image restoration module 500 can perform restoration training through deep learning using a plurality of image data pairs.

[0114] Figure 17 is a schematic block diagram showing a method of repairing degraded image data using a trained image restoration module. Referring to Figure 17 , the second image data 452 generated by the capture of the camera module 105 can be transmitted to the trained image restoration module 500'. The image restoration module 500' can be in a state trained using a large number of image data pairs as described in reference to Figure 16 , and can generate third image data 501 by modifying the received second image data 452.

[0115] According to an embodiment of the present disclosure, by manipulating a pattern mounting portion included in an image data acquisition device, the dummy pattern 151 can be positioned at a first position on the camera module 105 or at a second position other than the upper portion of the camera module 105. Accordingly, the camera module 105 can generate both image data deteriorated by the dummy pattern and image data of good quality that is not deteriorated without changing the resolution, position, or viewing angle. Accordingly, the result of restoration training for image restoration can be improved, and thus, the quality of the restored image data generated by the image restoration module can also be improved.

[0116] The drawings referred to so far and the detailed description of the present disclosure herein may be merely examples of the present disclosure, may be used only to describe the present disclosure, and may not be intended to limit the meaning and scope of the present disclosure described in the claims. Accordingly, those skilled in the art will understand that various modifications and other equivalent embodiments are possible from these drawings and the detailed description. Accordingly, the true scope of the present disclosure should be determined by the technical spirit of the appended claims.

Claims

1. An image data acquisition device, comprising: a camera module configured to generate image data; as well as The pattern mounting portion is configured to selectively position a dummy pattern on the camera module.

2. The image data acquisition device according to claim 1, wherein: The generated image data includes first image data and second image data, The first image data is generated when the dummy pattern is in a first position to allow external light to pass through the dummy pattern and enter the camera module, and The second image data is generated with the dummy pattern in a second position to allow external light to enter the camera module without passing through the dummy pattern.

3. The image data acquisition device according to claim 2, further comprising: a housing in which the camera module is positioned, Wherein, the pattern mounting portion is positioned on the housing.

4. The image data acquisition device according to claim 2, wherein: The pattern installation part includes: The dummy pattern corresponds to a display pattern set on the under-screen camera; and A main body is mechanically coupled to the dummy pattern.

5. The image data acquisition device according to claim 4, wherein: The dummy pattern is arranged on the dummy pattern carrier part, The main body includes a guide to move the dummy pattern carrier portion, and The dummy pattern carrier portion is engaged with the guide to move to one of the first position and the second position.

6. The image data acquisition device according to claim 5, wherein: The guide includes a groove formed as a straight line on an inner surface of a cavity formed in the body, and The dummy pattern carrier portion includes a protrusion engaged with the groove.

7. The image data acquisition device according to claim 5, wherein: The guide includes a protrusion formed as a straight line on an inner surface of a cavity formed in the body, and The dummy pattern carrier portion includes a groove engaged with the protrusion.

8. The image data acquisition device according to claim 4, wherein: The dummy pattern is arranged on the dummy pattern carrier part, The pattern mounting portion further includes a hinge coupled to the main body to move the dummy pattern carrier portion, and The dummy pattern carrier portion is configured to move to one of the first position and the second position by opening or closing about the hinge.

9. The image data acquisition device according to claim 4, wherein: The dummy pattern is arranged on the dummy pattern carrier part, The pattern mounting portion further includes a fixing pin coupled to the main body to move the dummy pattern carrier portion, and The dummy pattern carrier portion is configured to move from the first position to the second position by rotating clockwise around the fixing pin.

10. The image data acquisition device according to claim 4, wherein: The dummy pattern is arranged on the dummy pattern carrier part, The pattern mounting portion further includes a fixing pin coupled to the main body to move the dummy pattern carrier portion, and The dummy pattern carrier portion is configured to move from the first position to the second position by rotating counterclockwise about the fixing pin.

Images