Image forming apparatus

Abstract

An image forming apparatus according to an example of the present disclosure includes: a plurality of housings; a hinge element rotatably connecting the plurality of housings; a camera module group and a display, each shell is internally provided with the camera module group and the display, and the camera module group comprises at least one camera module; a sensor that continuously detects an angle between the plurality of housings; and an image processor that generates a synthesized image by synthesizing images captured by the camera module groups in each of the housings, and dynamically changes the synthesized image in response to a change in the detected angle during image capturing by the camera module groups of each of the housings.

Description

Technical Field

[0001] The present disclosure relates to an imaging device, and more particularly to a foldable imaging device. Background Art

[0002] Recently, foldable smartphones have been developed.

[0003] However, there is still room for improvement in enhancing the diversity and quality of images generated based on images captured by multiple camera modules in conventional foldable smartphones. Summary of the Invention

[0004] The present disclosure aims to solve at least one of the above technical problems. Accordingly, the present disclosure provides an imaging device.

[0005] According to the present disclosure, the above imaging device includes:

[0006] a plurality of housings;

[0007] a hinge element rotatably connecting the plurality of housings;

[0008] a camera module group and a display disposed in each housing, the camera module group including at least one camera module;

[0009] a sensor continuously detecting an angle between the plurality of housings;

[0010] an image processor generating a composite image by synthesizing images captured by the camera module groups in the respective housings and dynamically changing the composite image in response to a change in the detected angle during image capture by the camera module groups in the respective housings. Brief Description of the Drawings

[0011] These and / or other aspects and advantages of the embodiments of the present disclosure will become apparent and more readily understood from the following description with reference to the accompanying drawings, in which:

[0012] Figure 1 is a front view showing a configuration example of an imaging device according to a first embodiment of the present disclosure;

[0013] Figure 2 is a rear view showing a configuration example of an imaging device according to a first embodiment of the present disclosure;

[0014] Figure 3 is a perspective view showing a configuration example of an imaging device according to a first embodiment of the present disclosure;

[0015] Figure 4is a block diagram showing a configuration example of an imaging device according to a first embodiment of the present disclosure;

[0016] Figure 5 is a flowchart showing an operation example of an imaging device according to a first embodiment of the present disclosure;

[0017] Figure 6 is a side view showing an imaging device in an omnidirectional imaging mode in an operation example of an imaging device according to a first embodiment of the present disclosure;

[0018] Figure 7 is a diagram showing an omnidirectional image in an operation example of an imaging device according to a first embodiment of the present disclosure;

[0019] Figure 8 is a diagram showing a specific example of an imaging device in an omnidirectional imaging mode in an operation example of an imaging device according to a first embodiment of the present disclosure;

[0020] Figure 9 is a diagram showing a blurred image in an operation example of an imaging device according to a first embodiment of the present disclosure;

[0021] Figure 10 is a flowchart showing an operation example of an imaging device according to a second embodiment of the present disclosure;

[0022] Figure 11 is a diagram showing a panoramic image in an operation example of an imaging device according to a second embodiment of the present disclosure;

[0023] Figure 12 is a diagram showing a super-resolution image in an operation example of a modified imaging device according to a second embodiment of the present disclosure;

[0024] Figure 13 is a flowchart showing an operation example of an imaging device according to a third embodiment of the present disclosure;

[0025] Figure 14 is a perspective view showing a configuration example of an imaging device according to a fourth embodiment of the present disclosure;

[0026] Figure 15 is a diagram showing an augmented reality image in an operation example of an imaging device according to a fourth embodiment of the present disclosure;

[0027] Figure 16 is a perspective view showing a modified imaging device according to a fourth embodiment of the present disclosure. Detailed Description

[0028] Embodiments of the present disclosure will be described in detail, and examples of the embodiments will be shown in the drawings. Throughout the specification, the same or similar elements are denoted by the same reference numerals, and elements having the same or similar functions. The embodiments described herein with reference to the drawings are illustrative and are intended to illustrate the present disclosure, and should not be construed as limiting the present disclosure.

[0029] <First Embodiment>

[0030] An imaging device according to a first embodiment of the present disclosure will be described below. As Figures 1 to 3 shown, the imaging device 1 according to the first embodiment includes a first housing 2, a second housing 3, a hinge element 4, a first camera module group 5, a second camera module group 6, a first display 8, and a second display 9. In Figures 1 to 3 this, the first housing 2 and the second housing 3 are shown as examples of a plurality of housings. Further, both the first camera module group 5 and the second camera module group 6 are shown as examples of camera module groups provided in each housing. Further, both the first display 8 and the second display 9 are shown as examples of displays provided in each housing. However, in the imaging device 1 according to an example of the present disclosure, the number of housings, the number of camera module groups, and the number of displays are not limited to two. For example, the number of housings, the number of camera module groups, and the number of displays may be three or more. In other words, the imaging device 1 may include three or more housings foldable by a hinge element, and may include a camera module group and a display provided in each housing.

[0031] As Figure 4 shown, the imaging device 1 further includes a first optical mechanism 513 in the first group, a second optical mechanism 523 in the first group, a first optical mechanism 613 in the second group, a second optical mechanism 623 in the second group, a hinge sensor 11, an image processor 12, a main processor 13, a memory 14, an image interface 15, an audio interface 16, a speaker 17, a microphone 18, a communication interface 19, an antenna 20, and an input interface 21. The first camera module group 5, the second camera module group 6, the hinge sensor 11, the memory 14, the image interface 15, the audio interface 16, the communication interface 19, and the input interface 21 are all connected to the image processor 12 and the main processor 13 via a bus 22. The optical mechanisms 513, 523, 613, 623 may be an optical zoom mechanism (e.g., an autofocus mechanism of a lens or an optical image stabilizer) applied to a variable magnification optical device or a single-focus optical device.

[0032] The components of the imaging device 1 will be specifically described.

[0033] The first housing 2 and the second housing 3 have equal widths ( Figure 1 dimensions in the X direction in Figure 1the dimension in the Z direction). In Figure 1 In the example shown, each of the first housing 2 and the second housing 3 has a shape that is approximately square, where the width is almost equal to the height. However, the specific shapes and dimensions of the first housing 2 and the second housing 3 are not limited to Figure 1 those shown.

[0034] The first housing 2 and the second housing 3 are rotatably connected by a hinge element 4. In Figure 1 the example shown, the hinge element 4 is provided between one end of the first housing 2 in the +X direction and the other end of the second housing 3 in the -X direction. The first housing 2 and the second housing 3 can rotate about the hinge element 4 as the center of rotation, as shown by the arrow A in Figure 3 . Figure 1 and Figure 2 show the imaging device 1 in a fully open state, in which the first housing 2 and the second housing 3 are rotated until the angle θ (see Figure 3 ) between the first housing 2 and the second housing 3 reaches a maximum (i.e., 180°). The angle θ between the first housing 2 and the second housing 3 when the imaging device 1 is fully folded so that the first display 8 and the second display 9 are not visible is set as a reference value (0°). In Figure 1 , the single hinge element 4 that connects the first housing 2 and the second housing is shown as an example of a hinge element that rotatably connects multiple housings. However, in the imaging device 1 according to an example of the present disclosure, the number of hinge elements 4 is not limited to 1. For example, the imaging device 1 may have more than two hinge elements that connect three or more housings.

[0035] The first camera module group 5 is provided in the first housing 2. In Figure 1 the example shown, the first camera module group 5 includes a first camera module 51 in the first group and a second camera module 52 in the first group. However, the number of camera modules included in the first camera module group 5 is not limited to two, but may be one or more. The first camera module group 5 may include more than three camera modules.

[0036] The first camera module 51 in the first group is provided on a side of the first housing 2 opposite to the side where the hinge element 4 is located. In other words, the first camera module 51 in the first group is provided near one end of the first housing 2 in the -X direction. In Figure 4In the illustrated example, the first camera module 51 in the first group includes a first imaging lens assembly 511 and a first image sensor 512 in the first group. The first imaging lens assembly 511 in the first group has at least one lens. The first imaging lens assembly 511 in the first group refracts the light incident from the object (subject) side according to the refractive power and forms an image of the light on the imaging surface of the first image sensor 512 in the first group.

[0037] The first image sensor 512 in the first group may include a solid-state image sensor, such as a complementary metal oxide semiconductor (CMOS) or a charge-coupled device (CCD). The first image sensor 512 in the first group performs photoelectric conversion on the light imaged by the first imaging lens assembly 511 in the first group and outputs the resulting image data to the image processor 12. A filter, such as an infrared filter, may also be provided between the first imaging lens assembly 511 in the first group and the first image sensor 512 in the first group. Compared with the first camera module 61 in the second group, the second camera module 52 in the first group, and the second camera module 62 in the second group, the first camera module 51 in the first group may have a wider field of view. Alternatively, the field of view of the first camera module 51 in the first group may be equal to the field of view of the first camera module 61 in the second group.

[0038] In Figure 1 the illustrated example, the second camera module 52 in the first group is disposed in the first housing 2 on the side where the hinge element 4 is located with respect to the first camera module 51 in the first group. In other words, the second camera module 52 in the first group is arranged to be near the first camera module 51 in the first group in the +X direction. Moreover, in Figure 1 the illustrated example, the second camera module 52 in the first group is disposed at the same position as the first camera module 51 in the first group in the Z direction. In Figure 4 the illustrated example, the second camera module 52 in the first group includes a second imaging lens assembly 521 and a second image sensor 522 in the first group. The second imaging lens assembly 521 in the first group has at least one lens. The second imaging lens assembly 521 in the first group refracts the light incident from the object side according to the refractive power of the lens and forms an image of the light on the imaging surface of the second image sensor 522 in the first group. The second image sensor 522 in the first group may include a solid-state image sensor, such as CMOS or CCD, which performs photoelectric conversion on the light imaged by the second imaging lens assembly 521 in the first group and outputs the resulting image data to the image processor 12. A filter, such as an infrared filter, may be provided between the second imaging lens assembly 521 in the first group and the second image sensor 522 in the first group.

[0039] The second camera module group 6 is disposed within the second housing 3. In Figure 1 the example shown, the second camera module group 6 includes a first camera module 61 in the second group and a second camera module 62 in the second group. However, the number of camera modules included in the second camera module group 6 is not limited to two, but may be one or more. The second camera module group 6 may include three or more camera modules.

[0040] The first camera module 61 in the second group is disposed on a side of the second housing 3 opposite to the side where the hinge element 4 is located. In other words, the first camera module 61 in the second group is disposed near one end of the second housing 3 in the +X direction. In Figure 1 the example shown, when the imaging device 1 is in the fully open state, the first camera module 61 in the second group has a positional relationship in which it is as far apart as possible from the first camera module 51 in the first group in the +X direction. The first camera module 61 in the second group is disposed at the same position as the first camera module 51 in the first group in the Z direction. In Figure 4 the example shown, the first camera module 61 in the second group includes a first imaging lens assembly 611 in the second group and a first image sensor 612 in the second group, and the first imaging lens assembly 611 in the second group has at least one lens. The first imaging lens assembly 611 in the second group refracts light incident from the object side according to the refractive power of the lens and forms an image of the light on the imaging surface of the first image sensor 612 in the second group.

[0041] The first image sensor 612 in the second group may include a solid-state image sensor such as a CMOS or a CCD, which performs photoelectric conversion on the light imaged by the first imaging lens assembly 611 in the second group and outputs the resulting image data to the image processor 12. A filter such as an infrared filter may be disposed between the first imaging lens assembly 611 in the second group and the first image sensor 612 in the second group. Moreover, a camera module having the same structure as the first camera module 51 in the first group may be used as the first camera module 61 in the second group. In this case, existing camera modules (the first camera module 51 in the first group) can be effectively utilized.

[0042] According to the specifications of the field of view angle, the second camera module 52 in the first group or the second camera module 62 in the second group may be omitted. The reason is that when an image with a sufficient field of view angle in terms of specifications can be obtained by synthesizing images from multiple camera modules 51, 61, the camera module 52 or 62 can be selected to be omitted from the perspective of cost or volume. More desirably, each camera module for synthesizing the captured images is disposed at the same position in the Z direction and directly opposite to each other in the X direction with respect to the hinge portion 4 to reduce the load of image processing.

[0043] InFigure 1 In the example shown, in the second group, the second camera module 62 in the second housing 3 is disposed on the side where the hinge element 4 is located with respect to the first camera module 61 in the second group. In other words, the second camera module 62 in the second group is arranged to be near the first camera module 61 in the second group in the -X direction. Moreover, in Figure 1 the example shown, the second camera module 62 in the second group is disposed at the same position as the first camera module 61 in the second group in the Z direction. In Figure 4 the example shown, the second camera module 62 in the second group includes a second imaging lens assembly 621 in the second group and a second image sensor 622 in the second group. The second imaging lens assembly 621 in the second group has at least one lens. The second imaging lens assembly 621 in the second group refracts the light incident from the object side according to the refractive power of the lens and forms an image of the light on the imaging surface of the second image sensor 622 in the second group. The second image sensor 622 in the second group may include a solid-state image sensor such as a CMOS or a CCD, which performs photoelectric conversion on the light imaged by the second imaging lens assembly 621 in the second group and outputs the resulting image data to the image processor 12. A filter such as an infrared filter may be disposed between the second imaging lens assembly 621 in the second group and the second image sensor 622 in the second group.

[0044] As Figure 2 shown, the first display 8 is disposed within the first housing 2. The first display 8 has a display surface on the rear surface (i.e., the end surface in the +Y direction) of the first housing 2, which is opposite to the front surface (i.e., the end surface in the -Y direction) of the first housing 2, and the front surface is provided with holes for the first imaging lens assembly 511 and the second imaging lens assembly 521 in the first group to admit light.

[0045] As Figure 2 shown, the second display 9 is disposed within the second housing 3. The second display 9 has a display surface on the rear surface (i.e., the end surface in the +Y direction) of the second housing 3, which is opposite to the front surface (i.e., the end surface in the -Y direction) of the second housing 3, and the front surface is provided with holes for the first imaging lens assembly 611 and the second imaging lens assembly 621 in the second group to admit light. In Figure 2 the example shown, the second display 9 is arranged to be adjacent to the first display 8 in the +X direction.

[0046] The first display 8 and the second display 9 display a composite image of the images captured by the first camera module group 5 and the second camera module group 6. In other words, the first display 8 and the second display 9 display a composite image obtained by synthesizing the images captured by the camera module groups 5 and 6 of the respective housings 2 and 3.

[0047] The image captured by the first camera module group 5 can be captured by one of the camera modules included in the first camera module group 5. Alternatively, the image captured by the first camera module group 5 can be captured by each of two or more camera modules included in the first camera module group 5. In Figure 1 the example shown, the image captured by the first camera module group 5 can be the image captured by the first camera module 51 in the first group or the second camera module 52 in the first group, or the image captured by both the first camera module 51 in the first group and the second camera module 52 in the first group.

[0048] The image captured by the second camera module group 6 can be captured by one of the camera modules included in the second camera module group 6. Alternatively, the image captured by the second camera module group 6 can be captured by each of two or more camera modules included in the second camera module group 6. In Figure 1 the example shown, the image captured by the second camera module group 6 can be the image captured by the first camera module 61 in the second group or the second camera module 62 in the second group, or the image captured by both the first camera module 61 in the second group and the second camera module 62 in the second group.

[0049] The composite image can be obtained by synthesizing an image captured by one of the camera modules included in the first camera module group 5 and an image captured by one of the camera modules included in the second camera module group 6. In other words, the composite image can be obtained by synthesizing an image captured by one of the camera modules included in the camera module group 5 of the housing 2 and an image captured by one of the camera modules included in the camera module group 6 of the housing 3.

[0050] Alternatively, the composite image can be obtained by synthesizing a first composite image and a second composite image. The first composite image is obtained by synthesizing images captured by two or more camera modules included in the first camera module group 5, and the second composite image is obtained by synthesizing images captured by two or more camera modules included in the second camera module group 6. In other words, the composite image can be obtained by synthesizing images captured by two or more camera modules included in the camera module group 5 of the housing 2 and images captured by two or more camera modules included in the camera module group 6 of the housing 3.

[0051] If the first optical mechanism 513 in the first group is an optical zoom mechanism applied to a variable magnification optical device, the first optical mechanism 513 in the first group can change the focal length of the first camera module 51 in the first group. In this case, the first optical mechanism 513 in the first group includes, for example, an actuator that drives at least one lens of the first imaging lens assembly 511 in the first group in the optical axis direction. The actuator of the first optical mechanism 513 in the first group includes, for example, a drive source (such as a motor) and a driving force transmission member (such as a gear), and the driving force transmission member transmits the driving force of the drive source to at least one lens of the first imaging lens assembly 511 in the first group. The first optical mechanism 513 in the first group can be an optical mechanism applied to a single-focus optical device.

[0052] If the second optical mechanism 523 in the first group is an optical zoom mechanism applied to a variable magnification optical device, the second optical mechanism 523 in the first group can change the focal length of the second camera module 52 in the first group. In this case, the second optical mechanism 523 in the first group includes, for example, an actuator that drives at least one lens of the second imaging lens assembly 521 in the first group in the optical axis direction. The actuator of the second optical mechanism 523 in the first group includes, for example, a drive source (such as a motor) and a driving force transmission member (such as a gear), and the driving force transmission member transmits the driving force of the drive source to at least one lens of the second imaging lens assembly 521 in the first group. The second optical mechanism 523 in the first group can be an optical mechanism applied to a single-focus optical device.

[0053] If the first optical mechanism 613 in the second group is an optical zoom mechanism applied to a variable magnification optical device, the first optical mechanism 613 in the second group can change the focal length of the first camera module 61 in the second group. In this case, the first optical mechanism 613 in the second group includes, for example, an actuator that drives at least one lens of the first imaging lens assembly 611 in the second group in the optical axis direction. The actuator of the first optical mechanism 613 in the second group includes, for example, a drive source (such as a motor) and a driving force transmission member (such as a gear), and the driving force transmission member transmits the driving force of the drive source to at least one lens of the first imaging lens assembly 611 in the second group. The first optical mechanism 613 in the second group can be an optical mechanism applied to a single-focus optical device.

[0054] If the second optical mechanism 623 in the second group is an optical zoom mechanism applied to a variable magnification optical device, the second optical mechanism 623 in the second group can change the focal length of the second camera module 62 in the second group. In this case, the second optical mechanism 623 in the second group includes, for example, an actuator that drives at least one lens of the second imaging lens assembly 621 in the second group in the optical axis direction. The actuator of the second optical mechanism 623 in the second group includes, for example, a drive source (such as a motor) and a driving force transmission member (such as a gear), and the driving force transmission member transmits the driving force of the drive source to at least one lens of the second imaging lens assembly 621 in the second group. The second optical mechanism 623 in the second group can be an optical mechanism applied to a single-focus optical device.

[0055] The hinge sensor 11 is provided in the hinge element 4. The hinge sensor 11 continuously detects the angle between the first housing 2 and the second housing 3 (i.e., the angle between the housings 2, 3). The detected angle between the first housing 2 and the second housing 3 is output to the image processor 12. The function of the hinge sensor 11 can be partially implemented by the main processor 13. In this case, the hinge sensor 11 can output a detection signal for detecting the angle between the first housing 2 and the second housing 3, and the main processor 13 can calculate the angle between the first housing 2 and the second housing based on the detection signal. The specific form of the hinge sensor 11 is not limited. For example, the hinge sensor 11 can be a rotary encoder.

[0056] The image processor 12 is provided in at least one of the first housing 2 and the second housing 3. The image processor 12 generates a composite image by combining the images captured by the first camera module group 5 and the images captured by the second camera module group 6. The images captured by the first camera module group 5 are input from at least one of the first image sensor 512 and the second image sensor 522 in the first group, and the images captured by the second camera module group 6 are input from at least one of the first image sensor 612 and the second image sensor 622 in the second group.

[0057] For example, the image processor 12 can generate a composite image of the following images: the images captured by at least one of the camera modules included in the first camera module group 5 (i.e., the first camera module 51 and the second camera module 52 in the first group); and the images captured by at least one of the camera modules included in the second camera module group 6 (i.e., the first camera module 61 and the second camera module 62 in the second group).

[0058] Alternatively, the image processor 12 may generate a first composite image by synthesizing the images captured by each of two or more camera modules included in the first camera module group 5 (i.e., the first camera module 51 in the first group and the second camera module 52 in the first group). In addition, the image processor 12 may generate a second composite image by synthesizing the images captured by each of two or more camera modules included in the second camera module group 6 (i.e., the first camera module 61 in the second group and the second camera module 62 in the second group). Then, the image processor 12 may generate a composite image by synthesizing the first composite image and the second composite image.

[0059] During the image capture by the first camera module group 5 and the second camera module group 6, the image processor 12 dynamically changes the composite image in response to a change in the detected angle between the first housing 2 and the second housing 3. The composite image of the images captured by the first camera module group 5 and the second camera module group 6 may be a video or a still image. The period during which the first camera module group 5 and the second camera module group 6 capture images may be during video capture or during a preview before still image capture.

[0060] The image processor 12 may dynamically change the composite image by dynamically changing the aspect ratio of the composite image of the images captured by the first camera module group 5 and the second camera module group 6, which will be displayed on the first display 8 and the second display 9. In this case, in the lateral direction along the rotation direction of the first housing 2 and the second housing 3, the image processor 12 may make the aspect ratio when the detected angle between the first housing 2 and the second housing 3 is small larger than the aspect ratio when the detected angle between the first housing 2 and the second housing 3 is large.

[0061] Moreover, when the detected angle between the first housing 2 and the second housing 3 is minimum (i.e., 0°), the image processor 12 may generate an omnidirectional image as the composite image of the images captured by the first camera module group 5 and the second camera module group 6. In this case, preferably, the shooting field of view angle of the first camera module group 5 and the shooting field of view angle of the second camera module group 6 are both 180 degrees or more. In view of video stitching, it is further preferred that the shooting field of view angle is 190 degrees or more. The omnidirectional image may be a video or a still image. In the case where the user sets the imaging mode to capture an omnidirectional image through the input interface 21, the image processor 12 may generate an omnidirectional image.

[0062] Further, in response to a change in the detected angle between the first housing 2 and the second housing 3, the image processor 12 may dynamically change the blurring state of the background in the composite image of the images captured by the first camera module group 5 and the second camera module group 6. In other words, in response to a change in the detected angle between the first housing 2 and the second housing 3, the image processor 12 may dynamically change the blurred image that blurs the background of the object. In this case, in response to a change in the detected angle between the first housing 2 and the second housing 3, the image processor 12 may dynamically change the depth map to dynamically change the blurring state of the background formed based on the depth map. When the user sets the imaging mode to capture a blurred image through the input interface 21, the image processor 12 may generate a blurred image.

[0063] Further, in response to a change in the detected angle between the first housing 2 and the second housing 3, the image processor 12 may dynamically change the parameters for synthesizing the images captured by the first camera module group 5 and the second camera module group 6. The parameter may be an affine parameter for performing an affine transformation on the images captured by the first camera module group 5 and the second camera module group 6.

[0064] The main processor 13 controls the entire process of the imaging device 1. The main processor 13 not only controls the imaging process, but also controls the calling process and the communication process. The main processor 13 may be used together with the hinge sensor 11 to detect the angle between the first housing 2 and the second housing 3. In response to a change in the detected angle between the first housing 2 and the second housing 3, the image processor 12 may control the driving of the optical mechanisms 513, 523, 613, 623 to dynamically change the focal lengths of the first camera module group 5 and the second camera module group 6. By coordinating the image processor 12 and the main processor 13, the driving of the optical mechanisms 513, 523, 613, 623 can be controlled more precisely, thereby dynamically changing the focal length.

[0065] The memory 14 stores programs and data for processing by the image processor 12 and the main processor 13. The image processor 12 and the main processor 13 read and execute the programs and data stored in the memory 14 to implement various functions.

[0066] The image interface 15 outputs the image generated by the image processor 12 to the first display 8 and the second display 9.

[0067] The audio interface 16 outputs the audio data received through the antenna 20 and the communication interface 19 to the speaker 17. The audio interface 16 also outputs the audio data received through the microphone 18 to the communication interface 19.

[0068] The input interface 21 receives operation inputs from the user. The input interface 21 can be, for example, a graphical user interface (GUI) displayed on the first display 8 and the second display 9, or a mechanical switch.

[0069] Next, an operation example of the imaging device 1 according to the first embodiment will be described. First, as Figure 5 shown, the hinge sensor 11 starts continuously detecting the angle between the first housing 2 and the second housing 3 (step S1).

[0070] Next, when the detected angle between the first housing 2 and the second housing 3 is 0° (step S2: Yes), the image processor 12 generates an omnidirectional image as a composite image of the images captured by the first camera module group 5 and the second camera module group 6 (step S3). The omnidirectional image can be a video or a still image. The image processor 12 can generate the omnidirectional image under the condition that the imaging mode has been preset by the user through the input interface 21 to capture the omnidirectional image.

[0071] For example, as Figure 6 shown, when the imaging device is in a fully closed state, the detected angle between the first housing 2 and the second housing 3 becomes 0°. In this case, as Figure 7 shown, the image processor 12 generates an omnidirectional image as the composite image IC. Moreover, as Figure 8 shown, when the detected angle between the first housing 2 and the second housing 3 is 0°, the imaging device 1 can also be used as a wearable monitoring / surveillance camera hung around a child's neck or shoulder.

[0072] On the other hand, as Figure 5 shown, when the detected angle between the first housing 2 and the second housing 3 is greater than 0° (step S2: No), the image processor 12 generates a blurred image that blurs the background of the object as a composite image of the images captured by the first camera module group 5 and the second camera module group 6 (step S4). The blurred image can be a video or a still image. The processor 12 can also generate the blurred image under the condition that the imaging mode has been preset by the user through the input interface 21 to capture the blurred image.

[0073] In Figure 9In the example shown, the image processor 12 generates a depth map that represents the distribution of distances from the imaging device 1 to the object within the field of view angle. The image processor 12 generates the depth map based on the inter-pixel parallax between the image IR captured by the first camera module group 5 and the image IL captured by the second camera module group 6. The image processor 12 sets the aspect ratio and the affine parameters adapted to the detected angle between the first housing 2 and the second housing 3. For example, the image processor 12 can generate the aspect ratio and the affine parameters adapted to the detected angle based on the correspondence relationship between the angle between the first housing 2 and the second housing 3, the aspect ratio, and the affine parameters. Then, the image processor 12 generates a blurred image as the composite image IC based on the generated depth map and the set aspect ratio and affine parameters.

[0074] After generating the blurred image, as Figure 5 shown, the image processor 12 determines whether the detected angle between the first housing 2 and the second housing 3 has changed during the image capture by the first camera module group 5 and the second camera module group 6 (step S5). Specifically, when the blurred image generated in step S4 is a video, for example, when capturing consecutive images of a video, the image processor 12 determines whether the detected angle between the first housing 2 and the second housing 3 has changed. When the blurred image generated in step S4 is a still image, for example, when taking a preview before capturing the next still image, the image processor 12 determines whether the detected angle between the first housing 2 and the second housing 3 has changed.

[0075] If the detected angle between the first housing 2 and the second housing 3 has changed (step S5: Yes), the image processor 12 changes the depth map, the aspect ratio, and the affine parameters (step S6).

[0076] After changing the depth map, the aspect ratio, and the affine parameters, the image processor 12 generates the blurred image again based on the generated depth map, the aspect ratio, and the affine parameters (step S4).

[0077] On the other hand, if the detected angle between the first housing 2 and the second housing 3 has not changed, the process is terminated. In this case, the image processor 12 continues to generate the blurred image based on the current depth map, the aspect ratio, and the affine parameters.

[0078] As described above, according to the first embodiment, the hinge sensor 11 continuously detects the angle between the first housing 2 and the second housing 3. The processor 12 generates a composite image of the images captured by the first camera module group 5 and the second camera module group 6. The processor 12 also dynamically changes the composite image in response to the detected angle during the image capture by the first camera module group 5 and the second camera module group 6. Thereby, a good composite image can be obtained in response to the change in the angle between the first housing 2 and the second housing 3, which allows improving the diversity and quality of the composite image generated from the images captured by the camera modules 5 and 6.

[0079] Furthermore, according to the first embodiment, when any one of the optical mechanisms 511, 513, 521, 523 includes an optical zoom mechanism, the focal lengths of the first camera module group 5 and the second camera module group 6 can be dynamically changed in response to the change in the detected angle between the first housing 2 and the second housing 3. Thereby, a favorable focal length can be set in response to the change in the angle between the first housing 2 and the second housing 3, which further allows improving the diversity and quality of the composite image.

[0080] Furthermore, according to the first embodiment, in response to the change in the detected angle between the first housing 2 and the second housing 3, the image processor 12 can dynamically change the aspect ratio of the composite image to be displayed on the first display 8 and the second display 9. Thereby, a favorable aspect ratio can be set in response to the change in the angle between the first housing 2 and the second housing 3, which further allows increasing the diversity and quality of the composite image.

[0081] Furthermore, according to the first embodiment, in the lateral direction along the rotation direction of the first housing 2 and the second housing 3, the image processor 12 can make the aspect ratio when the detected angle between the first housing 2 and the second housing 3 is small larger than the aspect ratio when the detected angle between the first housing 2 and the second housing 3 is large. Thereby, in response to the change in the angle between the first housing 2 and the second housing 3, a more favorable aspect ratio can be set, which further improves the diversity and quality of the composite image.

[0082] Furthermore, according to the first embodiment, when the detected angle between the first housing 2 and the second housing 3 is the smallest (0°), the image processor 12 can generate an omnidirectional image as the composite image. In this way, the diversity of the composite image can be further increased.

[0083] Furthermore, according to the first embodiment, in response to the change in the detected angle between the first housing 2 and the second housing 3, the image processor 12 can dynamically change the blur state of the background in the composite image. Thereby, in response to the change in the detected angle between the first housing 2 and the second housing 3, a favorable blur state can be formed, which allows further improving the diversity and quality of the composite image.

[0084] Further, according to the first embodiment, in response to a change in the detected angle between the first housing 2 and the second housing 3, the image processor 12 may dynamically change the depth map to dynamically change the blur state of the background formed based on the depth map. Thus, in response to a change in the angle between the first housing 2 and the second housing 3, a favorable blur state can be easily formed, which allows the diversity and quality of the synthesized image to be easily improved.

[0085] Further, according to the first embodiment, in response to a change in the detected angle between the first housing 2 and the second housing 3, the image processor 12 may continuously change the parameters for synthesizing the images captured by the first camera module group 5 and the second camera module group 6. Thus, in response to the angle between the first housing 2 and the second housing 3, favorable parameters can be set, which allows the diversity and quality of the synthesized image to be further improved.

[0086] Further, according to the first embodiment, in response to a change in the detected angle between the first housing 2 and the second housing 3, the image processor 12 may continuously change the affine parameters for performing an affine transformation on the images captured by the first camera module group 5 and the second camera module group 6. Thus, in response to a change in the angle between the first housing 2 and the second housing 3, favorable affine parameters can be set, which allows the diversity and quality of the synthesized image to be further improved.

[0087] Further, according to the first embodiment, at least one camera module in the first camera module group 5 is disposed on a side of the first housing 2 opposite to the side where the hinge element 4 is located, and at least one camera module in the second camera module group 6 is disposed on a side of the second housing 3 opposite to the side where the hinge element 4 is located. Thus, the parallax between the images captured by the first camera module group 5 and the second camera module group 6 can be increased, which allows a blurred image to be generated with high precision.

[0088] As the second camera module 52 in the first group, it may select a lens having a wider field of view angle or a telephoto-side field of view angle compared to the lens of the first camera module 51 in the first group, or a camera module having different functions (such as distance measurement, infrared function, monochromatic, etc.) may be installed. As the second camera module 62 in the second group, it may select a lens having a wider field of view angle or a telephoto-side viewing angle compared to the lens of the first camera module 61 in the second group, or a camera module having different functions (such as distance measurement, infrared function, or monochromatic, etc.) may be installed.

[0089] <Second Embodiment>

[0090] Next, the imaging device 1 according to the second embodiment will be described mainly focusing on the differences from the first embodiment. The imaging device 1 generates a panoramic image when the detected angle between the first housing 2 and the second housing 3 is greater than 0°.

[0091] In the first embodiment, when the detected angle between the first housing 2 and the second housing 3 is greater than 0°, a blurred image is generated as a composite image.

[0092] In contrast, in the second embodiment, as Figure 10 shown, when the detected angle between the first housing 2 and the second housing 3 is greater than 0° (step S2: No), the image processor 12 generates a panoramic image as a composite image of the images captured by the first camera module group 5 and the second camera module group 6 (step S7). The panoramic image can be a panoramic video or a panoramic still image. When generating a panoramic video as the panoramic image, the image processor 12 cuts out the still image of each frame from the videos captured by the first camera module group 5 and the second camera module group 6, and synthesizes the cut-out still images to generate a panoramic image. Under the condition that the user has preset the imaging mode to capture a panoramic image through the input interface 21, the image processor 12 can generate a panoramic image.

[0093] In Figure 11 the example shown, the image processor 12 sets the aspect ratio and affine parameters adapted to the detected angle between the first housing 2 and the second housing 3. Then, the image processor 12 generates a panoramic image based on the set aspect ratio and affine parameters as a composite image IC of the image IR captured by the first camera module group 5 and the image IL captured by the second camera 6.

[0094] After generating the panoramic image, as Figure 10 shown, the image processor 12 determines whether the detected angle between the first housing 2 and the second housing 3 has changed during the image capture by the first camera module group 5 and the second camera module group 6 (step S8). If the detected angle between the first housing 2 and the second housing 3 has changed (step S8: Yes), the image processor 12 changes the aspect ratio and affine parameters (step S9).

[0095] After changing the aspect ratio and affine parameters, the image processor 12 generates a panoramic image again based on the changed aspect ratio and affine parameters (step S7).

[0096] On the other hand, if the detected angle between the first housing 2 and the second housing 3 has not changed, the process is terminated. In this case, the image processor 12 continues to generate a panoramic image based on the current aspect ratio and affine parameters.

[0097] As Figure 12As shown, when generating a panoramic image, the image processor 12 can generate a super-resolution image as the panoramic image by performing feature point mapping based on the parallax image of the image IR captured by the first camera module group 5 and the image IL captured by the second camera module group 6. In this super-resolution image, the number of pixels is increased by synthesizing the pixel P1 in the image IR captured by the first camera module group 5 and the pixel P2 in the image IL captured by the second camera module group 6. When generating the super-resolution image, in response to the detected change in the angle between the first housing 2 and the second housing 3, the image processor 12 can dynamically change the feature point mapping to dynamically change the super-resolution image.

[0098] According to the second embodiment, the image processor 12 can generate a panoramic image as a composite image and can dynamically change the panoramic image in response to the detected change in the angle between the first housing 2 and the second housing 3. Thus, in response to the change in the angle between the first housing 2 and the second housing 3, a good panoramic image can be generated, which allows for further improving the diversity and quality of the composite image.

[0099] Furthermore, according to the second embodiment, the image processor 12 can generate a super-resolution image as a composite image and can dynamically change the super-resolution image in response to the detected change in the angle between the first housing 2 and the second housing 3. Thus, in response to the change in the angle between the first housing 2 and the second housing 3, a good super-resolution image can be generated, which allows for further improving the diversity and quality of the composite image.

[0100] <Third Embodiment>

[0101] Next, the imaging device 1 according to the third embodiment will be mainly described with a focus on the differences from the first embodiment. The imaging device 1 generates a 3D image when the detected angle between the first housing 2 and the second housing 3 is greater than 0°.

[0102] As Figure 13 shown, in the third embodiment, when the detected angle between the first housing 2 and the second housing 3 is greater than 0° (step S2: No), the image processor 12 generates a three-dimensional (3-Dimensional, 3D) image as a composite image of the images captured by the first camera module group 5 and the second camera module group 6 (step S10). The 3D image can be a video or a still image. Under the condition that the user has preset the imaging mode to capture a 3D image through the input interface 21, the image processor 12 can generate a 3D image.

[0103] For example, the image processor 12 may generate a depth map based on the disparity of each pixel between the images captured by the first camera module group 5 and the second camera module group 6. The image processor 12 may also set an aspect ratio and affine parameters adapted to the detected angle between the first housing 2 and the second housing 3. Then, the image processor 12 generates a 3D image based on the generated depth map and the set aspect ratio and affine parameters. The 3D image may be a stereoscopic image that can be stereoscopically presented through special glasses or the like.

[0104] After generating the 3D image, the image processor 12 determines whether the detected angle between the first housing 2 and the second housing 3 has changed during the image capture by the first camera module group 5 and the second camera module group 6 (step S11). If the detected angle between the first housing 2 and the second housing 3 has changed (step S11: Yes), the image processor 12 changes the depth map, aspect ratio, and affine parameters (step S12).

[0105] After changing the depth map, aspect ratio, and affine parameters, the image processor 12 generates a 3D image again based on the changed depth map, aspect ratio, and affine parameters (step S10).

[0106] On the other hand, if the detected angle between the first housing 2 and the second housing 3 has not changed, the process is terminated. In this case, the image processor 12 continues to generate a 3D image based on the current depth map, aspect ratio, and affine parameters.

[0107] According to the third embodiment, the image processor 12 generates a 3D image as a composite image and dynamically changes the 3D image in response to a change in the detected angle between the first housing 2 and the second housing 3. Thus, in response to a change in the angle between the first housing 2 and the second housing 3, a good 3D image can be generated, which allows further improving the diversity and quality of the composite image.

[0108] It should be noted that the above first embodiment, second embodiment, and third embodiment can be implemented independently or appropriately combined.

[0109] <Fourth Embodiment>

[0110] Next, the imaging device 1 according to the fourth embodiment, which generates an augmented reality image or a virtual reality image, will be mainly described with a focus on the differences from the first embodiment.

[0111] As Figure 14 shown, the imaging device 1 according to the fourth embodiment is used by being attached to the glasses 21 in a state where the angle between the first housing 2 and the second housing 3 is maximum (180°).

[0112] As Figure 15As shown, when the angle between the first housing 2 and the second housing 3 is at its maximum, the image processor 12 generates an Augmented Reality Image (AR image) as a composite image IC. The augmented reality image is an image obtained by superimposing the image (digital information) generated by the image processor 12 on the composite image of the images captured by the first camera module group 5 and the second camera module group 6 (i.e., the surrounding image in the real world). The augmented reality image generated without holding the device (such as a smartphone) to the object is sometimes referred to as a Mixed Reality Image (MR image). Under the condition that the user has preset the operation mode to generate an augmented reality image through the input interface 21, the image processor 12 can generate an augmented reality image. The image processor 12 can also generate a Virtual Reality Image (VR image) instead of the augmented reality image. The augmented reality image and the virtual reality image can be a video or a still image.

[0113] Moreover, as Figure 16 shown, the imaging device 1 can be attached to the glasses 21 and used when the angle between the first housing 2 and the second housing 3 is in a state between the minimum and the maximum. In this case, the image processor 12 can generate a wide-angle virtual reality image as a composite image. The wide-angle virtual reality image can be a video or a still image.

[0114] According to the fourth embodiment, the image processor 12 generates an augmented reality image or a virtual reality image as a composite image. Thus, the diversity of the composite image can be further improved.

[0115] In the description of the embodiments of the present disclosure, it should be understood that terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise" and "counterclockwise" should be interpreted as referring to the orientation or position described or shown in the drawings under discussion. These related terms are only used to simplify the description of the present disclosure and do not indicate or imply that the device or element involved must have a specific orientation, or be constructed or operated in a specific orientation. Therefore, these terms should not be construed as limiting the present disclosure.

[0116] In addition, terms such as "first" and "second" are used herein for illustrative purposes and are not intended to indicate or imply relative importance or significance, nor do they imply the number of the technical features referred to. Therefore, the features defined by "first" and "second" may include one or more of such features. In the description of the present disclosure, unless otherwise specified, "a plurality" means two or more.

[0117] In the description of the embodiments of the present disclosure, unless otherwise specified or limited, terms such as "installed", "connected", "coupled", etc. are used in a broad sense and may be, for example, fixedly connected, detachably connected or integrally connected; they may also be mechanically or electrically connected; they may also be directly connected or indirectly connected through an intermediate structure; they may also be internal communication between two elements, and those skilled in the art can understand according to specific circumstances.

[0118] In the embodiments of the present disclosure, unless otherwise specified or limited, the structure in which the first feature is "above" or "below" the second feature may include embodiments in which the first feature is in direct contact with the second feature, and may also include embodiments in which the first feature and the second feature are not in direct contact with each other, but are in contact through additional features formed therebetween. Further, the first feature being "on", "above" or "at the top of" the second feature may include the following embodiments: wherein the first feature is directly or obliquely "on", "above" or "at the top of" the second feature, or simply means that the height of the first feature is higher than the height of the second feature; and the first feature being "under", "below" or "at the bottom of" the second feature may include the following embodiments: wherein the first feature is directly or obliquely "under", "below" or "at the bottom of" the second feature, or simply means that the height of the first feature is lower than the height of the second feature.

[0119] In the above description, various embodiments and examples are provided to implement different structures of the present disclosure. To simplify the present disclosure, some elements and settings are described above. However, these elements and settings are only examples and are not intended to limit the present disclosure. In addition, in different examples of the present disclosure, reference numerals and / or reference signs may be repeated. This repetition is for the sake of simplicity and clarity and does not involve the relationship between different embodiments and / or settings. In addition, examples of different processes and materials are provided in the present disclosure. However, those skilled in the art should understand that other processes and / or materials may also be applied.

[0120] Throughout the specification, references to "an embodiment", "some embodiments", "exemplary embodiments", "an example", "a specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. Thus, the appearance of the above terms throughout the specification is not necessarily referring to the same embodiment or example of the present disclosure. In addition, the specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0121] Any process or method described in a flowchart or otherwise herein can be understood to include one or more modules, segments, or portions of code for executable instructions implementing specific logical functions or steps in the process, and the scope of the preferred embodiments of the present disclosure includes other implementations, where those skilled in the art should understand that those functions can be implemented in an order different from the order shown or discussed, including implementing in substantially the same order or the reverse order.

[0122] The logic and / or steps described otherwise herein or shown in a flowchart, such as a particular sequence list of executable instructions for implementing a logical function, can be embodied specifically in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device (e.g., a computer-based system including a processor or other system capable of obtaining instructions from and executing the instructions of the instruction execution system, apparatus, and device). As used in the specification, a "computer-readable medium" can be any device suitable for including, storing, communicating, propagating, or transporting a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples of the computer-readable medium include, but are not limited to: an electrical connection having one or more wires (electronic device), a portable computer case (magnetic device), a random access memory (RAM), a read only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium on which a program can be printed, as for example, the paper or other suitable medium can be optically scanned and then, if necessary, edited, decrypted, or processed in other suitable ways to obtain the program in electronic form and then stored in a computer memory.

[0123] It should be understood that each part of the present disclosure can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by an appropriate instruction execution system. For example, if implemented by hardware, as in another embodiment, the steps or methods can be implemented by one or a combination of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a data signal logic function, a dedicated integrated circuit having a logic gate circuit with an appropriate combination, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0124] Those skilled in the art will appreciate that all or part of the steps in the above exemplary method of the present disclosure may be implemented by commanding related hardware using a program. The program may be stored in a computer-readable storage medium, and when running on a computer, the program includes one step or a combination of multiple steps in the method embodiment of the present disclosure.

[0125] In addition, each functional unit of the embodiment of the present disclosure may be integrated into a processing module, or these units may be separate physical existences, or two or more units may be integrated into a processing module. The integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, the integrated module may be stored in a computer-readable storage medium.

[0126] The above storage medium may be a read-only memory, a disk, a CD, etc.

[0127] Although embodiments of the present disclosure have been shown and described, those skilled in the art will appreciate that these embodiments are illustrative and are not to be construed as limiting the present disclosure, and that changes, modifications, substitutions and variations may be made to these embodiments without departing from the scope of the present disclosure.

Claims

1. An imaging device, comprising: a plurality of housings; a hinge element rotatably connecting the plurality of housings; a camera module group and a display, the camera module group and the display being provided in each housing, the camera module group including at least one camera module; a sensor continuously detecting an angle between the plurality of housings; an image processor generating a composite image by synthesizing images captured by the camera module groups in the respective housings, and dynamically changing the composite image in response to a change in the detected angle during the capturing of images by the camera module groups in the respective housings.

2. The imaging device according to claim 1, wherein The imaging device further includes an optical zoom mechanism that dynamically changes the focal length of the camera module group in each housing in response to the change in the detected angle.

3. The imaging device according to claim 1, wherein The image processor dynamically changes the aspect ratio of the composite image to be displayed on the display in each housing in response to the change in the detected angle.

4. The imaging device according to claim 3, wherein In a lateral direction along the rotational direction of the plurality of housings, the image sensor makes the aspect ratio when the detected angle is smaller greater than the aspect ratio when the detected angle is larger.

5. The imaging device according to claim 1, wherein, When the detected angle is the smallest, the image processor generates an omnidirectional image as the composite image.

6. The imaging device according to claim 1, wherein The image processor dynamically changes the blur state of the background in the composite image in response to the change in the detected angle.

7. The imaging device according to claim 6, characterized in that, In response to the change in the detected angle, the image processor dynamically changes the depth map to dynamically change the blur state of the background formed based on the depth map.

8. The imaging device according to claim 1, wherein, In response to the change in the detected angle, the image processor dynamically changes the parameters for synthesizing the images captured by the camera module groups in the respective housings.

9. The imaging device according to claim 8, wherein The parameters are used to perform an affine transformation on the images captured by the camera module groups in the respective housings.

10. The imaging device according to claim 1, characterized in that, The image processor generates a panoramic image as the composite image and dynamically changes the panoramic image in response to the change in the detected angle.

11. The imaging device according to claim 1, wherein The image processor generates a 3D image as the composite image and dynamically changes the 3D image in response to the change in the detected angle.

12. The imaging device according to claim 1, wherein The image processor generates a super-resolution image as the composite image and dynamically changes the super-resolution image in response to the change in the detected angle.

13. The imaging device according to claim 1, wherein The image processor generates a video as the composite image and dynamically changes the video in response to the change in the detected angle.

14. The imaging device according to claim 1, characterized in that, When the detected angle is the largest, the image processor generates an augmented reality image or a virtual reality image as the composite image.

15. The imaging device according to claim 14, characterized in that, The image processor generates the augmented reality image or the virtual reality image in a state where the imaging device is attached to glasses.

16. The imaging device according to claim 1, wherein, When the detected angle is between the minimum angle and the maximum angle, the image processor generates a virtual reality image as the composite image.

17. The imaging device according to claim 1, wherein The plurality of housings includes at least a first housing and a second housing. A first camera module group is disposed in the first housing. The second housing is connected to the first housing through the hinge element. A second camera module group is disposed in the second housing. Wherein, at least one camera module of the first camera module group is disposed on a side of the first housing opposite to the side where the hinge element is located. Wherein, at least one camera module of the second camera module group is disposed on a side of the second housing opposite to the side where the hinge element is located.

18. The imaging device according to claim 1, wherein, The composite image is synthesized from images captured by one of the camera modules included in the camera module groups in each housing.

19. The imaging device according to claim 1, characterized in that, The composite image is synthesized from images captured by two or more camera modules included in the camera module groups in each housing.

Images