Multi-camera camera module, camera system, electronic device and imaging method
By using optical zoom and relative position adjustment of the multi-camera module, the problem of insufficient clarity in both foreground and background in the viewfinder was solved, achieving higher quality image synthesis effects.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NINGBO SUNNY OPOTECH CO LTD
- Filing Date
- 2021-06-15
- Publication Date
- 2026-05-12
AI Technical Summary
When existing camera modules capture both foreground and background scenes, it is difficult to improve the overall image clarity. In particular, due to the fixed focal length of the camera module, changes in the distance between the subject and the camera device can cause some parts of the image to be unclear.
It adopts a multi-camera module structure, is equipped with a camera unit with optical zoom function, and adjusts the relative position relationship between the first camera unit and the second camera unit through a moving mechanism. Combined with the optical axis tilt design, it achieves optical zoom to capture clear images at different depths of field.
Through optical zoom and relative position adjustment, the multi-camera module can capture clearer images, significantly improve the quality of synthesized images, and provide better imaging results.
Smart Images
Figure CN116114243B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of camera modules, and more particularly to multi-camera camera modules, camera systems, electronic devices, and imaging methods. Background Technology
[0002] With the widespread adoption of mobile electronic devices, the technology behind camera modules used in these devices to help users acquire images (such as videos or pictures) has developed and progressed rapidly. Especially with the development of smartphones, consumers' demands for shooting functions have become increasingly diverse, and their requirements for image quality have also increased, posing greater challenges to camera modules.
[0003] In order to meet consumers' requirements for shooting functions and image quality, in recent years, camera modules have undergone changes from single-camera camera modules to multi-camera camera modules. For example, some manufacturers combine wide-angle modules and telephoto modules in structural configuration to form dual-camera camera modules.
[0004] A dual-camera module consisting of a wide-angle module and a telephoto module synthesizes images from the images captured by the wide-angle module and the telephoto module. However, since the telephoto module has a fixed focal length and a small field of view, its compensation for the image captured by the wide-angle module is limited, making it difficult to improve the image clarity.
[0005] To address the challenge of improving overall image clarity when both foreground and background scenes are captured in the viewfinder of a camera module, several design solutions have been proposed. For example, adding an additional module to the existing dual-camera module (e.g., adding a module with a suitable focal length and field of view); or equipping the camera module with optical image stabilization (OIS) to adjust the position of the optical lens relative to the image sensor and capture multiple images, which are then combined to improve clarity.
[0006] However, none of these solutions can fundamentally solve the aforementioned technical problems. The reason is that in the above-mentioned camera module design, the focal length of the camera module is fixed, while the distance between the target and the camera device changes constantly. In the multiple images obtained, some parts of the target will be unclear, making it difficult to improve the imaging quality of the synthesized image.
[0007] Therefore, a new module structure design is needed to solve the above problems and provide users with a better shooting experience. Summary of the Invention
[0008] One advantage of this application is that it provides a multi-camera module, a camera system, an electronic device, and an imaging method. The structure of the multi-camera module enables it to perform optical zoom based on the distance between itself and the target. This allows the multi-camera module to capture clear images of the target at different depths of field when the viewfinder includes both foreground and background, resulting in a better imaging effect in the final synthesized image.
[0009] Another advantage of this application is that it provides a multi-camera module, a camera system, an electronic device, and an imaging method. The multi-camera module is equipped with a camera unit with optical zoom function, and the relative positional relationship between the camera unit with optical zoom function and other camera units can be changed so that the multi-camera module can acquire clear images of the subject at different depths of field, thereby resulting in a final image with better imaging effect.
[0010] Another advantage of this application is that it provides a multi-camera module, camera system, electronic device and imaging method, wherein the relative positional relationship between the camera unit with optical zoom function and other camera units can be changed by a moving mechanism, so that the camera unit with optical zoom function can better compensate (or, in other processing ways) the images captured by other camera units, so that the final synthesized image has a better imaging effect.
[0011] Another advantage of this application is that it provides a multi-camera module, camera system, electronic device, and imaging method, wherein the optical axis set by the camera unit with optical zoom tends to be tilted towards the optical axis set by other camera units, so that the imaging range of the camera unit with optical zoom function can be better aligned with the part to be processed in the image captured by other camera modules. Thus, the camera unit with optical zoom function can better compensate for the image captured by other camera units, thereby improving the imaging effect of the final synthesized image.
[0012] Other advantages and features of this application will become apparent from the following description and can be realized by means and combinations particularly pointed out in the claims.
[0013] To achieve at least one of the above objectives or advantages, this application provides a multi-camera module, comprising:
[0014] The first camera unit is equipped with a first optical axis; and
[0015] The second camera unit, which has zoom capability, is equipped with a second optical axis; and
[0016] A moving mechanism configured to adjust the relative positional relationship between the first camera unit and the second camera unit.
[0017] In the multi-camera module according to this application, the moving mechanism is configured to adjust the relative positional relationship between the first camera unit and the second camera unit based on an adjustment instruction, the adjustment instruction being generated based on the area to be processed in the first image of the target acquired by the first camera unit.
[0018] In the multi-camera module according to this application, the second optical axis is tilted in a direction tending towards the first optical axis to form an angle with the first optical axis.
[0019] In the multi-camera module according to this application, the angle between the first optical axis and the second optical axis is 0.1° to 45°.
[0020] In the multi-camera module according to this application, the angle between the first optical axis and the second optical axis ranges from 0.1° to 10°.
[0021] In the multi-camera module according to this application, the second camera unit is mounted on the moving mechanism so as to drive the second camera unit through the moving mechanism to change the relative positional relationship between the first camera unit and the second camera unit.
[0022] In the multi-camera module according to this application, the moving mechanism includes: a housing, a carrier suspended within the housing for carrying the second camera unit, and a coil-magnet pair disposed between the carrier and the housing and corresponding to each other.
[0023] In the multi-camera module according to this application, the moving mechanism further includes a ball bearing installed between the carrier and the housing, so that the carrier is suspended within the housing by means of the ball bearing.
[0024] In the multi-camera module according to this application, the moving mechanism further includes an elastic element extending between the inner wall of the housing and the outer wall of the carrier, so that the carrier is suspended within the housing by means of the elastic element.
[0025] In the multi-camera module according to this application, the first field of view of the first camera unit is greater than 60°, and the maximum second field of view of the second camera unit is less than 30°.
[0026] According to another aspect of this application, a camera system is also provided, comprising:
[0027] The multi-camera module as described above; and
[0028] A processor communicatively connected to the multi-camera module, wherein the processor is configured to generate the adjustment instruction based on the region to be processed in a first image of the target acquired by the first camera unit.
[0029] In the camera system according to this application, the processor is further configured to fuse a first image of the target captured by the first camera unit and a second image of the target captured by the second camera unit to obtain a fused image.
[0030] According to another aspect of this application, an electronic device is also provided, which includes the multi-camera module as described above.
[0031] According to another aspect of this application, an imaging method for a camera system is also provided, comprising:
[0032] Obtain a first image of the target captured by the first camera unit and a second image of the target captured by the second camera unit;
[0033] Determine at least one region to be processed in the first image;
[0034] Based on the relative positional relationship between the mapped image of the second image to the first image and the region to be processed, an adjustment instruction is generated;
[0035] Based on the adjustment command, the moving mechanism is driven to move the second camera unit to the position of the mapped image of the second image captured by the second camera unit corresponding to the area to be processed in the first image;
[0036] Control the second camera unit to perform optical zoom and obtain a zoomed second image of the subject;
[0037] Based on the relative positional relationship between the mapped image of the zoomed second image onto the first image and the area to be processed, a second adjustment instruction is generated;
[0038] Based on the second adjustment command, a moving mechanism is driven to move the second camera unit, wherein, during the movement of the second camera unit, at least one zoomed second image of the target captured by the second camera unit is obtained; and
[0039] The first image and the zoomed second image are merged to obtain a merged image.
[0040] In the imaging method according to this application, determining at least one region to be processed in the first image includes: determining at least one region in the first image with relatively low imaging quality as the at least one region to be processed.
[0041] In the imaging method according to this application, determining at least one region to be processed in the first image includes: receiving a region to be processed designation instruction; and, in response to the region to be processed designation instruction, determining at least one region to be processed in the first image.
[0042] In the imaging method according to this application, determining at least one region to be processed in the first image includes: determining at least one region to be processed in the first image based on default settings.
[0043] In the imaging method according to this application, a second adjustment instruction is generated based on the relative positional relationship between the mapped image of the zoomed second image onto the first image and the region to be processed, including: determining the number of pixels Mx and My of the region to be processed in the X and Y directions set in the first image; determining the number of pixels Nx and Ny of the mapped image in the X and Y directions set in the first image; and generating the second adjustment instruction based on Mx, My, Nx, and Ny.
[0044] In the imaging method according to this application, generating the second adjustment instruction based on Mx, My, Nx and Ny includes: generating the second adjustment instruction in response to Nx > Mx and Ny > My, wherein the adjustment instruction is used to drive the moving mechanism to move the second camera unit such that the center of the mapped image is aligned with the center of the area to be processed.
[0045] In the imaging method according to this application, generating the second adjustment instruction based on Mx, My, Nx, and Ny includes: determining a first integer multiple relationship between Mx and Nx in response to Mx being greater than Nx; determining a second integer multiple relationship between My and Ny in response to My being greater than Ny; and generating the second adjustment instruction based on the first integer multiple relationship and the second integer multiple relationship, wherein the second adjustment instruction is used to drive the moving mechanism to move the second camera unit along the X direction by at least a first integer multiple; and to drive the moving mechanism to move the second camera unit along the Y direction by at least a second integer multiple.
[0046] In the imaging method according to this application, during the movement of the second camera unit, obtaining at least one zoomed second image of the target captured by the second camera unit includes: obtaining a zoomed second image of the target captured by the second camera unit for each movement, thereby obtaining multiple zoomed second images; wherein, fusing the first image and the zoomed second images to obtain a fused image includes: fusing the first image and the multiple zoomed second images to obtain the fused image.
[0047] In the imaging method according to this application, an adjustment instruction is generated based on the relative positional relationship between the mapped image of the second image to the first image and the region to be processed, including: determining the relative positional relationship between the center of the region to be processed and the center of the mapped image; and generating the adjustment instruction based on a pre-calibrated correspondence table between the relative position of the center of the region to be processed and the mapped image and the translation position of the second camera unit.
[0048] In the imaging method according to this application, a second adjustment instruction is generated based on the relative positional relationship between the mapped image of the zoomed second image onto the first image and the region to be processed, including: determining the relative positional relationship between the center of the region to be processed and the center of the mapped image; and generating the second adjustment instruction based on a pre-calibrated correspondence table between the relative position of the center of the region to be processed and the mapped image and the translation position of the second camera unit.
[0049] The further objectives and advantages of this application will become fully apparent from the following description and accompanying drawings.
[0050] These and other objects, features and advantages of this application are fully apparent from the following detailed description, the accompanying drawings and the claims. Attached Figure Description
[0051] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.
[0052] Figure 1 The illustration shows a schematic diagram of a multi-camera module according to an embodiment of this application.
[0053] Figure 2 Another schematic diagram of the multi-camera module according to an embodiment of this application is shown.
[0054] Figure 3 The illustration shows a schematic diagram of the viewfinder of the first camera unit and the second camera unit of the multi-camera module according to an embodiment of this application.
[0055] Figure 4 The illustration shows yet another schematic diagram of the multi-camera module according to an embodiment of this application.
[0056] Figure 5The figure shows a schematic diagram of the second camera unit in the multi-camera module according to an embodiment of the present application.
[0057] Figure 6 The illustration shows a modified embodiment of the second camera unit in the multi-camera module according to an embodiment of this application.
[0058] Figure 7 The illustration shows a flowchart of an imaging method according to an embodiment of this application.
[0059] Figure 8 The illustration shows a schematic diagram of a camera system according to an embodiment of this application.
[0060] Figure 9 The illustration shows a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation
[0061] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.
[0062] Exemplary multi-camera module
[0063] like Figure 1 and Figure 2 As shown, a multi-camera module 10 according to an embodiment of this application is illustrated. The multi-camera module 10 is configured to perform optical zoom based on its distance from the target object. This allows the multi-camera module 10 to capture clear images of the target object at different depths of field when both near and far views are included in the viewfinder, resulting in a superior final image. Specifically, the multi-camera module 10 according to this application is equipped with a camera unit having optical zoom functionality. Furthermore, the relative positional relationship between the camera unit with optical zoom functionality and other camera units can be changed to allow the multi-camera module 10 to capture clear images of the target object at different depths of field, thereby achieving a superior final image.
[0064] It is worth noting that, in this embodiment, the camera unit with zoom function and other camera units refer to different camera units that are structurally integrated in the multi-camera module 10, rather than structurally separate camera modules. Specifically, in the multi-camera module 10, two or more camera units are integrally formed by processes such as molding to create the multi-camera module 10, and the multi-camera module 10 is connected as a whole to other peripheral devices, such as an image processor.
[0065] like Figure 1 and Figure 2 As shown, the multi-camera module 10 according to an embodiment of this application includes a first camera unit 11, a second camera unit 12, and a moving mechanism 13 configured to adjust the relative positional relationship between the first camera unit 11 and the second camera unit 12. The second camera unit 12 is a camera unit with optical zoom function (i.e., the focal length of the first camera unit 11 can be adjusted). That is, in this embodiment, a multi-camera module 10 including two camera units is taken as an example. Of course, in other examples of this application, a greater number of camera units may be included, which is not limited to this application.
[0066] like Figure 1 As shown, in the multi-camera module 10, the first camera unit 11 is implemented as a conventional camera module with a fixed equivalent focal length, and the second camera unit 12 is a camera module with optical zoom capability. More specifically, as... Figure 1 As shown, the second camera unit 12 includes a photosensitive chip 121, at least one lens group 122 located on the photosensitive path set by the photosensitive chip 121, and a driving component 123 for driving at least some of the lenses in the at least one lens group 122 to perform optical zoom.
[0067] More specifically, in such Figure 1 In the illustrated example, the at least one lens group 122 includes a first lens group 124 and a second lens group 125, and the driving assembly 123 includes a first driving element 127 and a second driving element 128. The first driving element 127 drives at least a portion of the lenses in the first lens group 124 to move for optical zoom, and the second driving element 128 drives the entire second lens group 125 to move for optical focusing, thereby compensating for the image quality degradation after optical zoom and ensuring that the second camera unit 12 has relatively superior imaging quality after optical zoom. That is, in this embodiment, the at least one lens group 122 includes a compensation lens group (the second lens group 125) and a zoom lens group (the first lens group 124), and the driving assembly 123 includes a zoom driver (the first driving element 127) and a focus driver (the second driving element 128).
[0068] It should be understood that in the embodiments of this application, the at least one lens group 122 may also include more lens groups, for example, it may also include a third lens group 126, the position of which is fixed as a fixed lens group, which is not limited by this application.
[0069] Furthermore, in order to reduce the size of the second camera unit 12 in the height direction, as shown in... Figure 1 In the illustrated example, the second camera unit 12 further includes a reflective element 129 (e.g., a prism, mirror, etc.) disposed on the light-sensing path of the photosensitive chip 121 for deflecting the imaging light. That is, in the example shown... Figure 1 In the illustrated example, the second camera unit 12 is implemented as a periscope camera module.
[0070] It is worth mentioning that, in the embodiments of this application, the second camera unit 12 can be implemented as a traditional upright camera module, and this is not limited to this application. Meanwhile, the second camera unit 12 can also achieve optical zoom in other ways. For example, in other examples of this application, the optical lens of the second camera unit 12 is a liquid lens, which can achieve optical zoom by changing the surface shape of the liquid lens through electrical current, and this is also not limited to this application.
[0071] In particular, such as Figure 2 As shown, in the multi-camera module 10, the first camera unit 11 has a relatively large field of view, that is, it has a larger imaging window (or, the first camera unit 11 has a larger viewfinder, capable of capturing scenes within a larger spatial range), while the second camera unit 12 has a relatively small field of view compared to the first camera unit 11, that is, the imaging window of the second camera unit 12 is smaller. Figure 3 As shown, when the first camera unit 11 and the second camera unit 12 simultaneously capture images of the target, the imaging windows of the first camera unit 11 and the second camera unit 12 at least partially overlap. More specifically, the imaging window of the second camera unit 12 is smaller than the imaging window of the first camera unit 11, and if the two are positioned at an appropriate distance, the imaging window of the second camera unit 12 is located within the imaging window of the first camera unit 11. Therefore, when the target is captured by the multi-camera module 10, the images of the target captured by the first camera unit 11 and the second camera unit 12 are related in content. Thus, a fused image with better imaging effect can be obtained by synthesizing the first image of the target captured by the first camera unit 11 and the second image of the target captured by the second camera unit 12.
[0072] Accordingly, in this embodiment, the first field of view of the first camera unit 11 is greater than 60°, while the maximum second field of view of the second camera unit 12 is less than 30°. It should be understood that during the optical zoom process of the second camera unit 12, the second field of view of the second camera unit 12 will change, but its maximum field of view will not exceed 30°.
[0073] Furthermore, although the first image of the target captured by the first camera unit 11 and the second image of the target captured by the second camera unit 12 are related in content when the multi-camera module 10 captures the image of the target, in the actual image synthesis process, the area to be processed in the first image may not be related to the content of the second image. For example, in one image fusion scheme, a region with low imaging quality in the first image is set as the region to be processed. Ideally, the content of the second image should correspond to the region to be processed and have high imaging quality. In this way, the image effect of the subject having high imaging quality globally can be obtained by fusing the first image and the second image. However, in the actual imaging process, the correspondence between the second image and the region to be processed is determined by the physical positional relationship between the first camera unit 11 and the second camera unit 12 (i.e., the relative positional relationship between the first camera unit 11 and the second camera unit 12). That is, when the relative positional relationship between the first camera unit 11 and the second camera unit 12 does not meet the preset requirements, the second image will not correspond to the region to be processed in the second image, and thus a better visual effect cannot be obtained through image fusion processing.
[0074] To meet the requirements of subsequent image processing, the relative positional relationship between the first camera unit 11 and the second camera unit 12 can be adjusted in this embodiment. Specifically, the positional change between the first camera unit 11 and the second camera unit 12 is achieved through a moving mechanism 13, such as... Figure 1 As shown.
[0075] Specifically, such as Figure 1As shown, in this embodiment, the second camera unit 12 is mounted on the moving mechanism 13 to drive the second camera unit 12 to change the relative positional relationship between the first camera unit 11 and the second camera unit 12. Specifically, in this embodiment, the moving mechanism 13 is configured to adjust the relative positional relationship between the first camera unit 11 and the second camera unit 12 based on an adjustment command. The adjustment command is generated based on the area to be processed in the first image of the target acquired by the first camera unit 11. That is, in this embodiment, the translation structure is configured to adjust the relative positional relationship between the first camera unit 11 and the second camera unit 12 based on the requirements of subsequent image processing.
[0076] Figure 5 The illustration shows a schematic diagram of the second camera unit 12 in the multi-camera module 10 according to an embodiment of this application. Figure 5 As shown, in this embodiment of the application, the moving mechanism 13 includes: a housing 131; a carrier 132 suspended within the housing 131 and used to support the second camera unit 12; and coil-magnet pairs 133 disposed between the carrier 132 and the housing 131 and corresponding to each other, wherein, when turned on, the coil-magnet pairs 133 can drive the carrier 132 to move the second camera unit 12. Specifically, as... Figure 5 As shown, the moving mechanism 13 further includes a ball bearing 134A installed between the carrier 132 and the housing 131, so that the carrier 132 is suspended within the housing 131 by means of the ball bearing 134A.
[0077] Figure 6 The illustration shows a modified embodiment of the second camera unit 12 in the multi-camera module 10 according to an embodiment of this application. For example... Figure 6 As shown, in this modified embodiment, the moving mechanism 13 further includes an elastic element 134B extending between the inner wall of the housing 131 and the outer wall of the carrier 132, so that the carrier 132 is suspended within the housing 131 by means of the elastic element 134B. In a specific embodiment, the elastic element 134B may be implemented as an elastic element 134B such as a leaf spring, spring, or sheet spring.
[0078] It should be understood that in this embodiment, the position of the first camera unit 11 remains fixed, while the position of the second camera unit 12 is adjusted by the moving mechanism 13 to change the relative positional relationship between the first camera unit 11 and the second camera unit 12. Of course, in other examples of this application, the above technical objective can also be achieved in other ways. For example, the position of the second camera unit 12 can be kept fixed, while the position of the first camera unit 11 can be set to be adjustable; or, simultaneously, the positions of both the first camera unit 11 and the second camera unit 12 can be set to be adjustable.
[0079] It is worth mentioning that, in this embodiment, when the position of the first camera unit 11 is fixed and the position of the second camera unit 12 is adjustable, preferably, the imaging window of the second camera unit 12 can be inclined towards the side of the imaging window of the first camera unit 11 away from the second camera unit 12. Thus, when the second camera unit 12 is moved multiple times, the imaging window of the second camera unit 12 can cover any part of the imaging window of the entire first camera unit 11. In a specific example of this application, the imaging window of the second camera unit 12 can be more inclined towards the central area of the imaging window of the first camera unit 11, so that the second image of the target captured by the second camera unit 12 and the first image of the captured image by the first camera unit 11 can have a higher correlation in content. That is, in this embodiment, preferably, the second optical axis X2 set by the second camera unit 12 is tilted towards the first optical axis X1 set by the first camera unit 11, forming an angle with the first optical axis X1, such as... Figure 4 As shown. Specifically, in the embodiments of this application, the included angle between the first optical axis X1 and the second optical axis X2 is 0.1° to 45°, and more preferably, the included angle is in the range of 0.1° to 10°.
[0080] In summary, the multi-camera module 10 described in the embodiments of this application is explained, wherein the structural configuration of the multi-camera module 10 enables it to perform optical zoom based on the distance between itself and the subject, so that when the viewfinder simultaneously includes both close-up and distant views, the multi-camera module 10 can capture clear images of the subject at different depths of field, so that the final synthesized image has a better imaging effect.
[0081] In particular, in this embodiment of the application, the multi-camera module 10 is configured with a camera unit with optical zoom function, and the relative positional relationship between the camera unit with optical zoom function and other camera units can be changed so that the multi-camera module 10 can acquire clear images of the subject at different depths of field, thereby the final synthesized image has a better imaging effect.
[0082] As described above, in this embodiment of the application, the moving mechanism 13 is configured to adjust the relative positional relationship between the first camera unit 11 and the second camera unit 12 based on an adjustment instruction, wherein the adjustment instruction is generated based on the area to be processed in the first image of the target acquired by the first camera unit 11.
[0083] To illustrate the movement of the moving mechanism 13 (i.e., the change in the relative positions of the second camera unit 12 and the first camera unit 11), the imaging method applicable to the multi-camera module 10 will be described below.
[0084] Schematic imaging method
[0085] Figure 7 The illustration shows a flowchart of an imaging method suitable for the multi-camera module 10 according to an embodiment of the present application.
[0086] like Figure 7 As shown, the imaging method according to the embodiments of this application includes the following steps: S110, obtaining a first image of the target captured by the first camera unit 11 and a second image of the target captured by the second camera unit 12; S120, determining at least one region to be processed in the first image; S130, generating an adjustment command based on the relative positional relationship between the mapping image of the second image onto the first image and the region to be processed; S140, driving the moving mechanism 13 based on the adjustment command to move the second camera unit 12 until the mapping image of the second image captured by the second camera unit 12 onto the first image corresponds to the region to be processed. S150, controlling the second camera unit 12 to perform optical zoom and obtain a zoomed second image of the subject; S160, generating a second adjustment command based on the relative positional relationship between the mapped image of the zoomed second image to the first image and the area to be processed; S170, driving the moving mechanism 13 to move the second camera unit 12 based on the second adjustment command, wherein, during the movement of the second camera unit 12, at least one zoomed second image of the subject acquired by the second camera unit 12 is obtained; and S180, fusing the first image and the zoomed second image to obtain a fused image.
[0087] In step S110, a first image of the target captured by the first camera unit 11 and a second image of the target captured by the second camera unit 12 are obtained. As mentioned above, in this embodiment, when the first camera unit 11 and the second camera unit 12 simultaneously capture images of the target, the imaging window of the first camera unit 11 and the imaging window of the second camera unit 12 at least partially overlap. More specifically, the imaging window of the second camera unit 12 is smaller than the imaging window of the first camera unit 11, and if the two are positioned at an appropriate distance, the imaging window of the second camera unit 12 is located within the imaging window of the first camera unit 11. Therefore, when the target is captured by the multi-camera module 10, the images of the target captured by the first camera unit 11 and the second camera unit 12 are related in content, so a fused image with better imaging effect can be obtained by synthesizing the first image of the target captured by the first camera unit 11 and the second image of the target captured by the second camera unit 12.
[0088] In step S120, at least one region to be processed in the first image is determined. Here, in this embodiment, the selection of the region to be processed is related to the final image compositing effect. For example, when the final image compositing effect is set to generate an image with high imaging quality of the subject globally, the region to be processed can be set as the region in the first image whose imaging quality needs to be compensated; that is, the region in the first image with low imaging quality is determined as the region to be processed. As another example, when the final image compositing effect is set to blur the background portion of the scene in which the subject is photographed, the region to be processed can be set as the middle region of the first image (usually the middle region corresponds to the subject).
[0089] Furthermore, after determining the selection criteria for the region to be processed, the at least one region to be processed in the first image can be determined at least in the following manner.
[0090] In one example of this application, the process of determining at least one region to be processed in the first image includes: determining at least one region in the first image with relatively low imaging quality as the at least one region to be processed. In specific implementations, the at least one region in the first image with relatively low imaging quality can be determined as the at least one region to be processed using functions such as the Brenner gradient function, the Tenengrad gradient function, or the Laplacian gradient function. It is worth noting that, in this embodiment of the application, a region with low imaging quality can represent a region with low sharpness in the image.
[0091] In another example of this application, the process of determining at least one region to be processed in the first image includes: first, receiving a region-to-be-processed designation instruction; and then, in response to the region-to-be-processed designation instruction, determining at least one region to be processed in the first image. That is, in this example, the region to be processed is manually set, specifically determined by a user applying a designation instruction, wherein the designation instruction includes clicking a corresponding region of the first image, double-clicking a corresponding region of the first image, etc., which is not limited to this application.
[0092] In yet another example of this application, determining at least one region to be processed in the first image includes: determining at least one region to be processed in the first image based on default settings. That is, in this example, at least one region to be processed in the first image is determined based on the system's default settings.
[0093] It is worth mentioning that when the area to be processed is selected by the user or set by the system default, the second camera unit 12 can perform automatic optical zoom based on the operation in step S150 described later, or it can perform optical zoom by the user selecting the zoom ratio or by the system default zoom ratio.
[0094] In step S130, an adjustment command is generated based on the relative positional relationship between the mapped image of the second image onto the first image and the area to be processed. Here, the adjustment command is used to drive the moving mechanism 13 to move the second camera unit 12 to the position where the mapped image of the second image acquired by the second camera unit 12 corresponds to the area to be processed.
[0095] That is, after determining at least one region to be processed in the first image, the relative positional relationship between the first camera unit 11 and the second camera unit 12 is changed so that the content of the second image of the target captured by the second camera unit 12 corresponds to the region to be processed. For example, in the above example of this application, the region to be processed is the area in the first image whose imaging quality needs to be compensated. Accordingly, the purpose of step S130 is to generate an adjustment instruction for driving the moving mechanism 13 to move the second camera unit 12 so that the content of the second image of the target captured by the second camera unit 12 corresponds to the region to be processed.
[0096] Here, the specific process of generating adjustment instructions based on the relative positional relationship between the mapping image of the second image to the first image and the region to be processed in step S160 will reappear, so it will not be elaborated on here.
[0097] In step S140, based on the adjustment command, the moving mechanism 13 is driven to move the second camera unit 12 to a position where the mapped image of the second image acquired by the second camera unit 12 corresponds to the area to be processed in the first image. That is, based on the adjustment command, the moving mechanism 13 is driven to move the second camera unit 12 so that the content of the second image of the target acquired by the second camera unit 12 corresponds to the area to be processed.
[0098] In step S150, the second camera unit 12 is controlled to perform optical zoom and obtain a zoomed second image of the target. As mentioned above, in this embodiment, the second camera unit 12 has optical zoom capability. Therefore, in this embodiment, the second camera unit 12 can perform optical zoom based on the distance between itself and the target or the sharpness of the second image, so that the second camera unit 12 can acquire a second image of the target with relatively high imaging quality.
[0099] In step S160, a second adjustment instruction is generated based on the relative positional relationship between the mapped image of the zoomed second image to the first image and the area to be processed. The second adjustment instruction is used to drive the moving mechanism 13 to drive the second camera unit 12.
[0100] Specifically, in one example of this application, the process of generating a second adjustment instruction based on the relative positional relationship between the mapped image of the zoomed second image onto the first image and the area to be processed first includes: determining the number of pixels Mx and My in the X and Y directions of the area to be processed in the first image; then determining the number of pixels Nx and Ny in the mapped image in the X and Y directions of the first image; and then generating the second adjustment instruction based on Mx, My, Nx, and Ny.
[0101] More specifically, in this embodiment, the process of generating the second adjustment instruction based on Mx, My, Nx, and Ny when Nx > Mx and Ny > My includes: generating the second adjustment instruction in response to Nx > Mx and Ny > My, wherein the adjustment instruction is used to drive the moving mechanism 13 to move the second camera unit 12, so that the center of the mapped image is aligned with the center of the area to be processed. That is, when the imaging window of the second camera unit 12 can cover the area to be processed in the imaging window of the first camera unit 11, the second camera unit 12 is moved so that the center of the imaging window of the second camera unit 12 coincides with the center of the area to be processed in the imaging window of the first camera unit 11 (it is worth mentioning that, in specific implementations, near-coincidence is sufficient).
[0102] Conversely, when Nx is less than Mx or when Nx is less than My, the integer multiple relationships between Mx and Nx and My and Ny are calculated respectively (if there is a remainder, the multiple is increased by 1), and the number of times the second camera unit 12 needs to move in the X and Y directions is obtained, so that the second camera unit 12 can be moved multiple times, so that the multiple imaging windows of the second camera unit 12 can cover the area to be processed in the imaging window of the first camera unit 11. Accordingly, the process of generating the second adjustment instruction based on Mx, My, Nx, and Ny further includes: determining a first integer multiple relationship between Mx and Nx in response to Mx being greater than Nx; determining a second integer multiple relationship between My and Ny in response to My being greater than Ny; and generating the second adjustment instruction based on the first integer multiple relationship and the second integer multiple relationship, wherein the second adjustment instruction is used to drive the moving mechanism 13 to move the second camera unit 12 along the X direction by at least a first integer multiple; and to drive the moving mechanism 13 to move the second camera unit 12 along the Y direction by at least a second integer multiple.
[0103] It is worth mentioning that, in other examples of this application, the displacement of the second camera unit 12 can also be determined in other ways. For example, the displacement of the second camera unit 12 can be determined based on the center position of the area to be processed in the first image acquired by the first camera unit 11. Specifically, the center position of the area to be processed can be set as (x1, y1), then the required translation amount d(x, y) of the second camera unit 12 is k(x1, y1), where k is a translation parameter, which can be calculated from the parameters of the second camera unit 12 and the first camera unit 11. The relevant parameters include the angle between the optical axes of the second camera unit 12 and the first camera unit 11, the field of view of the first camera unit 11 and the second camera unit 12, the image plane size, etc.
[0104] Alternatively, the translation parameter k can be obtained through calibration using a target plate. A target plate is placed in front of the multi-camera module 10. The focal length of the second camera unit 12 is changed, altering its zoom ratio. The second camera unit 12 is then translated, and the translation amount is recorded. This yields the translation amount of the mapping image of the second image captured by the second camera module onto the first image at that zoom ratio. Then, by changing the zoom ratio of the second camera unit 12, the translation amount of the mapping image of the second image captured by the second camera module onto the first image is obtained at multiple different optical zoom ratios. The translation parameter k is calculated using these multiple sets of data.
[0105] For example, the translation amount of the second camera unit 12 can be determined by constructing a zoom ratio-translation amount comparison table. Specifically, a target plate is placed in front of the multi-camera module 10. By changing the focal length of the second camera unit 12 to change its zoom ratio, the second camera unit 12 is translated and its translation amount is recorded. This yields the translation amount of the mapping image of the second image captured by the second camera module onto the first image at that zoom ratio. By changing the zoom ratio of the second camera unit 12, the translation amount of the mapping image of the second image captured by the second camera module onto the first image at multiple different optical zoom ratios is obtained, resulting in a zoom ratio-translation amount comparison table. The second camera unit 12 can then use this comparison table to obtain the relationship between its translation amount and the image translation amount at different zoom ratios.
[0106] Accordingly, in this embodiment of the application, an adjustment instruction is generated based on the relative positional relationship between the mapped image of the second image to the first image and the region to be processed, including: determining the relative positional relationship between the center of the region to be processed and the center of the mapped image; and generating the adjustment instruction based on a pre-calibrated correspondence table between the relative position of the center of the region to be processed and the mapped image and the translation position of the second camera unit 12.
[0107] Accordingly, in this embodiment, a second adjustment instruction is generated based on the relative positional relationship between the mapped image (the zoomed second image mapped to the first image) and the region to be processed. This includes: determining the relative positional relationship between the center of the region to be processed and the center of the mapped image; and generating the second adjustment instruction based on a pre-calibrated correspondence table between the relative position of the center of the region to be processed and the mapped image and the translation position of the second camera unit 12.
[0108] In step S170, based on the second adjustment command, the moving mechanism 13 is driven to move the second camera unit 12, wherein, during the movement of the second camera unit 12, at least one zoomed second image of the subject is obtained by the second camera unit 12.
[0109] Specifically, in a practical implementation, each time the second camera unit 12 is moved, a zoomed second image of the target can be acquired through the second camera unit 12 to obtain multiple zoomed second images.
[0110] Accordingly, in conjunction with steps S130 and S140, as well as steps S160 and S170, it can be seen that in the imaging method according to the embodiments of this application, the relative positional relationship between the second camera unit 12 and the first camera unit 11 is changed twice: once before the second camera unit 12 performs optical zoom, and again after the second camera unit 12 performs optical zoom. That is, in the imaging method according to the embodiments of this application, the position of the second camera unit 12 is changed twice by the moving mechanism 13. The purpose of the first change is to move the second camera unit 12 to a position that roughly corresponds to the area to be processed for optical zoom, and the purpose of the second change is to ensure that the imaging window of the second camera unit 12 can completely cover the area to be processed. Specifically, when the imaging window of the second camera unit 12 is larger than the area to be processed, the purpose of another movement is achieved by moving the second camera unit 12 once; when the imaging window of the second camera unit 12 is smaller than the area to be processed, the purpose of another movement is achieved by moving the second camera unit 12 multiple times so that the combined window formed by the movement of the imaging window of the second camera unit 12 completely covers the area to be processed.
[0111] In step S180, the first image and the zoomed second image are merged to obtain a merged image. Accordingly, in this embodiment of the application, merging the first image and the zoomed second image to obtain a merged image includes: merging the first image and the multiple zoomed second images to obtain the merged image.
[0112] In summary, the imaging method based on the embodiments of this application has been clarified, wherein the implementation of the imaging method relies on the optimization and improvement of the structural configuration of the multi-camera module 10. That is, the optimization of the structural configuration of the multi-camera module 10 provides the necessary hardware foundation for the implementation of the imaging method, thereby providing users with a better visual experience through the imaging method and the hardware configuration of the multi-camera module 10.
[0113] illustrative camera system
[0114] According to another aspect of this application, a camera system is also provided.
[0115] Figure 8 The illustration shows a schematic diagram of the camera system according to an embodiment of this application.
[0116] like Figure 8 As shown, the camera system 30 includes a multi-camera module 10 as described above and a processor 20 communicatively connected to the multi-camera module 10. The processor 20 is configured to generate the adjustment instruction based on the region to be processed in the first image of the target acquired by the first camera unit 11. Accordingly, upon receiving the adjustment instruction, the moving mechanism 13 adjusts the relative positional relationship between the first camera unit 11 and the second camera unit 12 based on the adjustment instruction.
[0117] Accordingly, in this embodiment, the processor 20 is further configured to fuse the first image of the target captured by the first camera unit 11 and the second image of the target captured by the second camera unit 12 to obtain a fused image.
[0118] Indicative electronic devices
[0119] According to another aspect of this application, an electronic device 100 is also provided.
[0120] Figure 9 The illustration shows a perspective view of an electronic device 100 according to an embodiment of this application.
[0121] like Figure 9 As shown, the electronic device 100 according to an embodiment of this application includes an electronic device body 90 and a multi-camera module 10 as described above, assembled on the electronic device body 90. In a specific implementation, the multi-camera module 10 is preferably disposed on the back of the electronic device body 90 to be configured as a rear camera module. Of course, it can also be disposed on the front of the electronic device body 90 to be configured as a front camera module.
[0122] like Figure 9 As shown in this embodiment, the main body 90 of the electronic device includes a screen and an integrated circuit. The screen can be used to display image data collected by the multi-camera module 10, and the integrated circuit can be used to process the image data collected by the multi-camera module 10 to control the multi-camera module 10 to realize its imaging function.
[0123] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and any modifications or variations of the embodiments of the present invention may be made without departing from the stated principles.
Claims
1. A multi-camera module, characterized in that, include: First camera unit; A second camera unit with zoom capability; as well as A moving mechanism configured to adjust the relative positional relationship between the first camera unit and the second camera unit, wherein the imaging windows of the first camera unit and the second camera unit at least partially overlap, wherein... The moving mechanism is configured to: Before the second camera unit performs optical zoom, the relative positional relationship between the first camera unit and the second camera unit is adjusted so that the second camera unit corresponds to the position of the area to be processed for optical zooming. The first camera unit is configured to acquire a first image of the target so that a processor communicatively connected to the multi-camera module can generate adjustment instructions based on the area to be processed in the first image. The moving mechanism is configured to move the second camera unit according to the adjustment instructions to the position of the mapped image of the second image acquired by the second camera unit corresponding to the area to be processed in the first image. After the second camera unit performs optical zoom, the relative positional relationship between the first camera unit and the second camera unit is adjusted so that the imaging window of the second camera unit or the combined window formed by the movement of the imaging window completely covers the area to be processed. During the movement of the second camera unit, at least one zoomed second image of the target acquired by the second camera unit is obtained. The processor is further configured to fuse the first image and the zoomed second image to obtain a fused image.
2. The multi-camera module according to claim 1, wherein, The second optical axis of the second camera unit is tilted in a direction that tends toward the first optical axis of the first camera unit, so as to form an angle with the first optical axis.
3. The multi-camera module according to claim 2, wherein, The angle between the first optical axis and the second optical axis is 0.1° to 45°.
4. The multi-camera module according to claim 2, wherein, The angle between the first optical axis and the second optical axis ranges from 0.1° to 10°.
5. The multi-camera module according to claim 1, wherein, The second camera unit is mounted on the moving mechanism so that the moving mechanism drives the second camera unit to change the relative positional relationship between the first camera unit and the second camera unit.
6. The multi-camera module according to claim 5, wherein, The moving mechanism includes: a housing, a carrier suspended within the housing for carrying the second camera unit, and a coil-magnet pair disposed between the carrier and the housing and corresponding to each other.
7. The multi-camera module according to claim 6, wherein, The moving mechanism further includes ball bearings installed between the carrier and the housing, so that the carrier is suspended within the housing by means of the ball bearings.
8. The multi-camera module according to claim 6, wherein, The moving mechanism further includes an elastic element extending between the inner wall of the housing and the outer wall of the carrier, so that the carrier is suspended within the housing by means of the elastic element.
9. The multi-camera module according to claim 1, wherein, The first field of view of the first camera unit is greater than 60°, and the maximum second field of view of the second camera unit is less than 30°.
10. A camera system, characterized in that, include: The multi-camera module according to any one of claims 1-9; as well as A processor communicatively connected to the multi-camera module, wherein the processor is configured to generate the adjustment instruction based on the region to be processed in a first image of the target acquired by the first camera unit.
11. The camera system according to claim 10, wherein, The processor is further configured to fuse a first image of the target captured by the first camera unit and a second image of the target captured by the second camera unit to obtain a fused image.
12. An electronic device, characterized in that, include: The multi-camera module according to any one of claims 1-9.
13. An imaging method for a camera system, characterized in that, include: Obtain a first image of the target captured by the first camera unit and a second image of the target captured by the second camera unit; Determine at least one region to be processed in the first image; Based on the relative positional relationship between the mapped image of the second image to the first image and the region to be processed, an adjustment instruction is generated; Based on the adjustment command, the moving mechanism is driven to move the second camera unit to the position of the mapped image of the second image captured by the second camera unit corresponding to the area to be processed in the first image; Control the second camera unit to perform optical zoom and obtain a zoomed second image of the subject; Based on the relative positional relationship between the mapped image of the zoomed second image onto the first image and the area to be processed, a second adjustment instruction is generated; Based on the second adjustment command, a moving mechanism is driven to move the second camera unit so that the imaging window of the second camera unit can cover the area to be processed in the imaging window of the first camera unit. During the movement of the second camera unit, at least one zoomed second image of the target object is obtained through the second camera unit. The first image and the zoomed second image are merged to obtain a fused image.
14. The imaging method according to claim 13, wherein, Determining at least one region to be processed in the first image includes: determining at least one region in the first image with relatively low imaging quality as the at least one region to be processed.
15. The imaging method according to claim 13, wherein, Determining at least one region to be processed in the first image includes: Receive instructions specifying the region to be processed; and In response to the instruction specifying the region to be processed, at least one region to be processed in the first image is determined.
16. The imaging method according to claim 13, wherein, Determining at least one region to be processed in the first image includes: determining at least one region to be processed in the first image based on default settings.
17. The imaging method according to claim 13, wherein, Based on the relative positional relationship between the mapped image of the zoomed second image onto the first image and the area to be processed, a second adjustment instruction is generated, including: Determine the number of pixels Mx and My in the X and Y directions of the region to be processed in the first image; Determine the number of pixels Nx and Ny of the mapped image in the X and Y directions set in the first image; and Based on Mx, My, Nx, and Ny, the second adjustment instruction is generated.
18. The imaging method according to claim 17, wherein, Based on Mx, My, Nx, and Ny, the second adjustment instruction is generated, including: In response to Nx > Mx and Ny > My, a second adjustment instruction is generated, wherein the adjustment instruction is used to drive the moving mechanism to move the second camera unit so that the center of the mapped image is aligned with the center of the area to be processed.
19. The imaging method according to claim 17, wherein, Based on Mx, My, Nx, and Ny, the second adjustment instruction is generated, including: In response to Mx being greater than Nx, a first integer multiple relationship between Mx and Nx is determined; In response to My being greater than Ny, a second integer multiple relationship between My and Ny is determined; Based on the first integer multiple relationship and the second integer multiple relationship, a second adjustment instruction is generated, wherein the second adjustment instruction is used to drive the moving mechanism to move the second camera unit along the X direction by at least a first integer multiple; and to drive the moving mechanism to move the second camera unit along the Y direction by at least a second integer multiple.
20. The imaging method according to claim 19, wherein, During the movement of the second camera unit, at least one zoomed second image of the subject captured by the second camera unit is obtained, including: each time the second camera unit moves, a zoomed second image of the subject captured by the second camera unit is obtained, so as to obtain multiple zoomed second images; The process of fusing the first image and the zoomed second image to obtain a fused image includes: fusing the first image and the multiple zoomed second images to obtain the fused image.
21. The imaging method according to claim 13, wherein, Based on the relative positional relationship between the mapped image of the second image to the first image and the region to be processed, an adjustment instruction is generated, including: Determine the relative positional relationship between the center of the region to be processed and the center of the mapped image; and The adjustment command is generated based on the correspondence table between the relative position of the center of the region to be processed and the mapped image and the translation position of the second camera unit.
22. The imaging method according to claim 13, wherein, Based on the relative positional relationship between the mapped image of the zoomed second image onto the first image and the area to be processed, a second adjustment instruction is generated, including: Determine the relative positional relationship between the center of the region to be processed and the center of the mapped image; and The second adjustment instruction is generated based on the correspondence table between the center of the region to be processed and the mapped image, which is pre-calibrated, and the translation position of the second camera unit.