Imaging device, medium, and system
The described imaging system uses multiple focal length cameras and an active distance sensor to ensure all subjects at varying distances are in focus by controlled image capture and integration, addressing the challenge of blurred subjects in conventional systems.
Patent Information
- Application Number
- PCT/JP2025/005885
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-02-20
- Filing Date
- 2025-02-20
- Publication Date
- 2025-08-28
AI Technical Summary
Conventional imaging systems struggle to capture images where multiple subjects at different distances are all in focus, often resulting in some subjects being blurred.
An imaging system equipped with multiple cameras of varying focal lengths and an active distance measurement sensor to identify subject distances, allowing controlled image capture and integration of focused images from each camera.
Enables capturing images where multiple subjects at different distances are all in focus by using a combination of cameras and distance measurement to adjust focus and integrate focused areas.
Smart Images

Figure JP2025005885_28082025_PF_FP_ABST
Abstract
Description
IMAGING DEVICE, MEDIUM, AND SYSTEM
[0001] The present disclosure relates to an imaging system, an imaging device, an imaging method, and a program.Background
[0002] In recent years, some portable information processing devices such as smartphones and tablet terminals include a plurality of cameras such as a wide-angle camera and a telephoto camera. For example, PTL 1 discloses a technique of determining depth information from an image captured by a wide-angle camera and an image captured by a telephoto camera, and performing a blurring process on the image according to the depth information.
[0003] Japanese Laid-open Patent Publication No. 2020-535764Summary
[0004] However, in the above-described conventional technology, when a plurality of subjects at different distances is imaged, an image in which only one subject is in focus, and another subject is blurred is captured.
[0005] Therefore, the present disclosure proposes an imaging system, an imaging device, an imaging method, and a program capable of capturing an image in which a plurality of subjects is in focus even in a case where the plurality of subjects is at different distances.
[0006] According to the present disclosure, an imaging system including: a plurality of cameras having different focal lengths; a distance measurement sensor configured to perform distance measurement in an imaging range of the plurality of cameras; an identification unit configured to identify distances to a plurality of subjects included in the imaging range, from a result of the distance measurement by the distance measurement sensor; an imaging control unit configured to perform control to capture an image with a camera corresponding to a distance to each subject among the plurality of cameras; and an integration unit configured to integrate images captured by cameras corresponding to distances to respective subjects.
[0007] Fig. 1 is a schematic view of a smartphone according to the embodiment of the present disclosure.Fig. 2 is a schematic view of a smartphone according to the embodiment of the present disclosure.Fig. 3 is a block diagram illustrating a configuration example of a smartphone according to the embodiment of the present disclosure.Fig. 4A is an explanatory diagram for describing an imaging method according to the first comparative example.Fig. 4B is an explanatory diagram for describing an imaging method according to the second comparative example.Fig. 5 is an explanatory diagram for describing an imaging method of the present disclosure.Fig. 6 is a flowchart illustrating a processing procedure of imaging processing executed by the smartphone according to the embodiment of the present disclosure.Fig. 7 is a block diagram illustrating a configuration example of a smartphone according to a modification of the present disclosure.Fig. 8 is a block diagram illustrating a configuration example of a smartphone according to a modification of the present disclosure.Fig. 9 is a block diagram illustrating a configuration example of a smartphone according to a modification of the present disclosure.Fig. 10 is a block diagram illustrating a configuration example of a smartphone according to a modification of the present disclosure.Fig. 11 is a block diagram illustrating a configuration example of a smartphone according to a modification of the present disclosure.Fig. 12A is a view illustrating an example of arrangement of a plurality of cameras and a distance measurement sensor according to a modification of the present disclosure.Fig. 12B is a view illustrating an example of arrangement of a plurality of cameras and a distance measurement sensor according to a modification of the present disclosure.Fig. 12C is a view illustrating an example of arrangement of a plurality of cameras and a distance measurement sensor according to a modification of the present disclosure.Fig. 13 is a block diagram illustrating a configuration example of a smartphone according to a modification of the present disclosure.Fig. 14 is an explanatory diagram for describing an imaging method according to a modification of the present disclosure.Fig. 15 is a hardware configuration diagram illustrating an example of a computer that implements functions of a smartphone.
[0008] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same parts are denoted by the same reference signs, and a duplicate description will be omitted.
[0009] In the following description, a case where the technology of the present disclosure is applied to a smartphone 10 will be described as an example. Furthermore, in the following description, an application for imaging operated on the smartphone 10 may be referred to as a "camera application".
[0010] Further, the present disclosure will be described in the order of the following items. 1. Configuration of smartphone 1-1. Schematic configuration of smartphone 1-2. Functional configuration of smartphone 2. Specific example of imaging 2-1. First comparative example 2-2. Second comparative example 2-3. Imaging technique of the present disclosure 3. Processing procedure 4. Modifications 5. Hardware configuration 6. Conclusion
[0011] <<1. Configuration of smartphone>> <1-1. Schematic configuration of smartphone> Fig. 1 is a schematic view of a smartphone 10 according to an embodiment of the present disclosure. Fig. 2 is a schematic view of the smartphone 10 according to the embodiment of the present disclosure. Note that Fig. 1 illustrates a configuration of the front face of the smartphone 10. Furthermore, Fig. 2 illustrates a configuration of the back face of the smartphone 10.
[0012] The smartphone 10 is a portable information processing device having a camera function. In the present embodiment, the smartphone 10 corresponds to the imaging system and the imaging device of the present disclosure. As illustrated in Fig. 1, a touch screen TS is provided on the front face of the smartphone 10. The touch screen TS is a device in which a touch panel and a display are integrated. The touch screen TS detects a touch operation by the user. As a detection method of the touch screen TS, known methods such as a capacitance method, a resistive film method, a face acoustic wave method (or an ultrasonic method), an infrared method, an electromagnetic induction method, and a load detection method are used.
[0013] The touch screen TS has a rectangular shape. At one end of the smartphone 10 in the longitudinal direction, for example, a camera 16 and an earpiece 15 are provided. The camera 16 includes, for example an inward in-camera mainly used by a user to capture an image of the user himself / herself. A speaker is provided in the earpiece 15. Although not illustrated, at the other end of the smartphone 10, for example, a mouthpiece having a microphone is provided.
[0014] On the side face of the smartphone 10 along the long side of the touch screen TS, for example, a volume key 11, a fingerprint sensor 12, a power key 13, and a camera key 14 are provided. The volume key 11 is a hardware key that performs volume adjustment. The fingerprint sensor 12 is a device that reads a fingerprint. The power key 13 is a hardware key that turns on / off power. A camera key 14 is a hardware key that activates a camera application. The camera key 14 is used as a shutter at the time of imaging. The camera key 14 is provided, for example, at a position closer to the mouthpiece (opposite to the earpiece 15) than the central portion of the side face of the smartphone 10.
[0015] Furthermore, as illustrated in Fig. 2, a plurality of cameras 17 and a distance measurement sensor 18 are provided on the back face of the smartphone 10. In the example of Fig. 2, for example, three cameras 17a to 17c are provided as the plurality of cameras 17. The plurality of cameras 17 is outward out-cameras that capture images of the surroundings of the user, such as the direction of the user's line of sight. The plurality of cameras 17 have different focal lengths. For example, the camera 17a is, for example, an ultra-wide-angle camera having a focal length of about 16 mm. Furthermore, the camera 17b is, for example, a standard wide-angle camera having a focal length of about 24 mm. Furthermore, the camera 17c is, for example, a telephoto camera having a focal length of about 85 mm to 125 mm.
[0016] The distance measurement sensor 18 is a sensor adopting a so-called active type distance measuring method of emitting laser light, observing reflected light of the emitted laser light, and measuring a distance to an object. The distance measurement sensor 18 can measure a distance in an imaging range by the plurality of cameras 17. The distance measurement range measured by the distance measurement sensor 18 is only required to include the imaging range of the plurality of cameras 17, and may be wider than the imaging range of the plurality of cameras 17. That is, the distance measurement sensor 18 may be capable of distance measurement in a wide range beyond the range that can be captured by the plurality of cameras 17.
[0017] A flash light 19 and a red green blue clear-infrared (RGBC-IR) sensor 20 are provided in the vicinity of the camera 17. The flash light 19 is a device that lights up at the time of imaging to brightly illuminate a subject to be imaged. The RGBC-IR sensor 20 is a device that detects a component of a light source in an imaging environment at the time of imaging. The RGBC-IR sensor 20 acquires, for example, infrared information around the smartphone 10, and identifies a light source environment such as outdoor, an incandescent lamp, or a fluorescent lamp.
[0018] The smartphone 10 is equipped with a camera application, and images can be captured by the plurality of cameras 17. When the camera application is activated, the smartphone 10 captures a moving image with any of the cameras 17a to 17c, and displays the captured moving image on the touch screen TS as a live view. The smartphone 10 receives designation of a subject in a live view image displayed on the touch screen TS. The subject may be a person or an object such as a flower or a thing other than a person. The smartphone 10 can designate a plurality of subjects. In addition, the smartphone 10 receives an image capturing instruction on the touch screen TS.
[0019] When capturing an image with the smartphone 10, the user activates a camera application, designates a position of a subject in a live view image displayed on the touch screen TS, and instructs to capture an image. When receiving an image capturing instruction on the touch screen TS, the smartphone 10 captures an image with the plurality of cameras 17.
[0020] <1-2. Functional configuration of smartphone> Next, a functional configuration of the smartphone 10 will be described. Fig. 3 is a block diagram illustrating a configuration example of the smartphone 10 according to the embodiment of the present disclosure. Note that Fig. 3 illustrates only components necessary for describing features of the embodiment of the present disclosure, and omits description of general components.
[0021] In other words, each component illustrated in Fig. 3 is functionally conceptual, and does not necessarily have to be physically configured as illustrated. For example, a specific form of distribution and integration of each block is not limited to the illustrated form, and all or part thereof can be functionally or physically distributed and integrated in an any unit according to various loads, usage conditions, and the like.
[0022] In the description using Fig. 3, the description of the already described components may be simplified or omitted.
[0023] As illustrated in Fig. 3, the smartphone 10 includes a camera group 30, a distance measurement sensor 18, a reception unit 31, and an application processor (AP) 32.
[0024] The camera group 30 is a plurality of cameras mounted on the smartphone 10, and includes the above-described plurality of cameras 17 (cameras 17a to 17c). Each of the plurality of cameras 17 includes an image sensor 40 and a lens 41. The image sensor 40 is, for example, an RGB sensor provided with a sensor unit in which a plurality of light receiving pixels each having a light receiving element that detects any of red (R), green (G), and blue (B) is disposed in a two-dimensional matrix shape. A distance range suitable for imaging is determined for each of the plurality of cameras 17. For example, the plurality of cameras 17 is configured such that distance ranges in which subjects are in focus are different by at least one of the image sensor 40 and the lens 41.
[0025] The distance measurement sensor 18 includes a light emitting unit 18a and a light receiving unit 18b.
[0026] The light emitting unit 18a emits laser light. Examples of such a light emitting unit 18a include a vertical cavity surface emitting laser (VCSEL) that emits laser light as a face light source. Note that the light emitting unit 18a is not limited to the VCSEL, and may be, for example, an edge emitting laser or the like. The light emitting unit 18a is directed in an imaging direction of the plurality of cameras 17, and emits laser light to an imaging range of the plurality of cameras 17.
[0027] The light receiving unit 18b detects reflected light emitted from the light emitting unit 18a and reflected by an object. The light receiving unit 18b includes a pixel array unit in which a plurality of light receiving pixels each having a light receiving element that detects reflected light is disposed in a two-dimensional matrix shape. As the light receiving element, for example, a single photon avalanche diode (SPAD) can be used. The light receiving unit 18b is directed in an imaging direction of the plurality of cameras 17, and detects reflected light from an imaging range of the plurality of cameras 17.
[0028] The distance measurement sensor 18 is a direct time of flight (dToF) sensor that calculates the distance to the object based on the elapsed time from the light emission of the light emitting unit 18a to the detection of the reflected light by the light receiving unit 18b. In the distance measurement sensor 18, the light emitting unit 18a emits laser light to the imaging range of the plurality of cameras 17, and the light receiving unit 18b receives reflected light from the imaging range, thereby measuring the distance to the object for each object included in the imaging range. The distance measurement sensor 18 generates and outputs distance data indicating the distance to the object. The distance data may be distance image data configured by the distance to the object, or may be point cloud data indicating the surface shape of the object by three-dimensional coordinates of a plurality of observation points on the surface.
[0029] The reception unit 31 receives various instructions from the user. The reception unit 31 may receive an instruction by a physical operation by the user, may receive an instruction by recognizing a voice of the user, or may receive an instruction by recognizing a gesture of the user. For example, the reception unit 31 receives various inputs from the user via the touch screen TS. For example, when capturing an image, the reception unit 31 displays a live view image and receives designation of a subject in the image of the live view. Furthermore, the reception unit 31 receives an image capturing instruction.
[0030] The AP 32 is an example of a processor such as a central processing unit (CPU) or a micro processing unit (MPU) that executes an operating system (OS) or various application programs of the smartphone 10. The AP 32 includes a processor and a memory, and controls each unit of the smartphone 10. The AP 32 functions as various processing units and storage units by the operation of the program. For example, the AP 32 includes a distance sensor control unit 50, a distance data storage unit 51, an identification unit 52, an imaging control unit 53, an imaging data storage unit 54, and an integration unit 55.
[0031] When receiving the designation of the subject in the image of the live view, the reception unit 31 stores the position of the designated subject in the image in the distance data storage unit 51. When receiving an image capturing instruction, the reception unit 31 instructs the distance sensor control unit 50 to start image capturing.
[0032] When start of imaging is instructed, the distance sensor control unit 50 controls the distance measurement sensor 18 and performs distance measurement by the distance measurement sensor 18. The distance measurement sensor 18 performs distance measurement in an imaging range of the plurality of cameras 17 under the control of the distance sensor control unit 50 to output distance data.
[0033] The distance data storage unit 51 stores the distance data output from the distance measurement sensor 18. In addition, the distance data storage unit 51 stores setting information. In the setting information, conditions at the time of distance measurement by the distance measurement sensor 18 are stored. Furthermore, the position of the designated subject is stored in the setting information. In a case where a plurality of subjects is designated, the position of each designated subject is stored in the setting information.
[0034] The identification unit 52 identifies the distance to the subject included in the imaging range, from the result of distance measurement by the distance measurement sensor 18. For example, the identification unit 52 reads the distance of the portion corresponding to the position of the subject stored in the setting information of the distance data storage unit 51 from the distance of each area in the imaging range stored in the distance data of the distance data storage unit 51, and identifies the read distance as the distance to the subject. In a case where a plurality of subjects is designated, the identification unit 52 reads the distance of the portion corresponding to the position of the subject for each subject, and identifies the read distance as the distance to the subject.
[0035] The imaging control unit 53 controls the plurality of cameras 17 to control image capturing. For example, the imaging control unit 53 performs control to capture an image with each camera 17 corresponding to the distance of each subject identified by the identification unit 52 among the plurality of cameras 17. The imaging control unit 53 simultaneously captures images with the camera 17 corresponding to the distance of each subject among the plurality of cameras 17. For example, the imaging control unit 53 captures an image by focusing on a subject in each distance range using a camera in which the distance of each subject is the distance range. For example, the imaging control unit 53 images the subject using the camera 17 having a longer focal length as the subject is farther. Each camera 17 captures an image by the image sensor 40 under the control of the imaging control unit 53 to output image data of the captured image.
[0036] The imaging data storage unit 54 stores the image data output from each camera 17. In addition, the imaging data storage unit 54 stores setting information. In the setting information, imaging conditions when an image is captured, such as an F value and a shutter speed of each camera 17, are stored.
[0037] The integration unit 55 integrates the image data stored in the imaging data storage unit 54 to generate image data. For example, the integration unit 55 determines an area in which an image of image data is in focus for each piece of image data. For example, in the focused area, the luminance greatly changes at the contour of the object. The integration unit 55 evaluates a luminance change of a contour of an object appearing in the image of the image data, and determines an area in which the image is in focus. The integration unit 55 may use any determination method as the method of determining the focused area as long as the focused area can be determined.
[0038] The integration unit 55 integrates the areas in which the images of the image data are in focus with respect to the imaging ranges of the plurality of cameras 17, and generates image data of the imaging range. For example, the integration unit 55 generates image data by integrating the areas in which images of other image data are in focus based on an image of an imaging range of any image data. Note that the image integration method may be any integration method as long as focused areas can be integrated. The image data generated by the integration unit 55 is displayed as a captured image on the touch screen TS, and is stored as the captured image.
[0039] <<2. Specific example of imaging>> Next, an imaging method of the present disclosure and an imaging method of comparative examples will be described with reference to Figs. 4A, 4B, and 5. Hereinafter, a case where the subject is a person will be described as an example, but the subject may be an object such as a flower or a thing other than a person.
[0040] <2-1. First comparative example> Fig. 4A is an explanatory diagram illustrating an imaging method according to a first comparative example. In the first comparative example, a case where imaging is performed by a smartphone 90 will be described. The smartphone 90 is provided with a wide-angle camera 91, a telephoto camera 92, and a distance sensor 93. In the first comparative example, three persons of a child 80a, a man 80b, and a woman 80c are included in imaging ranges of the wide-angle camera 91 and the telephoto camera 92. In Fig. 4A, the child 80a, the man 80b, and the woman 80c are away from the smartphone 90 in this order. The child 80a is located at a close distance from the smartphone 90. The man 80b is located at an intermediate distance farther away from the smartphone 90 than the child 80a. The woman 80c is located farther away from the smartphone 90 than the man 80b. When instructed to capture an image, the smartphone 90 captures an image with a camera suitable for the distance to the subject among the wide-angle camera 91 and the telephoto camera 92. That is, the smartphone 90 captures an image by the wide-angle camera 91 or the telephoto camera 92 according to the distance to the subject. Fig. 4A illustrates a case where the smartphone 90 images the child 80a as a subject. When imaging is instructed, the smartphone 90 acquires distance data by the distance sensor 93 and identifies the distance to the child 80a as a subject. Then, the smartphone 90 captures an image with a camera corresponding to the distance to the subject of the wide-angle camera 91 and the telephoto camera 92. For example, in the first comparative example, among the wide-angle camera 91 and the telephoto camera 92, the wide-angle camera 91 corresponding to the distance to the child 80a captures an image. In this case, an image in which the man 80b and the woman 80c are out of focus and the man 80b and the woman 80c are blurred is captured.
[0041] <2-2. Second comparative example> Fig. 4B is an explanatory diagram illustrating an imaging method according to a second comparative example. In the second comparative example, a case where the distance sensor 93 is a passive sensor in the smartphone 90 will be described. The passive distance sensor 93 use a method of measuring a distance of an object based on parallax by a stereo camera. Fig. 4B illustrates a case where the smartphone 90 images the child 80a as a subject in a state where the background is dark. When imaging is instructed, the smartphone 90 measures the distance by the distance sensor 93.
[0042] However, the passive distance sensor 93 cannot correctly set the distance when the background is dark. As a result, the smartphone 90 cannot correctly identify the distance to the child 80a as the subject, and performs imaging based on an incorrect distance. In this case, an image in which the man 80b and the woman 80c are out of focus and the man 80b and the woman 80c are blurred is captured.
[0043] <2-3. Imaging method of the present disclosure> Fig. 5 is an explanatory diagram illustrating an imaging method of the present disclosure. In Fig. 5, a case where an image is captured by the cameras 17a to 17c of the smartphone 10 according to the embodiment will be described. In Fig. 5, three persons including the child 80a, the man 80b, and the woman 80c are included in the imaging range of the cameras 17a to 17c of the smartphone 10. Also in Fig. 5, the child 80a, the man 80b, and the woman 80c are away from the smartphone 10 in this order. The child 80a is located at a close distance from the smartphone 10. The man 80b is located at an intermediate distance farther away from the smartphone 10 than the child 80a. The woman 80c is located farther away from the smartphone 10 than the man 80b. The smartphone 10 receives designation of a subject to be imaged. In Fig. 5, the man 80b, the woman 80c, and the child 80a are designated as subjects. When imaging is instructed, the smartphone 10 acquires distance data by the distance measurement sensor 18. Here, the smartphone 10 emits laser light, and measures a distance to an object by observing reflected light of the emitted laser light, that is, by a so-called active type distance measurement sensor 18. As a result, for example, even in a case where the background is dark, the smartphone 10 can measure the distance to the object, and can identify the distances to the man 80b, the woman 80c, and the child 80a as the subjects.
[0044] The smartphone 10 captures an image with the camera 17 corresponding to the distance of the subject among the cameras 17a to 17c. In Fig. 5, the distance to the child 80a is within the distance range of imaging by the camera 17a which is an ultra-wide-angle camera, and the camera 17a corresponds to imaging of the child 80a. In addition, the distance to the man 80b is within a distance range of imaging of the camera 17b which is a wide-angle camera, and the camera 17b corresponds to imaging of the man 80b. Furthermore, the distance to the woman 80c is within the distance range of imaging by the camera 17c which is a telephoto camera, and the camera 17c corresponds to imaging of the woman 80c. The smartphone 10 focuses the camera 17a on the child 80a, focuses the camera 17b on the man 80b, focuses the camera 17c on the woman 80c, and simultaneously captures images with the cameras 17a to 17c. The smartphone 10 integrates the images of the image data captured by the cameras 17a to 17c to generate image data. For example, the smartphone 10 integrates the areas in which the images of the image data are in focus to generate image data of the imaging range. As a result, the smartphone 10 can capture an image in which a plurality of subjects is in focus even in a case where the plurality of subjects is at different distances. For example, the smartphone 10 can capture an image in which the man 80b, the woman 80c, and the child 80a are in focus.
[0045] <<3. Processing procedure>> Next, a processing procedure of imaging processing executed by the smartphone 10 according to the embodiment of the present disclosure will be described with reference to Fig. 6. Fig. 6 is a flowchart illustrating a processing procedure of imaging processing executed by the smartphone 10 according to the embodiment of the present disclosure. When capturing an image with the smartphone 10, the user activates a camera application, designates a position of a subject in a live view image displayed on the touch screen TS, and instructs to capture an image. The imaging processing illustrated in Fig. 6 is executed when an image capturing instruction is received.
[0046] The distance sensor control unit 50 controls the distance measurement sensor 18 and performs distance measurement by the distance measurement sensor 18 (Step S10). The distance measurement sensor 18 performs distance measurement in an imaging range of the plurality of cameras 17 under the control of the distance sensor control unit 50 to output distance data. The distance data storage unit 51 stores the distance data output from the distance measurement sensor 18.
[0047] The identification unit 52 identifies the distance to the subject included in the imaging range, from the result of distance measurement by the distance measurement sensor 18 (Step S11). For example, the identification unit 52 reads the distance of the portion corresponding to the position of the subject stored in the setting information of the distance data storage unit 51 from the distance of each area in the imaging range stored in the distance data of the distance data storage unit 51, and identifies the read distance as a distance to the subject. In a case where a plurality of subjects is designated, the identification unit 52 reads the distance of the portion corresponding to the position of the subject for each subject, and identifies the read distance as a distance to the subject.
[0048] The imaging control unit 53 controls the plurality of cameras 17 to capture an image (Step S12). For example, the imaging control unit 53 performs control to capture an image with each camera 17 corresponding to the distance of each subject identified by the identification unit 52 among the plurality of cameras 17. The imaging control unit 53 simultaneously captures images with the camera 17 corresponding to the distance of each subject among the plurality of cameras 17. Each camera 17 captures an image by the image sensor 40 under the control of the imaging control unit 53 to output image data of the captured image. The imaging data storage unit 54 stores the image data output from each camera 17.
[0049] The integration unit 55 integrates the image data stored in the imaging data storage unit 54, generates image data (Step S13), and ends the process. For example, the integration unit 55 integrates the areas in which the images of the image data are in focus with respect to the imaging ranges of the plurality of cameras 17, and generates image data of the imaging range.
[0050] <<4. Modifications>> The embodiment of the present disclosure described above can include several modifications.
[0051] In the embodiment of the present disclosure described above, a case where designation of a subject to be imaged is received from a user is described as an example. However, the present invention is not limited thereto. The smartphone 10 may select the subject. Fig. 7 is a block diagram illustrating a configuration example of the smartphone 10 according to a modification of the present disclosure. The AP 32 further includes a selection unit 56. The selection unit 56 selects a plurality of subjects to be imaged from a plurality of objects included in an imaging range of the plurality of cameras 17. For example, the selection unit 56 performs image processing of recognizing an object on an image captured by any of the plurality of cameras 17, and selects a subject from objects appearing in the image. For example, the selection unit 56 performs face recognition on the image of the live view, and selects the recognized face portion as the subject. The selection unit 56 stores the position of the selected subject in the setting information of the distance data storage unit 51. The identification unit 52 identifies the distance to the subject stored in the setting information of the distance data storage unit 51 among the plurality of objects included in the imaging range. As a result, the smartphone 10 can capture an image in which a person is in focus with the person as the subject even when the user does not designate the person as the subject. Furthermore, even in a case where a plurality of subjects is at different distances, the smartphone 10 can capture an image in which the plurality of subjects is in focus. For example, when imaging the man 80b, the woman 80c, and the child 80a illustrated in Fig. 5, the smartphone 10 can capture an image in which the man 80b, the woman 80c, and the child 80a are in focus.
[0052] Furthermore, in the embodiment of the present disclosure described above, the case of receiving the designation of the subject to be imaged from the user is described as an example. However, the present invention is not limited thereto. The identification unit 52 may identify a plurality of subjects in a predetermined order from a plurality of objects included in an imaging range of the plurality of cameras 17 and identify distances to the identified plurality of subjects. The predetermined order may be any order. For example, the predetermined order may be the ascending order of distance, the descending order of distance, or the priority may be determined by dividing the imaging range into a plurality of ranges, and the order may be the descending order of priority. The predetermined order may be predetermined or designated by the user. In addition, the predetermined order may be an order selected by the reception unit 31. When an image is captured, a subject is often closely located. Therefore, the identification unit 52 may identify distances of a plurality of subjects in ascending order of distance with respect to the plurality of objects included in the imaging range of the plurality of cameras 17. For example, the identification unit 52 may identify distances with a predetermined number of areas as the subjects in ascending order of distance from the distance of each area in the imaging range stored in the distance data of the distance data storage unit 51. The predetermined number may be a fixed value or a variable value. The predetermined number may be settable by the user.
[0053] Furthermore, in the embodiment of the present disclosure described above, a case is described as an example in which the plurality of cameras 17 is configured such that the distance ranges in which the subjects are in focus are different. However, the present invention is not limited thereto. Each of the plurality of cameras 17 may be configured to be able to adjust the focal point with the lens 41. Fig. 8 is a block diagram illustrating a configuration example of the smartphone 10 according to a modification of the present disclosure. For example, the plurality of cameras 17 is configured such that the lens 41 is movable along the optical axis, and the focal point is adjustable with the lens 41. The imaging control unit 53 captures an image by adjusting the focal point of the lens 41 with respect to each subject by the camera 17 corresponding to the distance of each subject among the plurality of cameras 17. As a result, the smartphone 10 can capture an image in which a plurality of subjects is in focus even in a case where the plurality of subjects is at different distances. For example, when imaging the man 80b, the woman 80c, and the child 80a illustrated in Fig. 5, the smartphone 10 can capture an image in which the man 80b, the woman 80c, and the child 80a are in focus.
[0054] Furthermore, in the embodiment of the present disclosure described above, the case where the AP 32 functions as the distance sensor control unit 50, the distance data storage unit 51, the identification unit 52, the imaging control unit 53, the imaging data storage unit 54, and the integration unit 55 is described as an example. However, the present invention is not limited thereto. The distance sensor control unit 50, the distance data storage unit 51, the identification unit 52, the imaging control unit 53, the imaging data storage unit 54, and the integration unit 55 may be realized by a plurality of APs or a plurality of arithmetic units. Fig. 9 is a block diagram illustrating a configuration example of the smartphone 10 according to a modification of the present disclosure. The smartphone 10 further includes an AP 33 and an arithmetic unit 34. The AP 32, the AP 33, and the arithmetic unit 34 each include a processor and a memory, and function as various processing units and storage units when a program operates. The AP 32 mainly controls processing related to the distance measurement sensor 18, and has functions of the distance sensor control unit 50 and the distance data storage unit 51. The AP 33 mainly controls processing related to the plurality of cameras 17, and has functions of the identification unit 52, the imaging control unit 53, and the imaging data storage unit 54. The arithmetic unit 34 mainly control image processing, and has the function of the integration unit 55. Even in this case, the smartphone 10 can capture an image in which a plurality of subjects is in focus even when the plurality of subjects is at different distances. For example, when imaging the man 80b, the woman 80c, and the child 80a illustrated in Fig. 5, the smartphone 10 can capture an image in which the man 80b, the woman 80c, and the child 80a are in focus.
[0055] Furthermore, in the embodiment of the present disclosure described above, the case where the smartphone 10 integrates images captured by the plurality of cameras 17 is described as an example. However, the present invention is not limited thereto. For example, a server device different from the smartphone 10 may integrate images. That is, the disclosed technology may be implemented by a configuration of a system including the smartphone 10 and the server device. Fig. 10 is a block diagram illustrating a configuration example of the smartphone 10 according to a modification of the present disclosure. The smartphone 10 can communicate with a server device 60 via a network such as a mobile telephone network or the Internet. The server device 60 is implemented as, for example, a cloud server. The Smartphone 10 transmits image data captured by each camera 17 to the server device 60. For example, the AP 32 transmits the image data and the setting information stored in the imaging data storage unit 54 to the server device 60. The server device 60 has a function of the integration unit 55. The integration unit 55 integrates the image data received from the smartphone 10 to generate image data. The server device 60 has higher processing capability than the smartphone 10. Therefore, by performing image processing of integrating image data by the server device 60, a wide range of integration processing can be performed with high accuracy.
[0056] On the other hand, in a case where the smartphone 10 integrates the images captured by the plurality of cameras 17 as in the above-described embodiment, the smartphone 10 does not require communication power with the server device 60, so that low power consumption can be suppressed. In addition, since the smartphone 10 does not communicate with an external device, security can be secured, and occurrence of communication cost can be suppressed.
[0057] Furthermore, in the embodiment of the present disclosure described above, the case where the integration unit 55 determines areas in which respective images captured by the plurality of cameras 17 are in focus and integrates the areas in which the respective images are in focus is described as an example. However, the present invention is not limited thereto. The integration unit 55 may integrate portions corresponding to the positions of the subjects in the respective images captured by the plurality of cameras 17. Further, the integration unit 55 may integrate portions in which the respective images are in focus and integrate blurred portions in which images other than the subject portion are out of focus. For example, the integration unit 55 may integrate portions other than the subject, such as background blurring. Fig. 11 is a block diagram illustrating a configuration example of the smartphone 10 according to a modification of the present disclosure. The integration unit 55 acquires a portion corresponding to the position of the subject used in identifying the distance to the subject from the identification unit 52. Note that the integration unit 55 may acquire a portion corresponding to the position of the subject from the setting information in the distance data storage unit 51. The integration unit 55 integrates the portions corresponding to the positions of the subjects in the respective images captured by the plurality of cameras 17. For example, the integration unit 55 integrates portions corresponding to the positions of the subjects in respective images based on the image in the imaging range of any image data to generate image data. Portions of each image corresponding to the position of the subject are captured with the subject focused. Therefore, by integrating the portions corresponding to the positions of the subjects in the respective images, the smartphone 10 can capture an image in which a plurality of subjects is in focus even in a case where the plurality of subjects is at different distances. For example, when imaging the man 80b, the woman 80c, and the child 80a illustrated in Fig. 5, the smartphone 10 can capture an image in which the man 80b, the woman 80c, and the child 80a are in focus.
[0058] Furthermore, in the embodiment of the present disclosure described above, the case where the plurality of cameras 17 is linearly disposed and the distance measurement sensor 18 is disposed outside the plurality of cameras 17 is described as an example. However, the present invention is not limited thereto. Each camera 17 is preferably disposed adjacent to the distance measurement sensor 18. In addition, it is preferable that the cameras 17 and the distance measurement sensors 18 are disposed so that distances between respective cameras 17 and the distance measurement sensors 18 are equal. Fig. 12A is a diagram illustrating an example of arrangement of the plurality of cameras 17 and the distance measurement sensor 18 according to a modification of the present disclosure. Fig. 12A illustrates an example of arrangement of the two cameras 17 and the distance measurement sensor 18. The distance measurement sensor 18 is disposed between the two cameras 17 and is adjacent to the two cameras 17. As described above, by disposing the distance measurement sensor 18 adjacent to the two cameras 17, the parallax between the distance measurement sensor 18 and each camera 17 can be reduced.
[0059] In a case where the plurality of cameras 17 is linearly disposed and the distance measurement sensor 18 is disposed outside the plurality of cameras 17, the distance measurement sensor 18 is preferably disposed adjacent to the wide-angle camera. Fig. 12B is a diagram illustrating an example of arrangement of the plurality of cameras 17 and the distance measurement sensor 18 according to a modification of the present disclosure. Fig. 12B illustrates an example of arrangement of the two cameras 17 and the distance measurement sensor 18. In Fig. 12B, two cameras 17 are disposed adjacent to each other on the left and right. A right camera 17x is a wide-angle camera. The distance measurement sensor 18 is disposed adjacent to the right camera 17x. Fig. 12B illustrates a positional deviation between the subject present at the center of the angle of view of the camera 17x and the center of the distance measurement range measured by the distance measurement sensor 18 due to the parallax between the camera 17x and the distance measurement sensor 18. When there is no parallax between the camera 17x and the distance measurement sensor 18, there is no positional deviation. In a case where the distance measurement sensor 18 is not adjacent to the camera 17x and the parallax is large, the positional deviation is large. In a case where the distance measurement sensor 18 is adjacent to the camera 17x and the parallax is small, the positional deviation is small. Fig. 12C is a diagram illustrating an example of arrangement of the plurality of cameras 17 and the distance measurement sensor 18 according to a modification of the present disclosure. In Fig. 12C, two cameras 17 are disposed adjacent to each other on the left and right. A right camera 17y is a telephoto camera. The distance measurement sensor 18 is disposed adjacent to the right camera 17y. Fig. 12C illustrates an example of an image captured by the camera 17x due to the parallax between the camera 17x and the distance measurement sensor 18. The camera 17x can capture substantially the same image in both a case where there is no parallax with the distance measurement sensor 18 and a case where there is a large parallax with the distance measurement sensor 18 without being adjacent to the distance measurement sensor 18. As described above, in the telephoto camera, the influence of the parallax with respect to the distance measurement sensor 18 on the imaging result is small. Therefore, the distance measurement sensor 18 is preferably disposed adjacent to the wide-angle camera. As a result, when integrating a plurality of images captured by the plurality of cameras 17, it is possible to obtain an integration result with less blur.
[0060] Furthermore, in the embodiment of the present disclosure described above, a case where a plurality of subjects is imaged by the plurality of cameras 17 and the captured images are integrated is described as an example. However, the present invention is not limited thereto. A plurality of subjects may be imaged by one camera 17 at different imaging timings, and the captured images may be integrated. Fig. 13 is a block diagram illustrating a configuration example of the smartphone 10 according to a modification of the present disclosure. The smartphone 10 includes one camera 17. The camera 17 is configured such that the focal length of the lens 41 can be changed, and imaging from ultra-wide angle to telephoto can be performed by adjusting the focal length. The imaging control unit 53 controls the camera 17 to capture an image. For example, the imaging control unit 53 adjusts the focal point of the lens 41 and performs control to sequentially image each subject identified by the identification unit 52. As a result, the smartphone 10 can capture an image in which a plurality of subjects is in focus even in a case where the plurality of subjects is at different distances. The imaging data storage unit 54 stores each piece of image data output from the camera 17. The integration unit 55 integrates the image data stored in the imaging data storage unit 54 to generate image data. Fig. 14 is an explanatory diagram for describing an imaging method according to a modification of the present disclosure. In Fig. 14, a case where an image is captured by one camera 17 of the smartphone 10 according to a modification of the present disclosure will be described. The camera 17 is configured to be capable of capturing an image from an ultra-wide angle to a telephoto by adjusting a focal length. In Fig. 14, three persons, i.e., the child 80a, the man 80b, and the woman 80c, are included in the imaging range of the camera 17 of the smartphone 10. In Fig. 14, the child 80a, the man 80b, and the woman 80c are away from the smartphone 10 in this order. The child 80a is located at a close distance from the smartphone 90. The man 80b is located at an intermediate distance farther away from the smartphone 90 than the child 80a. The woman 80c is located farther away from the smartphone 90 than the man 80b. The smartphone 10 receives designation of a subject to be imaged. In Fig. 14, the man 80b, the woman 80c, and the child 80a are designated as subjects. When imaging is instructed, the smartphone 10 acquires distance data by the distance measurement sensor 18. The smartphone 10 images each of the man 80b, the woman 80c, and the child 80a by the camera 17 with the imaging timing shifted. For example, the smartphone 10 adjusts the camera 17 to an ultra-wide angle by changing the focal length, and captures an image by focusing the camera 17 on the child 80a. Next, the smartphone 10 changes the focal length to adjust the camera 17 to a wide angle, and captures an image by focusing the camera 17 on the man 80b. Next, the smartphone 10 changes the focal length to adjust the camera 17 to telephoto, and captures an image by focusing the camera 17 on the woman 80c. The smartphone 10 integrates the images of the image data captured by the camera 17 at different imaging timings to generate image data. As a result, the smartphone 10 can capture an image in which a plurality of subjects is in focus even in a case where the plurality of subjects is at different distances. Furthermore, in a case where the subject is stationary, the smartphone 10 can obtain an optimum integration result since there is no parallax in the images of image data captured by the camera 17 with the imaging timing shifted. Furthermore, the smartphone 10 can reduce the volume and surface area of the portion occupied by the camera 17 when including one camera 17. Furthermore, the smartphone 10 can reduce the manufacturing cost when including one camera 17.
[0061] Furthermore, in the embodiment of the present disclosure described above, the case where the camera 17 is a camera that captures a color image in RGB is described as an example. However, the present invention is not limited thereto. The camera 17 may be a camera that captures a black-and-white image. Furthermore, the camera 17 may be a camera that captures an infrared image. Furthermore, the image sensor 40 of the camera 17 may be a stacked type image sensor in which a sensor unit for detecting light and a circuit unit for reading out the electric charge accumulated in the sensor unit are stacked. Moreover, the image sensor 40 may be a flat image sensor in which the circuit unit is formed in the same layer as the sensor unit, such as around the sensor unit. The stacked image sensor can reduce the chip area. This allows the smartphone 10 to have many cameras 17 arranged thereon, and therefore allows the arrangement of the cameras 17 corresponding to many focal lengths, and an image in which a plurality of subjects at various distances is in focus can be captured. Since the flat image sensor is cheaper than the stacked image sensor, the manufacturing costs of the camera 17 can be reduced. Also in the distance measurement sensor 18, the light emitting unit 18a and the light receiving unit 18b may be a stacked type or a flat type. In a case where the distance measurement sensor 18 is of a stacked type, since the number of simultaneous distance measurement points can be increased or the number of pixels can be increased, the distance to a large number of subjects can be measured. Furthermore, in a case where the distance measurement sensor 18 is of a stacked type, it is possible to realize distance measurement with high resolution and to increase focus selection by the user. On the other hand, in a case where the distance measurement sensor 18 is of a flat type, the manufacturing cost of the distance measurement sensor 18 can be reduced because the flat type is inexpensive, as compared with the stacked type.
[0062] Furthermore, in the embodiment of the present disclosure described above, the case where the camera 17 and the distance measurement sensor 18 are separately disposed is described as an example. However, the present invention is not limited thereto. The camera 17 and the distance measurement sensor 18 may be integrated. For example, the image sensor 40 and the distance measurement sensor 18 of the camera 17 may be stacked to form one chip, and the camera 17 and the distance measurement sensor 18 may be integrated. In this case, there is no parallax between the distance measurement sensor 18 and the camera 17, so that optimal integration results can be obtained. Furthermore, the camera 17 and the distance measurement sensor 18 can be realized by one sensor, which improves area efficiency.
[0063] Further, in the above-described embodiment of the present disclosure, the case where the distance measurement sensor 18 performs distance measurement by the dToF method has been described as an example. However, the present invention is not limited thereto. The distance measurement method of the distance measurement sensor 18 may be an indirect time of flight (iToF) method, a structured light method, or a frequency modulated continuous wave (FMCW) method. The dToF method can measure a distance to a long distance subject. Furthermore, the dToF method is effective in a case where a short distance and a long distance are mixed in the distance of the subject to be imaged. Since the iToF method has high resolution, it is possible to acquire distance information with a large number of subjects and to widen the range of focus selection. In addition, since the iToF method has high distance measurement accuracy, it is possible to select the camera 17 optimal for the distance to the subject. Since the structured light method has high distance measurement accuracy at a short distance, it is possible to select the camera 17 optimal for a subject at a short distance. In the FMCW method, the distance to the subject can be measured with high accuracy as compared with that in the ToF method, so that the optimum focal length can be calculated.
[0064] Further, in the above-described embodiments of the present disclosure, it is also possible to manually perform all or part of the process described as being performed automatically of respective processes described, alternatively, it is also possible to automatically perform all or part of the process described as being performed manually by a known method. In addition, the processing procedure, specific name, and information including various pieces of data and parameters illustrated in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various types of information illustrated in each figure are not limited to the illustrated information.
[0065] Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as illustrated in the figure. That is, the specific form of distribution / integration of each device is not limited to the one illustrated in the figure, and all or part of the device can be functionally or physically dispersed / integrated in any unit according to various loads and usage conditions.
[0066] In addition, the above-described embodiments of the present disclosure can be appropriately combined in a region in which the processing contents do not contradict each other. Further, the order of each step illustrated in the sequence diagram or the flowchart of the present embodiment can be changed as appropriate.
[0067] <<5. Hardware configuration>> Furthermore, the smartphone 10 and the server device 60 according to the embodiment of the present disclosure described above are realized by a computer 1000 having a configuration as illustrated in Fig. 15, for example. The smartphone 10 will be described as an example. Fig. 15 is a hardware configuration diagram illustrating an example of the computer 1000 that implements the functions of the smartphone 10. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, a secondary storage device 1400, a communication interface 1500, and an input / output interface 1600. Respective units of the computer 1000 are connected by a bus 1050.
[0068] The CPU 1100 operates based on a program stored in the ROM 1300 or the secondary storage device 1400, and controls each unit. For example, the CPU 1100 develops a program stored in the ROM 1300 or the secondary storage device 1400 in the RAM 1200, and executes processing corresponding to various programs.
[0069] The ROM 1300 stores a boot program such as a basic input output system (BIOS) executed by the CPU 1100 when the computer 1000 is activated, a program depending on hardware of the computer 1000, and the like.
[0070] The secondary storage device 1400 is a computer-readable recording medium that non-transiently records programs executed by the CPU 1100, data used by the programs, and the like. Specifically, the secondary storage device 1400 is a recording medium that records a program of imaging processing according to the embodiment of the present disclosure, which is an example of the program data 1450.
[0071] The communication interface 1500 is an interface for the computer 1000 to be connected to an external network 1550. For example, the CPU 1100 receives data from another device or transmits data generated by the CPU 1100 to another device via the communication interface 1500.
[0072] The input / output interface 1600 is an interface that connects an input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a microphone or a touch panel via the input / output interface 1600. In addition, the CPU 1100 transmits data to an output device such as a display or a speaker via the input / output interface 1600. Furthermore, the input / output interface 1600 may function as a media interface that reads a program or the like recorded in a predetermined recording medium (medium). The medium is, for example, an optical recording medium such as a digital versatile disc (DVD) or a phase change rewritable disk (PD), a magneto-optical recording medium such as a magneto-optical disk (MO), a tape medium, a magnetic recording medium, a semiconductor memory, or the like.
[0073] For example, in a case where the computer 1000 functions as the smartphone 10 according to the embodiment of the present disclosure, the CPU 1100 of the computer 1000 implements the function of the AP 32 by executing a program loaded on the RAM 1200. In addition, the secondary storage device 1400 stores a program according to the present disclosure and data in the distance data storage unit 51 and the imaging data storage unit 54. The CPU 1100 reads the program data 1450 from the secondary storage device 1400 and executes the program data, but as another example, the program may be acquired from another device via the external network 1550.
[0074] <<6. Conclusion>> As described above, according to an embodiment of the present disclosure, the smartphone 10 (corresponding to an example of an "imaging system" and an "imaging device") includes the plurality of cameras 17, the distance measurement sensor 18, the identification unit 52, the imaging control unit 53, and the integration unit 55. The plurality of cameras 17 have different focal lengths. The distance measurement sensor 18 performs distance measurement in an imaging range of the plurality of cameras 17. The identification unit 52 identifies distances to a plurality of subjects included in the imaging range, from the result of distance measurement by the distance measurement sensor 18. The imaging control unit 53 performs control to capture an image with each camera 17 corresponding to the distance of each subject among the plurality of cameras 17. The integration unit 55 integrates the respective images captured by the cameras 17 corresponding to the distances of the respective subjects. As a result, the smartphone 10 can capture an image in which a plurality of subjects is in focus even in a case where the plurality of subjects is at different distances.
[0075] The embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments as they are, and various changes can be made without departing from the gist of the present disclosure. Moreover, the components over different embodiments and modifications may be suitably combined.
[0076] Further, the effects in each embodiment described in the present specification are merely examples and are not limited, and other effects may be present.
[0077] 10, 90 Smartphone 16, 17, 17a-17c, 17x, 17y Camera 18 Distance measurement sensor 18a Light emitting unit 18b Light receiving unit 19 Flash light 20 Red green blue clear-infrared (RGBC-IR) sensor 30 Camera group 31 Reception unit 32, 33 AP 34 Arithmetic unit 40 Image sensor 41 Lens 50 Distance sensor control unit 51 Distance data storage unit 52 Identification unit 53 Imaging control unit 54 Imaging data storage unit 55 Integration unit 56 Selection unit 60 Server device 80a Child 80b Man 80c Woman 91 Wide-angle camera 92 Telephoto camera 93 Distance sensor
Claims
1. A device comprising: a memory; and an electronic processor configured to: generate an integrated image by integrating a first image and a second image together, wherein only a portion of a first subject is in focus in the first image, and wherein only a portion of a second subject is in focus in the second image, and store the integrated image in the memory, wherein the portion of the first subject and the portion of the second subject are in focus in the integrated image.
2. The device according to claim 1, further comprising: a first camera having a first focal length; a second camera having a second focal length that is different from the first focal length; and a distance sensor configured to perform distance measurements on a plurality of subjects, the plurality of subjects including the first subject and the second subject, wherein the electronic processor is further configured to: control the distance sensor to determine a first distance of the first subject, control the distance sensor to determine a second distance of the second subject, generate the first image by controlling the first camera to capture the first subject based on the first distance, and generate the second image by controlling the second camera to capture the second subject based on the second distance.
3. The device according to claim 2, wherein the first image and the second image are captured simultaneously.
4. The device according to claim 2, wherein the first camera is a telephoto camera, and wherein the second camera is a wide-angle camera.
5. The device according to claim 2, wherein, to generate the first image by controlling the first camera to capture the first subject based on the first distance, and generate the second image by controlling the second camera to capture the second subject based on the second distance, the electronic processor is further configured to: control the first camera to perform a first focus adjustment on only the portion of the first subject based on the first distance, and control the second camera to perform a second focus adjustment on only the portion of the second subject based on the second distance.
6. The device according to claim 1, further comprising: a single camera; and a distance sensor configured to perform distance measurements on a plurality of subjects, the plurality of subjects including the first subject and the second subject, wherein the electronic processor is further configured to: control the distance sensor to determine a first distance of the first subject, control the single camera to perform a first focus adjustment on only the portion of the first subject based on the first distance, responsive to performing the first focus adjustment, generate the first image by controlling the single camera to capture the first subject, control the distance sensor to determine a second distance of the second subject, control the single camera to perform a second focus adjustment on only the portion of the second subject based on the second distance, and responsive to performing the second focus adjustment, generate the second image by controlling the single camera to capture the second subject.
7. The device according to claim 6, wherein the single camera has an adjustable focal length.
8. The device according to claim 1, wherein the electronic processor is further configured to: receive the first image from a first computing device, store the first image in the memory, receive the second image from a second computing device, and store the second image in the memory.
9. The device according to claim 8, wherein the first computing device and the second computing device are the same cloud server.
10. A non-transitory computer-readable medium comprising instructions that, when executed by an electronic processor cause the electronic processor to perform a set of operations comprising: generating an integrated image by integrating a first image and a second image together, wherein only a portion of a first subject is in focus in the first image, and wherein only a portion of a second subject is in focus in the second image; and storing the integrated image in a memory; wherein the portion of the first subject and the portion of the second subject are in focus in the integrated image.
11. The non-transitory computer-readable medium according to claim 10, wherein the set of operations further includes controlling a distance sensor to determine a first distance of the first subject; controlling the distance sensor to determine a second distance of the second subject; generating the first image by controlling a first camera to capture the first subject based on the first distance; and generating the second image by controlling a second camera to capture the second subject based on the second distance.
12. The non-transitory computer-readable medium according to claim 11, wherein the first image and the second image are captured simultaneously.
13. The non-transitory computer-readable medium according to claim 11, wherein the first camera is a telephoto camera, and wherein the second camera is a wide-angle camera.
14. The non-transitory computer-readable medium according to claim 11, wherein generating the first image by controlling the first camera to capture the first subject based on the first distance, and generating the second image by controlling the second camera to capture the second subject based on the second distance, the set of operations further includes controlling the first camera to perform a first focus adjustment on only the portion of the first subject based on the first distance, and controlling the second camera to perform a second focus adjustment on only the portion of the second subject based on the second distance.
15. The non-transitory computer-readable medium according to claim 10, wherein the set of operations further includes determining a first distance of the first subject; controlling a single camera to perform a first focus adjustment on only the portion of the first subject based on the first distance; responsive to performing the first focus adjustment, generating the first image by controlling the single camera to capture the first subject; determining a second distance of the second subject; controlling the single camera to perform a second focus adjustment on only the portion of the second subject based on the second distance; and responsive to performing the second focus adjustment, generating the second image by controlling the single camera to capture the second subject.
16. The non-transitory computer-readable medium according to claim 15, wherein the single camera has an adjustable focal length.
17. The non-transitory computer-readable medium according to claim 10, wherein the set of operations further includes receiving the first image from a first computing device; controlling the memory to store the first image; receiving the second image from a second computing device; and controlling the memory to store the second image.
18. A system comprising: a memory; and an electronic processor configured to: generate an integrated image by integrating a first image and a second image together, wherein only a portion of a first subject is in focus in the first image, and wherein only a portion of a second subject is in focus in the second image, and store the integrated image in the memory, wherein the portion of the first subject and the portion of the second subject are in focus in the integrated image.
19. The system according to claim 18, further comprising: a plurality of subjects including the first subject and the second subject; and a computing device, wherein the computing device includes the electronic processor; a first camera having a first focal length; a second camera having a second focal length that is different from the first focal length; and a distance sensor configured to perform distance measurements on the plurality of subjects, wherein the electronic processor is further configured to: control the distance sensor to determine a first distance of the first subject, control the distance sensor to determine a second distance of the second subject, generate the first image by controlling the first camera to capture the first subject based on the first distance, and generate the second image by controlling the second camera to capture the second subject based on the second distance.
20. The system according to claim 18, further comprising: a plurality of subjects including the first subject and the second subject; and a computing device, wherein the computing device includes the electronic processor; a single camera; and a distance sensor configured to perform distance measurements on the plurality of subjects, wherein the electronic processor is further configured to: control the distance sensor to determine a first distance of the first subject, control the single camera to perform a first focus adjustment on only the portion of the first subject based on the first distance, responsive to performing the first focus adjustment, generate the first image by controlling the single camera to capture the first subject, control the distance sensor to determine a second distance of the second subject, control the single camera to perform a second focus adjustment on only the portion of the second subject based on the second distance, and responsive to performing the second focus adjustment, generate the second image by controlling the single camera to capture the second subject.
Citation Information
Patent Citations
Method, mobile terminal, and storage medium for dual camera-based imaging
JP2020535764A
Image capture for later refocusing or focus-manipulation
US20130044254A1
Method and apparatus for generating an all-in-focus image
US20150062370A1
Image fusion for scenes with objects at multiple depths
US20220237813A1