Camera with scanning beam path folding element for automotive or monitoring applications

Abstract

Systems and methods for operating and using the same, including a plurality of dual aperture zoom digital cameras for an automobile or monitoring that should have a scanning optical path folding element (OPFE). In some embodiments, a dual aperture zoom digital camera includes a wide angle camera having a wide field of view (FOVW), a wide angle sensor, and a wide angle lens, where the wide angle camera is operative to output wide angle image information, a telephoto camera, and a processing unit. The remote camera has a far field of view (FOVT) less than the wide field of view and has a remote sensor, a remote lens having a remote lens optical axis and a scanning beam path folding element, and the processing unit is operative to detect an object of interest (OOI) from wide angle image and / or remote image information, and directing the remote camera to move a far field of view to acquire remote image information on the object of interest.

Description

[0001] Related Applications

[0002] This application is a divisional application of application number 201980039157.6, filed on July 4, 2019, and entitled “Camera with scanning optical path folding element for automotive or surveillance applications”. This application claims priority to U.S. Provisional Patent Application No. 62 / 693,951 filed on July 4, 2018, the entire contents of which are incorporated herein by reference. Technical Field

[0003] Embodiments disclosed herein relate generally to digital cameras, and particularly to thin zoom digital cameras. Background Art

[0004] Host devices with two cameras (also called "dual cameras" or "dual aperture cameras") are known, see, for example, U.S. Patent No. 9,185,291. The two cameras have lenses of different focal lengths and have respective image sensors that operate simultaneously to capture images. Even though each lens / sensor combination is aimed at looking in the same direction, each lens / sensor combination will capture images of the same scene with a different field of view (FOV). As used herein, "FOV" is defined by the tangent of the angle between a line intersecting the lens and parallel to the lens' optical axis and a line between the lens and any object captured at the respective image corners. For simplicity, "image sensor" will be referred to as "sensor" below.

[0005] Dual-aperture zoom cameras are also known, where one camera has a "wide" field of view (FOV W ), while the other camera has a narrow or “far” field of view (FOV T ), see, for example, U.S. Patent No. 9,185,291. These cameras are respectively called wide-angle and telephoto cameras, including respective wide-angle and telephoto sensors. These sensors provide separate wide-angle and telephoto images, respectively. The wide-angle image captures a wide field of view and has a lower spatial resolution than the spatial resolution of the telephoto image capturing a far field of view. The images can be merged (fused) together to form a composite image. In the composite image, the central portion is formed by combining relatively high spatial resolution images captured by a lens / sensor combination with a longer focal length, and the peripheral portion is formed by the peripheral portion of a relatively low spatial resolution image captured by a lens / sensor combination with a shorter focal length. The user selects the desired zoom amount, and the composite image is used to interpolate the values ​​from the selected zoom amount to provide a corresponding zoom image. Hereinafter, in this specification, the use of "resolution" refers to image spatial resolution, which represents the resolving power of the camera determined by the focal length of the lens, its aperture diameter, and the sensor pixel size.

[0006] Dual aperture cameras are known in which one image (usually a telephoto image) is obtained by folding the optical path, see, for example, commonly invented and commonly owned U.S. Patent Application No. 14 / 455,906, which teaches zoom digital cameras, including "vertical" (optical axis pointing directly at the subject or scene) wide angle cameras and "folded" telephoto cameras, see also below Figure 2 B. The optical axis of the folded camera is substantially perpendicular (orthogonal) to the optical axis of the upright camera. The folded camera can be autofocus and optically stabilized by moving its lens or by tilting an optical path folding (reflecting) element ("OPFE"), such as a prism or mirror inserted in the optical path between its lens and the corresponding sensor. For simplicity, the OPFE is collectively referred to as a "prism" hereinafter, and it should be understood that the term can refer to any other optical path folding (reflecting) element that can perform the function of folding the optical path as described herein.

[0007] For example, PCT patent application PCT / IB2016 / 056060, entitled "Dual Aperture Zoom Digital Camera User Interface", discloses a user interface for operating a dual aperture digital camera included in a host device, the dual aperture digital camera including a wide-angle camera and a telephoto camera, the user interface including a screen, a visible frame, and a device, the screen is configured to display an image of at least one icon and a scene acquired by at least one telephoto camera and a wide-angle camera, the visible frame defines a telescopic field superimposed on a wide-angle image defined by a wide field of view, and the device is used to switch the screen from displaying a wide-angle image to displaying a telephoto image. The user interface also includes a device for switching the screen from displaying a telephoto image to displaying a wide-angle image. The user interface may also include a device for acquiring a telephoto image, a device for storing and displaying the acquired telephoto image, a device for simultaneously acquiring a wide-angle image and a telephoto image, a device for storing and displaying a wide-angle image and a telephoto image, respectively, a focus indicator for the telephoto image, and a focus indicator for the wide-angle image.

[0008] Object recognition is known and describes the task of finding and identifying objects in images or video sequences. Many methods have been implemented to accomplish this task in computer vision systems. Such methods can rely on appearance-based methods, using example images under varying conditions and large libraries of models, and / or feature-based methods that search to find a feasible match between object features and image features, such as by using detection and matching of surface patches, corners, and edges. An algorithm that analyzes consecutive frames and outputs the object's motion between frames can be used to track recognized objects in a preview or video feed.

[0009] The problem of motion-based object tracking can be divided into two parts:

[0010] (1) Detect moving objects in each frame. This can be done by combining an object recognition algorithm for identifying and tracking specific objects (such as faces), or, for example, by detecting any moving objects in the scene. The latter can be combined with a background subtraction algorithm based on a Gaussian mixture model and morphological operations applied to the resulting foreground mask to remove noise. Blob analysis can later detect groups of multiple connected pixels that may correspond to multiple moving objects.

[0011] (2) Associating detections corresponding to the same object over time, for example using a motion estimation filter such as a Kalman filter.

[0012] In automotive or surveillance applications involving cameras, it would be advantageous to have the ability to examine a specific area of ​​interest at high resolution. If addressed by a single camera, the required spatial resolution would force the single camera to have a sensor with a very large number of pixels.

[0013] Therefore, it is necessary to identify a specific region of interest in an image with a large field of view and steer a camera with a narrow field of view toward that location. Summary of the invention

[0014] In various embodiments, systems including a dual aperture zoom digital camera with a scanning optical path folding element for automotive or surveillance applications and methods for operating and using the same are provided.

[0015] In various exemplary embodiments, various systems are provided, the systems comprising: a wide-angle camera having a wide field of view FOV W The invention also includes a wide-angle sensor and a wide-angle lens, wherein the wide-angle camera is operated to output wide-angle image information; a telephoto camera having a telescopic field of view (FOV) smaller than the wide field of view. T The invention also includes a telephoto sensor, a telephoto lens having a telephoto lens optical axis and a scanning optical path folding element (OPFE); and a processing unit, which is operated to detect an object of interest (OOI) from wide-angle image and / or telephoto image information, and direct the telephoto camera to move the telescopic field of view to obtain telephoto image information on the object of interest.

[0016] In an exemplary embodiment, the system is installed in a vehicle and the processing unit is further operative to calculate a required action measure or response required by the vehicle.

[0017] In an exemplary embodiment, the system further includes an actuator for tilting the optical path folding element to move or scan the far field of view.

[0018] In an exemplary embodiment, the processing unit is operative to direct the telephoto camera to move the telescopic field of view to substantially a center of the wide field of view.

[0019] In an exemplary embodiment, the processing unit is operative to direct the telephoto camera to move the telescopic field of view to substantially a center of the object of interest.

[0020] In an exemplary embodiment, the processing unit is operative to receive steering information from a steering wheel of the vehicle and direct the telephoto camera to move or scan the telephoto field further based on the steering information.

[0021] In an exemplary embodiment, the processing unit is operative to receive steering information from a steering wheel of the vehicle, and the actuator further tilts the optical path folding element to move or scan the far field of view based on the steering information.

[0022] In an exemplary embodiment, the wide field of view covers a road in front of the vehicle, the object of interest is a curve in the road, and the processing unit is operative to move the far field of view to follow the curve in the road.

[0023] In an exemplary embodiment, the vehicle includes a cabin, wherein the object of interest is located within the cabin, and the optical path folding element can be tilted to provide an extended telephoto camera field of view (FOV E ).

[0024] In an exemplary embodiment, the object of interest is a driver of the vehicle and the required action measurement or response is based on the driver's line of sight.

[0025] In an exemplary embodiment, the object of interest is a child, and wherein the required action measurement or response is a warning that the child is not wearing a seat belt.

[0026] In an exemplary embodiment, the required action measurement or response includes an action measurement or response selected from a group consisting of changing the speed and / or route of the vehicle, operating an internal alarm to the driver of the vehicle, operating an external alarm, sending a data message or calling an Internet / cloud-based service / police / road assistance service, and combinations thereof.

[0027] In an exemplary embodiment, the object of interest is a human face.

[0028] In an exemplary embodiment, the processing unit is operative to instruct the telephoto camera to move to a specific position of the face for facial recognition.

[0029] In an exemplary embodiment, the processing unit is operative to instruct the telephoto camera to move the telescopic field of view to scan portions of the wide field of view in two directions.

[0030] In an exemplary embodiment, the scanning is performed by tilting the scanning optical path folding element with a settling time of the optical path folding element between 5-50 milliseconds.

[0031] In an exemplary embodiment, the processing unit is operative to detect the object of interest from the wide-angle image information and / or the telephoto image information, and direct the telephoto camera to move a telescopic field of view to acquire information on the object of interest in an automatic tracking mode.

[0032] In an exemplary embodiment, the wide-angle image information and the telephoto image information may be fused together to form a composite image or a composite video stream.

[0033] In an exemplary embodiment, each composite image has the same field of view.

[0034] In an exemplary embodiment, a composite image is formed by stitching multiple telescopic images. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Non-limiting examples of the embodiments disclosed herein are described below with reference to the drawings listed after this paragraph. The same structure, element, or component appearing in multiple drawings is generally labeled with the same number in all drawings in which it appears. The drawings and descriptions are intended to illustrate and clarify the embodiments disclosed herein and should not be considered to be limiting in any way.

[0036] Figure 1A An embodiment of the system disclosed herein is shown;

[0037] Figure 1B An example of the elements of a dual camera is shown in perspective;

[0038] Figure 2 Schematically shows Figure 1A The usage of the system;

[0039] Figure 3 Shows Figure 2 A flowchart of the method in the use case;

[0040] Figure 4A Another embodiment of the system disclosed herein is shown;

[0041] Figure 4B Another embodiment of the system disclosed herein is shown;

[0042] Figure 5 Schematically shows Figure 4Aor the use of 4B's system;

[0043] Fig. 6A Schematically shows Figure 4A or a method of using the system in 4B;

[0044] Figure 6B Schematically shows Figure 4A or another method of using the system in 4B;

[0045] Figure 6C Schematically shows Figure 4A or another method of using the system in 4B;

[0046] Fig. 7A Another embodiment of the system disclosed herein is shown;

[0047] Figure 7B Another embodiment of the system disclosed herein is shown;

[0048] Figure 8 Shows Fig. 7A and Figure 7B The carriage part and usage of the system;

[0049] Fig. 9A Schematically shows Fig. 7A A systematic approach in

[0050] Fig. 9B Schematically shows Figure 7B Another method of using the system;

[0051] Fig. 10A Another embodiment of the system disclosed herein is shown;

[0052] Fig. 10B shows the resolution of the image obtained with known digital zoom;

[0053] Fig. 10C Shows the use Fig. 10A The resolution of the image obtained by the system using "optical" zoom. DETAILED DESCRIPTION

[0054] Figure 1AAn embodiment of a system disclosed herein and numbered 100 is shown. System 100 can be installed in or attached to a vehicle 102. System 100 includes a telephoto camera 104, a wide-angle camera 106, and a processing unit ("processor") 108. The vehicle can be, for example, a car, bus, truck, motorcycle, coach, or any type of known vehicle. Processing unit 108 can be a CPU, GPU, ASIC, FPGA, or any other processor capable of performing graphical analysis. When used in conjunction with a vehicle, a system like system 100 can also be referred to as an "advanced driver assistance system" or ADAS.

[0055] The combination of telephoto camera 104 and wide angle camera 106 may be referred to as a “dual camera” and is referenced as 110 . Figure 1B An example of the elements of the dual camera 110 is shown in perspective view. The wide angle camera 106 includes a wide angle sensor 132 and a wide angle lens 134 having a wide angle lens optical axis 136. The wide angle sensor 132 is characterized by a wide angle sensor active area size and a wide angle sensor pixel size. The wide angle lens 134 is characterized by a wide angle sensor active area size and a wide angle sensor pixel size. W Alternatively, in one embodiment, the wide-angle lens 134 may have a fixed (constant) EFL. W . Optionally, the wide angle lens 134 can be fixed at a constant distance from the wide angle sensor 132 (fixed focus). Optionally, the wide angle lens 134 can be coupled to a focus mechanism (e.g., an auto focus (AF) mechanism) that can change the distance between the wide angle lens 134 and the wide angle sensor 132 to provide a non-fixed (variable) focus. Wide Sensor Area and EFL W The combination of the wide field of view (FOV W ). According to some examples, the wide field of view can be 50-100 degrees in the horizontal vehicle-facing direction.

[0056] Telephoto camera 104 includes a telephoto sensor 122 and a telephoto lens 124 having a telephoto lens optical axis 138. Telephoto sensor 122 is characterized by a telephoto sensor active area size and a telephoto sensor pixel size. Telephoto lens 124 is characterized by a telephoto sensor active area size and a telephoto sensor pixel size. TAlternatively, in one embodiment, the telephoto lens 124 may have a fixed (constant) EFL. In some embodiments, the telephoto lens may be fixed at a constant distance from the telephoto sensor 122 (fixed focus). Alternatively, the telephoto lens may be coupled to a focusing mechanism (e.g., an AF mechanism) that can vary the distance of the telephoto lens 124 to the telephoto sensor 122 (non-fixed focus). Telephoto sensor area and telephoto lens EFL T The combination of determines the far field of view (FOVT). According to some examples, the far field of view can be between 10-30 degrees in the horizontal direction facing the vehicle. Therefore, the far field of view is smaller (narrower) than the wide field of view.

[0057] The telephoto camera 104 also includes an optical path folding element 126, such as a mirror or a prism. The reflective surface of the optical path folding element 126 is tilted 45 degrees at a stationary point relative to the optical axis 138 of the telephoto lens. The telephoto camera 104 also includes an actuator (motor) 128. The actuator 128 can tilt the reflective surface of the optical path folding element 126 relative to the stationary point by up to ±α degrees (an exemplary α may be up to 10, 20, 40 or 70 degrees). That is, the actuator 128 can tilt or scan the optical path folding element and perform telescopic field of view together with it. The actuator 128 can be, for example, a stepper motor or a voice coil motor (VCM) as described in the commonly owned patent application PCT / IB2017 / 057706.

[0058] In some examples, the wide angle camera 106 and the telephoto camera 104 face the front of a vehicle and share at least some of their respective fields of view. Typically, the wide field of view is directed toward the front direction (direction of travel) away from the vehicle and is substantially symmetrical with respect to the sides of the vehicle. In one mode of operation, the telephoto camera is operated using actuator 128 to scan the wide field of view (FOV W ) within the long-range field of view (FOV T ). In some examples, scanning of the far field of view is used to enable the telephoto camera to take a closer look at a detected potential object of interest (OOI) previously detected from the wide angle and / or telephoto images, see details below.

[0059] Figure 2 Schematically shows Figure 1A100 is used in a system 100. A dual camera 110 is mounted on the front of a vehicle 102. For example, a triangle 204 represents a wide field of view in a horizontal plane, i.e., as a horizontal wide field of view or "HFVOW". In the wide field of view, a "viewing distance" 206 is defined as the maximum distance that allows the system 100 to detect a potential object of interest using an image from the wide angle camera. An "object of interest" can be, for example, a hazard, another car, a hole or obstacle in the road, a pedestrian, a road bend, a road sign, etc. "Recognition distance" 208 is defined as the minimum distance that allows the system 100 to recognize all the information needed to make a decision using an image from the wide angle camera. As is known in the art. According to one example, the object of interest can be a road sign that is observable but unreadable within the viewing distance. According to one example, an object of interest can be observed within the viewing distance, but only recognized or distinguished as a road sign or a pedestrian within the recognition distance. In other words, if an object of interest is located before the observation distance 206 (closer than the wide angle camera) but farther than the identification distance 208, the system 100 can use an image from the wide angle camera to calculate that the object of interest is located in the wide field of view, but cannot fully calculate the required required action measurement or response (see below).

[0060] According to some examples, the action measurement or response of the system 100 may include one or more or a combination of the following: changing the speed and / or route of the vehicle 102, operating an internal alarm to the driver of the vehicle, operating an external alarm, sending a data message or calling an Internet / cloud-based service / police / road assistance service, etc. For example, a triangle 210 represents a far field of view in a horizontal plane, i.e., a horizontal FVO T (HFVO T ). According to one example, HFOV W Can be in the range of 70-180 degrees, and HFOV T It can be in the range of 15-45 degrees. According to another example, HFOV W Can be in the range of 140-180 degrees, and HFOV T It can be in the range of 15-70 degrees. Therefore, the output image of the telephoto camera may have a higher resolution than the output image of the wide-angle camera. For example, the resolution of the output image of the telephoto camera may be 3 to 20 times higher than the output image of the wide-angle camera, and therefore, the recognition distance 212 of the telephoto camera may be 3 to 20 times farther than the recognition distance 208 of the wide-angle camera.

[0061] exist Figure 2In the example shown in (a) of FIG. 1 , the vehicle 102 approaches the object of interest 202. The object of interest 202 is located between the observation distance 206 and the recognition distance 208. Although the object of interest 202 is observable by the wide-angle camera, it may not be recognizable (i.e., the wide-angle camera captures the object of interest 202 at a resolution that is too low to be recognized, classified, or processed relative to what is required by the system 100). Figure 2 (b) and then change the POV T Scanning to face the object of interest 202 allows the telephoto camera to capture the object of interest 202 in more detail (eg, “recognize” it).

[0062] Figure 3 Shows Figure 2 An example of a detailed flow chart of the method of operation of the system 100 is as follows:

[0063] Step 302: Exemplarily, the wide-angle camera (eg, 106) acquires a plurality of wide-angle images. In some alternative embodiments, the plurality of images may also or alternatively be acquired by the telephoto camera.

[0064] Step 304: Exemplarily, the wide-angle camera sends the wide-angle image acquired in step 302 to a processing unit (eg, 108) for analysis. In some alternative embodiments, the multiple telephoto images acquired in step 302 may also or alternatively be sent to the processing unit for analysis.

[0065] Step 306: The processing unit detects that there is an object of interest 202 in front of the vehicle 102, but more details are needed to resolve or decide a course of action.

[0066] Step 308: The processing unit directs the telephoto camera (e.g., 104) to face the telephoto field of view toward the object of interest 202 (i.e., by scanning the telephoto camera) to acquire and receive multiple images of the object of interest 202 at higher quality and / or higher resolution. The processing unit may then have more information on the object of interest in order to fully calculate the desired action measure or response.

[0067] In some examples, the telephoto camera may be a camera equipped with a motor to drive the entire camera. In some examples, the telephoto camera may be a folded camera as described in co-owned patent application PCT / IB2016 / 057366, wherein the optical path folding element is operable to change (i.e., scan) a telephoto camera point of view (POV). In some examples, the telephoto camera can only scan in one dimension (1D) (i.e., along a line). In some examples, the telephoto camera can scan in two dimensions (2D), i.e., scan an area. In some examples, the motor used for scanning may be a VCM, a shape memory alloy (SMA) motor, a piezoelectric motor, a stepper motor, or a DC motor. In some examples, the telephoto camera and / or the wide-angle camera may be integrated with an optical image stabilizer (OIS) to compensate for vehicle vibrations.

[0068] Figure 4A An embodiment of a system numbered 400 is shown mounted or attached to a vehicle 402. Optionally, vehicle 402 may have a steering wheel 416. In some vehicles, a joystick (not shown) may replace the steering wheel, and the following description is related to both. Compared to system 100, system 400 only includes a telephoto camera 404 (similar to telephoto camera 104) and a processing unit 408 (similar to processing unit 108).

[0069] Figure 4B Another embodiment of a system numbered 400' is shown, which is similar to system 100, that is, in addition to telephoto camera 404, it also includes a wide-angle camera 406. The following description refers to systems 400 and 400'. The telephoto camera faces the front side of the vehicle. Figure 5 As shown, in system 400, the telephoto camera 404 can be operated to change the POV T Angle / direction, as shown by arrow 502, thereby achieving the FOV E An "effective" FOV that is larger than the far field of view. According to system 400 ( Fig. 6A ), a processing unit continuously commands the telephoto camera to continuously change the viewpoint direction or angle from left to right, and vice versa (602), and the telephoto camera rotates according to the received command (604). In the system 400 ( Figure 6B ), the processing unit follows the steering wheel or joystick: when the user turns the steering wheel / joystick (612) to the left, the telephoto camera POV (POV T ) moves to the left, and when the user turns the steering wheel / joystick to the right, the telephoto camera POV T Move right. According to the third optional use case of system 400 ( Figure 6C), the processing unit can use an image recognition algorithm to identify road curves and change the far field of view to follow the road.

[0070] Fig. 7A An embodiment of another system (similar to system 400) is shown, generally designated 700, that may be mounted on or attached to a vehicle 702. Vehicle 702 includes a vehicle compartment 716.

[0071] Figure 7B Yet another system embodiment is shown numbered 700', similar to system 100, and including a wide angle camera 706 (similar to wide angle camera 106) in addition to telephoto camera 704. Wide angle camera 706 may also be mounted in vehicle cabin 716 to face object of interest 802.

[0072] Figure 8 Shows Fig. 7A and 7B 716 and the use of the system. The wide angle camera 706 is not shown. The telephoto camera 704 with a far field of view faces the interior of the vehicle 716 and an object of interest 802, such as a passenger. The telephoto camera 704 can be scanned to allow the effective FOV (FOVE) to be larger than the far field of view.

[0073] Fig. 9A The main steps of the method of use of the system 700 are shown in the flow chart. The long-range camera 704 is operated to change angle / direction and scan the vehicle cabin 716. The processing unit 708 is operated to identify an object of interest 802 (e.g., passenger body, face, eyes, etc.). The processing unit 708 can be further operated to direct the long-range camera 704 to face the object of interest 802. The data obtained by the long-range camera is used to identify hazards (e.g., the driver is not looking at the road, the driver is asleep, the passenger is not wearing a seat belt, the child is not in a child seat, etc.). Fig. 9B The main steps of the method of use of the system 700' are shown in a flow chart. The processing unit directs the telephoto camera to the object of interest 802 using data from the wide angle camera and the telephoto camera.

[0074] Fig. 10AAn embodiment of another system disclosed herein and numbered 1000 is shown. The surveillance camera 1000 includes a telephoto camera 1002, a wide-angle camera 1004, and a processing unit 1006, and can be used for surveillance, and is therefore also referred to as a "surveillance camera". The telephoto camera 1002 and the wide-angle camera 1004 are part of a dual camera 1010. These components can be similar or even the same as the wide-angle camera and telephoto camera and processor described in the above embodiments. The surveillance camera 1000 and the processing unit 1006 may include software and algorithms to detect objects of interest (e.g., faces) in a wide field of view and turn the telephoto camera in the X and Y directions in the wide-angle image towards these objects of interest to enhance the image or video quality of these objects and enable their analysis (e.g., for facial recognition if the objects are faces).

[0075] In one embodiment, the processing unit 1006 may instruct the telephoto camera 1002 to continuously scan a portion of the wide field of view. In one embodiment, the processing unit 1006 may instruct the telephoto camera 1002 to move to a specific position (as shown in FIG. 9 ). The tilt and stabilization time of the prism may occur within 5 to 50 milliseconds. In addition, the telephoto camera 1002 may be pointed from one region of interest (ROI) to another region of interest (ROI) every 1 second or faster or slower. Fig. 10B An imaged scene captured by the wide angle camera 1004 is shown and then digitally zoomed, and Fig. 10C An example of an imaged scene acquired by a wide angle camera 1004 (left) and then optically zoomed on the ROI (right) by a directed telephoto camera 1002 is shown. Fig. 10B Compared to the digital zoom image in Fig. 10C The zoomed image in shows a significant resolution gain, allowing, for example, facial recognition of a person in the ROI.

[0076] As is known in the art, wide angle and telephoto images and / or video streams may be recorded during the auto-tracking mode and may be fused together to form a composite image or a composite video stream. This fusion may be applied on a camera host device (e.g., a mobile electronic device of any type that includes a system or camera disclosed herein). Alternatively, the wide angle and telephoto images or video streams may be uploaded to the cloud for application of the fusion operation. Each composite image may also have the same field of view by scanning with a telephoto camera, stitching multiple telephoto images to provide a "stitched" telephoto image, and then fusing the stitched telephoto image with the wide angle image. This is advantageous because the wide angle image captures the entire scene simultaneously, while the telephoto images to be stitched together are continuous, and therefore motion or occlusion in the scene may be overcome if necessary. Stitching of telephoto images and / or fusion of stitched telephoto images with wide angle images may also be performed in the cloud.

[0077] Although the disclosure has been described in terms of certain embodiments and generally associated methods, variations and substitutions of the embodiments and methods will be apparent to those skilled in the art. It should be understood that the disclosure is not limited to the specific embodiments described herein, but only to the scope of the appended claims.

Claims

1. A system, characterized in that: The system comprises: One or more cameras, wherein the one or more cameras are configured to provide a first video stream of a plurality of first images and a second video stream of a plurality of second images, wherein the plurality of first images have first image information and a first field of view FOV1, and the plurality of second images have second image information and a second field of view FOV2 smaller than the first field of view FOV1, and wherein the second field of view FOV2 is configured to scan within the first field of view FOV1; and A processing unit is configured to detect an object of interest from the first image information, and when the object of interest is farther than a distance allowing the object of interest to be identified in the first image information, direct the second field of view FOV2 to scan within the first field of view FOV1 to obtain the second image information on the object of interest.

2. The system according to claim 1, characterized in that: The first field of view FOV1 has a range of 50-100 degrees.

3. The system according to claim 1, characterized in that: The second field of view FOV2 has a range of 10-30 degrees.

4. The system according to claim 1, characterized in that: The plurality of first images have a first spatial resolution, and wherein the plurality of second images have a second spatial resolution higher than the first spatial resolution.

5. The system according to claim 1, characterized in that: The processing unit is configured to direct the second field of view FOV2 to scan to substantially a center of the first field of view FOV1.

6. The system according to claim 1, characterized in that: The processing unit is configured to direct the second field of view FOV2 to scan substantially a center of the object of interest.

7. The system of claim 1, wherein: The processing unit is configured to direct the second field of view FOV2 to scan to a specific position of the object of interest.

8. The system of claim 1, wherein: The object of interest is a face and the second image information is used for face recognition.

9. The system of claim 1, wherein: The processing unit is configured to direct the second field of view FOV2 to scan a portion of the first field of view FOV1 in two directions.

10. The system of claim 1, wherein: The processing unit is configured to direct the second field of view FOV2 to scan within the first field of view FOV1 to acquire the second image information on the object of interest in an automatic tracking mode.

11. The system of claim 1, wherein: The processing unit is installed in a vehicle, and wherein the processing unit is configured to calculate a required action measure or response required by the vehicle.

12. The system according to claim 11, characterized in that: The object of interest is a face of a driver of the vehicle, and wherein the required action measurement or response is based on a line of sight of the driver.

13. The system of claim 11, wherein: The object of interest is a face of a child, and wherein the desired action measurement or response is a warning that the child is not wearing a seat belt.

14. The system of claim 11, wherein: The required action measure or response is selected from a group consisting of changing the speed and / or route of the vehicle, operating an internal alarm to the driver of the vehicle, operating an external alarm, sending a data message or calling an Internet / cloud-based service / police / road assistance service, and combinations thereof.

15. The system of claim 11, wherein: The processing unit is configured to receive steering information from a steering wheel of the vehicle, and further direct the second field of view FOV2 to scan based on the steering information.

16. The system of claim 11, wherein: The first field of view FOV1 covers a road in front of the vehicle, wherein the object of interest is a road curve, and wherein the processing unit is configured to direct the second field of view FOV2 to follow the road curve.

17. The system of claim 1, wherein: The first image information and the second image information are fused together to form a composite image or a composite video stream.

18. The system of claim 17, wherein: Each composite image has the same field of view.

19. The system of claim 17, wherein: A composite image is formed by stitching the plurality of second images.

20. A system, characterized in that: The system comprises: One or more cameras, wherein the one or more cameras are configured to provide a first video stream of a plurality of first images and a second video stream of a plurality of second images; and A processing unit is configured to record the first video stream and the second video stream during an automatic tracking mode, and merge the recorded first video stream and the second video stream together to form a composite image or a composite video stream.

Images