Method for imaging a scene, computer program, device, apparatus and vehicle implementing such a method

Abstract

The invention relates to a method (200) for imaging a scene (S) with an apparatus comprising multiple camera modules, said method (200) comprising the following steps: - acquiring (102), by way of at least two different camera modules of said apparatus, at least two images of said scene (S) with different field angles, and - storing (108) an image, referred to as enriched image, constructed with at least two of said images taken with different field angles. It also relates to a computer program and an imaging device implementing such a method, and to an apparatus and a vehicle comprising such an imaging device.

Description

DESCRIPTION Title: Method for imaging a scene, computer program, device, apparatus and vehicle implementing such a method.

[0001] The present invention relates to a method for imaging a scene, and in particular for imaging a scene with different field angles. It also relates to a computer program and a device implementing such a method. It further relates to an apparatus and a vehicle implementing such a method.

[0002] The field of invention is the field of taking images of a scene. State of the art

[0003] Techniques are known for representing a scene from a stack of images taken at different focusing distances, called focus or depths of field in the present application. For example, the technique called focus bracketing, or focus stacking, is known, which makes it possible to capture a stack of images of a scene, each image being captured at a different depth of field. Then, the scene is represented on a display medium, with a greater depth of field, by exploiting the stack of images.

[0004] These techniques provide a new way of imaging a scene since it allows more information to be collected from the scene when said scene is imaged, in order to then represent the scene in a richer way on a display screen.

[0005] An aim of the present invention is to propose a new technique for imaging a scene in a rich manner.

[0006] Another aim of the invention is to propose a solution for imaging a scene in a richer manner and providing enriched data representing said scene which can be used in the context of a scene analysis process, without going through a visual representation of said scene.

[0007] Another aim of the invention is to propose a solution for imaging a scene making it possible to represent the scene on a display surface in a richer manner.

[0008] Another aim of the invention is to propose a solution for imaging a scene making it possible to represent the scene on a display surface in a more customizable manner. Statement of the invention

[0009] The invention proposes to achieve at least one of the aforementioned aims by a method for imaging a scene with an apparatus comprising several camera modules, each camera module comprising an optical lens associated with an image sensor, said method comprising the following steps: - acquisition, by at least two different camera modules of said apparatus, of at least two images of said scene with different field angles, and - storage of an image, called an enriched image, constructed with at least two of said images taken with different field angles.

[0010] The invention proposes imaging a scene with several camera modules to take several images of said scene, acquired with different field angles. Thus, the scene is imaged according to different field angles. The acquisition of several images of the scene with different field angles allows the acquisition of more information of the scene, and therefore to obtain an enriched image of the scene which is richer in terms of scene information, compared to a single image. In addition, the information contained in the enriched image is different from that contained in a stack of images obtained by focus bracketing.

[0011] Furthermore, images acquired with different field angles can then be used to create a representation of the scene on a display surface, such as a device's display screen or a surface associated with a projector, in a richer manner.

[0012] With an enriched image obtained according to the invention, it is possible to offer a personalized display of the scene for a user. Indeed, the latter can choose the field angle from which he wishes to display the scene. Thus, with the same enriched image obtained by the invention, different users can observe the scene from different field angles.

[0013] Furthermore, the invention allows a dynamic display of a scene for each user. Indeed, in addition to being able to choose the field angle according to which the scene is displayed, each user has the possibility of navigating from one field angle to another, thus modifying the display of the scene. In other words, the user has the possibility of navigating in the scene from an enriched image of the scene obtained by the present invention.

[0014] Furthermore, the invention proposes a solution for imaging a scene in a richer manner and providing enriched data representing said scene. This data can be used by any system or device as part of a data analysis process or scene analysis, without necessarily requiring a visual representation of the scene on a display medium. Such data analysis can be used, for example, in the context of: - an automated decision-making process in an autonomous or semi-autonomous vehicle, - a medical diagnostic process, - a process of monitoring or tracking a target object, - etc.

[0015] By image we mean a digital image, and in particular a raster image.

[0016] Field of view refers to the angle at which the scene is imaged, relative to the optical axis of a lens or camera module. Typically, current devices, and in particular current smartphones, can be equipped with different camera modules with different field of view angles. For example, a smartphone can be equipped with: - a wide-angle camera module, in particular a field angle greater than 60°, for example a x0.5 camera module opening between 90° and 120°; - a standard field angle camera module, in particular an intermediate field angle (45°~60°), for example an xl camera module; - a telephoto camera module, in particular a field angle of less than 45°: this module can also be equipped with a zoom allowing different field angles to be obtained with said telephoto camera module (typically between 25° and 5° field angle for an optical zoom labeled x2 to x5).

[0017] Advantageously, the acquisition step can comprise simultaneous acquisition, by two different camera modules, of two images with different field angles.

[0018] In this case, both camera modules acquire two images of the scene with two different field angles at the same time. Both camera modules can be commanded to start image acquisition at the same time. Depending on the field angle, exposure time, and different resolution, the acquisition of an image by one camera module may take longer than the acquisition of the other image by the other camera module.

[0019] For example, if the apparatus includes a wide-angle camera module and a telephoto camera module, said camera modules may be controlled to acquire an image of the scene at the same time. In another example, if the apparatus includes a wide-angle camera module, a standard field-angle camera module, and a telephoto camera module, said three camera modules may be controlled to simultaneously acquire three images of a scene, each with a different field of view. Of course, these examples are in no way limiting.

[0020] Alternatively, or in addition, the acquisition step may comprise a sequential acquisition, by two different camera modules, of two images with different field angles.

[0021] In this case, the two camera modules acquire two images of the scene with two different field angles at different times. Preferably, said two times are close to each other so that the imaged scene does not change between said two times, or changes very little. For example, said two times may be separated by a duration less than or equal to 100 ms, and in particular 50 ms, and even more particularly 10 ms.

[0022] The two camera modules can be controlled to begin image acquisition at time-shifted instants. In particular, the two camera modules can be controlled so that one of the camera modules begins image acquisition before the other of the camera modules has finished image acquisition. Alternatively, the two camera modules can be controlled so that one of the camera modules begins image acquisition after the other of the camera modules has finished image acquisition.

[0023] For example, if the apparatus includes a wide-angle camera module and a telephoto camera module, the camera modules may be controlled to acquire an image of the scene sequentially over time. In another example, if the apparatus includes a wide-angle camera module, a standard field-angle camera module, and a telephoto camera module, at least two of the camera modules may be controlled to acquire images of a scene, each with a different field angle, sequentially over time. Of course, these examples are not limiting in any way.

[0024] Alternatively, or in addition, the acquisition step may comprise a sequential acquisition, by a camera module, of several images with different field angles, said method comprising a modification of the field angle of said camera module between the acquisition of said images.

[0025] In this case, the camera module has an adjustable / changeable field angle to produce images at at least two different field angles.

[0026] According to a non-limiting exemplary embodiment, if the device comprises a telephoto camera module capable of acquiring images with different field angle values, said telephoto camera module can be controlled to sequentially acquire at least two images of a scene, each with a different field angle. To do this, the camera module is configured at a first field angle and an image of the scene is taken with this first field angle, then the camera module is configured at a second field angle and a second image of the scene is taken with this second field angle, and so on if necessary.

[0027] Alternatively, or in addition, the acquisition step may include: - an acquisition, by a camera module, of at least two images at different field angles, - an acquisition, by another camera module, of at least two images at the same field angle, different from the field angles of said images taken by said one camera module.

[0028] These acquisitions can be carried out sequentially.

[0029] Preferably, the two camera modules can be synchronized so that image acquisitions by the camera modules can be performed simultaneously. In other words, the two camera modules can be controlled so that: - simultaneously, at a first instant, the first camera module takes an image at a first field angle and the second camera module takes an image at a second field angle different from the first field angle, then - simultaneously, at a second instant, the first camera module takes an image at a third field angle, different from the first and second field angles, and the second camera module takes an image at the second field angle. The two images taken by the second camera module with the same field of view can be used for different purposes. For example, these two images can be used to detect possible change(s) in the scene between the first and second instants. According to another example, these two images can be used to align the images taken with different field of view by the first camera module.

[0030] According to an example of implementation, the device can include three camera modules: - a first wide-angle camera module, with a fixed field angle, - a second standard camera module, with a fixed field angle or an adjustable field angle, continuously or discretely, between two extreme field angle values ​​different from that of the first camera module, and - a third telephoto camera module, with an adjustable field angle, continuously or discreetly, different from those of the first and second camera modules.

[0031] With such a device, the acquisition step can include: - at a first moment and carried out simultaneously: ■ acquisition of an image of the scene with the first camera module at its unique field angle, ■ acquisition of an image of the scene with the second camera module configured at its first field angle value, and ■ acquisition of an image of the scene with the third camera module configured at its first field angle value; then - at a second moment and carried out simultaneously: ■ optionally, acquisition of an image of the scene with the first camera module at its unique field angle, ■ acquisition of an image of the scene with the second camera module configured at its second field angle value, and ■ acquisition of an image of the scene with the third camera module configured at its second field angle value; then - at a third moment and carried out simultaneously: ■ optionally, acquisition of an image of the scene with the first camera module at its unique field angle, ■ optionally, acquisition of an image of the scene with the second camera module always configured at its second field angle value, and ■ acquisition of an image of the scene with the third camera module configured at its third field angle value. In this case, the first camera module takes three images with the same field of view, and the second camera module takes three images, two of which are at the same field of view, and the third camera module takes three images, all of which are acquired with different field of view.

[0032] Alternatively, with such a device, the acquisition step may include: - at a first moment and carried out simultaneously: ■ acquisition of an image of the scene with the first camera module at its unique field angle, ■ acquisition of an image of the scene with the second camera module configured at its first field angle value, and ■ acquisition of an image of the scene with the third camera module configured at its first field angle value; then - at a second instant and carried out simultaneously: ■ acquisition of an image of the scene with the second camera module configured at its second field angle value, and ■ acquisition of an image of the scene with the third camera module configured at its second field angle value; then - at a third instant: ■ acquiring an image of the scene with the third camera module configured at its third field angle value; then In this case, the first camera module takes a single image. The second camera module takes two images with different field angles. The third camera module takes three images with different field angles.

[0033] Of course, the above examples are in no way limiting and other alternatives are possible without departing from the scope of the invention.

[0034] According to embodiments, the enhanced image may be a stack of images, comprising at least two images acquired at different field angles.

[0035] According to embodiments, the enhanced image may comprise only the images acquired at different field angles.

[0036] According to embodiments, the enriched image may comprise all the images acquired during the acquisition step, i.e. the images acquired at different field angles, and also, potentially, several images acquired with the same field angle. This latter case makes it possible to detect, by analyzing several images acquired at the same field angle, changes that may occur in the scene during the performance of the acquisition step, i.e. during the acquisition of the images of the scene. In addition, the images acquired with the same field angle may further be used, for example, to correct possible aberrations caused by a change in field angle, or to align the images acquired with different field angles.

[0037] According to embodiments, the enriched image can be represented by a set of value matrices, each matrix representing one of the images forming the enriched image.

[0038] In this case, each matrix may include the values ​​captured by the sensor used to acquire said image, and each matrix may be associated with a value indicating the field angle with which said image was acquired. The value representing the field angle may be stored in association with the value matrix, or directly in each cell of the matrix. For example, each cell of the matrix may include the values: (AV, R, G, B), with - AV a value indicating the field angle with which the image was acquired, for example AV=45° for a standard lens, - R, G and B respectively the values ​​of the three colors red, green and blue captured by a pixel of the image sensor.

[0039] According to embodiments, the enriched image can be represented by a single matrix of values. In this case, at least one cell, in particular several cells of the matrix can comprise several sets of values, each set of values ​​indicating on the one hand the value representing the field angle and the values ​​captured by the pixel of the sensor for this field angle. For example, at least one cell of the matrix can include the values: {(AVl, Rl, Gl, Bl), ... ,(AVn, Rn, Gn, Bn)} with, with n an integer greater than or equal to 2, and - AVi,..., AV n values ​​indicating the field angles, for example AVi=45°, AV2=75°, etc. - Ri, Gi and Bi respectively the values ​​of the three colors red, green and blue, captured by a pixel of the image sensor, with the field angle AV. Of course some cells in the matrix may only have one set of values, because some parts of the scene may only be captured at one angle of view, for example wide angle.

[0040] According to a non-limiting exemplary embodiment, the single matrix may for example be sized so as to cover the widest field angle available in the data set, but with a resolution (number of cells per row and per column) corresponding at least to the resolution of the best angularly resolved field angle. For example, if the data set contains a wide angle producing 2048x1600 pixels at 120°x93°, a standard lens producing 4096x3072 pixels at 45°x33.7° field angle and a telephoto lens, or a zoom, producing 2048x1600 pixels at 5°x3.9° field, the single matrix may cover at least the 120°x93° of the widest field available and the resolution (pixels / 0field) must be finer than the finest resolution available, in this example along the horizontal axis Max{2048 / 120° = 17pix / °, 4096 / 45°=91pix / °, 2048 / 5°=409pix / °} = 409 pix / °. Proceeding in the same way on the vertical axis, the single matrix will have at least 120°x96° of field and 409 pixels per degree of field, i.e. 50000x40000 data elements. Each cell of the matrix can include the values ​​(AV, R, G, B) corresponding to the same position in the field and calculated from each original image among those which cover the target point. The single matrix being in this example of uniform resolution, we will proceed to fill its values ​​by interpolation, calculating the R, G, B values ​​of each cell of the final matrix, by searching in each raw image, the neighborhood of pixels corresponding to the same direction in the field, and by performing an interpolation of the pixels of the neighborhood of the source image (linear or bicubic or other) to determine the RGB value of an element of the matrix of the enriched image. We note that in this implementation, the resolution of the final matrix being at least equal to that of the source data, we preserve substantially the information of the best-resolved photo in the field region covered by the telephoto lens; and we fill the areas not subject to the wide zoom effect which were only captured by the least resolved lens by interpolation, which corresponds neither more nor less to what we would have seen by zooming in the basic wide-angle image. In the field areas which were intercepted by more than one shot, we preserve the multiplicity of values ​​(AV, , G, B) which we make available for a display or interpretation strategy dedicated to this format,

[0041] Alternatively, flattening can be applied to retain only a single data point. In spirit, flattening should retain only the RGB data of the highest resolution lens covering a given field region. In the implementation, state-of-the-art image blending methods are applied to soften the visually rectilinear character of the field edge of a narrow lens when represented in the matrix covering the widest field. Typically, luminance correction, geometry correction, and blending of the data from the highest resolution image with the image of the previous resolution level are applied, with a weighting that gradually varies from zero to 1 as one goes deeper into the covered field; or that varies according to patterns that follow the contours naturally present in the scene making the transition less noticeable.All of these approaches are considered state-of-the-art in image stitching and therefore will not be further detailed here.

[0042] By "big zoom" or "big zoom effect" is meant a zoom of very large amplitude, exceeding what is achievable with a single optical formula. For an optical formula, one can go from the minimum zoom corresponding to a full-frame visualization, to a zoomed visualization up to the limit where the pixels are so enlarged that they become too perceptible and end up forming an uninterpretable mush. In comparison, the solution proposed by the present invention covers zoom minimum of the image of the optical formula, for example the wide angle formula x0.5, up to the maximum zoom of the telephoto optical formula x5, therefore an amplitude increased by the optical factor x0.5:x5 i.e. xlO.

[0043] At least one camera module may have an adjustable field angle, either discretely or continuously. In other words, at least one camera module may be equipped with, and associated with, a mechanism for modifying / adjusting the field angle of said camera module. Such a camera module may be used to acquire an image of the scene at a field angle that can be changed from one scene to another. Such a camera module may be used to sequentially acquire several images of the same scene at several field angles.

[0044] At least one camera module can have a fixed, non-adjustable field of view.

[0045] In other words, at least one camera module may not be equipped with, or associated with, a mechanism for modifying the field angle of said camera module. In this case, said camera module always performs image acquisition with the same field angle.

[0046] According to embodiments, the acquisition step can be carried out by three camera modules.

[0047] In particular, one of the three camera modules may be a wide-angle module, in particular a camera module with a field angle greater than 60°.

[0048] In particular, one of the three camera modules may be a standard field angle camera module, in particular a camera module with a field angle equal to 45°.

[0049] In particular, one of the three camera modules may be a telephoto camera module, in particular a camera module with a field angle of less than 45°.

[0050] Of course, the device can be equipped with a different number of camera modules.

[0051] According to another aspect of the invention, an image is provided, stored on a non-transitory storage medium, obtained by the method according to the invention.

[0052] The enriched image can be a 2D image.

[0053] The enhanced image can be a 3D image.

[0054] The enriched image may be in the form of a single matrix. Alternatively, the enriched image may be in the form of a stack of multiple matrices, each of the matrices representing an image that is part of the enriched image.

[0055] According to another aspect of the invention, there is provided a computer program comprising executable instructions which, when executed by a computing device, implement all the steps of the method according to the invention.

[0056] The computer program can be in any computer language, such as machine language, C, C++, JAVA, Python, etc.

[0057] According to another aspect of the invention, there is provided a device for imaging a scene comprising: - several camera modules each comprising an optical lens associated with an image sensor, and - at least one computing unit; configured to implement all the steps of the method according to the invention.

[0058] At least one camera module can acquire an image of the scene with a field angle different from that(those) of at least one other camera module.

[0059] Preferably, each of the camera modules can acquire an image of the scene with a different field angle than those of the other camera modules.

[0060] At least one camera module has a non-adjustable, fixed field angle.

[0061] At least one camera module has an adjustable / modifiable field angle, continuously or discretely. In this case, said camera module is equipped with, or is associated with, a mechanism for modifying the field angle of said camera module.

[0062] Generally speaking, the device according to the invention may include all the characteristics described above with reference to the imaging method according to the invention, and which are not repeated here exhaustively for the sake of conciseness.

[0063] The device according to the invention may comprise a control unit configured / programmed to control and command the camera modules of said device in order to image the scene in accordance with the method according to the invention.

[0064] The control unit may be a software module, such as an application or a computer program. Alternatively, the control unit may be a hardware module, such as a computer chip, a processor, etc. Alternatively, the control unit may be a combination of at least one software module and at least one hardware module.

[0065] The device according to the invention may further comprise a unit for processing the acquired images in order to obtain an enriched image. This processing unit may be configured / programmed to carry out at least one of the following steps: - align at least two images together, - combine / merge at least two images, - generate an enriched image, for example represented by a single matrix of values, from several images, - etc.

[0066] The processing unit may be a software module, such as an application or a computer program. Alternatively, the processing unit may be a hardware module, such as a computer chip, a processor, etc. Alternatively, the processing unit may be a combination of at least one software module and at least one hardware module.

[0067] The control unit and the processing unit may be separate units. Alternatively, the control unit and the processing unit may be integrated within a single unit.

[0068] At least one of the control and processing units can be integrated into another module, such as a photo application installed on the device.

[0069] The device according to the invention can be integrated into any type of device such as a smartphone, a tablet, a computer, a calculator, a processor, a computer chip, etc.

[0070] According to another aspect of the invention, there is provided an apparatus comprising an imaging device according to the invention.

[0071] The apparatus according to the invention may further comprise at least one means for displaying an image which may be any type of display means.

[0072] For example, the display means may be a display screen, such as a touch screen, etc.

[0073] For example, the display means may be a projector projecting images onto a display surface.

[0074] In particular, the device may be a user device such as a smartphone, tablet, etc.

[0075] In particular, the device may be a computer-type user device.

[0076] In particular, the device may be a television.

[0077] In particular, the device may be a camera, a video camera, a camcorder, etc.

[0078] In particular, the device may be a virtual reality headset and / or an augmented reality headset.

[0079] In particular, the device may be a medical imaging device. More particularly, the device may be an endoscope, an ultrasound device, etc.

[0080] Of course, the invention is not limited to the devices just listed.

[0081] According to another aspect of the present invention, there is provided a vehicle equipped with an imaging device according to the invention.

[0082] The vehicle may include at least one means for displaying an image which may be any type of display means.

[0083] For example, the display means may be a display screen, such as a touch screen, etc.

[0084] For example, the display means may be a projector projecting images onto a display surface.

[0085] According to embodiments, the vehicle may be a land vehicle, such as a car, semi-autonomous, autonomous or not.

[0086] According to embodiments, the vehicle may be a flying vehicle, such as a drone, an airplane, a helicopter, semi-autonomous, autonomous or not.

[0087] According to embodiments, the vehicle may be a maritime vehicle, such as a boat or a submarine, semi-autonomous, autonomous or not.

[0088] According to another aspect of the present invention, there is provided a method of representing a scene on a display medium, with an enriched image acquired with the imaging method according to the invention, said method comprising at least one iteration of the following steps: - display of an image acquired according to a field angle, - receiving field angle modification data, and - change the field angle of the displayed scene.

[0089] The modification data may be entered, in particular manually, by a user observing the displayed scene, for example through a user interface such as a keyboard, mouse, touch screen, etc.

[0090] Alternatively, or in addition, the modification data may be detected in an automated manner by at least one sensor arranged to detect a target area / position in which a user is interested, such as a sensor detecting the position aimed by the user's gaze, the position aimed by a finger or hand of a user, etc.

[0091] Thus, the representation of the scene can be carried out from the enriched image acquired in accordance with the present invention. The representation of the scene on a display surface can be carried out according to a determined strategy, depending for example on an area of ​​interest of a user, and / or a zoom level desired by the user, who can thus navigate in the displayed scene.

[0092] Following an example of realization: - the representation of the scene can start with an image captured by a wide-angle camera module, for example 0.5x; - when the user zooms towards the center of the image, in principle containing the most interesting area of ​​the imaged scene, a zoom effect can be obtained by displaying the image captured with the standard camera module, - when the user continues to zoom, a wide zoom effect can be achieved by displaying the image captured with the telephoto camera module. Multiple levels of high zoom can be achieved if multiple images are captured at different field angles by the standard camera module, and / or by the telephoto camera module, as described above. Such a zoom level can, for example, reach 5x with a camera equipped with a telephoto camera module with up to a 5x field of view, although the initially displayed image covers the field of view of the 0.5x lens.

[0093] The different images, or areas of the scene found on the different images, can be refocused / merged with a fusion algorithm, such as a refocusing and progressive fading algorithm: starting for example from the widest angle image, to display the standard field angle image then the image with the telephoto field angle. Description of figures and embodiments

[0094] Other advantages and characteristics will appear on examining the detailed description of non-limiting embodiments, and the attached drawings in which: - FIGURE 1 is a schematic representation of a non-limiting exemplary embodiment of a method according to the invention; - FIGURE 2 is a schematic representation of another non-limiting example of embodiment of a method according to the invention; - FIGURE 3 is a schematic representation of a non-limiting exemplary embodiment of a device according to the invention; - FIGURES 4a-4c are schematic representations of non-limiting exemplary embodiments of an apparatus according to the invention; - FIGURE 5 is a schematic representation of a non-limiting exemplary embodiment of a vehicle according to the invention; and - FIGURE 6 is a schematic representation of a non-limiting exemplary embodiment of a method of representing a scene according to the invention.

[0095] It is understood that the embodiments which will be described below are in no way limiting. In particular, it will be possible to imagine variants of the invention comprising only a selection of characteristics described below isolated from the other characteristics described, if this selection of characteristics is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art. This selection includes at least one characteristic of functional preference without structural details, or with only part of the structural details if it is only this part which is sufficient to confer a technical advantage or to differentiate the invention from the state of the prior art.

[0096] In particular, all the variants and embodiments described can be combined with each other if there is no technical obstacle to this combination.

[0097] In the figures and in the rest of the description, the elements common to several figures retain the same reference.

[0098] FIGURE 1 is a schematic representation of a non-limiting exemplary embodiment of an imaging method according to the present invention.

[0099] The method 100 of FIGURE 1 can be used to image a scene S, with an imaging device, or apparatus comprising an imaging device, comprising several camera modules capable of imaging the scene with different field angles, in order to obtain an enriched image of said scene.

[0100] The imaging device comprises at least two camera modules capable of imaging the scene with at least two different field angles.

[0101] The method 100 comprises a step 102 of acquiring several images of the scene by the camera modules of the apparatus, at least two of the images being acquired with two different field angles. All the images can be acquired with different field angles. Only a portion of the images can be acquired with different field angles. The images can be acquired simultaneously or sequentially. A portion of the images can be acquired sequentially and another portion of the images can be acquired simultaneously.

[0102] At least two images acquired with two different field angles can be acquired simultaneously by two different camera modules. At least two images acquired with two different field angles can be acquired sequentially by two different camera modules, or by the same camera module.

[0103] In any case, the acquisition step 102 provides several images of the scene S acquired with different field angles, for example 0.5x, 1x, 2x, 5x, etc.

[0104] In an optional step 104, the images acquired in step 102 may be processed to be re-registered with each other. This re-registering makes it possible to correct movement defects of the device used when the scene is imaged. This re-registering also makes it possible to correct any movements of an object in the scene.

[0105] The registration can be carried out using images acquired with the same field of view. The registration can include detection of object(s), or point(s), of interest in the scene and repositioning of the images with reference to these objects. The registration can use other techniques well known to those skilled in the art and which will not be detailed here. Indeed, the registration of an image of a scene relative to another image of the same scene can be done according to various techniques documented in the prior art.

[0106] In a step 106, the images acquired, and possibly processed in step 110, are combined / merged to obtain an enriched image IE of the scene. Preferably, this step combines / merges only the images acquired with different field angles, but it is also possible to combine / merge images acquired with the same field angle, for example to improve the brightness of the scene or for any other reason.

[0107] According to an exemplary embodiment, the images are combined into an image stack. In this case, each matrix may comprise the values ​​captured by the sensor used for the acquisition of said image, and each matrix may be associated with a value indicating the field angle with which said image was acquired. The value representing the field angle may be stored in association with the matrix, or directly in each cell of the matrix. For example, at least one cell of the matrix may comprise the values: (AV, R, G, B), with - AV a value indicating the field angle with which the image was taken, for example AV=45° for a standard lens, and - R, G and B respectively the values ​​of the three colors red, green and blue captured by a pixel of the image sensor.

[0108] According to another embodiment, the enriched image can be represented by a single matrix of values, for example the single matrix described above.

[0109] During a step 108 the enriched image IE, namely the stack of matrices representing a stack of images or the single matrix comprising the data relating to several images, is stored in a memory, within the device, or remotely from the device, for example on a remote server or in the cloud.

[0110] This enriched image IE allows the scene to be represented in a richer manner and includes more information relating to the scene compared to a conventional image. Thus, the enriched image can be used to represent the scene on a display surface, such as a display screen or a projection surface, in a manner individualized for a user and allows said user to navigate in the scene represented on the display surface. [YES] FIGURE 2 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present invention.

[0112] The method 200 of FIGURE 2 includes all of the steps of the method 100 of FIGURE 1.

[0113] In the non-limiting example of method 200 of FIGURE 2, the scene is imaged with an apparatus comprising three camera modules: - a first camera module, called a wide-angle camera module, designed to image a scene with a 0.5x field angle; - a second camera module, called a standard camera module, designed to image a scene with a field angle of lx; - a third camera module, called a telephoto camera module, capable of imaging a scene with several field angles, for example example several field angles ranging from 2x to 5x. This telephoto camera module is equipped with a mechanism allowing the field angle of said module to be modified, continuously or discreetly between 2x and 5x. Of course, this example is given for information purposes only and is in no way limiting.

[0114] In the method 200 of FIGURE 1, the acquisition step 102 comprises several acquisition steps carried out sequentially one after the other.

[0115] During a step 202, three images are acquired simultaneously, namely: - an image acquired by the wide-angle camera module, at 0.5x; - an image acquired by the standard camera module at lx; and - an image acquired by the telephoto camera module with a 2x field of view. These three images are memorized.

[0116] During a step 204 carried out after step 202, simultaneously: - an image acquired by the telephoto camera module with a 3x field of view. - optionally, an image acquired by the standard camera module at lx; and - optionally, an image acquired by the 0.5x wide-angle camera module. This or these images are memorized.

[0117] During a step 206 carried out after step 204, simultaneously: - an image acquired by the telephoto camera module with a 5x field of view. - optionally, an image acquired by the standard lx camera module; and - optionally, an image acquired by the 0.5x wide-angle camera module. This or these images are memorized.

[0118] Thus, the acquisition step 102 of the method 200 provides images acquired with different field angles, some of these images being acquired simultaneously and others sequentially. In this non-limiting example, step 102 provides at least five images acquired with different field angles, and at most nine images including: - five captured at different field angles, - three with the field angle 0.5x, and - three with the field angle lx.

[0119] According to other examples not shown, the acquisition step 102 may only comprise step 202. Although extended, other embodiment examples are possible, in particular depending on the number of camera modules and the field angle values ​​with which each of these camera modules is capable of imaging the scene.

[0120] FIGURE 3 is a schematic representation of a non-limiting exemplary embodiment of an imaging device according to the present invention.

[0121] The device 300 of FIGURE 3 can be used to implement the method according to the invention for imaging a scene, and in particular any one of the methods 100 or 200 of FIGURES 1 and 2.

[0122] The device 300 comprises several camera modules for imaging a scene. Each camera module is formed by an optical lens and an image sensor, for example of the CCD or CMOS type. In the example shown, the device comprises “n” camera modules 302i-302 n , with n>2. For example n = 3. Current smartphones can include up to 3 camera modules for example.

[0123] At least one of the 302i-302 camera modules nmay have an adjustable field angle, continuously or discretely. In other words, at least one 302i camera module may be equipped with a, associated with a, mechanism (not shown) for modifying / adjusting the field angle of said camera module, continuously or discretely. Such a camera module may be used to perform sequential acquisition of several images of the same scene with several field angles.

[0124] At least one 302i camera module may have a field angle of fixed, non-adjustable value. In other words, at least one 302i camera module may not be equipped with, or not be associated with, a mechanism for modifying the field angle of said camera module. In this case, said camera module always acquires images with the same field angle.

[0125] The device 300 further comprises a computing unit 304 cooperating with the camera modules 302i-302 nto implement the steps of the method according to the invention.

[0126] The computing unit 304 may include a control unit 306 configured / programmed to control and command the camera modules 302i-302 n in order to image the scene in accordance, for example, with a previously defined sequence. This sequence may define the number of images to be acquired, the field angle for acquiring each image, the camera module used for acquiring each image, and the order in which the images are acquired, in the case where images are acquired sequentially. In particular, this control unit 306 may be arranged to control the camera modules 302i-302 n in order to carry out the image acquisition step 102 of the methods 100 and 200.

[0127] The computing unit 304 may comprise a processing unit 308 configured / programmed to process the acquired images in order to obtain an enriched image. This processing unit 308 may be configured to carry out at least one of the following steps: - align at least two images with each other, - combine / merge at least two images, - generate an enriched image, represented by a single matrix of values, from several images, - etc. In particular, this processing unit 308 can be arranged to carry out step 106, and optionally step 104, of the methods 100 and 200.

[0128] At least one of the units 304-308 may be a software module, such as an application or a computer program.

[0129] Alternatively, at least one of the units 304-308 may be a hardware module, such as a computer chip, processor, etc.

[0130] According to yet another alternative, at least one of the units 304-308 may be a combination of at least one software module and at least one hardware module.

[0131] Units 306-308 can be separate units. Alternatively, units 306-308 can be integrated within a single module.

[0132] At least one of the units 304-308 can be integrated into an existing application within a device equipped with the device according to the invention, such as for example a photo application installed on said device.

[0133] Of course, the device according to the invention may comprise other components / organs than those described above. For example, in certain cases, the device 300 may comprise a means for storing the acquired images and / or the enriched image obtained.

[0134] The device according to the invention can be, or be integrated into, any type of device such as a smartphone, a tablet, a computer, etc.

[0135] FIGURE 4a is a schematic representation of a non-limiting exemplary embodiment of an apparatus according to the present invention.

[0136] The apparatus 410 of FIGURE 4a comprises means configured to implement the method according to the invention, and in particular any one of the methods 100 or 200.

[0137] The device 410 is for example a Smartphone or a tablet.

[0138] The apparatus 410 comprises a device according to the invention, and in particular the device 300 of FIGURE 3.

[0139] In the example shown in FIGURE 4a, the imaging device 300 comprises a non-limiting number of three camera modules: 302i, 3022 and 302s, arranged for example on the side of a rear face of the device.

[0140] The apparatus 410 may further comprise a display screen 412 equipped with a detection surface 414, for example capacitive.

[0141] FIGURE 4b is a schematic representation of another non-limiting exemplary embodiment of an apparatus according to the present invention.

[0142] The apparatus 420 of FIGURE 4b comprises means configured to implement the method according to the invention, and in particular any one of the methods 100 or 200.

[0143] The device 420 is for example a virtual reality headset and / or an augmented reality headset.

[0144] The helmet 420 comprises a device according to the invention, and in particular the device 300 of FIGURE 3.

[0145] In the example shown in FIGURE 4b, the imaging device 300 comprises a non-limiting number of three camera modules: 302i, 3022 and 302s, arranged for example at a lateral edge of the front face of the helmet 420.

[0146] The headset 420 may further include a display screen 422.

[0147] The headset may further include a sensor (not visible in FIGURE 4b) for detecting the position aimed by an eye, or eyes, of the user on said display screen 422.

[0148] FIGURE 4c is a schematic representation of a non-limiting exemplary embodiment of an apparatus according to the present invention.

[0149] The apparatus 430 of FIGURE 4c comprises means configured to implement the method according to the invention, and in particular any one of the methods 100 or 200.

[0150] The device 430 is a medical imaging device, such as an endoscope, an ultrasound device, etc.

[0151] The medical imaging apparatus 430 comprises a device according to the invention, and in particular the device 300 of FIGURE 3.

[0152] In the example shown in FIGURE 4c, the imaging device 300 comprises a non-limiting number of two camera modules: 302i and 3022, arranged for example in the distal part of a probe 432 intended to be introduced into the body of a living being or intended to come into contact with the body of a living being.

[0153] The medical imaging device 430 may further comprise a display screen 434 equipped with a detection surface 436, for example capacitive.

[0154] FIGURE 5 is a schematic representation of a non-limiting exemplary embodiment of a vehicle according to the present invention.

[0155] The vehicle 500 of FIGURE 5 comprises means configured to implement the invention, and in particular any one of the methods 100 or 200.

[0156] Vehicle 500 is a land vehicle, for example a car.

[0157] The vehicle 500 comprises a device according to the invention, and in particular the device 300 of FIGURE 3.

[0158] In the example shown in FIGURE 5, the imaging device 300 comprises a non-limiting number of four camera modules: 302i, 3022, 302s and 3044. These camera modules 3021-3024 can for example be arranged in the upper part of a front windshield of said vehicle, or just above the front windshield. Of course, other positions are possible.

[0159] The vehicle 500 may further comprise a display screen 502 equipped with a detection surface 504, for example capacitive, arranged in the passenger compartment of the vehicle 500.

[0160] FIGURE 6 is a schematic representation of a non-limiting exemplary embodiment of a method of representing a scene on a display surface according to the present invention.

[0161] The method 600 of FIGURE 1 can be used to represent a scene on a display surface with an enriched image IE of the scene S, acquired in accordance with the method according to the invention.

[0162] The enriched image IE may be the enriched image acquired by the method 100 of FIGURE 1 or by the method 200 of FIGURE 2.

[0163] The method 600 can be implemented in any type of device equipped with a display screen, such as for example any of the devices 410, 420 and 430 of FIGURES 4a, 4b and 4c. The method 600 can be implemented in any type of vehicle equipped with a display screen, such as for example the vehicle 500 of FIGURE 5.

[0164] The representation method according to the invention can be implemented in the same device / vehicle as that with which the scene was imaged, or in a device / vehicle other than that with which the scene was imaged. In the latter case, the scene is imaged with a first device / vehicle and the enriched image is transmitted, directly or via other equipment, to the device / vehicle in which the scene is represented with the enriched image.

[0165] The method 600 comprises a step 602 of displaying, on the display surface, the scene according to a given field angle, for example the largest field angle. Thus, the entire scene is represented on the display surface and one or more users can observe the scene on the display surface.

[0166] During a step 604, data modifying the field angle of the scene is received.

[0167] The field of view modification data may be entered manually by a user observing the displayed scene, for example using a user interface such as a keyboard, mouse, touch screen, etc.

[0168] Alternatively, or in addition, the modification data may be detected in an automated manner. For example, the apparatus / vehicle performing the representation of the scene may comprise at least one sensor arranged to detect a target area / position in which a user is interested, such as a sensor detecting the position targeted by the user's gaze, the position targeted by a finger or a hand of a user, etc. Of course, These examples are given for informational purposes only and are in no way limiting.

[0169] Depending on the modification data received, the field angle of the scene is modified, during a step 606. For example, an image of the same scene acquired with another field angle is selected / generated from the enriched image. This image with another field angle is then displayed on the display surface.

[0170] Steps 604-606 may be repeated as many times as desired, to allow a user to achieve an individualized display of the scene by changing the field angle at which the scene is represented. Thus, the user can navigate through the scene by changing the field angle at which the scene is displayed as many times as desired.

[0171] Following a non-limiting example of implementation: - the representation of the scene can start with an image captured by a wide-angle camera module, for example 0.5x; - when the user zooms towards the center of the image, in principle containing the most interesting area of ​​the imaged scene, a zoom effect can be obtained by displaying the image captured with the standard camera module, - when the user continues to zoom, a wide zoom effect can be achieved by displaying the image captured with the telephoto camera module, Multiple zoom levels can be achieved if multiple images are captured at different angles of view by the standard camera module, and / or by the telephoto camera module, as described above. Such a zoom level can, for example, reach 5x with a device equipped with a 0.5x wide-angle camera module and a telephoto camera module capable of reaching up to a 5x angle of view, although the initially displayed image covers the field of view of the 0.5x lens.

[0172] The different images, or areas of the scene found on the different images, can be re-joined / merged with a fusion algorithm, such as a re-joining and progressive fading algorithm: starting with example of the widest angle image, to display the standard field angle image and then the image with the telephoto field angle.

[0173] The representation of the scene on a display surface can thus be carried out according to a determined strategy, depending for example on a user's area of ​​interest, and / or a zoom level desired by the user, who can thus navigate in the displayed scene.

[0174] Of course, the invention is not limited to the examples which have just been described.

Claims

CLAIMS 1. Method (100; 200) for imaging a scene (S) with a device comprising several camera modules (302i-302 n ), each camera module (302i-302 n ) comprising an optical lens associated with an image sensor, said method (100;200) comprising the following steps: - acquisition (102), by at least two camera modules (302i-302 n ) different from said device, of at least two images of said scene (S) with different field angles, and - storage (108) of an image, called an enriched image, constructed with at least two of said images taken with different field angles.

2. Method (100; 200) according to the preceding claim, characterized in that the acquisition step (102) comprises a simultaneous acquisition (202), by two different camera modules (302i-302 n ), of two images at different field angles.

3. Method (100; 200) according to any one of the preceding claims, characterized in that the acquisition step (102) comprises a sequential acquisition (202, 204, 206), by two different camera modules (302i-302 n ), of two images at different field angles.

4. Method (100;200) according to any one of the preceding claims, characterized in that the acquisition step (102) comprises a sequential acquisition (202,204,206), by a camera module, of several images at different field angles, said method (100;200) comprising a modification of the field angle of said camera module between the acquisition of said images.

5. Method (100; 200) according to any one of the preceding claims, characterized in that the acquisition step (102) comprises: - an acquisition (202,204,206), by a camera module, of at least two images at different field angles, - an acquisition (202,204,206), by another camera module, of at least two images at the same field angle, different from the field angles of said images taken by said one camera module.

6. Method (100; 200) according to any one of the preceding claims, characterized in that the enriched image (IE) is a stack of images, comprising at least two images acquired at different field angles.

7. Method (100; 200) according to any one of the preceding claims, characterized in that the acquisition step (102) is carried out by three camera modules, and in particular: - a wide-angle module, in particular greater than 60°, - a module with a standard field angle, in particular equal to 45°, and - a telephoto module, particularly less than 45°.

8. Image (IE), stored on a non-transitory storage medium, obtained by the method according to any one of the preceding claims.

9. Computer program comprising executable instructions which, when executed by a computing device, implement all the steps of the method (100;200) according to any one of the preceding claims.

10. Device (300) for imaging a scene (S) comprising: - several camera modules (302i-302 n ) each comprising an optical lens associated with an image sensor, and - at least one calculation unit (304); configured to implement all the steps of the method (100; 200) according to any one of claims 1 to 7.

11. Apparatus (410;420;430) comprising an imaging device (300) according to the preceding claim.

12. Vehicle (500) comprising an imaging device (300) according to claim 11.

13. Method (600) for representing a scene on a display medium, with an enriched image acquired with the method (100; 200) according to any one of claims 1 to 7, said method (600) comprising at least one iteration of the following steps: - display (602) of an image acquired according to a field angle, - receiving (604) a field angle modification data item, and - modification (606) of the field angle of the displayed scene.

Images