device and method for detecting regions of interest in an image

The method uses geographical prominence principles to refine region of interest detection by analyzing pixel depth distributions, addressing computational inefficiencies and improving contour accuracy in image processing.

FR3148856A1Inactive Publication Date: 2024-11-22ORANGE SA
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
FR2023004863
Authority / Receiving Office
FR · FR
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-16
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing methods for detecting regions of interest in images, particularly those using deep learning and Gaussian mixture models, are computationally intensive and often fail to accurately define the exact contours of identified objects, necessitating simpler and more efficient solutions.

Method used

A method employing geographical prominence principles to determine regions of interest by analyzing pixel depth distributions, identifying peaks and their relative heights, and selecting the foreground peak based on these relative heights to refine the region of interest.

Benefits of technology

This approach simplifies the detection of regions of interest by leveraging geographical prominence, providing accurate and efficient identification of object contours without requiring high computational resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

The invention relates to a method for detecting a region of interest in an image comprising: - identifying (E1) a first area of ​​the image, - obtaining (E2) a depth map of said first area of ​​the image, - obtaining (E3) at least one distribution of the pixels of said first area as a function of their depth, - determining (E4) at least one region of interest in said first area taking into account, for at least a first peak of the pixel distribution, a relative height of said first peak with respect to at least a second of said peaks of the pixel distribution, said second peak being the highest local minimum located in said pixel distribution between said first peak and another peak of said distribution higher than said first peak. Figure for the abstract: Fig. 4.
Need to check novelty before this filing date? Find Prior Art

Description

Title of the invention: Device and method for detecting a region of interest in an image Technical field

[0001] The present invention relates to a method and system for determining a region of interest in an image and finds its interest in many fields. Prior art

[0002] There are several methods for detecting and isolating regions of interest in images. A region of interest (also known by the acronym ROI for "Region Of Interest") can typically be a part of an image with remarkable properties. It can notably be an object present in the image, such as an object in which a user is interested or an object likely to interest a user or which can be the subject of subsequent automated processing (such as tracking). For example, it can be useful, in autonomous driving systems, to detect vehicles, road signs, people, present on the route. In logistics systems, it is important to be able to locate objects in warehouses, the objects can also be of different natures.In particular, there are interactive techniques in which a user selects the region of interest themselves, more or less accurately or precisely. There are also several methods based on image processing technologies. With the development of artificial intelligence and more specifically "deep learning" technologies, these region of interest detection techniques have improved. However, it sometimes happens that the region of interest thus identified does not completely correspond to an object and does not define its exact contours. Various techniques have been developed to refine the detection of a region of interest that would have been roughly identified by a user or by a first, at least partially automatic, image processing method.For example, there are techniques using depth maps and modeling the background by mixtures of Gaussians to identify the foreground of the region of interest. These approaches use advanced data structures (mixture of Gaussians and graphs) and are particularly computationally intensive. They can therefore only be implemented on certain computing architectures with very large capacities. There is therefore a need for simpler solutions to refine the determination of a region of interest. Statement of the invention

[0003] The invention proposes to overcome at least one drawback of the prior art by proposing a method for detecting a region of interest in an image comprising: - an identification of a first area of ​​the image, - obtaining a depth map of said first area of ​​the image, - obtaining at least one distribution of the pixels of said first zone as a function of their depth, - a determination of at least one region of interest in said first zone taking into account, for at least a first peak of the pixel distribution, a relative height of said first peak with respect to at least a second of said peaks of the pixel distribution, said second peak being the highest local minimum located in said pixel distribution between said first peak and another peak of said distribution higher than said first peak.

[0004] For simplification, in the present application the “relative height” of a peak will be described by the relative height of this peak in relation to this highest local minimum.

[0005] Thus, the present invention proposes, in an innovative manner, to apply a notion similar to that, recently introduced in the field of geography, of geographic prominence, to the detection of regions of interest in a digital image.

[0006] According to at least one embodiment, said determination takes into account the depth of said first peak.

[0007] According to at least one embodiment, determining at least one region of interest comprises - a determination (E32) of the relative height of peaks of said distribution, - a sorting of said peaks in a decreasing order of their relative height, - a selection (E33) of said first peak for which the following relative height peak has a greater depth.

[0008] According to at least one embodiment, determining at least one region of interest comprises: - a determination (E32) of the relative height of peaks of said distribution, - a sorting of said peaks in increasing order of their depth, - a selection of said first peak for which the next depth peak has a lower relative height.

[0009] According to at least one embodiment, said region of interest is determined by selecting (E41) all of the pixels between two minima framing said first peak selected in said second representation.

[0010] According to at least one embodiment, said distribution is a discrete or continuous distribution.

[0011] The characteristics presented in isolation in the present application in connection with certain embodiments of the method of the present application can be combined with each other according to other embodiments of the present method.

[0012] The invention also relates to a computer program comprising instructions for executing the steps of the method according to the invention, according to any one of its embodiments, when said program is executed by a computer.

[0013] The invention also relates to a computer-readable recording medium on which is recorded a computer program comprising instructions for executing the steps of the method according to the invention, according to any one of its embodiments.

[0014] The present invention also relates to a device comprising one or more processors configured together or separately for executing the steps of the method according to the invention, according to any one of its embodiments. Thus, the present invention relates to a device for detecting a region of interest in an image comprising one or more processors configured together or separately for: - identify a first area of ​​the image, - obtaining a depth map of said first area of ​​the image, - obtain at least one distribution of the pixels of said first zone according to their depth, - determining at least one region of interest in said first zone taking into account, for at least a first peak of the pixel distribution, a relative height of said first peak with respect to at least a second of said peaks of the pixel distribution, said second peak being the highest local minimum located in said pixel distribution between said first peak and another peak of said distribution higher than said first peak. Brief description of the drawings

[0015] Other characteristics and advantages of the present invention will emerge from the description given below, with reference to the accompanying drawings which illustrate an exemplary embodiment thereof without any limiting character.

[0016] [Fig-1] [Fig.l] represents a system implementing certain modes of rea use of the present invention,

[0017] [Fig.2] [Fig.2] shows a representation of an area encompassing a region of interest in an image,

[0018] [Fig.3] [Fig.3] represents an example of a depth map of the area in global of [Fig.2],

[0019] [Fig.4] [Fig.4] represents an embodiment of a method according to the invention,

[0020] [Fig.5] [Fig.5] represents a first embodiment of a distribution discrete pixels according to the invention,

[0021] [Fig.6] [Fig.6] represents a second embodiment of a continuous distribution of pixels according to the invention,

[0022] [Fig.7] [Fig.7] represents the concept of relative peak height as defined in the present application,

[0023] [Fig.8] [Fig.8] represents an embodiment of a sorting of the peaks in descending order of relative height,

[0024] [Fig.9] [Fig.9] represents an embodiment of sorting the peaks in ascending order of depth. Description of the embodiments

[0025] The present disclosure relates to the detection of a region of interest in an image. In this document, the term region of interest means any part of an image that is of interest or provides a benefit with respect to a given application. For example, in an application relating to the autonomous driving of vehicles, a region of interest may be an obstacle such as another vehicle, an object on the road, roadworks, pedestrians, etc.).

[0026] According to certain embodiments, a region of interest may correspond to an object that interests a user and this for different applications.

[0027] According to certain embodiments, the present disclosure relates to the general field of computer vision and is applicable to various applications using regions of interest.

[0028] [Fig.l] shows a system that may implement the present disclosure in certain embodiments.

[0029] A scene is captured by means 20 for capturing at least one image. A scene may be representative of an indoor or outdoor environment and include one or more objects, animals, people, background, etc. These capture means may take different forms depending on the embodiments and may in particular include: - one or more cameras such as stereographic cameras, and / or LIDAR (acronym for “light detection and ranging”), and / or RADAR (acronym for “radio detection and ranging”), and / or plenoptic cameras, and / or sensors using ultrasound known as ToF devices (acronym for “time of flight”).

[0030] These means for capturing (20) at least one image can be associated with processing means 10. The processing means 10 and the capturing means 20 can be included in the same single device 100 or belong to different devices coupled together. The processing means include in particular means for obtaining a depth map of at least one portion of the scene whose image was captured by the capture means. The processing means may include stereographic vision devices, radar vision devices, lidar vision devices or ToF type devices.

[0031] As illustrated by [Fig.l], the processing means may have the hardware architecture of a computer. Thus, the processing means 10 comprise, in particular, a processor 1, a random access memory 2, a read-only memory 3 and a non-volatile memory 4. They further comprise communication means 5, in particular for communicating with the capture device 20 and a user interface 30.

[0032] The read-only memory 3 constitutes a recording medium in accordance with at least one embodiment of the present disclosure, readable by the processor 1 and on which is recorded a computer program PROG in accordance with at least one embodiment of the present disclosure, comprising instructions for executing steps of the method for determining an area of ​​interest according to at least one embodiment of the present disclosure. The program PROG defines functional modules of the device.

[0033] The user interface 30 may allow a user to interact with the capture and processing means. The user interface may take various forms, including one or more screens, whether touch-sensitive or not, one or more keyboards or styluses or a tablet, mobile phone or computer. In [Fig.l], the user has selected an area encompassing the area of ​​interest.

[0034] [Fig. 2] represents an example of a scene filmed by the image capture means 20 (such as in this example a camera) and in which an area is delimited by the black rectangle. This area aims to encompass a region of interest (for example an object) whose location in this area is to be determined precisely by the present disclosure. This area may also be called the encompassing area hereinafter.

[0035] According to some embodiments, the area is delimited by a user using the user interface 30.

[0036] According to certain embodiments, the zone is automatically delimited by the processing means 10. This automatic delimitation can for example be based on methods based on artificial intelligence.

[0037] According to certain embodiments, the area is delimited both manually and automatically. A user can, for example, indicate to the processing means that he wishes to determine the area or areas comprising a certain type of object, a car for example, and the processing means are then responsible for detecting the cars in the image and delimiting the areas encompassing the cars present in the image.

[0038] According to certain embodiments, several encompassing zones can be identified, automatically by the processing means or manually by a user.

[0039] [Fig. 3] represents an example of a depth map of the bounding area illustrated in [Fig. 1]. According to certain embodiments, the depth map can be obtained per bounding area, for all or several bounding areas or for all or part of the captured image.

[0040] A depth map may consist of a grayscale representation of the image, in which the grayscale indicates the distance to the camera: the darker an area, the closer it is to the camera, or vice versa. A depth map is a visualization of a matrix (a two-dimensional array, for example) that associates, with each pixel, the distance (in meters or any other distance units) at which it is from the camera.

[0041] [Fig.4] represents an embodiment of a method according to the present disclosure.

[0042] The method according to at least one embodiment of the present disclosure can be implemented in a device such as the device 100 shown in [Fig.l].

[0043] One or more images of a scene are captured by the image capturing means identified in [Fig.l]. The present method may make it possible to identify one or more regions of interest in this or these images.

[0044] Thus, during a step E1, a first area of ​​the image is identified or selected. For example, a user selects on an image one or more areas encompassing the region(s) of interest in an approximate, or coarse, manner, that is to say that the encompassing area comprises at least one portion of the image which is not part of the region of interest. The encompassing area may be an area whose contours may approximately follow the contours of the region of interest and / or be of geometric shape. For example, when the user selects the area manually (using a stylus or a mouse for example), the encompassing area may be a rectangle around a region of interest (for example when the region of interest is itself rather rectangular in shape) (as shown in [Fig. 2]), a circle around a region of interest (for example when the region of interest is itself rather round)....This selection can be made by means of the user interface 30 identified in [Fig.l]. This selection can be made by surrounding the area of ​​interest, so as to encompass the area of ​​interest in its entirety. As mentioned previously, this selection can also be made by the processing means 10 automatically or semi-automatically, for example based on methods based on artificial intelligence. The processing means can be configured so as to identify certain objects, for example animals in a scene, by ap- . learning, and to delimit an area encompassing the animals for each animal detected.

[0045] The encompassing area thus delimited approximately, around the region of interest, therefore comprises a set of pixels, some of which are part of the region of interest to be isolated or identified.

[0046] Once the bounding areas have been obtained, a depth map is obtained for each or some of the bounding areas, step E2.

[0047] In certain embodiments, a global depth map of the captured image can be constructed and then this map can be segmented into several depth maps each representing a portion of the captured image, at least one of the depth maps representing a portion of the captured image and comprising at least one encompassing zone of at least one region of interest that can be used in the remainder of the method described below.

[0048] According to certain embodiments, when several bounding areas are identified, at least some of the steps of the method below can be implemented simultaneously in parallel for different identified bounding areas.

[0049] As mentioned previously, the depth maps can be obtained, for example, by the processing means 10 or by devices coupled to these processing means, by stereographic vision devices, radar vision devices, lidar vision devices or “time of flight capture” type devices known by the acronym ToF, English acronym for “time of flight”.

[0050] During the following steps, the method makes it possible to determine at least one region of interest in at least one encompassing zone from at least one depth map associated with this encompassing zone.

[0051] To do this, during a step E3, a distribution or a representation of the distribution of the pixels of the surrounding area is obtained as a function of their depth, from the depth map obtained for the surrounding area.

[0052] The distribution obtained may be of different natures. According to at least one embodiment, the distribution is a discrete distribution (for example in the form of a histogram). According to at least one embodiment, the distribution is a continuous distribution.

[0053] [Fig.5] illustrates, by way of example, a representation in the form of a histogram.

[0054] In this histogram, the x-axis represents the distance to the camera (therefore the depth), and the y-axis represents the number of pixels. The step or granularity chosen on the x-axis is 5 cm. The height of the bars in the histogram represents the number of pixels in each interval, the intervals on the x-axis can be [0, 5 cm]; [5, 10 cm]; [10, 15 cm]... Of course this granularity can vary, it can for example depend on the depth maximum, the greater the maximum depth, the greater the granularity can be. It can also depend on, or be indexed to, the size of the zone.

[0055] [Fig.6] illustrates a continuous representation. The abscissa axis represents the distance to the camera, therefore the depth, and the y-axis represents the number of pixels.

[0056] During a step E4, at least one region of interest is determined in at least one of the previously selected encompassing zones. This determination takes into account, for at least a first peak of the pixel distribution, a relative height of the first peak with respect to at least a second of the peaks of the pixel distribution, the second peak being the highest local minimum located in the pixel distribution between the first peak and another peak of the distribution higher than said first peak.

[0057] According to certain embodiments, the determination takes into account the depth of the first peak. Thus, it may be possible to select, for example, a peak in the foreground or a peak close to this foreground.

[0058] According to certain embodiments, step E4 may comprise obtaining (localizing or identifying) E41 peaks of the representation. The peaks of the histogram are identified in [Fig.5], the index i representing an interval number on the abscissa axis, a peak Pi is such that its height H; verifies:

[0059] Hi i< H; > Hi+1

[0060] According to certain embodiments, step E4 may comprise obtaining (or determining) E42 information or data relating to the height, also called relative height for simplification, of the peaks of the representation.

[0061] [Fig.7] illustrates the distribution of the relative heights of the peaks (as determined at step E42) of the pixel distribution.

[0062] In the illustrated example, the relative height of a peak is the minimum of the difference between the height of the peak and the minimum of the histogram between the peak and the peaks immediately above it to the right and left if these exist. The relative height of the highest peak corresponds to its height.

[0063] In [Fig.7], the relative height of a peak P; is calculated as follows: - we determine the highest peak on the right and the highest peak on the left of the current peak P, - we determine the minima MinG; and MinD; on each of the intervals defined by - MinG;: minima between the current peak and the highest peak on the left and - MinDi: minima between the current peak and the highest peak on the right, - MinDi has a height HD; and MinG; having a height HG. - the relative height PRO(Pi) of the current peak P; is given by:

[0064] PRCKP.) = Min( (Hi - HD; ); (H; - HG{ ))

[0065] The definition of the relative height of a peak given above also applies to a continuous distribution but the identification of the peaks may be different. For this purpose, the calculation of the relative height of a peak, in the case of a continuous representation, may consist of identifying the extrema of the distribution.

[0066] The maximum of a mathematical function verifies f (x) = 0 and / < 0; while the minimum of a function / verifies f (x) — 0 and f" > 0; f' and f" representing respectively the first and second derivatives of the function f.

[0067] The minima and maxima can be identified by solving the equation f (x) = 0

[0068] This equation can be solved: - analytically by solving the equation directly or - numerically using Newton's method.

[0069] The calculation of the second derivative then makes it possible to distinguish the minima from the maxima.

[0070] The minima and maxima being known, the relative height of a maximum can then be obtained by applying the previous definition.

[0071] The region of interest may be determined by selecting the foreground or near-foreground peak having the highest relative height, step E43. The region of interest may be determined from a representation of the relative height of the peaks of the pixel distribution. To do this, it may be possible to represent the distribution of relative heights (and no longer the height of the peaks of the pixel distribution as in [Fig.6]).

[0072] In Figures 8 and 9, 6 peaks denoted Pa, Pb, Pc, Pd, Pe, Pf which were previously identified during step E41 are shown schematically. If [Fig.5] represents a first distribution or a representation of a first distribution, then Figures 8 and 9 can represent a second, respectively a third, distribution or a representation of a second, respectively a third, distribution.

[0073] Each of these peaks is associated with a depth noted respectively for the peaks Pa, Pb, Pc, Pd, Pe, Pf, Prof(Pa), Prof(Pb), Prof(Pc), Prof(Pd), Prof(Pe), Prof(Pf).

[0074] Each of these peaks is associated with a relative height noted respectively for the peaks Pa, Pb, Pc, Pd, Pe, Pf, HR(Pa), HR(Pb), HR(PC), HR(Pd), HR(Pe), HR(Pf).

[0075] According to this example, we have:

[0076] HR(Pa) > HR(Ph) = HR(Pc) > HR(Pd) > HR(Pe) > HR(P^

[0077] Prof (Pb) < Prof (Pa) < Prof (Pd) < Prof (Pc) < Prof (Pf) < Prof (Pf)

[0078] According to a first embodiment of step E43, the peaks can be sorted, E431, in a decreasing order of their relative height as illustrated in [Fig.8], schematically, in order to facilitate the identification of the peak in the foreground or close to this foreground having the highest relative height.

[0079] The peaks are scanned in the decreasing direction of relative height and for each peak the depth of the current peak is compared with the depth of the peak following the current peak and the first peak for which the following peak has a greater depth is selected, step E432. Thus, the peak selected in this example is peak Pb, because peak Pa has a greater relative height but a lesser depth, and the following peaks all have a greater depth. It can be noted that peak Pc has an identical relative height but a greater depth, so it is not selected. Thus, peak Pb here represents the foreground peak.

[0080] According to a second embodiment of step E43, the peaks can be sorted, E431', in an increasing order of their depth as illustrated in [Fig.9], schematically, in order to facilitate the identification of the peak in the foreground or close to the foreground having the highest relative height.

[0081] The peaks are scanned in the increasing depth direction, the relative height of the current peak is compared with the relative height of the peak following the current peak and the first peak for which the following peak has a lower relative height is selected in step E432'. Thus, the peak selected in this example is the peak Pa, because the peak Pb which has a higher depth has a greater relative height. It can be noted that the peak Pc has an identical relative height but a higher depth, it is therefore not selected. Thus, the peak Pa is here a peak not in the foreground but close to the foreground. For example, it is thus possible to filter out parasitic elements present in the foreground.

[0082] It can be noted in this representation in ascending order of depth that, in certain embodiments, if two peaks have an identical associated depth, then at identical depth, the peak having a greater relative height is selected.

[0083] The area of ​​interest is determined from the representation obtained, step E43.

[0084] According to certain embodiments, the area of ​​interest is obtained by selecting, step E44, a set of pixels around the peak selected during step E43.

[0085] According to certain embodiments, this set of pixels is made up of the pixels between the two minima framing the peak selected during step E43.

[0086] Certain applications of this disclosure may be of interest in the manufacturing industry, and in particular in verifying the conformity of a part produced on production lines. The present disclosure can in fact help to precisely detect a part and therefore to determine its shape, its size and to verify whether it conforms to an expectation or to a specification and this in a manner at least partially automatic (without human intervention for example). Knowledge of the size and location of objects can also help to stabilize and specify the movements of robots handling these objects.

[0087] Other applications may relate to the field of logistics. Certain embodiments of the present disclosure may make it possible to track and locate goods in warehouses. Knowledge of the size of objects may help to estimate (for example to optimize) the storage space required for their storage and may therefore contribute to the management of flows in warehouses.

[0088] Other applications may concern the autonomous driving of vehicles by making it possible to determine obstacles on the road, for example, a region of interest which may represent an obstacle (other vehicles, objects, works, pedestrians, etc.).

[0089] Other applications may relate to the construction of maps of a physical environment to enable the navigation of robots, drones and autonomous vehicles in the presence of obstacles.

Claims

Claims

1. Method for detecting a region of interest in an image comprising: - an identification (E1) of a first zone of the image, - an obtaining (E2) of a depth map of said first zone of the image, - an obtaining (E3) of at least one distribution of the pixels of said first zone as a function of their depth, said distribution showing peaks - a determination (E4) of at least one region of interest in said first zone taking into account, for at least one first peak of the pixel distribution, a relative height of said first peak with respect to at least one local minimum determined as the highest minimum located in said pixel distribution between said first peak and another peak of said distribution higher than said first peak.

2. A detection method according to claim 1 wherein said determination takes into account the depth of said first peak.

3. Method according to claim 1 in which the determination of at least one region of interest comprises - a determination (E32) of the relative height of peaks of said distribution, - a sorting of said peaks in a decreasing order of their relative height in order to obtain a second distribution of pixels, - a selection (E33) of said first peak for which the following relative height peak has a greater depth.

4. Method according to claim 1 in which the determination of at least one region of interest comprises: - a determination (E32) of the relative height of peaks of said distribution, - a sorting of said peaks in an increasing order of their depth, - a selection of said first peak for which the following depth peak has a lower relative height.

5. Method according to claim 3 in which said region of interest is determined by selecting (E41) the set of pixels between two minima framing said first selected peak in said second distribution of pixels.

6. Method according to one of the preceding claims in which said distribution is a discrete or continuous distribution.

7. Computer program comprising instructions for executing the steps of the method according to one of claims 1 to 6 when said program is executed by a computer.

8. A computer-readable recording medium on which is recorded a computer program comprising instructions for carrying out the steps of the method according to one of claims 1 to 6.

9. Device for detecting a region of interest in an image comprising one or more processors configured together or separately to: - identify a first area of ​​the image, - obtain a depth map of said first area of ​​the image, - obtain at least one distribution of the pixels of said first area as a function of their depth - determine at least one region of interest in said first area taking into account, for at least one first peak of the pixel distribution, a relative height of said first peak with respect to at least one local minimum determined as the highest minimum located in said pixel distribution, between said first peak and another peak of said distribution higher than said first peak.

Citation Information

Patent Citations

  • Image processing apparatus, image processing method, and program

    US20130215107A1

  • Method and ui for z depth image segmentation

    US20150104101A1

  • Container use estimation

    US20190197719A1