Method and system for defining a region contour in an image with distorted lines

Abstract

Methods and systems are provided for defining a region contour in an image with distorted lines. The method includes displaying a first version of an image with distorted lines, wherein the lines in a first image portion of the first version of the image have reduced distortion, receiving user input defining a first contour portion of the region contour in the first version of the image, wherein the first contour portion includes one or more lines in the first image portion of the first version of the image, displaying another version of the image with distorted lines, wherein the lines in another image portion of the other version of the image have reduced distortion, and receiving user input defining another contour portion of the region contour in the other version of the image, wherein the other contour portion includes one or more lines in the other portion of the other version of the image.

Description

Field of the Invention

[0001] The present invention relates to the field of defining contours in images. Specifically, it relates to methods and systems for defining region contours in images with distorted lines. Background Art

[0002] In video surveillance applications, users typically want to define regions in the video images of a surveillance scene. To give a few examples, the region can be an area where an alarm should or should not be triggered if someone breaks in, an area where privacy shielding should be provided, or an area where loitering or objects should be detected.

[0003] The user can define such a region by drawing the contour of the region in the displayed video image. For the user, the simplest way is to define the contour by defining a plurality of lines that together form the contour of the region. This is also provided by commercial tools. In addition to simplicity, it is often the case that the desired region follows a straight line in the scene, such as a fence that defines the boundary of an alarm area or a window that should be privacy shielded.

[0004] However, problems arise when defining region contours in images with distorted lines, such as panoramic images. A panoramic image can be generated, for example, by capturing an image with a wide-angle lens. In this case, the lens will cause lines that are actually straight to appear curved in the resulting image. In another example, a panoramic image can be generated by stitching together a plurality of images captured from different viewpoints. In this case, the images are typically first projected onto a common three-dimensional surface, where they are stitched together. In a subsequent step, the stitched images represented on the three-dimensional surface are mapped onto a two-dimensional image plane so that they can be displayed on a conventional display screen. As a result of the mapping, the straight lines in the stitched images on the three-dimensional surface will appear distorted, i.e., curved in the resulting panoramic image. As a result, the desired region contour typically includes curved segments rather than straight lines. Therefore, the user cannot simply define the desired region contour by drawing straight lines in the image, and tracking the curved segments along the region is cumbersome in itself. Therefore, there is a need for methods to simplify the definition of contours in images with distorted lines. Summary of the Invention

[0005] In view of the above, an object of the present invention is to alleviate the above problems and provide methods and systems for simplifying the definition of contours in images with distorted lines.

[0006] This object is achieved by the appended independent claims. Example embodiments are given by the appended dependent claims.

[0007] According to a first aspect, there is provided a method for defining a region contour in an image having distorted lines. The method includes displaying a first version of the image having distorted lines, wherein the lines in a first image portion of the first version of the image have reduced distortion, receiving user input defining a first contour portion of the region contour in the first version of the image, wherein the first contour portion includes one or more lines in the first image portion of the first version of the image, displaying another version of the image having distorted lines, wherein the lines in another image portion of the another version of the image have reduced distortion, and receiving user input defining another contour portion of the region contour in the another version of the image, wherein the another contour portion includes one or more lines in the another image portion of the another version of the image.

[0008] Using this method, the region contour is defined by successively defining contour portions in different versions of the image. The different versions of the image differ in that the distortion of the lines is reduced in different image portions. Thus, the first version of the image has reduced distortion in the first image portion, which enables the input of the contour portion in the form of a straight line in the first image portion. The second version of the image, alternatively, has reduced distortion in the second image portion, which enables the input of the contour portion in the form of a straight line in the second image portion. Thus, by successively displaying different versions of the image having reduced line distortion in their different portions, it is possible to define the contour by drawing straight lines in the image.

[0009] The first version and the another version of the image having distorted lines are generated from the same image data, i.e., from the same pixel values, and thus both the first version and the another version are versions of the same image. However, the pixel positions in the first version of the image and in the another version of the image are displaced spatially relative to each other. Thus, the first version and the another version of the image can be referred to as spatially transformed versions of the same image. Specifically, the another version of the image can be referred to as a spatially transformed version of the first version of the image. The spatial transformation of the image means that the pixel positions in the image are subject to a spatial transformation that maps them to corresponding pixel positions in the spatially transformed version of the image.

[0010] Both the first version and the another version of the image have distorted lines. However, they have reduced line distortion in different image portions. The lines in an image portion of the image having reduced distortion means that they have reduced or lower line distortion compared to other portions of the image. Line distortion refers to the situation where a straight line in the scene is depicted as curved in the image. When the distortion of a line is reduced, we can say that it is corrected or straightened.

[0011] It should also be understood that at the end of the method, the region contour will be composed of the defined contour portions.

[0012] The method may further include repeating the steps of displaying another version of the image and receiving user input defining another contour portion of the region contour until the region contour is fully defined. In this way, the user is allowed to continuously define contour portions in different versions of the image until the user deems the contour to be complete. When the user provides input in this regard, the contour may be considered fully defined. Alternatively or additionally, when the contour of the region defines a closed region, such as when the starting point of the first contour portion and the end point of the last input contour portion meet to obtain a closed contour, the contour of the region may be considered complete.

[0013] In an exemplary embodiment, the first version of the image has reduced line distortion along a first horizontal line in the first version of the image, and another version of the image has reduced line distortion along another horizontal line in the other version of the image. Thus, the first image portion may extend along the first horizontal line in the first version of the image, and another image portion may extend along the other horizontal line in the other version of the image. Thus, it can be said that the first version of the image has a straightened horizontal line along the first horizontal line, and the other version of the image has a straightened horizontal line along the other horizontal line. This is particularly advantageous for panoramic images, which typically suffer from line distortion except along horizontal lines that can be set by the user and are referred to as "horizontal lines".

[0014] The first version of the image may be displayed in response to receiving user input indicating the first image portion, and another version may be displayed in response to receiving user input indicating the other image portion. In this way, the user can continuously define in which image portion the next displayed image version should have reduced line distortion.

[0015] The method may further include transforming a first contour portion defined in the first version of the image using a transformation between the first version of the image and another version of the image. This transformation refers to a spatial transformation between pixel positions in the image plane of the first version of the image and the image plane of the other version of the image. This allows calculating the pixel positions of the first contour portion in the image plane of the other version of the image. Additionally, it enables the transformed first contour portion to be displayed together with the other version of the image. In this way, when presenting a new version of the image to the user, the previously input output portion will move forward, and new image versions will be continuously presented. This improves the user experience as it enables the user to obtain an overview of the contour defined so far.

[0016] The method may further include merging the transformed first contour portion and another contour portion defined in the other version of the image into a common representation of the contour in the other version of the image. Thus, when contour portions are input, they can be continuously merged into a common representation.

[0017] The first contour portion defined in the first version can be represented by a plurality of first points, and the first contour portion is transformed by transforming the plurality of first points using a transformation. Representing a contour portion using a plurality of spatial points along the contour is a storage-efficient way of describing the contour portion. Usually, a few points on each line segment are sufficient. It also allows the contour portion to be transformed from the image plane space of one image version to the image plane of another image version at low computational cost, since only the positions of the points need to be spatially transformed.

[0018] Alternatively, the first contour portion can be transformed by applying a transformation to the first superimposed image after rendering of the first contour portion defined in the first version of the image. This allows the contour portion to be transformed from the image plane space of one image version to the image plane of another image version. This option can be used when the point representation of the first contour portion is not available.

[0019] The method can further include displaying the first contour portion together with the first version of the image, and displaying another contour portion together with another version of the image.

[0020] In an example embodiment, both the first version and the another version of the image are panoramic images. In some examples, the first version and the another version of the image are obtained by mapping an image represented on a three-dimensional surface to a two-dimensional image plane using a first mapping function and another mapping function, respectively. The three-dimensional surface can be a spherical or cylindrical surface, and a plurality of images captured by a panoramic camera are projected onto the surface so as to stitch the images together. In other examples, the first version and the another version of the image are obtained by respectively using a first undistortion function and another undistortion function to undistort a sheared portion of a wide-angle image. The wide-angle image can be captured, for example, by a wall-mounted camera through a wide-angle lens.

[0021] Regions in the image can be used as inclusion or exclusion zones for alerts, privacy masking, loitering detection, and / or object detection. Other applications of the region can also be envisioned.

[0022] According to a second aspect of the present invention, the above object is achieved by a system for defining a region contour in an image having distorted lines.

[0023] According to a third aspect of the present invention, the above object is achieved by a (non-transitory) computer-readable medium containing computer code instructions which, when executed by a processing device, cause the processing device to execute the method according to the first aspect.

[0024] The second and third aspects generally have the same features and advantages as the first aspect. It should also be noted that the present invention relates to all possible combinations of features, unless otherwise explicitly stated. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other objects, features, and advantages of the present invention can be better understood from the following illustrative and non - limiting detailed description of embodiments of the present invention with reference to the accompanying drawings, in which like reference numerals will be used for like elements, wherein:

[0026] Figure 1 Schematically shows a system for defining the contour of a region with distorted lines in an image according to an embodiment.

[0027] Figure 2 Schematically shows the undistortion of a wide - angle image using different undistortion functions according to an embodiment.

[0028] Figure 3 Schematically shows mapping an image presented on a three - dimensional surface to a two - dimensional image plane using different mapping functions according to an embodiment.

[0029] Figure 4 Is a flowchart of a method for defining the contour of a region in an image with distorted lines according to an embodiment.

[0030] Figures 5a to 5c Schematically illustrates Figure 4 User interface views of different stages of the flowchart method. DETAILED DESCRIPTION OF THE INVENTION

[0031] The present invention will be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown. The systems and devices disclosed herein will be described during operation.

[0032] Figure 1 Shows a system 1 for defining the contour of a region in an image with distorted lines. System 1 includes a processing unit 121 and a user interface 14. The processing unit 121 is operably connected to the user interface 14 via a wired or wireless connection.

[0033] The processing unit 121 can be arranged to operate in association with a memory 122. The processing unit 121 and the memory 122 can be part of a processing device 12. The memory 122 can serve as a non - transitory computer - readable medium storing computer code instructions executed by the processing unit 121. Specifically, the computer code instructions can cause the processing unit 121 to execute any method disclosed herein.

[0034] The user interface 14 includes a display 141 on which an image generated by the processing unit 121 can be displayed. The user interface 14 also includes a user input device 142, such as a computer mouse, a touch screen, etc., which allows the user to provide user input related to the displayed image. The user input is transmitted to the processing unit 121. To enable user interaction, the user interface 14 can implement a graphical user interface 143 through which an image is displayed to the user, and through which the user can define a region contour in the displayed image. Specifically, the graphical user interface 143 can be configured to allow the user to define parts of the contour, for example, by drawing one or more lines in the displayed image. The graphical user interface 143 can further allow the user to indicate an image part of the image, such as an image part extending along a horizontal line in the image. In the example shown, the graphical user interface 143 includes a slider that allows the user to indicate the vertical height or coordinates of the horizontal line. In response to receiving an indication of an image part of the image, the currently displayed image can be transformed to reduce line distortion in the image part. Specifically, the line distortion can be reduced along the indicated horizontal line.

[0035] The system 1 can be arranged to receive an image from the camera 16. The camera 16 can be a video camera that produces a panoramic image video sequence. In some embodiments, the processing unit 121 and the memory 122 are arranged in the camera 16, while in other embodiments, they are arranged outside the camera 16, such as in a video management system.

[0036] In Figure 2 In one example shown, the camera 16 captures an image of a scene by using a wide-angle lens (such as a fish-eye lens). As a result, the camera 16 generates a wide-angle image 21 of the scene. The wide-angle image 21 is represented on a two-dimensional image plane and generally has a circular shape. Since light is refracted by the wide-angle lens, the wide-angle image 21 is distorted or warped in the sense that straight lines in the scene will be curved in the wide-angle image 21. The processing unit 121 can generate a panoramic image 22 having a substantially rectangular shape from the wide-angle image 21. Specifically, it can generate panoramic images 22a, 22b from a clipped portion 211 of the wide-angle image 21. For a camera 16 arranged to provide a side view of a monitored scene, such as a camera mounted on a wall, the clipped portion 211 can be formed by those pixels in the wide-angle image 21 that depict scene points within a specific angular range above and below the horizontal center line 212 of the camera 16. In one example, the angular range can cover scene points located from 80 degrees below the horizontal center line 212 to 60 degrees above the horizontal center line. As Figure 2 shown, such a clipped portion 211 generally has the shape of an hourglass. Figure 2Each dashed line therein corresponds to a point in the scene at an angle above or below the horizontal center line 212 of the camera 16.

[0037] To generate the panoramic images 22a, 22b, the processing unit 121 may apply a transformation to the sheared portion 211, herein referred to as the undistortion functions f1, f2. Each undistortion function f1, f2 is a spatial mapping between a spatial point or pixel position in the sheared portion 211 and a spatial point or pixel position in the rectangular image plane of the panoramic images 22a, 22b. Depending on which undistortion function is used, line distortion can be reduced in different image portions of the resulting panoramic images 22a, 22b. Specifically, in Figure 2 the example shown, the choice of the undistortion functions f1, f2 affects the horizontal lines 23a, 23b in the panoramic images 22a, 22b, where line distortion will be reduced. In the panoramic image 22a, line distortion is reduced in the image portion extending along the first horizontal line 23a (herein located at the center of the panoramic image 22a), while in the panoramic image 22b, line distortion is reduced in the image portion extending along another horizontal line 23b (above the center of the panoramic image 22b in this example). How to implement and construct the undistortion function that reduces line distortion in a specific image portion is known per se to those skilled in the art and is thus not described in detail herein. For example, the "fisheye camera model" library in OpenCV (https: / / docs.opencv.org / 3.4 / db / d58 / group__calib3d__fisheye.html) can be used. In a practical implementation, a lookup table can be constructed that specifies which undistortion function to use to reduce line distortion in a given image portion, such as along a given horizontal line. The undistortion function can be specified according to the parameters of the undistortion function represented by parameters.

[0038] In Figure 3In another example shown, the images are panoramic images 32a, 32b, which are obtained by mapping an image 31 displayed on a three-dimensional surface 34 (such as a spherical surface or a cylindrical surface) onto a two-dimensional image plane. The mappings M1, M2 map the spatial points or pixel positions in the image 31 on the three-dimensional surface 34 to the spatial points or pixel positions in the two-dimensional image plane of the images 32a, 32b. The image 31 represented on the three-dimensional surface 34 can in turn be obtained by projecting multiple images of a scene captured by the camera 16 from different viewpoints. Specifically, the camera 16 can include multiple image sensors that are directed in different viewing directions and capture images of the scene simultaneously. From the multiple images, the image 31 can be generated by stitching the multiple images together. Stitching of the images generally includes using information describing how the multiple image sensors are arranged relative to each other to project the multiple images onto a common three-dimensional projection surface, such as the surface 34. This allows the pixels in the multiple images corresponding to the same point in the scene to be projected onto the same point on the three-dimensional surface 34, so that the multiple images obtained from different viewpoints can be combined into a stitched image 31.

[0039] Mapping the M1, M2 of the image 31 on the three-dimensional surface to the two-dimensional image plane causes the straight lines in the image 31 to be distorted in the resulting panoramic images 32a, 32b. This is shown by the dashed line in the image 31, which is distorted when the mappings M1, M2 are applied. However, depending on the choice of the mappings M1, M2, there will be an image portion extending along the horizontal lines in the panoramic images 32a, 32b, along which there is reduced line distortion, or even no line distortion, compared to other portions of the panoramic images 32a, 32b. As Figure 3 shown, the position of the horizontal line depends on which mapping is used. When using the mapping M1, it causes the lines in the image portion along the first horizontal line 33a to have a reduced amount of distortion. If a different mapping M2 is used, it causes the lines along the second horizontal line 33b to have a reduced amount of distortion. Therefore, by appropriately choosing the mappings M1, M2, a panoramic image 32 can be obtained that has reduced line distortion in a desired image portion, such as along the desired horizontal lines 33a, 33b. The relationship between the mapping and the position of the horizontal height is known per se to those skilled in the art and will not be described in more detail here, where the lines along the position of the horizontal height have reduced distortion. For example, the "image warping" library in OpenCV (https: / / docs.opencv.org / 4.x / d0 / dfa / group__stitching__warp.html) can be used. In a practical implementation, a lookup table can be constructed that specifies which mapping functions M1, M2 are used for a given horizontal line 33a, 33b.

[0040] Referring again toFigure 1 The user interface 14 enables the user to input an indication as to which image part of the image the line distortion should be reduced for. In particular, the user can indicate along which horizontal line in the image the distortion should be reduced. After such user input, the processing unit 121 can generate an image version with reduced line distortion in that image part by applying a suitable undistortion function or mapping as described above.

[0041] Reference will now be made to Figure 4 the flowchart of Figures 1 to 3 and Figures 5a to 5b to illustrate the operation of the system 1 when performing a method for defining a region contour in an image with distorted lines. As described above, the image can be a panoramic image obtained by undistorting a sheared part of a wide-angle image using an undistortion function or by mapping an image presented on a three-dimensional surface onto a two-dimensional image plane.

[0042] In step S04, a first version of the image is displayed on the user interface 14, such as in the view area of the graphical user interface 143 on the display 141. The first version of the image is generated by the processing unit 121. In the case of obtaining a panoramic image by mapping an image represented on a three-dimensional surface onto a two-dimensional image plane, the first version of the image can be generated by using the first mapping M1. In the case of obtaining a panoramic image by undistorting a wide-angle image, the first version of the image can be generated by undistorting the wide-angle image using the first undistortion function f1. Figure 5a shows a first version 51a of the image. The first version 51a of the image is generated so as to have reduced line distortion in its first image part 52a. This means that the distortion of the lines is reduced, i.e., the line distortion in the first image part 52a is lower compared to other image parts of the first version 51a of the image. This can be achieved, for example, by appropriately selecting the first mapping function or the first undistortion function, such as by using a look-up table. In the example shown, the first image part 52a extends horizontally across the first version 51a of the image, i.e., along the first horizontal line. It can be seen that the lines in the first image part 52a are less distorted than the lines in other parts of the first version 51a of the image. In other words, the lines in the first image part 52a are straightened or corrected. Although this effect is more obvious for the horizontal lines in the first version 51a, it also applies to the vertical lines in the image. The image part 52a can correspond to a rectangular image part as shown in Figure 5a or even a single horizontal line in the image. However, it should be understood that the image part can generally have any shape and position in the image plane.

[0043] In some embodiments, in response to step S02 of receiving a user input indicating a first image portion 52a, a first version 51a is generated and displayed. For example, the user input may indicate a vertical height 53a of a first horizontal line along which the first image portion 52a extends, such as a vertical coordinate. The vertical height 53a may be input via a graphical user interface, for example, by sliding a slider to a desired vertical height, as Figure 5a shown. However, it can be foreseen that there are many other ways to provide an input indicating the first image portion 52a, including drawing a line or a rectangle in the view area of the graphical user interface. In this way, the user can select in which image portion 52a the line should be straightened. In particular, the user can select a horizontal line in the panoramic image 52a that should be straightened. In a typical case, as described below, the user selects an image portion 52a in which he aims to define a contour portion of the region. In the example shown, the user indicates an image portion 52a that extends along the lower boundary of the depicted backyard. In response to receiving an indication of the first image portion 52a, for example, by using a look-up table, the processing unit 121 can appropriately select a first undistortion function f1 or a first mapping M1 to have reduced line distortion in the first image portion 52a of the first version 51a of the image.

[0044] In step S06, the processing unit 121 receives, via the user interface 14, a user input defining a first contour portion 54a of a region in the first version 51a of the image. For example, the user can define the first contour portion 54a by drawing a line in the first version 51a of the image. As shown, the first contour portion 54a can be displayed on the user interface 14 together with the first version 51a of the image. For example, the processing unit 121 can render a first overlay image depicting the first contour portion 54a. By way of example, pixels belonging to the first contour portion 54a can be assigned a predefined pixel value, such as the value 255 in the first overlay image. Then, by overlaying the first overlay image on the first version 51a of the image, the first overlay image can be displayed together with the first version 51a. In some embodiments, the first overlay image can be stored in a buffer for future use, which will be explained in further detail below.

[0045] The lines in the first portion 52a of the first version 51a of the image have reduced distortion, which enables the input of the first contour portion 54a. Thus, the user can specifically define the first contour portion 54a to include one or more lines 541a in the first image portion 52a. When the first image portion 52a extends along a horizontal line as Figure 5a shown, the one or more lines 541a will thus extend mainly in the horizontal direction in the first version 51a of the image. As Figure 5aAs further shown in , the first contour portion 54a may also include one or more lines 542a that extend beyond the first image portion 52a in the first version 51a of the image. The one or more lines 542a may correspond, for example, to lines in the scene that predominantly extend in the vertical direction. Generally, in panoramic images of the type described herein, the distortion of horizontal lines in the scene is more severe than the distortion of vertical lines in the scene. Thus, before defining the first contour portion 54a, the need to reduce the distortion of vertical lines or lines having a predominantly vertical direction in the scene is not as important as for horizontal lines in the scene.

[0046] In step S10 of the method, another version 51b of the image is displayed on the user interface 14, such as in the view area of the graphical user interface 143 on the display 141. Another version 51b of the image is generated by the processing unit 121. In the case of obtaining a panoramic image by mapping an image represented on a three-dimensional surface onto a two-dimensional image plane, another version 51b of the image can be generated by using a mapping M2 that is different from the first mapping M1 used to generate the first version 51a of the image. In the case of obtaining a panoramic image by undistorting a wide-angle image, another version 51b of the image can be generated by undistorting the wide-angle image using another undistortion function f2 that is different from the undistortion function f1 used to generate the first version of the image.

[0047] Another version 51b of the image is generated to reduce the line distortion in another image portion 52b of the another version 51b. The another image portion 52b is different from the first image portion 52a of the first version 51a. This means that the distortion of the lines is reduced, i.e., the line distortion is lower in the another image portion 52a compared to other image portions of the another version 51a of the image. This can be achieved, for example, by appropriately selecting the another mapping M2 or the another undistortion function f2, such as by using a look-up table. In the example shown, the another image portion 52b extends horizontally across the another version 51b of the image, i.e., along another horizontal line in the another version 51b of the image. In this case, the another image portion 52b corresponds to the upper boundary of the depicted backyard. It can be seen that the lines in the another image portion 52b are less distorted than the lines in other portions of the another version 51b of the image. Additionally, the lines in the another image portion 52b are less distorted than they are in the corresponding spatial portions of the first version 51a of the image.

[0048] In some embodiments, in response to step S08 of receiving a user input indicating another image portion 52b, another version 51b is generated and displayed. For example, the user input may indicate a vertical height 53b of another horizontal line along which another image portion 52b is to extend, such as a vertical coordinate. Similar to the explanation in connection with step S02, the vertical height 53b may be input via a graphical user interface, for example, by sliding a slider to the desired vertical height, as Figure 5b shown. In response to receiving an indication of another image portion 52b, the processing unit 121 may appropriately select another undistortion function f2 or another mapping M2, which will result in reduced line distortion in the another image portion 52b of the another version 51b of the image.

[0049] To enhance the user experience, it is desirable to display the first contour portion 54a together with the another version 51b of the image. However, before doing so, the first contour portion 54a needs to be transformed to map its pixel positions defined in the image plane of the first version 51a of the image to the corresponding pixel positions in the image plane of the another version 51b of the image. Generally, the another version 51b of the image can be regarded as a spatial transformation T of the first version 51a of the image. Specifically, the transformation T maps the spatial positions or pixel positions in the first version 51a of the image to the corresponding spatial positions or pixel positions in the second version 51b of the image. In the case where the first and second versions of the image are generated by applying the undistortion functions f1 and f2 to the wide-angle image respectively, the transformation T is given by T = f2(f1) -1 as given. In the case where the first and second versions of the image are generated by applying the mappings M1 and M2 to the image represented on the surface in the three-dimensional space respectively, the transformation T is given by T = M2(M1) -1 as given. By transforming the first contour portion 54a using the transformation T, that is, by mapping the pixel positions of the first contour portion 54a in the first version 51a of the image, it can be represented in the coordinate system of the another version 54b of the image. As a result, as Figure 5b shown, the transformed first contour portion can be displayed together with the another version 51b of the image. In this way, the contour portion defined in the previous displayed version of the image can be moved to the current displayed version of the image.

[0050] Different methods may be employed to transform the first contour portion 54a and display it together with the another version 51b of the image.

[0051] In a first method, a first contour portion 54a defined in a first version 51a has a vector representation. This means that both lines 541a and 542a that make up the first contour portion 54a are represented by vectors having a magnitude and a direction. Starting from this vector representation, the processing unit 121 can determine a plurality of first spatial points that are subsequently located along the first contour portion 54a. For the purpose of transforming the first contour portion 54a, the plurality of first points can be used to represent the first contour portion 54a. To transform the first contour portion, the processing unit 121 then applies a transformation T to the plurality of first points so that corresponding spatial points in a second version 51b of the image are found. If necessary, more points can be added after manual operation to represent the first contour portion 54a. In particular, if it is found that after applying the transformation T, a point located along a certain line segment in the first version 51a deviates from being located along the line segment when transformed to the second transformed view 51b, this may be necessary. If so, it means that more points should be added to represent the line segment. When added, the transformation T can be used to transform the additional points. Thus, one can start with a small number of points, such as three points per line segment, and add more points when needed.

[0052] After transforming the point representation of the first contour portion 54a to the coordinate system of the second transformed view 51b, the vector representation of the first contour portion 54a in the coordinate system of the second transformed view 51b can be defined according to vectors connecting subsequent points among the transformed plurality of first points. To display the transformed first contour portion 54a, the processing unit 121 can then render a further superimposed image, in which the transformed plurality of first points are interconnected by lines. As Figure 5b shown, by superimposing this another superimposed image on the second transformed view 51b, this another superimposed image can then be displayed together with the second transformed view 51b.

[0053] In a second method, the processing unit 121 does not apply the transformation T to a potential representation of the first contour portion 54a. Instead, it operates to use the transformation T to transform a first superimposed image that depicts the first contour portion 51a in the coordinate system of the first version 51a. As further mentioned above, such a first superimposed image is typically rendered in combination with the display of the first version 51a and can be retrieved from a buffer. For example, the pixel positions of pixels in the first superimposed image can be mapped to corresponding pixel positions in a second version 51b of the image using the transformation T. In this way, another superimposed image that depicts the first contour portion 51a in the coordinate system or image plane of the second transformed view 51b is generated. The second superimposed image can be added to the buffer and displayed as a superimposed image together with the second version 51b of the image, as Figure 5b shown.

[0054] Next, in step S12, the processing unit 121 receives a user input via the user interface 14, which defines another contour portion 54b of a region in another version 51b of the image. Similar to the explanation in connection with the first contour portion 54a, the user can define the another contour portion 54b by drawing a line in another version 51b of the image.

[0055] Since the lines in another portion 52b of another version 51b of the image have reduced distortion, the input of another contour portion 54b is achieved. Thus, at this stage of the method, the user can conveniently define another contour portion 54b to include one or more lines 541b in another image portion 52b. In the case where another image portion 52b extends along another horizontal line as shown in Figure 5c shown, the one or more lines 541b will thus extend mainly in the horizontal direction in another version 51b. Although not shown in the example of Figure 5c , it should be understood that another contour portion 54b can also include one or more lines that extend outside another image portion 52b in another version 51b of the image.

[0056] The processing unit 121 can merge another contour portion 54b with any previously received contour portion into a common representation of the region contour. In the example shown, the processing unit 121 can thus merge another contour portion 54b with the first contour portion 54a. To merge the contour portions 54a and 54b, they need to be represented in the same coordinate system or image plane. Thus, before merging, the contour portions 54a and 54b should be transformed to the same coordinate system. In some embodiments, the coordinate system can be the coordinate system of another version 51b of the image. In this case, the processing unit 121 will thus merge the first contour portion 54a with the transformed version of another contour portion 54b into a common representation of the contour described in the coordinate system of another version 51b of the image. In other embodiments, the merging can alternatively be performed in the coordinate system of the first version 51a. In other embodiments, the merging can be performed in the coordinate system of the original image data, such as in the coordinate system of the wide - angle image 21 of Figure 2 or Figure 3 the image 31 represented on a three - dimensional surface.

[0057] If the first contour portion 54a and another contour portion 54a have vector or point representations, the processing unit 121 can merge the vector or point representations into a common representation. For example, when transformed to the coordinate system of another version 51b of the image, the vector or point representation of the first contour portion 54a can be merged with the vector or point representation of another contour portion 54b.

[0058] As an alternative, if the transformed version of the first contour portion 54a is represented by the above-mentioned another superimposed image, the processing unit 121 may add another contour portion 54b to the same superimposed image. For example, pixels in the another superimposed image belonging to another contour portion 54b may also be assigned a predefined pixel value, such as the value 255. In this way, another superimposition will serve as a common representation of the first contour portion and the another contour portion. By applying a scan line algorithm to the further superimposition, the processing unit 121 can determine which pixel positions are within the region defined by the contour and which are outside the region. For example, this can be used to infer whether a pixel position is within an alert or analysis zone, or to fill pixel positions within the region in the superimposed image for privacy masking purposes. The scan line algorithm can also be used to convert the superimposed image of one or more contour portions into a point or vector representation of the contour portion.

[0059] As Figure 5c shown, the another contour portion 54b and the transformed version of the first contour portion 54a can be displayed together with another version 51b of the image in the user interface 14. For example, the processing unit 121 may display another version 51b of the image and a superimposed image depicting the transformed versions of the first contour portion 54a and the another contour portion 54b.

[0060] In step S14, the processing unit 121 may check whether the region contour has been completely defined, that is, whether the user has defined all parts of the contour. For example, when the region is completed, the user may provide an input to the processing unit 121 via the user interface 14. If it is found that the contour is not completely defined, steps S08, S10, and S12 are repeated until the user has defined the entire contour of the region. If the contour is completely defined, no other versions of the image are presented to the user. This is Figures 5a to 5c the case in the example where the contour is completely defined by the first contour portion 54a and the second contour portion 54b. Thus, when the contour has been completely defined, the contour consists of contour portions 54a, 54b, defined in different transformed views 51a, 51b of the image by user input. At this stage, the representation of the contour portion can be transformed into a reference coordinate system. Such a reference coordinate system may correspond to a predefined reference mapping Mref or a reference de-warping function fref.

[0061] It should be understood that those skilled in the art can modify the above embodiments in various ways and still utilize the advantages of the present invention shown in the above embodiments. Therefore, the present invention should not be limited to the embodiments shown, but should be defined only by the appended claims. In addition, as understood by those skilled in the art, the embodiments shown can be combined.

Claims

1. A method for defining a region contour in an image with distorted lines, characterized in that it includes: displaying (S04) a first version (51a) of the image with distorted lines, wherein the distortion of the lines in a first image part (52a) of the first version (51a) of the image is less than the distortion of the lines in other parts of the first version (51a) of the image, receiving (S06) a user input that defines a first contour part (54a) of the region contour in the first version (51a) of the image, wherein the first contour part (54a) includes one or more lines (541a) in the first image part (52a) of the first version (51a) of the image, displaying (S10) another version (51b) of the image with distorted lines, wherein the distortion of the lines in another image part (52b) of the another version (51b) of the image is less than the distortion of the lines in other parts of the another version (51b) of the image, and wherein the another image part (52b) is different from the first image part (52a), receiving (S12) a user input that defines another contour part (54b) of the region contour in the another version (51b) of the image, wherein the another contour part (54b) includes one or more lines (541b) in the another image part (52b) of the another version (51b) of the image, and repeating the steps of displaying (S10) another version of the image and receiving (S12) a user input that defines another contour part of the region contour until the region contour is fully defined.

2. The method according to claim 1, wherein, the first image part (52a) extends along a first horizontal line in the first version (51a) of the image, and the another image part (52b) extends along another horizontal line in the another version (51b) of the image.

3. The method according to claim 1, wherein, displaying (S04) the first version (51a) of the image in response to receiving a user input (S02) that indicates the first image part (52a), and displaying (S10) the another version (51b) of the image in response to receiving a user input (S08) that indicates the another image part (52b).

4. The method according to claim 1, further including: transforming (T) the first contour part (54a) defined in the first version (51a) of the image using a transformation (T) between the first version (51a) of the image and the another version (51b) of the image.

5. The method according to claim 4, further including: displaying the transformed first contour part (54a) together with the another version (51b) of the image.

6. The method according to claim 4, further including: Combine the transformed first contour portion (54a) and the other contour portion (54b) defined in the other version (51b) of the image into a common representation of the contour in the other version (51b) of the image.

7. The method according to claim 4, wherein, the first contour portion (54a) defined in the first version (51a) of the image is represented by a plurality of first points, and wherein the first contour portion (54a) is transformed by transforming the plurality of first points using the transformation (T).

8. The method according to claim 4, further comprising: rendering a first superimposed image of the first contour portion (54a) defined in the first version (51a) of the image, wherein the first contour portion (54a) is transformed by applying the transformation (T) to the rendered first superimposed image.

9. The method according to claim 1, further comprising: displaying the first contour portion (54a) together with the first version (51a) of the image, and displaying the other contour portion (54b) together with the other version (51b) of the image.

10. The method according to claim 1, wherein, the first version (51a) and the other version (51b) of the image are obtained by mapping an image represented on a three-dimensional surface (34) onto a two-dimensional image plane (32a, 32b) using a first mapping function (M1) and another mapping function (M2), respectively.

11. The method according to claim 1, wherein, the first version (51a) and the other version (51b) of the image are obtained by respectively undistorting a sheared portion (211) of a wide-angle image (21) using a first undistortion function (f1) and another undistortion function (f2).

12. The method according to claim 1, further comprising: using the region in the image as an inclusion or exclusion zone for alerts, privacy masking, loitering detection, and / or object detection.

13. A system (1) for defining a region contour in an image with distorted lines, characterized in that comprises: a processing unit (121) arranged to compute a first version (51a) of an image with distorted lines, wherein the distortion of the lines in the first image portion (52a) of the first version (51a) of the image is less than the distortion of the lines in other portions of the first version (51a) of the image, and to compute another version (51b) of the image with distorted lines, wherein the distortion of the lines in the other image portion (52b) of the other version (51b) of the image is less than the distortion of the lines in other portions of the other version (51b) of the image, and wherein the other image portion (52b) is different from the first image portion (52a), A user interface (14) is arranged to: display the first version (51a) of the image, and receive user input defining a first contour portion (54a) of the region contour in the first version (51a) of the image, wherein the first contour portion (54a) includes one or more lines (541a) in the first image portion (52a) of the first version (51a) of the image; display the other version (51b) of the image, and receive user input defining another contour portion (54b) of the region contour in the other version (51b) of the image, wherein the other contour portion (54b) includes one or more lines (541b) in the other image portion (52b) of the other version (51b) of the image; and repeat the steps of displaying the other version of the image and receiving user input defining another contour portion of the region contour until the region contour is fully defined.

14. A computer-readable medium comprising computer code instructions which, when executed by a processing device, cause the processing device to perform the method according to any one of claims 1 to 12.

Images