Image recording device

Abstract

The image recording device has an optoelectronic image sensor, a position sensor, a graphical user output interface, and an evaluation device. The image sensor acquires an image data set representing a scene located in front of the image sensor as imaged in an image area of ​​the image sensor. The position sensor detects the spatial position of the image area relative to a reference direction and provides position data specifying both the rotation angle and tilt angle of the image sensor. The evaluation device determines from the position data a projective transformation that maps the image data set onto a projection plane that is tilted relative to the image area in accordance with both the rotation and tilt. The evaluation device determines an image portion in the projection plane for the image data set mapped to the projection plane by the projective transformation, and displays the image portion in the graphical user output interface together with at least the area of ​​the captured scene that is within the image portion.

Description

[Technical Field]

[0001] The present invention relates to an image recording device including an optoelectronic image sensor and to a method of operating an image recording device including an optoelectronic image sensor. [Background technology]

[0002] An image recording device, such as a photo or video camera, including an optoelectronic image sensor captures a scene located in front of the image recording device onto an image area of ​​the image sensor and generates image data representing the scene captured in the image area. When capturing a scene, if the image area is tilted relative to a reference area of ​​the scene, e.g., the vertical plane of a house facade, the reference area of ​​the scene and the image area of ​​the image sensor may not be aligned parallel. This results in different areas of the reference area of ​​the scene being spaced differently from the image area of ​​the image sensor, resulting in different increments of size being captured in the image area. This results in perspective distortion of the scene captured in the image area. As a result, straight lines extending parallel to each other within the reference plane may be captured as, for example, so-called plunging lines. This can occur particularly when capturing images of tall buildings from a low vantage point.

[0003] The perspective distortion caused by tilting the image recording device can be reversed during subsequent processing of the image data by an inverse transform that compensates for the perspective distortion that occurred when the image data was recorded. In this regard, typically, straight lines that run parallel to each other in the original scene are identified in the image data, and the image data is then corrected by an inverse transform so that the identified straight lines also align parallel in the recorded scene.

[0004] However, this inverse transformation also distorts the margins of the image data, so the transformed image data must be cropped to avoid the distorted margins, reducing the usable image portion. This results in subsequent processing of the already recorded image data requiring cropping of image areas that were in the original recording and were important to the construction of the image, and are no longer usable. Summary of the Invention [Problem to be solved by the invention]

[0005] The object of the present invention is to enable a user of an image recording device to easily generate image data that remains inclusive of all image areas involved in the construction of the recording even after correcting for perspective distortions that occur during recording. [Means for solving the problem]

[0006] This object is met by an image recording device and a method for operating an image recording device according to the independent claims. Further developments are defined in the respective dependent claims.

[0007] The image recording device includes an optoelectronic image sensor, a position sensor, a graphical user output interface, and an evaluation device. The image sensor is configured to acquire an image data set representing a scene located in front of the image sensor as captured in an image area of ​​the image sensor. The position sensor is further configured to detect the spatial position of the image area relative to a reference direction and to provide position data identifying both a rotation angle of the image area around the optical axis of the image sensor, particularly relative to a direct projection of the reference direction onto the image area, at the time of acquisition of the image data set, and a tilt angle of the image area around a horizontal axis, particularly relative to the reference direction, at the time of acquisition of the image data set. In this regard, the horizontal axis is oriented perpendicular to the optical axis and perpendicular to the reference direction. The evaluation device is configured to determine from the position data a projective transformation that maps the image data set to a projection plane according to both the rotation and the tilt from the image area. In this regard, the projection plane is inclined with respect to the image area according to the tilt angle and intersects the image area along an intersection line rotated in the image area relative to a central axis of the image area according to the rotation angle. Furthermore, the evaluation device is configured to determine an image portion in the projection plane for the image data set mapped to the projection plane by the projective transformation, and to display the image portion in the graphical user output interface together with at least the area of ​​the scene captured in the image area that is within the image portion.

[0008] Within the framework of the present invention, it has been found that a simple and above all perspective-correct correction of perspective distortions is possible in that the position of the image sensor with respect to the recorded scene is automatically detected by a position sensor and a perspective transformation that compensates for the perspective distortion is calculated from the position data of the position sensor. As not only the tilt of the image sensor about the horizontal axis, i.e. the tilt of the image plane of the image sensor with respect to a reference plane of the scene aligned parallel to the reference direction, but also the rotation of the image sensor about its optical axis is taken into account in the perspective transformation, a perspective-correct correction of the captured scene is achieved, in particular even if the image sensor is rotated.

[0009] By displaying the image portion determined by the evaluation device together with the captured scene in the image area on the user output interface of the image recording device, the user can orient the image sensor already during recording of the scene in such a way that all image areas required for the construction of the recording are within the image portion even if the image data is subsequently cropped. In this respect, the user of the image recording device already receives direct feedback during recording about the image portion that will still be usable after correction of the perspective distortion, so that the user can already select all relevant recording parameters taking into account the image portion that will be available later.

[0010] The reference direction used to determine the projective transformation can be made independent of, among other things, the rotation and tilt of the image sensor determined by the position sensor. This makes the absolute spatial position of the projection plane also independent of the rotation of the image sensor about its optical axis and the tilt of the image sensor about its horizontal axis. Since the horizontal axis is oriented perpendicular to the optical axis and the reference direction, the optical axis also lies in a vertical plane, oriented perpendicular to the horizontal axis and formed by the reference direction and the optical axis, after application of the projective transformation. In this regard, the line of sight into the imaged scene also does not shift from the vertical plane after application of the projective transformation, which makes it easier to construct the image portion during scene recording compared to, for example, a transformation in which the projection plane always has the same orientation relative to the image sensor, regardless of the rotation of the image sensor about its optical axis.

[0011] The inclination of the projection surface relative to the image area may in particular depend on the tilt angle such that the inclination of the projection surface relative to the image area corresponds to the tilt angle or the tilt angle minus the residual angle. The inclination of the projection surface relative to the image area may in particular depend on the tilt angle such that the projection surface is tilted relative to the image area at a tilt angle not equal to zero and is aligned parallel to the image area at a tilt angle equal to zero.

[0012] Similarly, the rotation of the intersection line between the image region and the projection plane relative to the central axis of the image region may depend on the rotation angle such that the rotation of the intersection line relative to the central axis corresponds to the rotation angle. In particular, the rotation of the intersection line relative to the central axis may depend on the rotation angle such that the intersection line does not run parallel to the central axis at rotation angles not equal to zero, and runs parallel to the central axis of the image region at rotation angles equal to zero. And, for a rectangular image region, the intersection line does not run parallel to the margins of the image region at rotation angles different from zero.

[0013] The position sensor can be configured as a three-axis sensor that determines the spatial position of the image sensor along three mutually orthogonal axes. The position sensor can be stationary and disposed together with the image sensor within a common housing. For example, the position sensor can be configured as a gravity sensor or a gyro sensor. Alternatively, the reference direction can also be predefined by user input, e.g., by orienting the central axis of the image sensor along a desired reference direction and calibrating the position determination to a corresponding spatial location, e.g., in response to the user input.

[0014] The reference direction used in determining the spatial position of the image sensor may be a fixed, predefined direction, such as the direction of gravitational acceleration, in which case the horizontal axis is specifically aligned parallel to the horizon of the recorded scene, and the tilt angle specifies the tilt of the image region from its vertical orientation.

[0015] To ensure an unambiguous definition of the position of the horizontal axis when the reference direction is aligned parallel to the optical axis, the evaluation device can be configured to determine the horizontal axis as a direction aligned perpendicular to the reference direction and perpendicular to the optical axis only when the tilt angle corresponds to a maximum predetermined limit value. In this regard, the limit value can be less than 90°, for example, less than 45°. The limit value can be, for example, 1°, 3°, 5°, or 10°, less than 90°, or less than 45°. Furthermore, the evaluation device can be configured to determine the projective transformation and the image portion only for tilt angles that correspond to a maximum predetermined limit value. Alternatively, the evaluation device can be configured to align the horizontal axis parallel to a predetermined central axis of the image area when the predetermined limit value is exceeded.

[0016] The image recording device can be configured as a photographic camera or a video camera. The image recording device can particularly be configured as a mirrorless camera. To image the scene into the image area of ​​the image sensor, the image recording device can have imaging optics (such as a lens) that is stationary connected to the image sensor and a position sensor (e.g., a housing containing the image sensor and the position sensor) during imaging. The imaging optics can particularly be an exchangeable lens. The optical axis of the imaging optics can coincide with the optical axis of the image sensor.

[0017] The user output interface may be an electronic or optical viewfinder of an image recording device or screen. If it is an optical viewfinder, the image portion may be displayed superimposed on an area of ​​the captured scene shown in the viewfinder, allowing the image portion and at least the area of ​​the captured scene within the image portion to be simultaneously displayed in the user output interface configured as a viewfinder. The screen and / or viewfinder may also be located in a housing that includes an image sensor and a position sensor. Alternatively, at least the screen may be located separately from such a housing, for example, in a mobile device (e.g., a smartphone or tablet computer) connected to the image sensor and position sensor via a data link.

[0018] The evaluation device may include at least one programmable logic device (e.g., ASIC, FPGA, microchip, etc.). It may be located in the same housing as the image sensor and the position sensor. Alternatively, at least the components of the evaluation device may be located outside this housing, for example in an external processing device (e.g., a smartphone or tablet computer) of the image recording device connected to the housing via a data link.

[0019] The image sensor may be configured, for example, as a CCD sensor or a CMOS sensor. The area of ​​the image sensor that detects incident electromagnetic radiation and converts it into electronic signals to generate an image data set forms the image area of ​​the image sensor. The image area may in particular be rectangular.

[0020] The image portion can be determined, for example, as a rectangular image portion. To determine the image portion, the evaluation device can image at least all corner points of the image region onto the projection plane by a projective transformation. The evaluation device can particularly determine the image portion by simply mapping the corner points of the image region onto the projection plane, rather than the individual pixels of the image sensor. After determining the coordinates of the image portion in the projection plane, the evaluation device can image the coordinates of the image portion back onto the image region of the image sensor by a projective transformation inverse. This makes it possible to display the image portion in the untransformed scene on a graphical user output interface. In this regard, in particular, the image portion and all image data acquired by the image region of the image sensor can be displayed simultaneously.

[0021] Alternatively, the untransformed image portion can be displayed directly in the user output interface, along with the region of the imaged scene that is also mapped to the projection plane and that is within the image portion, allowing the image portion resulting after correction for perspective distortion to be displayed over the entire area in the user output interface, and therefore at a high magnification.

[0022] In a further development, the reference direction corresponds to the direction of gravitational acceleration. This allows for easy automatic determination of the spatial position of the image sensor. Typically, perspective distortion occurs when the imaged scene contains tall, vertical objects (e.g., buildings) recorded from a low or high viewpoint. And, particularly in the case of such perspective distortion, the perspective transformation can be determined without user input. In such a further development, the position sensor can be configured, for example, as an acceleration sensor for detecting the Earth's gravitational field.

[0023] In a further development, the projection plane is aligned parallel to the horizontal axis and / or parallel to a reference direction. When the projection plane is parallel to the horizontal axis, rotation of the image area around the optical axis is fully compensated. When the projection plane is parallel to the reference direction, tilt of the image area relative to the reference direction, e.g., vertical orientation determined from the gravitational acceleration, is fully compensated.

[0024] Alternatively, the projection plane can enclose a residual angle with the horizontal axis that is different from zero and smaller than the rotation angle, such that the rotation about the optical axis is not fully compensated. Alternatively or additionally, the projection plane can enclose a further residual angle with the reference direction that is different from zero and smaller than the tilt angle, such that the tilt of the image region about the horizontal axis is not fully compensated. In particular, in the case of under-correction, i.e., incomplete compensation of the tilt of the image region about the horizontal axis, the imaged scene may appear more natural after applying perspective correction than if the tilt were fully corrected.

[0025] In a further development, the horizontal axis intersects the optical axis at the centre of the image area, so that the projection plane also intersects the image area along the horizontal axis at the centre of the image area.

[0026] In a further development, the center of projection of the projective transformation is placed on the optical axis, which allows for perspective-correct compensation for perspective distortions that occur during imaging.

[0027] In a further development, the distance of the projection centers from the image field corresponds to the focal length, normalized to the diagonal of the image field, of the imaging optics of the image recording device, which images the scene onto the image sensor. This allows for complete correction of perspective distortions that occur during imaging. If the image recording device is configured to accommodate various imaging optics (e.g., interchangeable lenses), the evaluation device can be configured to determine the distance of the projection centers based on a given focal length of the imaging optics used during recording. In this regard, the focal length can be provided, for example, by the imaging optics (e.g., electronics of the imaging optics) and can be detected by the evaluation device. If the imaging optics is configured to capture the recorded scene with an adjustable focal length, for example, if the imaging optics is configured as a zoom lens, the evaluation device can be configured to detect the focal length set in each case.

[0028] In a further development, the evaluation device is configured to use only corner points of the image area projected onto the projection surface for determining the image portion. This allows the image portion to be determined particularly quickly and simply. In this further development of the image recording device, in particular, the image portion can be determined without projecting additional pixels of the image sensor, in particular without projecting all pixels of the image sensor onto the projection surface. This allows the image portion to be determined particularly quickly and with little calculation effort.

[0029] In a further development, the evaluation device is configured to determine the image portion based on a predefined aspect ratio. By predefining the aspect ratio, the image portion can then be determined automatically. For example, the predefined aspect ratio corresponds to the aspect ratio of the image area and / or the aspect ratio of the user output interface. For example, the aspect ratio can be permanently stored in the evaluation device or can be predefined by a user of the image recording device via a user input interface.

[0030] In a further development, for example, the predefined aspect ratio is different from the aspect ratio of the image region and / or the aspect ratio of the user output interface, and the evaluation device is configured to receive a user input via the user input interface for specifying the predefined aspect ratio, thereby allowing the image region to be automated in a particularly flexible manner.

[0031] In a further development, the evaluation device is configured to determine the image portion in the projection plane, e.g., as a rectangular portion, such that a first central axis of the image portion extends parallel to the horizontal axis and a second central axis of the image portion, oriented perpendicular to the first central axis, extends parallel to the reference direction, whereby the determined image portion in the projection plane is oriented in a manner that fully compensates for the rotation about the optical axis of the image sensor.

[0032] In a further development, the evaluation device is configured to determine the image portion such that at least two corners of the image portion lie on the margins of the image area mapped to the projection surface by the projective transformation if the predefined aspect ratio is maintained, whereby the image portion can be determined as a particularly large, in particular the largest usable, image portion while maintaining the predefined aspect ratio.

[0033] In a further development, the evaluation device is configured to determine the image portion independently of the position of the image sensor such that the center of the image area projected onto the projection plane by the projective transformation lies on the central axis of the image portion. This facilitates symmetrical imaging of the scene. Furthermore, areas of the imaged scene that lie within the image portion are particularly prone to image errors (e.g., distortions) in the determination of such an image portion, since the image errors introduced by the imaging optics typically increase with increasing distance from the optical axis and thus from the center of the image area.

[0034] In a further development, the evaluation device is configured to use the same algorithm for determining the image portion for both positive and negative tilt angles, and to apply the algorithm only if a positive tilt angle is present or only if a negative tilt angle is present to the projected image area inverted along the reference direction rather than to the image area projected onto the projection surface, so that only a small number of cases need to be distinguished to allow correction of both positive and negative tilt angles.

[0035] For each tilt angle at which the algorithm is applied to the inverted projected image region, i.e., either all positive tilt angles or all negative tilt angles, the relevant points of the image region (e.g., its corner points) can first be mapped to the projection plane and then inverted. After the algorithm is applied, the determined image portion can then be similarly inverted along the reference direction to obtain the position of the image portion in the original, uninverted projected image region.

[0036] In a further development, the evaluation device is configured to determine the corners of the image portion on the projection surface as intersections of the diagonals of the image portion, predefined by the aspect ratio, with the margins of the image area projected onto the projection surface, at least when the tilt angle is equal to zero. Such a graphical determination of the image portion allows for particularly easy determination of the image portion with low computational effort. When the tilt angle is equal to zero, the image portion determined in this way also corresponds to the largest possible image portion while maintaining the aspect ratio. In this regard, starting from the center of the image portion, the diagonals of the image portion can be determined, which center in particular can correspond to the center of the image area transformed onto the projection surface.

[0037] In a further development, the evaluation device is configured to determine the corners of the image portion of the projection plane as intersections of a diagonal of a half of the image portion predefined by the aspect ratio with a margin of the image area projected onto the projection plane, when at least the tilt angle differs from zero by at least a threshold value and the rotation angle is equal to zero, where in the projection plane the diagonal runs through the center of a further margin of the image portion aligned parallel to the horizontal axis. This also allows for a graphical determination of the image portion in a simple manner with low computational effort.

[0038] The center of the further margin of the image portion aligned parallel to the horizontal axis can be located in particular on a central axis of the image portion aligned parallel to the reference direction and extending through the center of the image area transformed into the projection plane. The image portion can be adjacent to the further margin of the image portion aligned parallel to the horizontal axis. The further margin can in particular be the longest margin of the image portion aligned parallel to the horizontal axis.

[0039] The evaluation device may be configured to determine the image portion as the intersection of a diagonal of the image portion predefined by the aspect ratio with the projected margin of the image area only if the tilt angle is equal to zero or less than a threshold value, and to determine the image portion as the intersection of half a diagonal of the image portion predefined by the aspect ratio with the projected margin of the image area only if the tilt angle is not equal to zero or differs from zero by at least a threshold value and the rotation angle is equal to zero or differs from zero by at most one further threshold value.

[0040] In a further development, the evaluation device is configured to display the image portion in an image area within the image portion in a graphical user output interface as a region of the image data set transformed into the projection plane by a projective transformation and by cropping the transformed image data set, e.g. by cropping the transformed image data set by a margin of the user output interface, so that the cropped and perspective-adjusted region of the image scene is directly displayed to the user.

[0041] In a further development, the evaluation device is configured to display the captured scene completely in the image area in a graphical user output interface without applying a projective transformation, whereby the evaluation device is configured to display the image portion by a frame superimposed on the captured scene, so that the captured scene in the image area of ​​the image sensor can be completely displayed to a user of the image recording device, and when preparing the recording the user can in particular also take into account areas of the scene that lie outside the image portion.

[0042] If the scene captured on the image plane is displayed in a graphical user output interface untransformed, i.e., without applying a projective transformation, the image portion determined on the projection plane can be displayed on the user output interface by applying an inverse transformation of the projective transformation. A frame surrounding the image portion can be displayed, in particular, by applying the inverse transformation. For example, the inverse transformation can be used as a further projective transformation to project the image portion from a projection plane having a projection center lying on the optical axis onto an image plane containing the image area.

[0043] Frames may be represented in a user output interface by individual lines. Further visual representation variations may include a grayed-out area surrounding the frame, or a color representation of the area outside the frame that differs from the color representation within the frame. Additionally, frames may generally also be represented by superimposed grid lines that may be aligned parallel and / or perpendicular to the frame's margins.

[0044] Such grid lines may be aligned parallel and / or perpendicular to the margins of the frame, particularly in the projection plane, and both the frame and the grid lines may be transformed by an inverse transformation into the image plane for display when the imaged scene is displayed untransformed in a user output interface.

[0045] In a further development, the evaluation device is configured to display the positions of the measurement points for determining the recording parameters of the image recording device on a user output interface, where the positions of the measurement points are displayed relative to the complete, untransformed scene captured in the image area. This allows, in particular, measurement points located outside the image portion to also be displayed and used to determine the recording parameters. In particular, the image recording device can be configured to take into account the measurement data of the measurement points even if they are located outside the image portion.

[0046] For example, the measurement points may be focus measurement points, exposure measurement points, measurement points for determining the color temperature for white balancing of an image dataset, etc. For example, the measurement points may be set by user input. For example, the user input may be detected by a mechanical control element (e.g., a button, switch, or control stick) of the image recording device. Alternatively or additionally, the user input may be detected by an electronic control element (e.g., a touch panel or touch screen).

[0047] In a further development, the image recording device includes a combined user interface including a user output interface and an overlaid position input interface for defining the position of the measurement point relative to the untransformed scene, wherein the combined user interface is configured to detect the position of the measurement point, for example, as a position in the untransformed scene where activation of the overlaid position input interface is detected. In this way, the position of the measurement point can be determined particularly easily and intuitively. For example, the combined user interface can be a touch screen, wherein the untransformed scene can be particularly displayed on the screen.

[0048] In a further development, the reference direction is in the projection plane. Alternatively, the projection plane may be tilted relative to the reference direction by a residual angle, where the residual angle is different from zero and smaller than the tilt angle. For example, the residual angle may be at most 10% of the tilt angle, e.g., at most 5%, at most 2%, or at most 1%. The residual angle may in particular be 2% of the tilt angle. At the same time, the projection plane may be aligned parallel to the horizontal axis. Since the tilt is only corrected up to a residual angle different from zero, an undercorrection of the tilt occurs, which may result in the corrected image data set appearing more natural than a complete correction, especially in the case of a large tilt.

[0049] In a further development, the image sensor is configured to acquire a series of image data sets representing the scene imaged in the image area at each successive point in time, the position sensor is configured to detect a respective spatial position of the image sensor for each image data set and to provide respective position data, and the evaluation device is further configured to determine a respective image portion for each image data set projected onto a respective projection plane by a projective transformation determined from the respective position data, and to successively display the respective image portions in the graphical user output interface together with at least the area of ​​the scene imaged in the image area that lies within the respective image portion.

[0050] This allows a user of the image recording device to define the scene captured by the image sensor based on the display of the user output interface. In particular, in such a further development, the user output interface can be configured as an electronic viewfinder or a screen arranged on the housing of the image recording device. For example, the image recording device can be configured to acquire each image data set and determine and / or display each image portion at a repetition rate of at least 5 images per second, e.g., at least 5, 10, 20, 40, 50, 60, or 120 images per second. In this regard, the position data can also be provided by the position sensor at a different frequency than the repetition rate, e.g., a lower frequency. In this case, the image portion can be determined and / or displayed multiple times, one after the other, based on the same position data.

[0051] 1. A method of operating an image recording device, comprising: acquiring an image dataset using an image sensor of the image recording device, the image dataset representing an image of a scene located in front of the image sensor in an image area of ​​the image sensor; Detecting the spatial position of the image region relative to a reference direction; - providing position data identifying both a rotation angle of said image field around an optical axis of said image sensor at the time of acquisition of said image data set, in particular relative to a direct projection of said reference direction onto said image field, and a tilt angle of said image field around a horizontal axis at the time of acquisition of said image data set, in particular relative to said reference direction, said horizontal axis being oriented perpendicular to said optical axis and perpendicular to said reference direction; determining a projective transformation from the position data, the projective transformation mapping the image data set onto a projection plane according to both the rotation from the image region and the tilt, the projection plane being tilted relative to the image region about the horizontal axis according to the tilt angle and intersecting the image region along an intersection line rotated in the image region relative to a central axis of the image region according to the rotation angle; determining an image portion in the projection plane for the image data set mapped to the projection plane by the projective transformation; A method is further specified that includes simultaneously displaying the image portion in a graphical user output interface of the image recording device at least an area of ​​the scene captured in the image area within the image portion.

[0052] The method can in particular be performed by the image recording device described above, and in this respect all further developments and technical advantages disclosed in relation to the image recording device also relate to the method described above and vice versa.

[0053] The invention will now be described with reference to the drawings, in which: [Brief explanation of the drawings]

[0054] [Figure 1] The image recording device is shown with a scene positioned in front of the image sensor of the image recording device. [Figure 2] The first, second, third and fourth spatial positions of the image recording device are shown. [Figure 3] The image recorder is shown with a representation of the scene on the user output interface of the image recorder. [Figure 4] 1 shows the recorded scene in projection onto a projection surface. [Figure 5] 3 shows the image portion of the scene determined in the projection plane. [Figure 6] A user output interface is shown with a first representation of an image portion and an area of ​​the scene located within the image portion. [Figure 7] A user output interface is shown along with a second representation of the image portion and the region of the scene located within the image portion. [Figure 8] 1 shows the projective transformation of the image area of ​​the image sensor onto the projection plane. [Figure 9] The image area and projection surface of the image sensor are shown. [Figure 10] 1 illustrates a method for displaying image portions on a user output interface of an image recording device. [Figure 11] The image area projected onto the projection surface is shown with first, second, and third image portions. [Figure 12] 1 illustrates a method for determining a first image portion. [Figure 13] 1 shows the first image portion and the projected image area for rotation and tilt angles different from zero. [Figure 14] 1 shows a first image portion when the tilt angle is equal to zero. [Figure 15] 1 illustrates a method for determining a second image portion. [Figure 16] 2 shows a second image portion and projected image area for rotation and tilt angles different from zero. [Figure 17] A second image portion is shown when the tilt angle is equal to zero. [Figure 18] 10 shows a diagrammatic representation of a method for determining a third image portion. [Figure 19] A further method for determining a third image portion is shown. [Figure 20] A method for determining the intersection points shown in FIG. 18 will be described. [Figure 21] The projected image area is shown along with additional intersection points. [Figure 22] FIG. 21 shows how to define further intersections. [Figure 23] The projected image area is shown along with additional intersection points. [Figure 24] A further method for determining image portions is shown. DETAILED DESCRIPTION OF THE INVENTION

[0055] 1 shows an image recording device 10 configured as a mirrorless system camera, with a scene 1 positioned in front of the image recording device 10. The image recording device 10 includes an optoelectronic image sensor 12 and imaging optics 18. The imaging optics 18 is configured as an interchangeable lens and images the scene 1 (including a building 2 disposed within the scene 1) onto an image area 30 of the image sensor 12 facing the imaging optics 18. In this regard, the optical axis 43 of the image sensor 30 coincides with the optical axis of the imaging optics 18.

[0056] The image recording device 10 further comprises an evaluation device 20 connected to the image sensor 12, a position sensor 14 of the image recording device 10, a first user interface 16 of the image recording device 10 configured as an electronic viewfinder, a second user interface 17 of the image recording device configured as a rear screen, and an electronic memory device 22. In this respect, the individual components of the image recording device 10 are arranged in a common housing 11.

[0057] The optoelectronic image sensor 12 and the position sensor 14 are in particular arranged immovably relative to one another within the housing 11. The position sensor 14 is configured as a three-axis acceleration sensor which determines the spatial position of the position sensor 14 and the image sensor 12 which is connected immovably relative to the position sensor 14 with respect to a reference direction 50 given by the acceleration of gravity 15 and transmits position data representative of the spatial position to the evaluation device 20.

[0058] 1, the image sensor 30 is tilted by a tilt angle 54 about a horizontal axis 121 that is oriented perpendicular to the reference direction 50 and perpendicular to the optical axis 43, so that the optical axis 43 intersects the building 2 at an angle other than a right angle. This causes the optical axis 43 to be offset from a vertical axis 123 that is oriented perpendicular to the horizontal axis 121 and perpendicular to the reference direction 50.

[0059] Furthermore, in the spatial position of the image recording device 10 shown in FIG. 1 , the image area 30 is not rotated about the optical axis 43 relative to the reference direction 50. Thus, the first central axis 41 of the image area 30 coincides with the horizontal axis 121, and the second central axis 42 of the image area 30, oriented perpendicular to the first central axis 41, is disposed in a vertical plane formed by the reference direction 50 and the optical axis 43. In this regard, the first and second central axes 41, 42 are each oriented perpendicular to the optical axis 43. The central axes 41, 42 and the optical axis 43 collectively form a Cartesian sensor coordinate system. Similarly, the horizontal axis 121, the vertical axis 122 extending parallel to the reference direction 50, and the longitudinal axis 123 also form a Cartesian coordinate system.

[0060] 2 shows the image recording device 10 at a first spatial position 115, a second spatial position 116, a third spatial position 117 and a fourth spatial position 118. The orientation of the sensor coordinate system is selected such that in each of the different positions 115, 116, 117, 118 the second central axis 42 subtends a smaller angle with the reference direction 50 than the first central axis 41. Furthermore, the orientation of the sensor coordinate system is selected such that the projection of the reference vector defining the reference direction 50 onto the second central axis 42 results in positive coordinate values.

[0061] As a result, in a first position 115 where the bottom side 111 of the image recording device 10 is oriented toward the reference direction 50, the first central axis 41 is oriented toward the right side 113, and the second central axis is oriented toward the bottom side 111 of the image recording device 10. In a second position 116 where the top side 112 of the image recording device 10 is oriented toward the reference direction 50, the first central axis 41 is oriented toward the left side 114 of the image recording device 10, and the second central axis 42 is oriented toward the top side 112. In a third position 117, the left side 114 of the image recording device 10 is oriented toward the reference direction 50, and as a result, the first central axis 41 is oriented toward the bottom side 111, and the second central axis 42 is oriented toward the left side 114 of the image recording device 10. In the fourth position 118, the right side 113 of the image recording device 10 is oriented toward the reference direction 50, such that the first central axis 41 is oriented toward the upper side 112 and the second central axis 42 is oriented toward the right side 113 of the image recording device 10.

[0062] 1, the image sensor 12 acquires a series of image data sets each representing the scene imaged in the image area 30 at successive times, and simultaneously each of the individual image data sets is displayed at least locally on the user output interfaces 16, 17 to enable a user of the image recording device 10 to select the area of ​​the scene 1 to be recorded.

[0063] Figure 3 shows, by way of example, a screen 17 with such an image dataset 100. The image dataset 100 is displayed untransformed on the screen 17 so that the parallel edges 105 of the building 2 are reproduced as plunging lines due to the tilt of the image sensor 12 about the horizontal axis 121.

[0064] The evaluation device 20 is configured to correct the perspective of the image data set 100 in each case by means of a projective transformation, taking into account the tilt angle 54 and the rotation angle of the image sensor 12 about the optical axis 43, where the rotation angle corresponds to the angle between the horizontal axis 121 and the first central axis 41. In this respect, both the tilt angle 54 and the rotation angle are determined from position data provided by the position sensor 14.

[0065] Figure 4 shows the result of such a projective transformation of the image dataset 100 shown in Figure 3. After applying the projective transformation, the two ends 105 of the building 2 extend parallel to each other and parallel to the vertical axis 122 and therefore parallel to the reference direction 50. As can be seen from Figure 4, the margins of the image dataset 100, which correspond to the margins of the image region 30, have been distorted by the projective transformation, so that the image dataset 100 no longer has rectangular margins.

[0066] In order to correct for distortions of the margins of the image region 30 when applying the projective transformation, the evaluation device 20 is configured to determine a rectangular image portion 60 as shown in Fig. 5. In this regard, the image portion 60 as shown in Fig. 5 is determined entirely within the image dataset 100. In alternative embodiments, the image portion 60 may also include areas outside the image dataset 100. The evaluation device 20 is further configured to display the image portion 60 and at least the area of ​​the scene 1 located within the image portion 60 on the user output interfaces 16, 17.

[0067] 6 shows a first display of image portion 60 and the region of scene 1 located within image portion 60 in second user output interface 17. In this regard, the region within image portion 60 is displayed untransformed. In addition to the region of scene 1 within image portion 60, the remainder of scene 1 captured in image region 30 is also displayed untransformed. Image portion 60 is reproduced as a frame superimposed on the displayed scene 1.

[0068] 6 also shows a first measurement point 91, which is displayed overlaid on the image data set 100 in the user output interface 17 and defines a focus measurement point for the autofocus function of the imaging optics 18. In this regard, the focus measurement point is located outside the image portion 60 such that the autofocus function focuses on an area of ​​the scene 1 that is outside the image portion 60. Furthermore, a second measurement point 92, which defines an exposure measurement point, is also displayed overlaid on the image data set 100. The second measurement point 92 is located within the image portion 60 such that an area of ​​the scene 1 that is within the image portion 60 is exposed.

[0069] The second user output interface 17 is configured as a touchscreen and is part of a combined user interface that includes a touch-sensitive position input interface overlaid on the user output interface 17. The positions of the measurement points 91, 92 can be defined by the position input interface. In this regard, the first measurement point can also be specifically located within the image portion 60 and / or the second measurement point 92 can be located outside the image portion 60.

[0070] 7 shows a second representation of image portion 60 and the area of ​​scene 1 captured by image sensor 12 located within image portion 60. In this regard, the area of ​​scene 1 within image portion 60 has been transformed and reproduced as shown in FIGS. 4 and 5 and is bounded by the margins of user output interface 17. In this regard, image portion 60 is displayed by showing only the area of ​​scene 1 captured by image sensor 12 that is within image portion 60.

[0071] In an alternative embodiment, the region of the scene 1 located within the image portion 60 can also be displayed by applying a projective transformation, and the image portion 60 can be displayed as a frame. In this regard, any region of the imaged scene 1 located outside the image portion 60 can also be transformed and reconstructed, for example as shown in Figure 5. In such a display, it is likewise possible to set measurement points 91, 92 both outside and inside the image portion.

[0072] 8 shows the projective transformation performed by the evaluation device 20 to determine the image portion 60 for a spatial position of the image recording device 10 shown in FIG. 1. In the projective transformation, the image field 30 of the image sensor 12 is projected onto the projection plane 120 with a projection center 125 located on the optical axis 43. In this respect, the projection plane 120 encloses an inclination angle 54 with the image plane 40 defined by the image field 30.

[0073] The projection plane 120 is oriented perpendicular to the vertical plane formed by the optical axis 43 and the reference direction 50, and extends through the center 39 of the image area 30. Furthermore, the reference direction 50 lies in the projection plane 120. Thus, the horizontal axis 121 shown in FIG. 1 and the similarly shown vertical axis 122 also lie in the projection plane 120. The projection plane 120 intersects with the image area 30 along the horizontal axis 121, so that the horizontal axis 121 forms the line of intersection between the projection plane 120 and the image area 30.

[0074] 1, there is no rotation of the image area 30 about the optical axis 43. This means that the horizontal axis 121 and the first central axis 41 of the image area 30 are coincident, and the second central axis 42 of the image area 30 is tilted relative to the vertical axis 122 by the tilt angle 54.

[0075] 9 shows the position of the image area 30 and the projection plane 120 for a rotation angle 52 different from zero. The projection plane 120 intersects the image area 30 along a horizontal axis 121, which is rotated by the rotation angle 52 about the optical axis 43 relative to the first central axis 41 of the image area 30. Furthermore, the central axis 41 of the image area 30 is rotated by the rotation angle 52 about the optical axis 43 relative to the orthogonal projection 51 of the reference direction 50 onto the image area 30 or image plane 40.

[0076] 9, the center of projection 125 has a spacing 126 from the center 39 of the image field 30. The spacing 126 is given by the focal length of the imaging optics 18. The focal length is normalized to the diagonal of the image field 30 and is equal to:

number

[0077] Tilt angle 54 specifies the tilt of image region 30 from projection plane 120 about horizontal axis 121, where tilt angle 54 originates from projection plane 120 and is defined as positive clockwise about horizontal axis 121 and negative counterclockwise about horizontal axis 121. For the tilt shown in FIG. 9, tilt angle 54 is negative.

[0078] The rotation angle 52 specifies the rotation of the normal plane (defined by the second central axis 42 of the image area 30 and the optical axis 43) of the image area 30 from the vertical plane (defined by the reference direction 50 and the optical axis 43) about the optical axis 43, where the rotation angle 52 originates from the vertical plane and is defined as positive for clockwise rotation about the optical axis 43 and negative for counterclockwise rotation about the optical axis 43. For the rotation shown in Figure 9, the rotation angle 52 is positive.

[0079] 10 shows a method 300 performed by the evaluation device 20 for displaying an image portion 60 on the user output interface 16, 17. The method 300 first involves acquiring 305 an image dataset 100 by the image sensor 12. Thereafter, the method 300 involves detecting 315 the spatial position of the image sensor 12 by the position sensor 14 and then providing position data by the position sensor 14. The provided position data is then transmitted to the evaluation device 12.

[0080] The evaluation unit 12 then determines 320 a projective transformation that transforms the image region 30 onto the projection plane 120 shown in Figure 8. The evaluation unit 12 determines, among other things, the spatial position of the projection plane 120 based on the tilt angle 54 and the rotation angle 52.

[0081] In the following description, a sensor coordinate system is used in which the X axis corresponds to the first central axis 41, the Y axis corresponds to the second central axis 42, and the Z axis corresponds to the optical axis 43. Furthermore, a reference coordinate system is used in which the X axis corresponds to the horizontal axis 121, the Y axis corresponds to the vertical axis 122, and the Z axis corresponds to the longitudinal axis 123. Furthermore, homogeneous coordinates are used.

[0082] Then, in the sensor coordinate system, a pixel in the image region 30 is given in homogeneous coordinates by the following equation:

number

[0083] In particular, the corner points of the image region 30 are given by:

number

[0084] In the method 300 shown in FIG. 10, these corner points are first transformed by the matrix transformation

number

number

number

[0085] The first rotation matrix represents the rotation of the image region 30 about the optical axis 43, where θ corresponds to the rotation angle 52. The second rotation matrix represents the tilt of the image region 30 about the horizontal axis 121, where φ corresponds to the tilt angle 54.

[0086] 8 is then performed, which is given by the XY plane of the reference coordinate system, taking into account both the tilt about tilt angle 54 and the rotation about rotation angle 52 with respect to this point. The projective transformation of each coordinate system onto the XY plane can be expressed in matrix notation as follows:

number

number

number

[0087] After applying the projection matrix, the projective transformation P xy-plane The coordinates determined by still have to be normalized by the fourth vector component according to the formula:

number

number

[0088] Thereafter, a determination 340 of the image portion 60 is made in the projection plane 120 by defining the four corners of the image portion 60 .

[0089] These corners of the image portion 60 are then transformed by the transformation M 3d image→3d world The inverse transformation of

number

[0090] When the image portion 60 is to be displayed together with the untransformed scene 1 imaged in the image region 30, a projective transformation 350 of the corner points is then performed in the sensor coordinate system by the center of projection 125 onto the image plane 40 (corresponding to the XY plane of the sensor coordinate system). The corner points of the image portion 60 thus transformed are then displayed 390 together with the untransformed scene 1 imaged in the image region 30.

[0091] 11 shows image region 30 projected onto projection plane 120 with first corner point 35 in the third quadrant of the reference coordinate system, second corner point 36 in the second quadrant of the reference coordinate system, third corner point 37 in the first quadrant of the reference coordinate system, and fourth corner point 38 in the fourth quadrant of the reference coordinate system. In this regard, projected image region 30 is shown as having a positive tilt angle 54 and a positive rotation angle 52.

[0092] In addition to image area 30, Figure 11 shows first image portion 81, second image portion 82, and third image portion 83, where each image portion 81, 82, 83 is entirely within image area 30 and forms an image portion that can be determined as image portion 60 by evaluation device 20. Each image portion 81, 82, 83 is rectangular and has a respective first margin 61, a respective second margin 62, a respective third margin 63, and a respective fourth margin 64. Each first margin 61 and each third margin 63 extend parallel to a horizontal axis 121, and each second and fourth margin 62, 64 extend parallel to a vertical axis 122. Furthermore, image portions 81, 82, 83 have a respective first central axis 75 aligned parallel to horizontal axis 121 and a respective second central axis 76 aligned parallel to vertical axis 122.

[0093] Each of the image portions 81, 82, and 83 is determined while maintaining a predetermined aspect ratio. Furthermore, each of the image portions 81, 82, and 83 is determined such that their respective second central axes 76 extend through the center 39 of the image area 30 projected onto the projection surface 120.

[0094] Additionally, the first image portion 81 is determined so that the center 85 of the first image portion 81 corresponds to the center 39 of the image area 30. The second image portion 82 is determined so that one of the margins 61, 63 of the second image portion 82, which extend parallel to the horizontal axis 121, is the same distance from the horizontal axis 121 as one of the corner points 35, 36, 37, 38 of the image area 30. In the case of a positive tilt angle 54, this is the corner point located closest to the horizontal axis 121 and having a negative Y coordinate, i.e., the second corner point 36 in the projection shown in FIG. 11 . In the case of a negative tilt angle 54, this is the corner point located closest to the horizontal axis 121 and having a positive Y coordinate. Finally, the third image portion 83 is determined so that at least two of its corners lie on the margins of the image area 30 while maximizing the area of ​​the third image portion 83.

[0095] 12 and 13 illustrate a method 400 performed during image portion 60 determination 340 when image portion 60 is determined as first image portion 81. In this regard, starting from center 39 of image area 30, a first diagonal 77 and a second diagonal 78 of first image portion 81 are determined (405). First diagonal 77 has a negative slope in the XY plane of the reference coordinate system, the magnitude of the negative slope corresponding to the reciprocal of a predefined aspect ratio for determining image portion 60. Second diagonal 78 has a positive slope in the XY plane, the magnitude of the positive slope corresponding to the reciprocal of a predefined aspect ratio. In this regard, the predefined aspect ratio is always defined as the extent of image portion 60 along horizontal axis 121 relative to the extent of image portion 60 along vertical axis 122.

[0096] Thereafter, calculation 410 is performed of eight intersections 46 between the diagonal lines 77, 78 and a line extending through the first and second corner points 35, 36 and the first margin 31 of the image area 30, a line extending through the second and third corner points 36, 37 and the second margin 32 of the image area 30, a line extending through the third and fourth corner points 37, 38 and the third margin 33 of the image area 30, and a line extending through the fourth and first corner points 38, 35 and the fourth margin 34 of the image area 30. In Figure 13, the intersection of the second diagonal line 78 and the line extending through the first margin 31 is not shown.

[0097] The method 400 then includes defining 415 the first image portion 81, where an intersection point 46 is determined that has a minimum spacing 47 from the center 39 of the image area 30. Thereafter, during the definition 415 of the first image portion 81, the corners of the first image portion 81 are determined such that they lie on the diagonals 77, 78 and their respective spacings from the center 39 of the image area 30 correspond to the minimum spacing 47.

[0098] 14 , when tilt angle 54 is zero, first image portion 81 determined by method 400 corresponds to third image portion 83. This means that when tilt angle 54 is zero, first image portion 81 is also determined by method 400 such that second central axis 76 of first image portion 81 extends through center 39 of image area 30 and such that first image portion 81 has a maximum area within image area 30 if the predefined aspect ratio is maintained. In this regard, two corners of first image portion 81 lie on margins 31, 32, 33, 34 of image area 30.

[0099] 15 and 16 illustrate a method 500 for determining the second image portion 82. In this regard, a check 505 is first made for the presence of a negative tilt angle 54. If a negative tilt angle 54 is present, a flip 510 of the image region 30 projected onto the projection plane 120 along the vertical axis 122 is first performed. This swaps the first corner point 35 with the fourth corner point 38, and the second corner point 36 with the third corner point 37, thereby ensuring that the margins 31, 33 of the image region 30 extending through the second and third quadrants of the reference coordinate system have a greater extent along the central axis 121 than the margins 31, 33 of the image region 30 extending through the first and fourth quadrants.

[0100] The center 74 of the first margin 61 of the second image portion 82 aligned parallel to the horizontal axis 121 is then determined (515) so as to lie along the vertical axis 122 at the same height as the corner points 35, 36 of the image area 30 in the second and third quadrants of the reference coordinate system located closest to the central axis 121. In the spatial position of the image sensor 12 shown in FIG. 16, this is the second corner point 36.

[0101] The method 500 then includes determining 520 the intersection points 46 of the first diagonal 72 with a line extending through the second, third, and fourth margins 32, 33, 34 of the image region 30, and the intersection points 46 of the second diagonal 73 with a line extending through the second, third, and fourth margins 32, 33, 34 of the image region 30. In alternative embodiments, only the intersection points 46 of the first diagonal 72 with a line extending through the third and fourth margins 33, 34 of the image region 30 and / or only the intersection points 46 of the second diagonal 73 with a line extending through the second and third margins 32, 33 of the image region 30 may be determined.

[0102] Each diagonal 72, 73 is a diagonal of half 70 of second image portion 82 separated by a second central axis 76 of second image portion 82, with each diagonal 72, 73 extending through a respective center 74. First diagonal 72 has a negative slope and second diagonal 73 has a positive slope, where the magnitude of each slope corresponds to twice the reciprocal of a predefined aspect ratio.

[0103] Thereafter, definition 525 of second image portion 82 is performed by determining the intersection point 46 having the minimum spacing 47 from center 74 and calculating the positions of the corners of second image portion 82 from the minimum spacing 47 and the predefined aspect ratio. Thereafter, if a negative tilt angle 54 is determined, a flip 590 of defined image portion 82 along vertical axis 122 is performed to compensate for the flip 510 of projected image area 30 performed at the start of method 500.

[0104] Thus, in method 600, the same algorithm for determining second image portion 82 is used for both positive and negative tilt angles 54, but the algorithm includes method steps 515, 520, 525 that occur between the first and last flips 510, 590. In an alternative embodiment, flips 510, 590 can also occur when a positive tilt angle 54 is determined, and method steps 515, 520, 525 are adapted so that center 74 is on third margin 63 of second image portion 82, which extends through the first and fourth quadrants of the reference coordinate system.

[0105] 17 , when rotation angle 52 is zero, second image portion 82 determined by method 500 corresponds to third image portion 83. This means that when rotation angle 52 is zero, second image portion 82 determined by method 500, like third image portion 83, is determined such that its second central axis 76 extends through center 39 of image area 30 and has the largest area within image area 30 when the predefined aspect ratio is maintained. In this regard, two corners of second image portion 82 lie on margins 31, 32, 33, and 34 of image area 30. However, unlike first image portion 81, center 85 of second image portion 82 does not correspond to center 39 of image area 30.

[0106] 18 shows a graphical representation of a method for determining the third image portion 83. The method includes first calculating an intersection point A between the first margin 31 of the image area 30 projected onto the projection surface 120 and the vertical axis 122, and calculating an intersection point E between the third margin 33 of the projected image area 30 and the vertical axis 122. The evaluation range for determining the third image portion 83 is then narrowed to the Y coordinate values ​​between the intersection points A and E.

[0107] Then, the margin function f 上 (y), f 右 (y), f 下 (y), and f 左 (y) is defined, where f 上 (y) specifies the dependence of the X coordinate of the first margin 31 of the image area 30 on the Y coordinate, and f 右 (y) specifies the dependence of the X coordinate of the second margin 32, and f 下 (y) specifies the dependence of the X coordinate of the third margin 33, and f 左 (y) specifies the dependence of the X coordinate of the fourth margin 34. For further calculations, the function f 上 (y), f 右 (y), f 下 (y), and f 左We use the absolute value of (y), which corresponds to mirroring the portions of the margins 31, 32, 33, 34 of the image area 30 extending in the third and fourth quadrants of the reference coordinate system about the vertical axis 122, and are expressed as a partly defined function h(y)=min(|f 上 (y)|,|f 右 (y)|,|f 下 (y)|,|f 左 (y)|) is used.

[0108] Then, in the interval ]0,x max ] along the X axis (where x max =max(h(y))), the X coordinate value x of a line extending parallel to the vertical axis 122 is determined by applying the following:

number

[0109] 19 shows a further method 600 for determining the third image portion 83. The method 600 first determines whether a negative tilt angle 54 exists (605). If so, first, the flip 510 of the projected image region 30 along the vertical axis 122 described in connection with FIG. 15 is performed. Intersection points A and E are then determined (615) as described above. If a negative tilt angle 54 does not exist, the method 600 proceeds directly to determining intersection points A and E (615).

[0110] Further intersection points B, C, and D shown in Figure 18 are then determined (620). The method shown in Figure 20 is used for this purpose. In this regard, it is first determined whether the Y coordinate of the first corner point 35 of the image area 30 is less than the Y coordinate of the second corner point 36 (626). If so, intersection point B is determined (628) as the intersection point between the first margin 31 and the fourth margin 34 of the projected image area 30 mirrored about the vertical axis 122, as shown in Figure 18. If not, intersection point B is determined (629) as the intersection point between the first margin 31 and the second margin 32 of the projected image area 30 mirrored about the vertical axis 122.

[0111] The method 620 then includes determining (630) intersection point C as the intersection of the second margin 32 or the fourth margin 34 mirrored about the vertical axis 122 with the horizontal axis 121. It is then determined (632) whether the Y coordinate of the fourth corner point 38 of the projected image area 30 is less than the Y coordinate of the third corner point 37 of the projected image area 30. If so, intersection point D is determined (634) as the intersection of the third margin 33 mirrored about the vertical axis 122 with the second margin 32 of the projected image area 30, as shown in FIG. 18 . Otherwise, intersection point D is determined (635) as the intersection of the third margin 33 and the fourth margin 34 of the projected image area 30 mirrored about the vertical axis 122.

[0112] A further method 600 for determining the third image portion 83 shown in FIG. 19 then includes defining 650 a first intersection point P1 shown in FIG. 21 and a second intersection point P2 also shown in FIG. 21, which define the boundaries of line segments of margins 31, 32, 33, 34 of the projected image area 30 that intersect with the third corner 67 of the determined third image portion 83.

[0113] Specification 650 is shown in Figure 22 and includes first determining 652 an intersection point F (shown in Figure 21) between first margin 31 of projected image area 30 and a line extending parallel to vertical axis 122 through intersection point C. Specification 650 then includes defining 654 an intersection point G (also shown in Figure 21) between first margin 31 and a line extending parallel to vertical axis 122 through intersection point D.

[0114] The connecting line CF is then compared with a predefined aspect ratio (656). If the ratio of twice the X coordinate value of the intersection point F to the length of the connecting line CF is less than or equal to the predefined aspect ratio, the first intersection point P1 is defined as intersection point B, and the second intersection point P2 is defined as intersection point C (657). Otherwise, the connecting line DG is compared with a predefined aspect ratio (658). If the ratio of twice the X coordinate value of the intersection point G to the length of the connecting line DG is greater than the predefined aspect ratio, the first intersection point P1 is defined as intersection point C, and the second intersection point P2 is defined as intersection point D (660) (shown in FIG. 21). Otherwise, the first intersection point P1 is defined as intersection point D, and the second intersection point P2 is defined as intersection point E (661).

[0115] Thereafter, as shown in Figure 23, an intersection point K between the line extending through intersection points A and B and the line extending through intersection points P1 and P2, and an intersection point L between the line extending through intersection points P1 and P2 and vertical axis 122 are determined (662). In this regard, the relative length ratios of the representation in Figure 23 differ from the length ratios shown in Figures 18 and 21.

[0116] Defining 650 then includes determining 664 the dimensions of the triangle defined by intersection points A, K, and L, where lengths a, b, and c shown in FIG. 23 are determined from the coordinates of intersection points A, K, and L. In this regard, length a corresponds to the difference in the Y coordinate values ​​of points A and K, length b corresponds to the difference in the Y coordinate values ​​of points K and L, and length c corresponds to the spacing of point K from vertical axis 122, which forms the Y axis.

[0117] Then, determine the angles γ and δ as follows:

number

number

[0118] The method 600 then includes determining 666 the location and dimensions of the third image portion 83. In this regard, first, the X coordinate value g of the second margin 62 of the third image portion 83 is calculated. The predefined aspect ratio is expressed as:

number

number

[0119] Then, the width w of the third image portion 83 along the horizontal axis 121 is w = 2 g, and the height h along the vertical axis 122 is h = d + e. Furthermore, the Y coordinate of the center 85 of the third image portion 83 is K.y + (ed) / 2, where K is the Y coordinate of the intersection point K, while the X coordinate of the center 85 of the third image portion 83 is equal to zero.

[0120] 19, the corners 65, 66, 67, 68 of the third image portion 83 shown in FIG. 18 are then calculated (685) from the position and dimensions of the third image portion 83. Finally, if a negative tilt angle 54 is present, the determined positions of the corners 65, 66, 67, 68 of the image portion 83 are flipped (590) along the vertical axis 122 to compensate for the flipping 510 of the corner points 35, 36, 37, 38 of the image region 30 that occurred at the start of the method 600.

[0121] Thus, in method 600, the same algorithm for determining the third image portion 83 is similarly used for both positive and negative tilt angles 54, but the algorithm includes method steps 615, 620, 650, 685 that occur between the first and last reversals 510, 590. In an alternative embodiment, the reversals 510, 590 can also occur when a positive tilt angle 54 is determined, and method steps 615, 620, 650, 685 must be adapted accordingly.

[0122] 24 illustrates a further method 700 by which the determination 340 of the image portion 60 may be performed. In this regard, similar to method 600, the image portion 60 is determined such that the image portion 60 has a predefined aspect ratio, its second central axis 76 extends through the projected center 39 of the image area 30, and the image portion 60 has a maximum area within the image area 30 projected onto the projection surface 120. In method 700, the image portion 60 is also particularly determined such that at least two of its corners 65, 66, 67, 68 lie on the margins 31, 32, 33, 34 of the projected image area 30.

[0123] The further method 700 first involves determining 705 the tilt angle 54. If the tilt angle 54 deviates from zero by at most a threshold value, in particular if the tilt angle 54 is equal to zero, the image portion 60 is determined by the method 400 described in connection with FIGS. 12-14. This allows the image portion 60 to be determined with particularly less effort than other methods 500, 600 in which the image portion 60 is similarly determined as the largest image portion having a predefined aspect ratio and a second central axis 76 extending through the center 39 of the image area 30 (see FIG. 14). Alternatively, if a negative tilt angle 54 is present, a flip 510 of the projected image area 30 along the vertical axis 122 is performed. For a positive tilt angle 54, the flip 510 is omitted.

[0124] A check 710 of the rotation angle 52 is then performed. If the rotation angle 52 differs from zero by at most a further threshold value, in particular if the rotation angle 52 is equal to zero, the image portion 60 is calculated according to method steps 515, 520, 525 of the method 500 described in relation to Figures 15 to 17. The calculation effort for determining the image portion 60 is likewise less than in method 600. As shown in Figure 17, in this case the determined image portion 60 also corresponds to the third image portion 83 of the largest area that can be determined according to method 600.

[0125] If checking 710 the rotation angle 52 reveals that the rotation angle 52 deviates from zero by more than a further threshold, the image portion 60 is calculated according to method steps 620, 650, 685 of the method 600. Finally, if the tilt angle 54 deviates from zero by more than a threshold and a negative tilt angle 52 is present, an inversion 590 of the determined image portion 60 is performed.

[0126] In alternative embodiments of the image recording device 10, the image portion 60 may be determined semi-automatically based on a center 85 of the image portion 60, which may be predefined by a user of the image recording device 10. Alternatively or additionally, the predetermined aspect ratio may also be predefined by user input. Furthermore, in alternative embodiments of the image recording device 10, the image portion 60 may be determined such that the image portion 60 includes areas located outside the projected image area 30 on the projection surface 120. In these cases, the evaluation device 20 may be configured to fill the areas located outside the projected image area 30 with calculated image information, such as interpolated image information.

[0127] Furthermore, the evaluation device 20 is configured to store reference data for perspective correction of the recorded image dataset 100 in the memory device 22. In this regard, the reference data can be stored separately from the image dataset 100 in a different file or together with the image dataset 100 in a common file, e.g., as metadata. For example, the reference data can include the tilt angle 54 and / or the rotation angle 52 and / or the focal length of the imaging optics 18, e.g., a normalized focal length (e.g., a focal length normalized to a 35 mm format). Alternatively or additionally, the reference data can include image portion data defining the position and size of the image portion 60.

[0128] Such image portion data may include, among other things, the positions of the corners 65, 66, 67, 68 of the image portion 60, and / or a predefined aspect ratio, and / or a relative size of the corrected image with respect to the image dataset 100. In this regard, for example, the positions of the corners 65, 66, 67, 68 may be stored in normalized form as a numerical value between 0 and 1, the predefined aspect ratio may be stored as a ratio of width to height, and the relative size may be stored as a ratio of the height of the image dataset 100 to the height of the corrected image. Furthermore, the reference data may include information specifying whether the stored image dataset 100 has already been corrected by a projective transformation. [Explanation of symbols]

[0129] Scene 1 2. Building 10 Image recording device 11. Cabinet 12 Image Sensor 14 Position Sensor 15 Gravitational acceleration 16 Primary User Output Interface 17 Secondary User Output Interface 18 Imaging optical system 20 Evaluation equipment 22 Memory Device 30 Image Area 31 First Margin 32 Second Margin 33 Third Margin 34 Fourth Margin 35 First corner point 36 Second corner point 37 Third corner point 38 Fourth Corner Point 39 center 40 image planes 41 First Central Axis 42 Second Central Axis 43 Optical axis 46 intersection 47 interval 50 Reference direction 51 Projection 52 rotation angle 54 Tilt angle 60 Image section 61 First Margin 62 Second Margin 63 Third Margin 64 Fourth Margin 65 First Corner 66 Second Corner 67 Third Corner 68 Fourth Corner 69 Diagonal 70 half 72 First Diagonal 73 Second Diagonal 74 center 75 First Central Axis 76 Second Central Axis 77 First Diagonal 78 Second Diagonal 81 First Image Part 82 Second Image Part 83 Third Image Part 85 center 86 First Intersection 87 Second Intersection 91 First measurement point 92 Second measurement point 100 image dataset 105 edge 111 upper side 112 Lower side 113 Right side 114 Left side 115 First Position 116 Second Position 117 Third Position 118 Fourth Position 120 Projection plane 121 horizontal axis 122 Vertical axis 123 Vertical Axis 125 Projection center 126 interval 127 Vertical plane 130 mirrored image areas 300 ways Obtained 305 image datasets 310 Detect spatial position 315 Provide location data 320 Determine the projective transformation 325 Transforming image corner coordinates to reference coordinates 330 Projecting image corners onto a projection surface 340 Determine image part 345 Back transform of image part corners to sensor coordinates 350 Backprojection of partial corners onto the image plane 390 Image Datasets and Image Portions 400 Method for determining a first image portion 405 Determine the diagonal 410 Calculating Intersections 415 Defines the first image part 500 How to determine the second image portion 505 Check for negative tilt angle 510 Invert projected image area 515 Determine the center 520 Determine the intersection point 525 Define the second image part 590 Invert image part 600 Method for determining a third image portion 605 Check for negative tilt angle 615 Determine the center 620 Determine intersections B, C, and D 626 Compare the Y coordinates of the first and second corner points 628 Determine intersection point B 629 Determine intersection point B 630 Determine intersection point C 632 Compare the Y coordinates of the third and fourth corner points 634 Determine intersection point D 635 Determine intersection point D 650 Define intersection points P1 and P2 652 Determine intersection point F 654 Determine intersection point G 656 Compare the connecting line CF with a predefined aspect ratio 657 Define intersection P1 as intersection B and intersection P2 as intersection C. 658 Compare the connecting line DG with a predefined aspect ratio 660 Define intersection P1 as intersection C and intersection P2 as intersection D. 661 Define intersection P1 as intersection D and intersection P2 as intersection E. 662 Determine the intersection points K and L 664 Determine the dimensions of a triangle 666 Determine the location and dimensions of the third image portion 685 Calculating corners of image parts 700 How to determine image parts 705 Check the inclination angle 710 Check the rotation angle

Claims

1. An image recording device (10) comprising an optoelectronic image sensor (12), a position sensor (14), a graphic user output interface (16, 17), and an evaluation device (20), the optoelectronic image sensor (12) is configured to acquire an image data set (100) representing an image of a scene (1) located in front of the optoelectronic image sensor (12) in an image area (30) of the optoelectronic image sensor (12); the position sensor (14) is configured to detect the spatial position of the image region (30) relative to a reference direction (50) and to provide position data identifying both a rotation angle (52) through which the image region (30) was rotated about an optical axis (43) of the optoelectronic image sensor (12) when the image data set (100) was acquired and a tilt angle (54) through which the image region (30) was tilted about a horizontal axis (121) when the image data set (100) was acquired; the horizontal axis (121) is oriented perpendicular to the optical axis (43) and perpendicular to the reference direction (50); the evaluation device (20) is configured to determine from the position data a projective transformation that maps the image data set (100) onto a projection plane (120) according to both the rotation and the tilt from the image region (30); the projection plane (120) is inclined with respect to the image area (30) according to the inclination angle (54), and intersects with the image area (30) along an intersection line rotated in the image area (30) with respect to a central axis (41, 42) of the image area (30) according to the rotation angle (52); the evaluation device (20) is configured to determine image portions (60, 81, 82, 83) in the projection plane (120) for the image data set (100) mapped onto the projection plane (120) by the projective transformation; The evaluation device (20) is configured to display the image portion (60, 81, 82, 83) in the graphic user output interface (16, 17) simultaneously with at least the area of ​​the scene (1) captured in the image area (30) that is within the image portion (60, 81, 82, 83).

2. 2. The image recording device (10) of claim 1, wherein the evaluation device (20) is configured to determine the image portions (60, 81, 82, 83) as, for example, rectangular portions, such that on the projection plane (120), a first central axis (75) of the image portions (60, 81, 82, 83) extends parallel to the horizontal axis (121) and a second central axis (76) of the image portions (60, 81, 82, 83), oriented perpendicular to the first central axis (75), extends parallel to the reference direction (50).

3. 3. The image recording device (10) of claim 1 or 2, wherein the evaluation device (20) is configured to determine the image portions (60, 81, 82, 83) independently of the position of the optoelectronic image sensor (12) so that the center (39) of the image area (30) projected onto the projection plane (120) by the projective transformation lies on the central axis (76) of the image portions (60, 81, 82, 83).

4. the projection center (125) of said projective transformation is located on said optical axis (43); An image recording device (10) as claimed in any one of claims 1 to 3, wherein the distance (126) of the projection centre (125) from the image area (30) corresponds, for example, to a focal length normalised to a diagonal of the image area (30) of an imaging optics (18) of the image recording device (10) that images the scene (1) onto the optoelectronic image sensor (12).

5. The image recording device (10) according to any one of claims 1 to 4, wherein the evaluation device (20) is configured to use only corner points (35, 36, 37, 38) of the image area (30) projected onto the projection surface (120) for determining the image portions (60, 81, 82, 83).

6. the evaluation device (20) is configured to determine the image portions (60, 81, 82, 83) based on a predefined aspect ratio; 6. The image recording device (10) of claim 1, wherein the predefined aspect ratio is different from, for example, an aspect ratio of the optoelectronic image sensor (12) and / or the graphical user output interface (16, 17), and the evaluation device (20) is configured to receive a user input for specifying the predefined aspect ratio, for example via a user input interface (17).

7. 7. The image recording device (10) of claim 6, wherein the evaluation device (20) is configured to determine the image portions (60, 81, 82, 83) such that at least two corners (65, 66, 67, 68) of the image portions (60, 81, 82, 83) are on the margins (31, 32, 33, 34) of the image area (30) mapped to the projection plane (120) by the projective transformation, when a predetermined aspect ratio is maintained.

8. 7. The image recording device (10) of claim 6, wherein the evaluation device (20) is configured to determine corners (65, 66, 67, 68) of the image portion (60, 81, 82, 83) on the projection surface (120) as intersections of diagonals (69) of the image portion (60, 81, 82, 83) predetermined by the aspect ratio with margins (31, 32, 33, 34) of the image area (30) projected onto the projection surface (120), at least when the tilt angle (54) is equal to zero.

9. the evaluation device (20) is configured to determine corners (65, 66, 67, 68) of the image portion (60, 81, 82, 83) in the projection plane (120) as intersections of diagonals (72, 73) of halves (70) of the image portion (60, 81, 82, 83) predetermined by the aspect ratio with margins (31, 32, 33, 34) of the image area (30) projected onto the projection plane (120), if at least the tilt angle (54) differs from zero by at least a threshold value and the rotation angle (52) is equal to zero, 7. The image recording device (10) of claim 6, wherein in the projection plane (120), the diagonal lines (72, 73) extend through the centers (74) of further margins (61, 62, 63, 64) of the image portions (60, 81, 82, 83) aligned parallel to the horizontal axis (121).

10. the evaluation device (20) is configured to display in the graphical user output interface (16, 17) the region of the scene captured in the image region (30) within the image portion as a region of the image data set (100) transformed onto the projection plane (120) by the projective transformation, 10. The image recording device (10) according to claim 1, wherein the evaluation device (20) is configured to display the image portion (60, 81, 82, 83) by cropping the transformed image data set (100), for example by cropping the transformed image data set (100) by a margin of the graphical user output interface (16, 17).

11. the evaluation device (20) is configured to display the captured scene (1) entirely in the image area (30) without the use of the projective transformation in the graphic user output interface (16, 17); 10. The image recording device (10) according to any one of claims 1 to 9, wherein the evaluation device (20) is configured to display the image portions (60, 81, 82, 83) by means of frames superimposed on the captured scene (1).

12. the evaluation device (20) is configured to display in the graphic user output interface (16, 17) the positions of measurement points (91, 92) for determining recording parameters of the image recording device (10), the positions of said measurement points (91, 92) are displayed relative to the complete, untransformed scene (1) imaged in said image field (30); The image recording device (10) of claim 11 further comprises a combined user interface (17) including, for example, the graphic user output interface and an overlaid position input interface for defining the positions of the measurement points (91, 92) relative to the unconverted scene (1), the combined user interface (17) being configured to detect the positions of the measurement points (91, 92) as positions within the unconverted scene (1) where activation of the overlaid position input interface is detected.

13. said reference direction (50) is in said projection plane (120), or The image recording device (10) of any one of claims 1 to 12, wherein the projection plane (120) is inclined with respect to the reference direction (50) by a residual angle, the residual angle being different from zero and smaller than the inclination angle (54).

14. the optoelectronic image sensor (12) is configured to acquire a series of image data sets representing the scene (1) imaged in the image field (30) at successive times; the position sensor (14) is configured to detect a respective spatial position of the optoelectronic image sensor (12) for each image data set (100) and provide respective position data; the evaluation device (20) is configured to determine a respective image portion (60, 81, 82, 83) for each image data set (100) projected onto a respective projection plane (120) by a projective transformation determined from the respective position data, 14. The image recording device (10) according to claim 1, wherein the evaluation device (20) is configured to display the respective image portions (60, 81, 82, 83) in succession in the graphic user output interface (16, 17) together with at least the area of ​​the scene (1) captured in the image area (30) within the respective image portion (60, 81, 82, 83).

15. A method (300) of operating an image recording device (10), comprising: acquiring (305) an image dataset (100) using an optoelectronic image sensor of the image recording device, the image data set (100) represents an image of a scene (1) located in front of the optoelectronic image sensor (12) in an image area (30) of the optoelectronic image sensor (12); Detecting (310) the spatial position of said image region (30) relative to a reference direction (50); providing (315) position data identifying both a rotation angle (52) through which the image region (30) was rotated about an optical axis (43) of the optoelectronic image sensor (12) during acquisition (305) of the image data set (100) and a tilt angle (54) through which the image region (30) was tilted about a horizontal axis (121) during acquisition of the image data set (100), the horizontal axis (121) is oriented perpendicular to the optical axis (43) and perpendicular to the reference direction (50); determining (320) a projective transformation from the position data, the projective transformation maps the image data set (100) onto a projection plane (120) according to both the rotation and the tilt from the image domain (30); the projection plane (120) is inclined relative to the image area (30) around the horizontal axis (121) in accordance with the inclination angle (54), and intersects with the image area (30) along an intersection line rotated in the image area (30) relative to a central axis (41, 42) of the image area (30) in accordance with the rotation angle (52); determining (340) image portions (60, 81, 82, 83) in the projection plane (120) for the image data set (100) mapped to the projection plane (120) by the projective transformation; and displaying (390) the image portion (60, 81, 82, 83) and at least the area of ​​the scene (1) captured in the image area (30) that is within the image portion (60, 81, 82, 83) in a graphical user output interface (16, 17) of the image recording device (10).

Images