METHOD AND MEASURING DEVICE FOR CORRECTING THE POSITION OF A MEASURING POINT

Description

[0001] The present invention relates to a method for correcting the position of a measurement point in a measurement image, in particular in an endoscopic and / or exoscopic and / or microscopic and / or laryngoscopic measurement image, according to the preamble of independent claim 1. The invention further relates to a method for outputting at least one statistical parameter for a moving measurement object, which is included in particular in an endoscopic and / or exoscopic and / or microscopic and / or laryngoscopic measurement image, according to the preamble of claim 6. The invention further relates to a measuring device with a processor configured to perform at least one of the methods according to the invention.

[0002] Optical visualization systems, such as microscopes, exoscopes, and endoscopes, allow for the display of a scene and / or work area where fine motor tasks and / or visual inspections are performed. In medical procedures, the work area is, for example, an operating field in an internal area of ​​the human body, such as within the thorax or head.

[0003] Exoscopy describes the observation and, if necessary, illumination of an operating field on a patient and / or an object field on any object, starting from a point away from, i.e., outside, the patient's body or away from the object.

[0004] Endoscopy is an imaging technique in which an endoscope is inserted into a cavity. The medical professional performing the procedure views the image captured by the endoscope on a monitor and uses this image to guide their actions. In medical procedures, such as minimally invasive ones, an endoscope is inserted into the body to obtain an internal image and display it on a monitor. Due to the often precise work required by medical professionals, it is desirable to have the most accurate, high-resolution image possible of the cavity and / or the area of ​​operation being performed.

[0005] The images captured by the endoscope and displayed on a screen are usually two-dimensional, so that even specialists are unable to accurately determine the dimensions and / or measurements of an object in the scene depicted due to the lack of depth information.

[0006] To solve this problem and / or to enable depth measurements within a single image, three-dimensional stereo endoscopes, stereo exoscopes, and / or microscopes are also known, in which a scene is captured from two different viewpoints by two image acquisition units. These two units capture an image of the scene being viewed, preferably synchronized for each unit of time and viewpoint. This results in so-called stereo image pairs with a common timestamp. Depth information can then be evaluated from such a stereo image pair using stereo reconstruction techniques.

[0007] During this stereo reconstruction, a disparity, i.e., a horizontal pixel offset, in the respective stereo image pairs is determined pixel by pixel by an algorithm. Objects at a great distance exhibit a low disparity, i.e., a small pixel offset, within a given image pair during stereo capture. Conversely, objects in the foreground exhibit a large disparity.

[0008] From the disparity, and with knowledge of further optical parameters of the image acquisition unit(s), a kind of depth map can be calculated for the stereo-reconstructed stereo image, which, for example, contains pixel-by-pixel depth information of the object being viewed. Using this depth information, it is possible, for instance, to determine the distance between any two pixels in Euclidean space.

[0009] Methods and devices are already known in the art that enable dimensional measurement between two measuring points in Euclidean space, thus incorporating depth information of the respective measuring points on a measurement object. In this process, a user selects two measuring points on a measurement object in a stereo image, for example via manual input, and calculates the distance between them. This allows the user to select points within the stereo image and subsequently receive the distance between these points, for example as a superimposed representation in the displayed stereo image.

[0010] From WO2021 / 138262 A1 and EP1965699 B1, a medical robotic system is known in which, generally speaking, superimposed information is displayed in measurement images using "telestration." Telestration describes the annotation and marking of images in the operating room by a mentor or another person who is monitoring the operation externally, for example, via live video. It is possible to display telestration graphics in a captured stereo image and, for example, assign them to a specific measurement object within that stereo image.

[0011] Furthermore, WO 2019 / 213432 A1 specifies that a user can select measurement points on a measurement object within a two-dimensionally displayed image. These points are then converted into three-dimensional measurement points by assigning disparity-based depth information. By assigning their respective depth information, the user can position such measurement points along the contour of the measurement object, even with only a two-dimensional representation of the object, thus enabling, for example, a dimensional measurement along the contour.

[0012] DE 10100335 B4 discloses a device for displaying a quantity obtained from an examination field, wherein the quantity is determined based on measurement data from an OCT device. The position of a marker in an image of an object can be updated when the object moves.

[0013] From JP 5777317 B2 a system for measuring respiratory movement in volume recordings is known, whereby values ​​such as minimum, maximum or a rate of change can be determined and output for the movement.

[0014] Furthermore, a solution approach is known from the prior art in which the user can select measurement points in real time during video recording of a measurement object using a stereo endoscope, in order to subsequently have the distance between these measurement points displayed as a superimposed representation in the endoscopic image. To compensate for possible movements of the image acquisition unit(s), the movement of the image acquisition unit(s) is detected by sensors, and the selected measurement points are optically tracked in the stereo image. Based on the sensor movement data, the measurement points in the image can be repositioned, thereby compensating for any movement of the image acquisition unit(s).

[0015] Although depth-information-based selection of measurement points on a measurement object is known from the prior art, the methods and systems are inadequate when the measurement object moves during observation. For example, in the medical field, when recording moving measurement objects, especially organs such as the lungs, heart, intestines, etc., the measurement object often moves back and forth while a measurement point is being selected, making reliable selection and / or aiming with a cursor difficult or even impossible. This makes it hard for the user to correctly define a measurement point in relation to the measurement object.

[0016] The observer and / or user also faces the challenge, particularly when measuring objects that move rhythmically and / or at a frequency, that a measuring distance between at least two measuring points defined by the observer in relation to the object is subject to temporal changes due to the object's movement. For example, a measured value for the measuring distance in a snapshot may not be displayed correctly because the object has already moved relative to its previous position, and thus the actual distance in the snapshot has either become smaller or larger. Therefore, the measurement accuracy of known dimensional measurement methods and systems is insufficient, and / or the accuracy achievable with them is regularly overestimated by the user.

[0017] The present invention is therefore based on the objective of overcoming the aforementioned disadvantages of the prior art. In particular, it is an objective to provide a method and a measuring device that increases and / or facilitates the accuracy of determining the position of a measuring point on a moving object. It is also an objective to provide a method and a measuring device that enables the most precise possible determination of a measuring distance and / or measuring accuracy in a dimensional measurement between two measuring points on an object, even when the object is moving, so that the user can be presented with the simplest possible, yet meaningful, information in a measurement image, particularly an endoscopic one.

[0018] At least one of the aforementioned problems is solved by a method having the features of independent claim 1. A supplementary solution to at least one of the aforementioned problems is provided by a method having the features of claim 6. Furthermore, the problems are solved by a measuring device according to the invention, which is configured to carry out at least one of the methods according to the invention.

[0019] Advantageous embodiments of the invention are specified in the dependent claims. It is understood that exemplary embodiments and configurations described with respect to the method according to claim 1 may relate in an equivalent, though not identical, form to the method according to claim 6, without being explicitly named for the latter. It is further understood that common linguistic transformations and / or a meaningful substitution of respective terms within the scope of usual linguistic practice, in particular the use of synonyms supported by generally accepted linguistic literature, are also encompassed by the present disclosure, without being explicitly mentioned in their respective formulations.

[0020] According to a first aspect of the invention, a method for correcting, in particular graphically, the position of a measuring point in a measurement image, especially in an endoscopic and / or exoscopic and / or microscopic and / or laryngoscopic measurement image, is specified.The procedure comprises at least the following steps: capturing a first measurement image of at least one moving object; determining the at least one measurement point with respect to the at least one moving object in the first measurement image; capturing at least one second measurement image of the at least one moving object following the first measurement image; calculating a position displacement vector between the at least one measurement point in the first measurement image and a corresponding image point in the second measurement image; and correcting a position of the at least one measurement point determined with respect to the first measurement image in the second measurement image based on the calculated position displacement vector.

[0021] Preferably, the first and / or second measurement image is a stereo-reconstructed stereo image, in which depth information is preferably made available for at least each pixel with respect to the at least one moving object being measured, based on a performed stereo reconstruction. To acquire the first stereo image, a measurement image of the moving object is preferably taken synchronously with respect to a predetermined time unit from two different viewpoints (predetermined by a stereo base between a first and a second image acquisition unit). The two temporally synchronized measurement images preferably form a stereo image pair with a common timestamp. Depth information can be evaluated pixel by pixel from each stereo image pair for each timestamp using stereo reconstruction methods.Preferably, during stereo reconstruction, a so-called disparity, i.e., a horizontal pixel offset, between the respective measurement images of the respective stereo image pair is determined pixel by pixel, i.e., per pixel, by an algorithm. Objects at a great distance preferably exhibit a low disparity, i.e., a small pixel offset, within a respective stereo image pair during stereo acquisition. In contrast, objects in the foreground exhibit a large disparity. From the pixel-by-pixel determined disparity, and with knowledge of further optical parameters of the image acquisition unit(s), a depth map can preferably be calculated for the stereo-reconstructed first and / or second stereo measurement image, which preferably includes pixel-by-pixel depth information of the object under observation.Using the depth information, it is possible, for example, to determine the distance between any two measurement points within the first and second stereo measurement images in Euclidean space.

[0022] Alternatively, the first and second measurement images can each be images in which depth information, i.e., the respective distance to points of an object, preferably to each pixel, is determined, particularly pixel by pixel, using a so-called time-of-flight sensor. Endoscopes with such sensors are known from the prior art, for example from EP 2 599 433 B1.

[0023] Alternatively, the first and second measurement images can each be images in which depth information, particularly pixel-by-pixel depth information, is determined based on so-called pseudostereoscopy. In this pseudostereoscopy, the depth information is derived from a stereo image comprising two time-separated individual images or frames of a video recording the movement of a measured object. In this case, the first and second measurement images are therefore each stereo image. The individual images are preferably captured by a conventional (mono) lens, preferably overlap, and can be used similarly to a stereo image pair or similarly to the images of a first and second stereo channel. The depth information, particularly pixel-by-pixel depth information, for each measurement image can be determined using stereo reconstruction methods.

[0024] Alternatively, the first and second measurement images could also be images where depth information, particularly pixel-by-pixel depth information, is determined using artificial intelligence. For example, it is possible to derive 3D information from 2D images using artificial intelligence. Such measurement images can be captured, for instance, by a (mono) lens or (mono) camera. From the captured two-dimensional images, three-dimensional image information, i.e., depth information, can then be deduced by applying an algorithm that, for example, represents an artificial neural network.

[0025] Alternatively, the first and second measurement images can each be images in which depth information, particularly pixel-by-pixel depth information, is determined based on the size ratios of previously known structures in the measurement scene. These structures could, for example, be part of an instrument and / or a marking on an instrument that is recognizable in the respective captured measurement image. This allows the camera's distance to the structure to be estimated based on a two-dimensional representation of the structure in the measurement image. This estimate can then preferably be used to determine the distance to other pixels in the measurement image.

[0026] According to the invention, the correction of the position change of at least one measuring point is preferably used to facilitate the selection of measuring points in a measurement image showing a moving object for the user. The tracking of a position change, at least between the first and the second measurement image, is preferably carried out by mathematical-optical tracking (optical tracking).

[0027] The first and second measurement images are each individual frames from a multitude of temporally successive measurement images, which, when combined, form a video recording of at least one object being measured. The time interval between the individual frames is preferably determined by a predetermined frame rate. It is understood that the first measurement image can, in principle, be any individual frame from such a video recording and / or video sequence and, of course, does not have to be an initially captured image. It is also understood that the second measurement image does not necessarily have to follow the first measurement image immediately in time; rather, one or more measurement images can be captured in between.The correction of the position of the at least one measurement point determined with respect to the first measurement image in the second measurement image on the basis of the calculated position displacement vector preferably takes place in real time, so that the user preferably cannot perceive this position displacement visually and it appears to him as if the at least one measurement point is attached to the at least one measured object.

[0028] In other words, the at least one measurement point is preferably continuously tracked between successive frames of a stereo live video recorded of the at least one object being measured. This makes it possible, for example, to correct the position of a cursor in the second measurement frame, which is moved by a user to determine, set, and / or select the at least one measurement point through at least the first measurement frame. More generally, it is possible to correct the change in position of the at least one measurement point in each subsequent measurement frame. Thus, it is possible to compensate for the movement of the at least one object being measured by tracking the measurement point accordingly. The at least one measurement point, represented, for example, as a cursor, therefore appears fixed to the at least one object being measured from the user's perspective.If the user wishes to move the cursor, i.e., the measuring point to be set relative to the object being measured, this movement is superimposed, according to the invention, with the calculated positional displacement resulting from the movement of the object being measured. In other words, the user moves the cursor relative to the moving object being measured, and not a pixel position of the cursor on a screen.

[0029] As an alternative to the solution according to the invention, it would be possible to select at least one measuring point on a still image. However, this would have the disadvantage that no temporal information about the distance of the measuring point from the image acquisition device would be available. This would lead to a temporally inaccurate measurement result. In addition, it would be cumbersome for the user to, for example, integrate an additional window into the workflow.

[0030] It is understood that the process steps according to the invention do not necessarily have to be carried out in the listed order, but that this order can also be changed. It is also possible that one or more intermediate steps can be carried out between one or more process steps.

[0031] In a preferred embodiment, the method according to the invention is characterized, according to the first aspect, in that a change in position between the at least one measuring point determined with respect to the first measurement image and the corresponding image point in the second measurement image is based on a movement of the movable object being measured, in particular a rhythmic and / or periodic and / or frequency-based movement. The method according to the invention thus differs from the prior art in particular in that a measuring point is not only corrected in its position with respect to the object being measured based on a change in the position of the measuring device. Instead, a position correction of the measuring point is carried out based on the movement of the object being measured, so that it appears to adhere to the object being measured in a visual sense. This was not possible in the prior art.The position displacement vector is therefore preferably calculated based on the movement of at least one object being measured. The rhythmic and / or periodic and / or frequency-based movement of the moving object causes it to continuously move relative to the measuring device used in the process, in particular relative to the two image acquisition units, and thus, among other things, continuously change its distance to the respective image acquisition unit. This also means that the depth information for the individual pixels in relation to the object being measured changes continuously between two measurement images. When a user determines a measurement point in relation to the object being measured, this determination is always based on specific depth information.According to the invention, in particular by calculating the position displacement vector, it is now possible to take into account the time-changing distance and / or depth information resulting from the movement of the object being measured, which can occur in all three spatial directions, when determining the measuring point. As a result, the measuring point remains fixed to the at least one object being measured.

[0032] In a further preferred embodiment, the method according to the invention is characterized in that the calculation of the position displacement vector is based on an optical flow algorithm and / or an elastic image registration algorithm and / or a point cloud registration algorithm and / or landmark-based tracking. It is understood that the algorithmic calculation methods listed here are not to be considered restrictive, but that other and / or complementary calculation methods can also be used to calculate the position displacement vector according to the invention.

[0033] In a further preferred embodiment, the method according to the invention is characterized in that the first and second measurement images are each individual frames of a video sequence. Preferably, the at least one movable object being measured is continuously captured as a video recording, in particular by an endoscopic and / or exoscopic and / or microscopic and / or laryngoscopic measuring device. Such a video recording of the at least one movable object being measured is preferably made by a first and a second image acquisition unit. The first image acquisition unit preferably records the object being measured from a first viewpoint. The second image acquisition unit preferably records the object being measured from a second viewpoint, wherein the first viewpoint differs from the second viewpoint.The first image acquisition unit preferably generates a multitude of individual frames for video recording of the object being measured, successively timed according to a predetermined frame rate (measured in frames per second). The second image acquisition unit preferably generates a multitude of individual frames for video recording of the object being measured, successively timed according to a predetermined frame rate (measured in frames per second). The frame rate of the first image acquisition unit preferably corresponds to the frame rate of the second image acquisition unit, so that the two image acquisition units record temporally synchronized frames.

[0034] In the inventive method according to the first aspect, the determination of the at least one measuring point with respect to the at least one movable object being measured is based on user input, which is made by moving a cursor on a display within the first measurement image. The user input can preferably be manual, semi-automatic, or automatic. For example, the user input can be manual using a joystick, keyboard, mouse, and / or touchscreen. Alternatively or additionally, the user input can also be partially automated, for example, computer-assisted. Automatic user input, particularly performed by a computer, is also possible.The method according to the invention is characterized, according to the first aspect, in that correcting the position of the at least one measuring point comprises superimposing a cursor motion vector for determining the measuring point with the position displacement vector. According to this embodiment, it is possible for the cursor to be moved to a targeted measuring point within the measurement image, for example, during live video acquisition of the object being measured. Preferably, the cursor is continuously corrected within the measurement image during such a movement, based on the position displacement vector, which is preferably dependent on a periodic and / or frequency-based movement of the object being measured.

[0035] In a further preferred embodiment, the method according to the first aspect of the invention is characterized in that the at least one measuring point is displayed on a user output device, preferably continuously and / or without interruption, in its corrected position. The user is thus preferably not shown the position correction itself. Consequently, the user only perceives that the at least one measuring point is attached to the at least one moving object being measured, i.e., that it always follows the object as it moves. The at least one measuring point therefore preferably moves back and forth in the (live) video recording in front of the user's eyes, along with the at least one object being measured.This allows the user to define the measuring point with greater accuracy in relation to at least one object being measured, since, for example, they can track a desired location where the measuring point is to be placed even during the movement of the object being measured and thus ensure that the measuring point has been correctly positioned in relation to the object being measured.

[0036] According to a preferred embodiment, it is possible to compensate for movement of the image acquisition device preferably used in the method according to the invention. For this purpose, among other things, a change in the position of at least one measuring point between the first and the second measurement image, which may be caused, for example, by movement of the image acquisition device and / or by movement of at least one movable object being measured, is tracked by methods of, in particular, optical, tracking. Preferably, in optical tracking, not the at least one measuring point is tracked, but the observed object being measured as such. For such tracking of the movement of the image acquisition device itself, a larger number of detected (measurement) points are preferably known, e.g., at particularly prominent structures such as edges or high-contrast areas of the object being measured.Preferably, at least two measurement points on the object being measured are known, and more preferably, a plurality of measurement points. This allows a translational movement in space and / or a rotation of the image acquisition device to be derived from the tracked measurement points. It is understood that other tracking methods can also be used. This allows a modified position displacement vector to be calculated, which can also take into account any movement of the image acquisition device. Preferably, the image acquisition device has at least one position sensor, for example, a gyroscope, a GPS sensor, and / or an optical sensor, which can detect a change in the position of the image acquisition device in the form of a sensor signal.The image acquisition device can also include a medical navigation system designed for optical or electromagnetic detection of the image acquisition device's position from outside the device. For this purpose, the image acquisition device can, for example, include a tracker or a magnetic field sensor in conjunction with a coil for electromagnetic detection. Such a sensor signal is preferably included in the calculation of the modified position displacement vector. This makes it possible to correct the position change of at least one measurement point in at least the second measurement image, so that the at least one measurement point appears to be fixed to the at least one moving object in the user-oriented representation, and thus preferably does not move relative to it. A further advantage of tracking and / or correcting the position of the at least one measurement point is that, if necessary,The user-selected measurement values ​​(e.g., a preferably three-dimensional position of the measurement point in a predetermined coordinate system) and / or other information can be averaged over time, particularly across multiple measurement images, at this measuring point. This allows measurement errors to be compensated.

[0037] In a second aspect of the invention, a method for outputting at least one statistical parameter for a moving object, which is included in particular in an endoscopic and / or exoscopic and / or microscopic image, is specified. The method according to the invention comprises at least the steps described below. The method comprises providing a plurality of temporally successive images of the at least one moving object by means of an image acquisition device, wherein each plurality of images includes, in particular, information about at least one measurement distance to be output with respect to the moving object. The measurement distance is calculated between a first measurement point, which is determined with respect to the at least one moving object, and a second measurement point, which is also determined with respect to the at least one moving object.The measuring section preferably defines a distance to be measured between the first and second measuring points. Furthermore, the method comprises determining, by statistical evaluation, at least one statistical parameter, in particular with respect to the at least one measuring section and / or the distance between the first and second measuring points, in the multitude of temporally successive measurement images, which are at least partially related to a rhythmic and / or periodic and / or frequency-based movement of the at least one moving object being measured. The method further comprises outputting the determined, at least one statistical parameter, by which the movement of the at least one moving object being measured is described at least partially, preferably in its entirety.The output of at least one statistical parameter is preferably provided to an operator and / or user of the method in graphical and / or textual and / or acoustic and / or tactile form. The method according to the second aspect of the invention is characterized in that a change in the position of the first measuring point and / or the second measuring point in the plurality of temporally successive measurement images, which is at least partially caused by a movement of the at least one movable measuring object, is corrected by the method according to the first aspect of the invention and according to its preferred embodiments.The two measuring points, which are preferably determined by the user in relation to the at least one object being measured, for defining the at least one measuring section, can thus be corrected in their position, preferably in real time, by the method according to the first aspect of the invention, such that at least one of the two measuring points appears to be visually attached to the at least one object being measured. It is possible that only one of the two measuring points is corrected in its position relative to the at least one object being measured by the method according to the invention.

[0038] Instead of the static analysis according to the invention of the multitude of temporally successive measurement images or the time series of at least one measurement section for periodic structures, an alternative solution would be to display the measurement results of each image to the user. However, this approach would have the disadvantage compared to the solution according to the invention that the displayed measurement intervals between the individual images would change rapidly or jump back and forth. This would make it difficult for the user to accurately read the measured values.

[0039] Thus, at least one of the aforementioned problems underlying the invention is solved by preferably determining a multitude of measurement results for each measuring section from temporally successive measurement images based on stereo reconstruction methods. The multitude of measurement results makes it possible, in particular, to depict the effect of a movement of the at least one moving object being measured on the at least one measuring section. The first and / or the second measuring point are preferably defined with respect to real points of the at least one object being measured and, optionally, its surroundings. If at least one of these real points moves due to a movement of the object being measured, the distance, in particular the Euclidean distance, between the two measuring points changes depending on the type and extent of the movement. The user can thus preferably select the at least two measuring points in real time.The method according to the invention preferably displays to the user a distance between the two selected measuring points, preferably as a graphic overlay in a measurement image, particularly preferably as a graphic overlay in a stereo video recording of the at least one measured object.

[0040] In the present method, a large number of measurement results for a distance between two measuring points on a test object can be determined, in particular a time series of measurement results, based on stereo reconstruction of measurement images. This large number of measurement results is evaluated, for example, by averaging, in order to obtain a time-averaged distance between two measuring points.

[0041] According to the invention, the movement of the object being measured, and thus a temporal change in distance, is detected by stereo reconstruction of a multitude of successive measurement images. The detected temporal progression of the distance, in particular the temporal progression of a change in distance, is statistically evaluated according to the invention in such a way that at least one statistical parameter characteristic of the object's movement can be captured in the multitude of successive measurement images and displayed to the user. This allows, for example, the determination of a periodicity, in particular a frequency and / or an amplitude, of the measured distance. Thus, according to the invention, it is possible to display to the user, preferably in addition to a static mean value, at least one statistical parameter, for example, a minimum and / or a maximum value and / or a movement frequency, for the measured distance.It should be noted that at least one statistical parameter can also be a time-based average. The inventors also recognized that this averaging can lead to insufficient measurement accuracy. The solution to this problem is the statistical evaluation and analysis of the time series of the at least one measurement section. Here, statistical evaluation functions are preferably applied to the time-based measurement profile, allowing the movement of the at least one object being described as precisely as possible. Through this statistical evaluation, changes in frequency, rhythm, and / or period of the movement can also be tracked, so that even with complex movement sequences, the user always receives an accurate description of the movement.

[0042] In a preferred embodiment, the method according to the second aspect of the invention is characterized in that the at least one statistical parameter comprises at least one frequency, one amplitude, one minimum value, one maximum value, one statistical mean, one standard deviation, and one statistical error indicator. Other statistical values ​​not explicitly mentioned in the preceding list may also be included. The selection of the output of the at least one statistical parameter preferably depends on the movement of the at least one movable object being measured. Preferably, several statistical parameters are output to the user so that the movement of the object being measured is described as precisely as possible. Particularly preferably, the statistical evaluation determines a time course of the at least one statistical parameter.

[0043] In a preferred embodiment, the method according to the second aspect of the invention is characterized in that a mean, resulting measurement error is output for the at least one statistical parameter. The output of the measurement error enables the user to determine whether a currently performed distance measurement still meets the desired accuracy requirements or whether, for example, the movement of the at least one moving object being measured has changed in such a way that a distance measurement with the required accuracy is no longer possible. The output of the measurement error includes a warning function for the user, which can be displayed, for example, as an acoustic, graphic, visual, and / or haptic indication.

[0044] In a preferred embodiment, the method according to the second aspect of the invention is characterized in that the first measurement image and / or the second measurement image and / or the plurality of measurement images are processed by stereo reconstruction of a respective stereo image pair based on optical and / or dimensional parameters of the image acquisition device. In this embodiment, the first and the second measurement image are preferably each a stereo image formed from a stereo image pair. The processing preferably includes a calculation, particularly pixel-wise, of depth information for each pixel in the first and / or second measurement image (stereo reconstruction) in order to enable a dimensional measurement between at least two measurement points selected from a recording scene.Stereo reconstruction preferably includes correction of distortion effects and / or transformation in each stereo image pair (rectification).

[0045] According to a third aspect of the invention, a measuring device is specified. The measuring device comprises an image acquisition device with at least one first and at least one second image acquisition unit spaced apart from it, and at least one evaluation unit. It is understood that the evaluation unit can be arranged outside or inside the image acquisition device and is configured to interact with the image acquisition device. Furthermore, the measuring device comprises a processor, in particular an evaluation and / or computing unit. The processor can preferably be included in the evaluation unit or arranged separately from it. The processor is preferably configured to perform at least some of the steps of the method according to the first aspect of the invention, including its embodiments.Alternatively or additionally, the processor is configured to perform at least some of the steps of the method according to the second aspect of the invention, including its embodiments.

[0046] According to the third aspect, the processor is preferably configured to capture and / or determine a first measurement image, preferably in the form of measurement image information, of at least one moving object being measured. Furthermore, the processor can be configured to provide the computing power that is technically necessary to graphically determine the at least one measurement point in relation to the at least one moving object being measured in the first measurement image. The processor is preferably configured to process at least one second measurement image of the at least one moving object being measured, which follows the first measurement image in time.Furthermore, the processor is configured to calculate a position displacement vector between the at least one measurement point in the first measurement image and a corresponding image point in the second measurement image, and to correct a position of the at least one measurement point in the second measurement image, which is defined with respect to the first measurement image, on the basis of the calculated position displacement vector.

[0047] Alternatively or additionally, according to the third aspect, the processor can preferably be configured to provide a plurality of temporally successive measurement images of the at least one moving object being measured. The image acquisition device is preferably configured to transmit a plurality of temporally successive stereo image pairs to the processor in the form of stereo image pair information. The processor is preferably configured to generate or provide a measurement image from each pair of measurement images by means of stereo reconstruction. The plurality of measurement images comprises at least one measurement path to be output with respect to the moving object being measured, which is determined between a first measurement point, which is defined with respect to the at least one moving object being measured, and a second measurement point, which is defined with respect to the at least one moving object being measured.The processor is preferably configured to calculate the distance of at least one measuring section. Furthermore, the processor is configured to determine, by statistical evaluation, at least one statistical parameter from the multitude of temporally successive measurement images, which is at least partially related to a periodic and / or frequency-based movement of the at least one moving object being measured. The processor is preferably configured to transmit the at least one statistical parameter or a temporal profile of the at least one statistical parameter to the user output device. The user output device is configured to output the at least one statistical parameter that at least partially describes the temporal movement of the at least one moving object being measured.

[0048] In a preferred embodiment, the measuring device comprises a stereoendoscope and / or a stereoexoscope and / or a stereomicroscope and / or a laryngoscope. Preferably, the measuring device is configured as a stereoendoscope and / or a stereoexoscope and / or a stereomicroscope and / or a laryngoscope.

[0049] In a preferred embodiment, the measuring device comprises a user input device and / or a user output device. The user input device may include a keyboard and / or a mouse and / or a joystick and / or a touchscreen and / or a touchpad and / or another manual input device. The user output device may include a screen and / or glasses and / or 3D glasses and / or augmented reality glasses.

[0050] Further advantages and details of the invention will become apparent from the following description of preferred embodiments of the invention and from the schematic drawings only.

[0051] They show: FIG 1: A schematic view of an embodiment of a measuring device according to the invention; FIG 2: An exemplary stereoscopic image in which an exemplary movable object is captured in a first position, with a measuring distance shown in relation to the object; FIG 3: An exemplary stereoscopic image in which an exemplary movable object is captured in a second position, with a measuring distance shown in relation to the object; FIG 4: A graphical representation of a time course of a measuring distance; FIG 5: The in FIG. 4 The graphical representation of the measurement distance with a superimposed representation of a time course of an average value of the measurement distance is shown; FIG 6: an exemplary frequency diagram obtained through statistical analysis; FIG 7: the in FIG. 4 The graphical representation of the measurement distance is shown, with a superimposed display of statistically calculated minima and maxima. FIG 8: An exemplary arrow representation of a cursor to illustrate a position displacement vector and a cursor movement vector resulting from user input between successive stereo measurement images.

[0052] Identical elements or elements with the same function are provided with the same reference numbers in the figures.

[0053] Fig. 1 Figure 1 shows a highly simplified schematic block diagram illustrating an exemplary embodiment of a measuring device 100. In this case, the measuring device 100 is implemented in a stereo endoscope 102. The measuring device 100 comprises an image acquisition device 104 with an evaluation unit 106. Furthermore, the image acquisition device 104 includes a first image acquisition unit 108 and a second image acquisition unit 110.

[0054] The first image acquisition unit 108 has a predetermined distance from the second image acquisition unit 110, which defines a stereo baseline of the stereo endoscope. The first and second image acquisition units 108 and 110 are preferably each a camera. The evaluation unit 106 is preferably configured to receive image data in the form of measurement images from the first and second image acquisition units 108 and 110 and to evaluate this data. The evaluation unit 106 particularly preferably includes at least one image processing processor (not shown in detail). It is understood that in other embodiments, the evaluation unit 106 can preferably be arranged outside the image acquisition device 104. The evaluation unit 106 is preferably designed as a so-called camera control unit (CCU).Preferably, at least one preprocessing of the captured measurement images 109, 111 can take place in the image acquisition device 104.

[0055] The first and second image acquisition units 108, 110 are each assigned a lens assembly 114. The lens assembly 114 includes, for example, a cover glass and optical units 116, 118 with apertures, which are assigned to the image acquisition units 108, 110. The optical units 116, 118 define the respective field of view of the image acquisition units 108, 110. Each of the two image acquisition units 108, 110 is assigned to an observation channel 120, 122. The observation channels 120, 122 are each configured to transmit the measured images in the form of signal-like image information to the evaluation unit 106 or to the at least one processor. A signal converter 124, 126 is assigned to each of the image acquisition units 108, 110 to provide the image information. The signal converters 124 and 126 are each designed to convert the optically captured measurement images into image information.For example, signal converters 124 and 126 are photochips.

[0056] The first image acquisition unit 108 is configured to acquire at least one initial image of at least one moving object 112. The object 112 is represented here by the letter P for illustrative purposes. However, the object 112 is typically, and preferably, a human or animal organ, another part of a human or animal body, or a component. The first image acquisition unit acquires the at least one image of the object 112, preferably from a first viewing angle.

[0057] The second image acquisition unit 110 is configured to acquire at least a second measurement image of the at least one moving measurement object 112. The first and second image acquisition units 108 and 110 are each configured to acquire the first and second measurement images, preferably synchronized in time. A first and second measurement image acquired in this way preferably forms a stereo image pair.

[0058] The evaluation unit 106 is designed to determine stereo image information from the stereo image pair or from the signal-based image information of the first and second measurement images using known stereo reconstruction methods. Depth information is available in the stereo image information for each captured pixel of at least one measurement object 112, which can be used, for example, to calculate a distance between two measurement points on the measurement object 112 in Euclidean space.

[0059] The stereo measurement image information can preferably be transmitted via a first and / or a second output channel 128, 130 to a user output device 132, through which the stereo measurement image information is provided to a user as a first (stereo) measurement image 134 and preferably displayed graphically. The user output device 132 can, for example, be a display. On the user output device 132, the at least one measurement object 112 is depicted in the form of an observation object 136. A user can preferably move a cursor 138 relative to the observation object 136 or to the virtualized measurement object 112 using a user input device (not shown) in order, for example, to determine a measurement point with respect to the measurement object.

[0060] It is understood that the measuring device 100 is configured to acquire a multitude of measurement images of the object being measured and thus to acquire or provide a multitude of stereo measurement images for each stereo image pair. Particularly preferably, the measuring device 100 is capable of recording a (live) video of the at least one object being measured 112, which is composed of a multitude of individual image pairs that are acquired sequentially at a predefined time interval (determined by the frame rate).

[0061] In Fig. 2 An example of a first stereo measurement image 134 is shown simplified in two dimensions, which was taken, for example, by the measuring device 100 according to the invention. It is evident that the object being measured is no longer the one shown in Fig. 1 The measurement object shown is not an example, but rather a human body part is represented as measurement object 112 by the first stereo image 134. The at least one measurement object 112 is represented here by the fingers of a hand. However, the at least one measurement object 112 can also comprise any other type of measurement object, for example, a part of an anatomy and / or one or more organs of the human or animal body. Alternatively, the at least one measurement object 112 can also be a component. According to Fig. 2 In the first stereo measurement image 134, an example distance 140 between a first measuring point 142 and a second measuring point 144 is shown, which is displayed to the user or operator visually both as a measuring distance 146 and additionally as a numerical value, in the example 29 mm, as graphic elements superimposed on the first stereo measurement image 134.

[0062] The first measuring point 142 is preferably determined and / or set by a user by means of the cursor 138, which is displayed on the user output device 132 and / or in the user's glasses, with respect to the at least one movable object 112, exemplified in the first stereo measurement image 134. The second measuring point 144 is preferably determined and / or set by a user by means of the cursor 138, which is displayed on the user output device 132 and / or in the user's glasses, with respect to the at least one movable object 112, exemplified in the first stereo measurement image 134.

[0063] In Fig. 3 As an example, a second (stereo) measurement image 148, which follows the first stereo measurement image 134 in terms of timing, is shown. According to the second stereo measurement image 148, at least one of the objects being measured, 112, is in a second position. The change in position of at least one of the objects being measured, 112, is due to a movement of the object being measured, 112, in this example, to a rhythmic and / or frequent opening and closing of the fingers of the hand. The first and second stereo measurement images 134 and 148 each represent an exemplary snapshot of this rhythmic or frequent movement.

[0064] The first and second stereo measurement images 134, 148 are each individual images of a multitude of temporally successive measurement images (see explanations regarding Fig. 1 ), which, when sequentially arranged, result in a video recording of the at least one measurement object 112. Each stereo measurement image of the plurality of stereo measurement images, including the first and second stereo measurement images 134, 148, results from a stereo image pair that is reconstructed using known stereo reconstruction methods, such that depth information is preferably available for each captured pixel of the at least one measurement object 112. From each stereo image pair and / or from the already stereo-reconstructed stereo measurement image, the distance 140 between the first and second measurement points 142, 144 in Euclidean space is calculated.

[0065] This results in a time series of measuring sections 146, which are exemplified in the in Fig. 4 The diagram shown illustrates this. The abscissa displays the frame number. The ordinate shows the changing distance in millimeters between the opening and closing fingers. The change in distance between two measuring points on the moving object 112, which are visually marked by the two measuring points 142 and 144, can be calculated independently of the optical or visual tracking of the measuring points 142 and 144 (their attachment to the object) using methods for determining optical flux and similar procedures.

[0066] As can be seen from a synthesis of the Figuren 2 und 3 As can be seen, both the first and second measuring points 142, 144 are corrected with respect to their position depending on the movement of the at least one measuring object 112 according to the method according to the invention and thus adhere visually to the measuring object 112 in the view displayed to the user of the measuring device 100. For this purpose, the open-source function cv2.calcOpticalFlowPyrLK() from openCV was used. However, other optical flow calculation methods and / or landmark-based tracking are also possible for position tracking and correction of the two measuring points 142, 144.

[0067] According to the in the Fig. 2 und 3 In the illustrated embodiment, the measuring distance 146 displayed to the user is not statistically evaluated according to the method also according to the invention. Instead, the distance 140 between the two measuring points 142, 144 is merely averaged over time. This time-averaged value, as shown in Fig. 3 As depicted, the user is shown an incorrect numerical value in the superimposed graphic element, in this case, for the distance 140 currently shown in the second stereo measurement image 148, for the distance 140. This erroneous display can be fundamentally improved by the method according to the second aspect of the invention.

[0068] In accordance with the erroneous representation of the distance 140 in the Fig. 3 The second stereo image 148 shown is in Fig. 5 the from Fig. 4 The known distance image sequence number is displayed together with a calculated mean value curve. It can be seen that, on average over time, a distance of 140 results with a mean value of approximately 29 mm. Due to the time averaging of the distance 140, the displayed numerical value is inaccurate, as shown in Fig. 3 The example shown is, however, delayed in terms of timing, which can lead to uncertainty regarding the accuracy of the display. The statistical error of this measurement is estimated at 3 mm and is therefore inaccurate compared to the evaluation made possible by the invention.

[0069] Instead of the in Fig. 5 In addition to the averaging shown as an example, the determined time series of the measuring section 146 is further analyzed according to the invention using statistical methods, for example by calculating a periodicity and / or an amplitude. The periodicity and / or an amplitude represent, by way of example, at least one statistical parameter. As an example, the result of such an analysis is Fig. 6 A periodic graph (maximum at 0.006 frames, corresponding to 170 frames) of the frequency analysis is shown. In addition, an example amplitude (displayed as minima and maxima) of the determined time series of the measurement section 146 was calculated and is shown in Fig. 7 displayed. By calculating these statistical parameters, the user can be shown minimum and maximum values ​​as well as, if applicable, frequencies of a movement of at least one object being measured, preferably graphically.

[0070] Particularly preferably, at least one vital parameter that is significant for the at least one measurement object 112 can be determined from the statistically analyzed movement of that object. If the at least one measurement object 112 is, for example, the heart of a human or an animal, the method according to the invention makes it possible to determine a respiratory rate and / or a heart rate based on the at least one statistical parameter. If the at least one measurement object 112 comprises at least one vein and / or artery located on the surface of the object, the method according to the invention makes it possible to determine a heartbeat based on the at least one statistical parameter.

[0071] In Fig. 8Figure 1 illustrates an application example of the method according to the first aspect of the invention. To determine the position of the cursor 138 for selecting measurement points 142, 144, two different displacement vectors are calculated. On the one hand, a position displacement vector 150 is determined between two successive stereo measurement images from optical tracking as described above. On the other hand, a movement vector 152 is determined from user input, preferably manual, e.g., via a joystick, keyboard, or mouse. The two vectors 150, 152 are superimposed, and the cursor 138 can be moved accordingly. This results in the cursor position changing only relative to the at least one measurement object 112. The cursor 138, or measurement point, thus preferably remains fixed to the measurement object as long as the user does not specify a different movement or perform a different input. Reference symbol list

[0072] 100 Measuring device 102 Stereo endoscope 104 Image acquisition device 106 Evaluation unit 108 First image acquisition unit 110 Second image acquisition unit 112 Object being measured 114 Lens assembly 116 Optical unit 118 Optical unit 120 Observation channel 122 Observation channel 124 Signal converter 126 Signal converter 128 First output channel 130 Second output channel 132 User output device 134 First measurement image 136 Observation object 138 Cursor 140 Distance 142 First measurement point 144 Second measurement point 146 Measuring distance 148 Second measurement image 150 Position displacement vector 152 Motion vector

Claims

1. A method for correcting a position of a measurement point in a measurement image, in particular in an endoscopic and / or exoscopic and / or microscopic measurement image, comprising at least the steps of: - Capturing a first measurement image (134) of at least one movable measurement object (112); - Determining the at least one measurement point (142, 144) in relation to the at least one movable measurement object (112) in the first measurement image (134); - Capturing at least one second measurement image (148), following the first measurement image (134) in time, of the at least one movable measurement object (112); - Calculating a position displacement vector (150) between the at least one measurement point (142, 144) in the first measurement image (134) and an image point, which corresponds to the at least one measurement point (142, 144), in the second measurement image (148); and - Correcting a position of the at least one measurement point (142, 144), which is determined in relation to the first measurement image (134), in the second measurement image (148) on the basis of the calculated position displacement vector (150), characterised in that determining the at least one measurement point (142, 144) in relation to the at least one movable measurement object (112) is carried out on the basis of a user input, namely by moving a cursor (138) within the first measurement image (134), and correcting the position of the at least one measurement point (142, 144) comprises a superposition of a movement vector (152) of the cursor (138) for determining the measurement point (142, 144) with the position displacement vector (150).

2. The method according to one of the preceding claims, characterised in that a position change between the at least one measurement point (142, 144), which is determined in relation to the first measurement image (134), and the image point, which corresponds thereto, in the second measurement image (148) is based on a movement of the movable measurement object (112).

3. The method according to one of the preceding claims, characterised in that calculating the position displacement vector (150) is carried out on the basis of an optical flow algorithm and / or an elastic image registration algorithm and / or a point cloud registration algorithm and / or a landmark-based tracking.

4. The method according to one of the preceding claims, characterised in that the first and second measurement images (134, 148) are each a single image of a stereovideo sequence.

5. The method according to one of the preceding claims, characterised in that the at least one measurement point (142, 144) is represented on a user output device (132), in particular always, in its corrected position.

6. A method for outputting at least one statistical characteristic variable relating to a movable measurement object (112), which is included in particular in an endoscopic and / or exoscopic and / or microscopic measurement image, wherein the method comprises at least the steps of: - Providing a plurality of temporally successive measurement images of the at least one movable measurement object (112), wherein the plurality of measurement images (134, 148) comprises at least one measurement path (146) which is to be output in relation to the movable measurement object (112) and which is calculated between a first measurement point (142), which is determined in relation to the at least one movable measurement object (112), and a second measurement point (144), which is determined in relation to the at least one movable measurement object (112); - Determining the at least one statistical characteristic variable in the plurality of temporally successive measurement images (134, 148), which is at least partially associated with a periodic and / or frequency-based movement of the at least one movable measurement object (112), by way of statistical evaluation; - Outputting the at least one statistical characteristic variable by way of which the movement of the at least one movable measurement object (112) is at least partially described, characterised in that a position change of the first measurement point (142) and / or of the second measurement point (144) in the plurality of temporally successive measurement images, which is caused at least partially by a movement of the at least one movable measurement object (112), is corrected by the method according to one of claims 1 to 5.

7. The method according to claim 6, characterised in that the at least one statistical characteristic variable comprises at least one of a frequency, an amplitude, a minimum value, a maximum value, a mean value, a standard deviation and a statistical error indicator.

8. The method according to claim 6 or 7, characterised in that a mean, resulting measurement error is output for the at least one statistical characteristic variable.

9. The method according to one of the preceding claims, characterised in that the first measurement image (134) and / or the second measurement image (148) and / or the plurality of measurement images are processed by stereo reconstruction of a respective stereo image pair on the basis of optical and / or dimensional parameters of the image capture device (104).

10. A measuring device (100) comprising: an image capture device (104) having at least one first and one second image capture unit (108, 110), spaced apart therefrom, an evaluation device (106), and a processor configured to perform the steps of the method according to one of claims 1 to 5 and / or the method according to one of claims 6 to 9.

11. The measuring device according to claim 10, characterised in that the measuring device (100) comprises a stereoendoscope (104) or a stereoexoscope or a stereomicroscope or a laryngoscope.

12. The measuring device according to claim 10 or 11, characterised in that it comprises a user input device and / or a user output device (132).