Method for determining object information in a medical examination and / or treatment order, medical examination and / or treatment order, computer program and electronically readable data carrier

Abstract

The invention relates to a method for determining object information in a medical examination and / or treatment arrangement (1) with at least one object (6, 7, 8, 9, 10, 19), wherein the object (6, 7, 8, 9, 10, 19) is detected by means of at least one camera (5) and by means of a control device (11) by evaluating camera data (26) of at least one camera image of the camera (5) an item of object information (29) comprising identification and / or position information of the object (6, 7, 8, 9, 10, 19) is determined, wherein the object (6, 7, 8, 9, 10, 19) is at least temporarily provided with a tracking device (17) and by means of the tracking device (17) an item of pose information (27) describing the position and / or orientation of the object (6, 7, 8, 9, 10, 19) is determined, which is used to determine the object information (29), in particular supporting.

Description

The invention relates to a method, in particular a computer-implemented method, for determining object information in a medical examination and / or treatment arrangement having at least one object, wherein the object is captured by means of at least one camera and object information comprising an identification and / or position information of the object is determined by means of a control device by evaluating at least camera data of at least one camera image of the camera. In addition, the invention relates to a medical examination and / or treatment arrangement, a computer program and an electronically readable data carrier.Medical examination and treatment arrangements, in particular their central examination and treatment devices, become increasingly complex and frequently have components or components which can be moved controllably by means of suitable actuators. While it is possible in principle to control such components by means of operator control elements on the user side, there is a trend to configure specific functions autonomously in examination and treatment arrangements. In both cases, it is expedient to identify other objects which may be located in the path of the movement in order to increase the safety and, in particular when using virtual three-dimensional models of objects, to know their position and orientation and / or, generally, their extents, for example in order to avoid collisions. Specifically, movement paths can be determined at least for certain types of objects in such a way that a collision is avoided.An example of a medical technology device that can be equipped with a certain level of autonomy is a mobile C-arm of an X-ray device. An X-ray radiator and an X-ray detector can be arranged opposite one another on the C-arm. The mobile C-arm now also comprises a mobile carrier, for example with a chassis, and can also have further degrees of freedom of movement, to which corresponding actuators are assigned. If a user now selects a desired position, for example in the form of a specific recording geometry, a control device of the examination and treatment arrangement, in particular of the X-ray device itself, can determine a movement trajectory using the available degrees of freedom in such a way that the mobile C-arm autonomously produces the desired recording geometry. The same also applies to permanently installed C-arms having degrees of freedom of movement of different types as components or portions of medical technology devices.In another specific example, it has been proposed to provide a specific level of autonomy for patient tables, in particular when specific positions have to be approached exactly, for example for coupling to an imaging device.For these and comparable cases, environment detection, in particular with respect to objects with which a collision can occur, is necessary. For this purpose, it has been proposed to provide cameras on the movable component and / or directed in another way to the range of movement of the movable component, which cameras record camera images, so that information on objects in the range of movement can be obtained from these at the time by evaluation. Here, frequently trained evaluation functions are used, which can comprise, for example, neural networks, in particular CNNs. For example, such evaluation functions are trained to divide objects contained in the camera data of the camera images into object classes, for example "mobile instrument table", "operating unit", "infusion stand" and the like. The machine learning is carried out on the basis of training data in which different specific configurations of objects of these object classes are contained. Therefore, trained evaluation functions, as is known in principle, provide possible object classes with associated reliability values, in particular probabilities. In addition to identifying objects, i.e. assigning an object class to an object contained in the camera image, it has also already been proposed to determine orientations and / or positions, in particular, poses, for example relative to the camera and / or an at least partially autonomously operated component, in a corresponding manner.The use of a range of probabilities or comparable reliability values is fundamentally less desirable, in particular in the case of safety functions. Instead, the aim is to provide evaluation results that are as unique as possible.The invention is based on the object of specifying an improved possibility for ascertaining object information of at least one object during the evaluation of camera images, in particular allowing an increase in reliability and robustness with respect to the results.This object is achieved according to the invention by a method, in particular a computer-implemented method, a medical examination and treatment arrangement, a computer program and an electronically readable data carrier according to the subordinate patent claims. Advantageous further developments are evident from the dependent claims.In a method of the type mentioned at the beginning, it is provided according to the invention that the object is provided at least temporarily with a tracking device and a pose information describing the position and / or the orientation of the object is determined by means of the tracking device, which pose information is used for determining the object information, in particular supporting it.It is therefore proposed to provide an object of a medical examination and treatment arrangement with a tracking device for the detection and tracking of the position and the orientation, i.e. the pose. In this case, it may be the case of an active, self-measuring tracking device or a passive tracking device in the sense of a marker, which can be understood as a pose marking device or a pose tracking marker.In the case of an active, self-measuring tracking device, the pose information is preferably determined in the same coordinate system in which work is also performed with regard to the camera data. For example, this can be achieved by means of a suitable calibration. In the case of a pose marking device as a tracking device, when the object is detected in a camera image, camera data of the pose marking device are also detected, by the evaluation of which extremely robust and reliable pose information describing the current position and / or the current orientation of the object can be determined.The tracking device can be installed on the object, but can also be realized, in particular integrated, as part of the object.While the pose information can fundamentally be incorporated directly into the object information in order to enrich it by a robust and reliably determined position and / or orientation of the object, it is preferred within the scope of the present invention to use the pose information in a supporting manner, in particular with regard to the reliability of the results of other evaluation functions, in particular trained evaluation functions. It has been recognized that at least as long as the evaluation functions, i.e. functions of image processing or image recognition (image recognition), do not fulfil or do not sufficiently fulfil requirements for the safety, which is highly relevant in particular in the medical field, a support functionality is extremely useful and advantageous in order to achieve the desired requirements. In other words, the robustness and reliability of the evaluation of the camera images with respect to the objects is significantly improved by the combination of conventional evaluation functions, i.e. image recognition software, with the tracker device, wherein the pose information, as will be explained in more detail below, can be used, for example, to improve the reliability values and / or to provide a certain redundancy.In particular, embodiments can provide that a reliability value is also determined for the pose information, wherein the use of the pose information takes place as a function of the reliability value of the pose information and / or a comparison of the reliability value of the pose information with a reliability value of the object information. Consequently, for example, different probability levels for specific properties of the object, in particular the orientation, can be introduced, on the basis of which the control device can decide by which instance an object, in particular its orientation, was detected correctly, in particular more reliably and / or more stably.Specifically, it can be provided that a trained evaluation function is used for evaluating the camera data, which evaluation function determines at least one object class of the object from output data with an associated reliability information item, in particular a reliability value. In this case, the object is thus identified as belonging to at least one object class.In general, a trained function maps cognitive functions that associate humans with other human brains. By training based on training data (machine learning), the trained function is able to adapt to new circumstances and detect and extrapolate patterns.Generally speaking, parameters of a trained function can be adapted by training. In particular, supervised learning, semi-supervised learning, unsupervised learning, reinforcement learning, and / or active learning may be used. In addition, representation learning (also known as "feature learning") can also be used. The parameters of the trained function can be adjusted iteratively by a plurality of training steps.A trained function may include, for example, a neural network, a support vector machine (SVM), a decision tree and / or a Bayesian network, and / or the trained function may be based on k-means clustering, Q-learning, genetic algorithms and / or assignment rules. In particular, a neural network may be a deep neural network, a convolutional neural network (CNN), or a deep CNN. Moreover, the neural network may be an Advanced Network, a Deep Advanced Network, and / or a Generative Advanced Network (GAN).With regard to the trained evaluation function, it is preferred if it comprises at least one neural network, in particular a CNN. Such embodiments have proven to be particularly advantageous in the evaluation of camera data.When using a pose marking device as tracker device, it is preferably designed in such a way that a corresponding tracking function can detect and analyze the pose marking device in the camera data without the use of artificial intelligence in order to determine the orientation and the position as accurately as possible. By contrast, machine learning / artificial intelligence is preferably used for the evaluation function, in particular with respect to the object class. However, cases are also conceivable in which a trained tracking function is used to ascertain the pose information. As already mentioned, reliability values can be assigned to the pose information, regardless of whether the tracking function is trained or not. Specific embodiments for tracking functions are known, for example, from the field of virtual reality tracking (VR tracking), where pose marking devices of this type are likewise used. Tracking devices with active sensor systems, which transmit the pose information wirelessly, for example, can also provide reliability values.When using a trained evaluation function, different variants are conceivable as to how the pose information can be used in a winning manner. Thus, in a first advantageous exemplary embodiment, it can be provided that the trained evaluation function uses at least a portion of the pose information, in particular an orientation of the object, as additional input data. In this case, the input data is therefore expanded by additional information which enables a more robust and reliable detection, in particular identification, of the object. Consequently, firstly the pose information is determined by means of the tracking device, at least the orientation of which enters as input data into the image recognition realized by the trained evaluation function. The trained evaluation function now knows that it has to search for an object in a specific pose, which reduces the number of possible objects and improves matching. The result is an identification of the object with a higher probability.In principle, it can be said that, when an object (of a specific object class) is detected, it is useful or even necessary for most functions which use the corresponding object information to also determine the pose of the object. Consequently, as already proposed in the prior art, it can be provided that, in addition to the object class, an orientation of the object and / or position of the object (as the position information) is also determined by at least one of the at least one trained evaluation function, which orientation is compared with the orientation and / or position of the object in accordance with the pose information for plausibility checking the evaluation result of all trained evaluation functions. In such an embodiment, use is therefore made of the fact that the trained evaluation functions used, in particular a single trained evaluation function, can also supply, in addition to the object class, own information about the pose as output data, in particular with regard to the orientation. The trained evaluation function thus determines an object class with an associated reliability value and an orientation / orientation class and / or position, and therefore the position information, in particular likewise with an associated reliability value. The pose, in particular orientation, determined by means of the tracking device can then be used to check whether the detected object according to the trained evaluation function has the same pose as the genuine object according to the pose information. In this way, image recognition, in particular identification of objects in the camera data, is achieved with redundancy that increases reliability and robustness, since it can be assumed that it is unlikely that an object and its pose are both incorrectly detected.Expediently, the tracking device can have an identification feature assigned to the object provided with it, by means of which identification information of the object is determined, which identification information is used for, in particular additional, plausibility checking of the determined object class. An actively measuring tracking device transmitting the pose information, in particular wirelessly, can transmit the identification information stored, for example, in a storage means of the tracking device, for example, with the pose information, which is also useful, in particular, when multiple tracking devices, in particular for multiple objects, are used. Alternatively or additionally, when the tracking device is designed as a pose marking device, it can be provided that it has an external identification feature for its unique identification, which is detected in the evaluation of the camera data and used for identifying the object. If the tracking devices are permanently assigned to objects, the assignment of the identification feature to the object can be stored, for example, in a database of the control device. In this context, it is also possible in particular to use generic identification features for object classes, and therefore to provide each object of the object class with a tracking device which has the identification feature for the object class, so that in this case the identification information describes an object class of the object. The object is then thus identified as an object of the object class, as is also done with a corresponding configuration of the trained evaluation function.In other words, it can be said that an object, in particular an object class, can additionally be determined independently of the camera data by a unique identification feature of the tracking device and a mapping, and therefore a mapping, between the identification feature, so that the combination of the identification feature of the tracking device with the image evaluation of the camera data creates a further redundancy, which can increase the robustness and reliability of the evaluation. This represents a direct, so-to-say genuine, redundancy with respect to the identification of the object.With regard to plausibility checking / redundancy both with regard to the pose information and with regard to the identification information, advantageous refinements of the invention can provide that the reliability value is adapted as a function of the at least one plausibility checking result. In this case, the redundancy is thus utilized to reevaluate robustness and reliability, in particular to indicate reliability values for the entire end result higher in the case of a match or at least substantially present match. If, however, the redundancy leads to different results being ascertained, the plausibility check result therefore shows a failed plausibility check, special measures can be provided. Thus, it can be provided that if at least one plausibility check fails, in particular if an implausibility condition is fulfilled, at least one associated measure is carried out, in particular an indication output to a user and / or a modification of the object information and / or a utility function using the object information. A user can therefore be warned that an inaccurate / failed detection of objects may be present, so that the user can, for example, himself take over the control of a medical technology component which uses the object information in a useful function. It is of course also conceivable to reflect this in the object information by modifying the same. For example, it can be provided, in particular in the case of safety-critical user functions, that the object information is changed to one or the object to which the highest safety standards are applied. An adaptation can also take place in the useful function by selecting higher safety standards for the object detected in an uncertain manner, for example. Such an adjustment can also be achieved implicitly by lowering the reliability values when an implausibility condition is fulfilled.In a specific example, in the case of at least partially autonomous movement of a component by the utility function, there can already be a distinction as to how specific object classes are treated. Thus, in some object classes, for example because of their doubts robustness against impacts, a significantly smaller safety distance can be provided than in other object classes which could be damaged, for example, which in turn can require a smaller safety distance than object classes relating to persons. Here, for example, in the event of a failure to plausibilize, i.e., in particular, fulfil the implausibility condition (which, generally speaking, can compare, for example, a distance measure between the tracking results and the results of the evaluation function with a threshold value), higher safety distances can be selected and the like, possibly also on the basis of a lowered reliability value.In a particularly expedient development of the present invention, it can be provided that the pose information and / or the identification information is stored together with the assigned camera data as training data set in which the pose information and / or the identification information defines the ground truth and is used for training at least one of the at least one trained evaluation function. If the pose information and / or the identification information are not sufficient to sufficiently define the ground truth, it can be provided that, in particular in the case of absent identification information of the object, the training data set with respect to the ground truth is supplemented by at least one information provided by the user. In such a configuration, it is conceivable, in particular when ascertaining the identification information and thus completely automatic determinableness of the ground truth, to record such training data sets in the actual use of the examination and / or treatment arrangement and to use them for further improving the trained evaluation function. However, it is also possible to use the tracking devices in a training phase, optionally also only in the training phase of machine learning, here specifically for annotation of camera data. In both cases, there is a significant improvement in the trained evaluation function and thus an improvement in the quality of the results thereof, the object information.As already mentioned, it can generally be provided that the tracking device is designed as a pose marking device, wherein the pose information is determined in the evaluation from camera data of the pose marking device. In this case, the pose marking device is expediently fastened or arranged on the object in such a way that at least one reference direction of the pose marking device, which reference direction can be unambiguously identified by the evaluation, corresponds to an excellent object direction of the object, wherein allocation information describing this correspondence is stored in the control device for use in the evaluation. In this way, ultimately a registration between the pose marking device (and thus a coordinate system of the object) and the control device is brought about with the corresponding evaluation unit, and corresponding information can be compared. In this case, for example, guidelines can be predefined as to how pose marking devices are to be fastened to respective objects of specific object classes which form the basis of the assignment information.Accordingly, it is expedient if, in the case of an actively measuring tracking device, the tracking device and the camera are registered with one another and / or calibrated to a jointly usable coordinate system. For this purpose, a calibration process can be carried out, for example, with a common calibration object. Since the camera and the object, and thus also the tracking device, move relative to one another, it is expedient to relate to an in particular fixed, commonly usable coordinate system, for example a static coordinate system of the examination and / or treatment arrangement and / or of a medical technology device thereof, for example an imaging device. If the camera is moved, position determination means for determining the position of the component (and thus of the camera) in the static coordinate system preferably exist on the part of the component carrying the camera. On the part of the object, the tracking device is in any case used to track its own position and orientation.In a specific embodiment, it can be provided that the camera and the control device form part of a control system of a medical technology device that is to be operated at least partially autonomously, wherein the object information is used to determine at least one control measure for the medical technology device. For example, a controlled movable component of an imaging device, in particular a mobile C-arm, and / or a mobile patient table, can be used as the medical technology device. In this case, at least one of the at least one camera can be provided on the component itself and / or at least one of the at least one camera can be directed externally to the component, in particular statically, onto an operating range of the component. For example, the utility function may be a guidance function for at least partially autonomous movement of the component, wherein object information may describe obstacles to be observed. For example, the utility function can use the object information to ascertain trajectories for the component or a portion of the component. In this case, for example, as already mentioned, different safety specifications (safety standards) can be used for different object classes, for example different safety distances, which can be greater for persons than for sensitive objects, wherein smaller safety distances can in turn be selected for less sensitive objects.In addition to the examples mentioned, the medical technology devices or components can, however, also be chosen differently. In particular, the invention can be applied to any controlled movable object, robots, support devices and the like.With regard to the tracking device, a large number of options have already been proposed in the prior art, which can also be used within the scope of the invention. In particular, tracking devices from the field of virtual reality tracking (VR tracking) can be used, i.e. at least one VR tracking device.If the tracking device comprises or is a pose marking device, it can be provided that the pose marking device marks the pose, in particular the orientation, on the basis of a shape of at least one marking body of the pose marking device and / or on the basis of a marking pattern. For this purpose, a large number of suitable approaches exist in the prior art (optical tracking, i.e. optical tracking), in particular also those which use components and components provided in any case on the object, and therefore allow an at least partially integrated configuration. If the object has, for example, a plate, rod or the like defining its orientation, this plate can have marking patterns which are easily recognizable in the camera data for improved optical visibility. A special shaping can also be used both for additional pose marking devices that can be attached to the object and for integrated pose marking devices.With regard to an actively measuring tracking device, this can use wireless tracking, inertial tracking, acoustic tracking, magnetic tracking as well as other tracking methods. In particular, the tracking device can have suitable sensor systems, for example receivers for wireless tracking signals, an inertial platform, transducers, magnetic field sensors and / or the like. In principle, tracking devices that use optical tracking can also be used.In addition to the method, the invention also relates to a medical examination and / or treatment arrangement, having at least one object, at least one camera for detecting the object, and a control device, which has an evaluation unit for ascertaining object information comprising an identification and / or position information of the object by evaluating camera data of at least one camera image of the camera, wherein the object is provided with a tracking device and a pose information describing the position and / or the orientation of the object can be ascertained by means of the tracking device, wherein the evaluation unit is designed to use the pose information for ascertaining the object information. In other words, the control device is designed to carry out the method according to the invention. All embodiments with respect to the method according to the invention can be transferred analogously to the examination and treatment arrangement according to the invention and vice versa, with which the already mentioned advantages can therefore likewise be obtained.In particular, in the case of the configuration of the tracking device as a pose marking device or comprising a pose marking device, it can be provided that the evaluation unit is designed to determine the pose information describing the position and / or the orientation of the object from camera data relating to the pose marking device and to use the pose information for determining the object information, in particular supporting it.The examination and treatment arrangement can be, for example, an angiography arrangement which has a C-arm X-ray device, the mobile C-arm of which is designed as an at least partially autonomously movable component, for which the control device and the tracking device form part of a control system. In addition to a mobile C-arm as a medical technology device, which can be controlled at least partially autonomously, a patient table as such a medical technology device or its component is also conceivable.The control device has in particular at least one processor and at least one storage means. Functional units can be formed by hardware and / or software in order to carry out steps of a method according to the invention, for example the evaluation unit already mentioned. Further functional units can comprise, for example, a plausibility checking unit, a training unit, a calibration unit and / or a useful function unit for executing a useful function using the object information. In the case of an actively measuring tracking device, wireless communication means are preferably present in order to transmit the pose information from the tracking device to the control device.As already mentioned, the at least one object can be, for example, an instrument table, an operating unit, a monitor unit, an infusion stand, an assistant, a patient, a patient support device, for example a patient table, and / or the like.A computer program according to the invention can be loaded directly into a storage means of a control device and has program means such that, when the computer program is executed on the control device, the latter is caused to carry out the steps of a method according to the present invention. The computer program can be stored on an electronically readable data carrier according to the present invention, which therefore has control information stored thereon, which comprises at least one computer program according to the invention and are configured such that, when the data carrier is used in a control device of an examination and treatment arrangement, said data carrier is configured to carry out a method according to the invention. The data carrier is in particular a non-transient data carrier, for example a CD-ROM.Further advantages and details of the present invention are evident from the exemplary embodiments described below and on the basis of the drawings. The following are shown: FIG. 1 schematically shows an examination and treatment arrangement according to the invention, FIG. 2 shows an exemplary object with a tracking device, FIG. 3 shows a flow chart of an exemplary embodiment of the method according to the invention, FIG. 4 is an explanatory diagram showing a first use of the pose information; and FIG. 5 is an explanatory diagram showing a second use of the pose information.FIG. 1 shows a schematic diagram of an examination and treatment arrangement 1 according to the invention. The examination and treatment arrangement 1 is in the present case, by way of example, an angiography arrangement which comprises an imaging device 2, specifically a C-arm X-ray device, with a mobile C-arm 3. The mobile C-arm 3 is an at least partially autonomously controllable medical technology device or component, which has controllable actuators, such as a chassis and rotary actuators for the C-arm. Thus, the mobile C-arm 3 can, for example, automatically and autonomously produce certain recording geometries for a patient on a patient table 4.In order to provide the necessary knowledge about its environment for the at least partially autonomous movement of the mobile C-arm 3, the examination and treatment arrangement 1 has a plurality of cameras 5, which record the operating range of the mobile C-arm 3 and thus also objects 6, 7, 8, 9 and 10 arranged within this operating range, including also the patient table 4. The cameras 5 can also be at least partially integrated into the mobile C-arm 3, the patient table 4 and / or the objects 6 to 10 and / or arranged on these.This also applies to the schematically shown control device 11, the components of which can also be arranged at least partially in or on the mobile C-arm 3, the patient table 4, the cameras 5 and / or objects 6 to 10.It should already be noted at this point that the mobile C-arm 3 is mentioned only by way of example for an at least partially autonomously controllable component and the following explanations can of course also be transferred to other such components or medical technology devices, for example the patient table 4, a treatment robot, a support device and the like.In addition to a storage means 13, the control device 11 has an evaluation unit 12 which evaluates the camera data of at least one camera image of the cameras 5 in order to determine object information relating to the objects 6 to 10 and the patient table 4. The object information describes, on the one hand, an object class of the respective object 6 to 10 or patient table 4 and its pose, i.e. position and orientation, as position information. They can also comprise further information, for example the extent, wherein it is however also conceivable to use virtual models which can describe the extent for the different object classes. With regard to the at least partially autonomous control of the mobile C-arm 3, the object classes are also assigned different safety requirements, for example different safety distances. For example, safety distances with respect to persons as objects, for example the patient, personnel or visitors, are significantly greater than with respect to robust objects, for example stable instrument tables.The object information is used in a user function unit 14 to determine a trajectory for the movement which meets the safety requirements, which trajectory can then be implemented accordingly by a control unit 15.In the present exemplary embodiment, a trained evaluation function is used by the evaluation unit 12 to evaluate the camera data in order to ascertain the object information. The trained evaluation function can comprise, for example, a neural network, for example a CNN. In order to train this, the control device 11 can optionally also comprise a training unit 16. In particular, as will be explained in more detail below, training data sets determined in the examination and treatment arrangement 1 itself can be used in this to train the trained evaluation function, in particular in an improving manner.For autonomous control processes, in particular in the context of medical applications, a high reliability and robustness of the environmental analysis with respect to objects 6 to 10, patient table 4 is important. In order to achieve improvements here, the objects 6 to 10 and also the patient table 4 are all provided in the present case with a tracking device 17, which can be fastened to the corresponding object 4, 6 to 10 and / or integrated therein. By means of the tracking devices 17, the pose of the corresponding assigned object 6 to 10, patient table 4, can be reliably determined and taken into account in the determination of the object information.The tracking devices 17 can be active-measuring tracking devices 17 that transmit the pose information, which is measured in particular by sensor, to the control device 11, in particular wirelessly, for which purpose a corresponding communication means 18 is shown on the part of the control device 11. Associated communication means of the tracking devices 17 are not shown for the sake of clarity. Additionally or alternatively, however, the tracking devices 17 can also be pose marking devices which are optically captured by the cameras 5, so that the evaluation unit 12 can determine the pose information by evaluating the camera data showing the pose marking device. Even if this is not shown in FIG. 1, the mobile C-arm 3 itself can of course also have a tracking device 17.FIG. 2 shows, by way of example, for an object 19, here an instrument table 20, a tracking device 17, which is arranged on a rolling frame 21 in a defined manner in such a way that a clear assignment of object directions of the object 19 to the pose information, and therefore reference directions of the tracking device 17, is possible, for which purpose, in an embodiment as a pose marking device, a corresponding assignment information item can be stored in the control device 11. In the case of actively measuring tracking devices 17, registration between the control device 11, in particular a coordinate system in which the camera data of the cameras 5 are evaluated, and the tracking device 17 can be achieved, for example, by a calibration process, for which purpose the control device 11 can have a calibration unit, not shown in detail here. A static coordinate system can be used here expediently.As merely indicated in FIG. 2, the tracking device 17 can have an identification feature 22, which can be identified electronically or, in an embodiment as a pose marking device, optically recognizable for the cameras 5 and can be assigned unambiguously to the object 19 or at least to its object class. In the case of an actively measuring tracking device 17, the identification feature 22 can be transmitted with the pose information via the communication means 18, which is expedient in any case for identifying the respective transmitter of the plurality of tracking devices 17. In the case of a pose marking device, the identification feature 22 can be recognized by the evaluation unit 12. By means of the identification feature, identification information of the respective object 6 to 10, 19, patient table 4 assigned to the pose information can be determined, which identification information can contain at least the object class and can likewise be used gainfully in the determination of the object information.In addition to the instrument table 20 and the patient table 4, other objects 6 to 10 are also conceivable. Thus, the object 6 can be, for example, an infusion stand, the object 7 can be, for example, a treatment robot, the object 8 can be, for example, a treatment person, the object 9 can be, for example, an operating unit, and the object 10 can be, for example, a monitor unit.FIG. 3 shows a general flow chart of an embodiment of the method according to the invention. In a step S 1, camera data of the objects 6 to 10, 19, patient table 4 are recorded in order to determine in a step S 2 by the trained evaluation function in the evaluation unit 12 for determining an object class (as identification of the object 6 to 10, 19, patient table 4) and a position and orientation of the object 6 to 10, 19, patient table 4 as position information, each with associated reliability values. Already in this case, the pose information (and optionally the identification information) determined in a step S 3 can be used in a expedient manner. The pose information is further used in a plausibility checking step S 4 before the final object information (containing at least object class (identification) and pose (position information)) is passed on to the utility function in step S 5, where it is used by the utility function unit 14.Plausibility checking step S 4 may be carried out in control device 11 by a plausibility checking unit 23 and is explained in greater detail below.FIG. 4 shows how the pose information can already be used early for enriching the input data 24 for the trained evaluation function 25. This is because not only the camera data 26 but also the pose information, which notifies the evaluation function 25 where an object 6 to 10, 19, patient table 4 is expected in which orientation, are provided there as input data 24 of the trained evaluation function 25, which simplifies and improves the detection.According to FIG. 5, the trained evaluation function 25 produces output data 28 which, as already mentioned, comprise the object class (with an associated reliability value) and a position and orientation, that is to say even a pose, as position information, again with an associated reliability value. In plausibility checking step S 4, the pose according to pose information 27 is now compared with the pose according to the position information of output data 28 for plausibility checking, and optionally the identification information is compared with the object class, likewise for plausibility checking. If an implausibility condition indicates failure of the plausibility check, at least one measure is carried out in a step S 4 a, in the present case the output of a warning to an operator who may wish to take over the control itself, and an adaptation of the object information 29 ultimately to be output, comprising at least the (optionally adapted) object class and the (optionally adapted) position information, for example by assigning an object class with higher safety requirements to the object table 4 which may be incorrectly identified 6 to 10, 19, in order to increase the safety. A measure can also influence the useful function and / or reduce reliability values. If the implausibility condition is not fulfilled, thus plausibility is given, the reliability value which is associated with the object class and the position information is increased since this indicates a higher reliability of this object information 29.In addition or as an alternative to the plausibility check by comparison described here, the redundancy can also be used differently, at least with regard to the pose, for example in a combination, for which purpose a reliability value can also be assigned to the pose information 27. Then, the result can be selected, for example, with respect to the reliability values.Finally, it is also possible, in particular when determining identification information, to compile training data sets for the training of the trained evaluation function 25, for example by the training unit 16, by means of the pose information 27 (and the identification information). In this case, a reliable ground truth is available by the pose information and the identification information. In particular, it is therefore conceivable to use the tracking devices 17 only temporarily in order to determine reliable training data sets and thus improve the quality of the output data 28 of the trained evaluation function 25 in such a way that the redundancy is no longer necessary (or at least not necessary for each object class).Although the invention has been illustrated and described in more detail by the preferred exemplary embodiment, the invention is not restricted by the disclosed examples and other variations can be derived therefrom by the person skilled in the art without departing from the scope of protection of the invention.Regardless of the grammatical sex of a certain term, individuals with male, female or other sex identity are included.

Claims

Method for determining object information in a medical examination and / or treatment arrangement (1) with at least one object (6, 7, 8, 9, 10, 19), wherein the object (6, 7, 8, 9, 10, 19) is detected by means of at least one camera (5) and an object information (29) comprising an identification and / or position information of the object (6, 7, 8, 9, 10, 19) is determined by means of a control device (11) by evaluating camera data (26) of at least one camera image of the camera (5), characterized in that the object (6, 7, 8, 9, 10, 19) is provided at least temporarily with a tracking device (17) and a pose information (27) describing the position and / or the orientation of the object (6, 7, 8, 9, 10, 19) is determined by means of the tracking device (17), which pose information is used for determining the object information (29), in particular supporting it.Method according to Claim 1, characterized in that a trained evaluation function (25) is used for evaluating the camera data, said function ascertaining at least one object class of the object (6, 7, 8, 9, 10, 19) with an associated item of reliability information as output data (28).Method according to Claim 2, characterized in that the trained evaluation function (25) uses at least part of the pose information (27) as additional input data (24).Method according to Claim 2 or 3, characterized in that, in addition to the object class, at least one of the at least one trained evaluation function (25) also determines an orientation of the object and / or position of the object (6, 7, 8, 9, 10, 19) as position information, which orientation is compared with the orientation and / or position of the object (6, 7, 8, 9, 10, 19) in accordance with the pose information (27) in order to plausibly check the evaluation result of all trained evaluation functions (25).Method according to one of Claims 2 to 4, characterized in that the tracking device (17) has an identification feature (22) which is assigned to the object (6, 7, 8, 9, 10, 19) provided with it and by means of which an identification information item of the object (6, 7, 8, 9, 10, 19) is determined, which identification information item is used for, in particular additional, plausibility checking of the determined object class.Method according to Claim 4 or 5, characterized in that the reliability value is adapted as a function of the at least one plausibility check result and / or, if at least one plausibility check fails, in particular if an implausibility condition is fulfilled, at least one associated measure is carried out, in particular an indication output to a user and / or a modification of the object information (29) and / or of a useful function using the object information (29).Method according to one of Claims 3 to 6, characterized in that the pose information (27) and / or the identification information is stored together with the assigned camera data (26) as a training data set in which the pose information (27) and / or the identification information defines the ground truth, and is used for training at least one of the at least one trained evaluation function (25).Method according to Claim 7, characterized in that, in particular in the case of absent identification information of the object (6, 7, 8, 9, 10, 19), the training data set relating to ground truth is supplemented by at least one information provided by the user.Method according to one of the preceding claims, characterized in that the tracking device (17) is designed as a pose marking device, wherein the pose information (27) is determined in the evaluation from camera data of the pose marking device.Method according to Claim 9, characterized in that the pose marking device is fastened or arranged on the object (6, 7, 8, 9, 10, 19) in such a way that at least one reference direction of the pose marking device, which reference direction can be identified unambiguously by the evaluation, corresponds to an excellent object direction of the object (6, 7, 8, 9, 10, 19), wherein allocation information describing this correspondence is stored in the control device (11) for use in the evaluation.Method according to one of the preceding claims, characterized in that, in the case of an actively measuring tracking device (17), the tracking device (17) and the camera (5) are registered with one another and / or calibrated to a commonly usable coordinate system.Method according to one of the preceding claims, characterized in that the camera (5) and the control device (11) form part of a control system of a medical technology device which is to be operated at least partially autonomously, wherein the object information (29) is used to determine at least one control measure for the medical technology device.Medical examination and / or treatment arrangement (1) comprising at least one object (6, 7, 8, 9, 10, 19), at least one camera (5) for detecting the object (6, 7, 8, 9, 10, 19) and a control device (11) comprising an evaluation unit (12) for determining object information (29) comprising an identification and / or position information of the object (6, 7, 8, 9, 10, 19) by evaluating camera data (26) of at least one camera image of the camera (5), characterized in that the object (6, 7, 8, 9, 10, 19) is provided with a tracking device (17) and a pose information (27) describing the position and / or the orientation of the object (6, 7, 8, 9, 10, 19) can be determined by means of the tracking device (17), wherein the evaluation unit (12) is designed to use the pose information (27) to ascertain the object information (29).Computer program which, when executed on a control device (11) of a medical examination and / or treatment arrangement (1), causes the latter to carry out the steps of a method according to one of Claims 1 to 12.Electronically readable data carrier on which a computer program according to claim 14 is stored.

Images