Data-to-sound interactive feedback

Abstract

A first aspect of the invention relates to a method for representing biomedical image data, the method comprising the steps: providing biomedical image data; assigning a region of interest (R) of the biomedical image data to a physical vibration model (116) of an acoustic model; mapping a data feature of the region of interest (R) to a parameter of the physical vibration model (116); and generating a sound output (112) of the acoustic model based on vibrations of the physical vibration model (116). Another aspect of the invention relates to a computer program. Yet another aspect of the invention relates to a system (20) for generating a sound output for representing biomedical image data.

Description

[0001] Technische Universitat Miinchen

[0002] Data-to-Sound Interactive Feedback

[0003] FIELD OF THE INVENTION

[0004] The present invention is in the field of data analysis and refers, in particular, to a data-to-sound interactive feedback approach. This approach is implemented by a method for generating a sound output from biomedical image data as well as by a related computer program and a related system. The approach may also be used for training a machine learning algorithm.

[0005] BACKGROUND OF THE INVENTION

[0006] The interpretation and analysis of data sets by human users is typically limited to visual perception, be it by direct visual inspection of the data or by visualization of the data by or on graphical representation means, such as screens or the like. This imposes upon users the limitation of not being able to use visual perception for other purposes while analyzing said (visualized) data.

[0007] In some fields, such as the medical field, this is a major limitation. For example, a surgeon may have to choose, during a surgical treatment, between concentrating his / her visual perception on data representation and / or data interpretation means, such as a screen showing a scanner or sensor result, or on the body of the patient receiving the surgical treatment.

[0008] For example in retinal surgery, fine anatomical structures need to be manipulated with micron level precision, while commonly an operating microscope provides primary visual access to the surgical site at the backside of the eye. An example of such a complex procedure is the peeling of an epiretinal membrane (ERM), an on average 6opm thin membrane that forms on top of the retina and, if left untreated, can lead to decreased central visual acuity. Before peeling this membrane with a microsurgical forceps, typically, a synthetic dye is administered to the retina. This dye stains the inner limiting membrane (ILM) and enables the surgeon to visually distinguish the ERM located on top of it. A challenging, yet crucial aspect of the subsequent procedure is to identify a suitable region to grasp the membrane and initiate the peeling. While the ERM is typically attached to the retina, a gap as small as 45 ± 8 pm between the ERM and the retina can occur, especially in patients where a large ERM stiffens and pulls the retina. Precise visualization and quantification of subtle elevations in the ERM are in most cases not possible through the microscopic view. However, optical coherence tomography (OCT) has been shown to provide the required micron-level resolution. Integrated with surgical microscopes, intraoperative OCT (iOCT) has shown benefits for surgical decision-making and surgery outcomes in ERM peeling procedures. Yet, a fundamental question regarding the integration of iOCT together with surgical microscopes is to find adequate channels for the simultaneous presentation of multimodal information. In currently available systems, the microscopic view and iOCT images are typically presented in juxtaposition. Moving the gaze between different visual monitors during such delicate tasks however increases the cognitive load on surgeons and affects their proprioception, potentially interfering with the surgical flow and slowing down the procedure. Therefore, more advanced and intuitive ways of providing multimodal information without distracting the focus during critical surgical phases need to be developed.

[0009] SUMMARY OF THE INVENTION

[0010] According to the invention, this problem is overcome with a method according to claim 1, a computer program according to claim 20 and a system according to claim 21. The dependent claims refer to particularly preferable embodiments of the invention.

[0011] A first aspect of the invention relates to a method for representing biomedical image data, the method comprising the steps: providing biomedical image data; assigning a region of interest of the biomedical image data to a physical vibration model of an acoustic model; mapping a data feature of the region of interest to a parameter of the physical vibration model; and generating a sound output of the acoustic model based on vibrations of the physical vibration model.

[0012] The invention allows to present information about the biomedical image data, in particular data features, as audible sound output. Thereby, a user, such as a surgeon, does not need to look upon a display to receive said information. Hence, a cognitive load is lowered, and decision-making of the user is improved.

[0013] The biomedical image data may comprise one or more biomedical images. The biomedical image data may be or may comprise one-dimensional data, two-dimensional data, three- dimensional data, and or higher dimensional data. In one or more preferred examples, the biomedical image data comprises one or more of an OCT-scan, a CT-scan, a PET scan, a SPECT scan, and an MRI scan. The OCT-scan may be or may comprise an OCT B-scan and / or an OCT A-scan.

[0014] In one or more specific examples, the biomedical image data is or comprises an OCT B-scan and / or the region of interest is or comprises an OCT A-scan.

[0015] Providing biomedical image data may comprise capturing and / or obtaining one or more biomedical images. In some embodiments, providing biomedical image data and / or one or more biomedical images may comprise using an OCT scanner, a CT scanner, a PET scanner, a SPECT scanner, an MRI scanner or the like.

[0016] Optical coherence tomography (OCT) may refer to an imaging technique based on interferometry with short-coherence-length light to obtain micrometer-level depth resolution. OCT may employ transverse scanning of a light beam to form two- and three-dimensional images from light reflected from biological tissue or other scattering media.

[0017] For instance, OCT is used in ophthalmology. OCT may also be used e.g. in cardiology, neurovascular applications, oncology, dermatology dentistry, as well as for other purposes.

[0018] Depending on the application and / or on the use case, the biomedical image data, e.g. an OCT- scan, may have a specific dimension. For instance, e.g. depending on the application and / or on the use case, the biomedical image data may be one-dimensional, two-dimensional, three- dimensional and / or higher-dimensional. One or more dimensions maybe a spatial dimension and / or a dimension in space. It may be provided that one dimension may be a temporal dimension. A spatial dimension may refer to a dimension in space.

[0019] It maybe provided that an OCT-scan and / or an OCT image may comprise one or more OCT A- scans. An OCT A-scan may be or may comprise substantially one-dimensional data. Alternatively or additionally, an OCT-scan and / or an OCT image may comprise one or more OCT B-scans, which maybe or may comprise two-dimensional data. An OCT B-scan maybe or may comprise more than one OCT A-scan. An OCT B-scan may be generated and / or formed by two or more OCT A-scans.

[0020] Alternatively or additionally, an OCT-scan and / or an OCT image may comprise or may be a volumetric OCT. A volumetric OCT may be three-dimensional, and / or may refer to a three- dimensional image, which may be or may comprise one or more OCT A-scans and / or one or more OCT B-Scans. A volumetric OCT may be generated and / or formed by two or more OCT B-scans.

[0021] Alternatively or additionally, an OCT-scan and / or an OCT image may comprise or may be a four-dimensional OCT. A four-dimensional OCT may comprise a time-series of volumetric OCT.

[0022] In some embodiments, one-dimensional biomedical image data may have one spatial dimension. Alternatively or additionally, in some embodiments, one-dimensional biomedical image data may have one temporal dimension, and / or may correspond to a single position in space for different points in time.

[0023] In some embodiments, two-dimensional biomedical image data may have two spatial dimensions. As an example, two-dimensional biomedical image data maybe or may comprise at least one OCT B-scan. Alternatively or additionally, in some embodiments, two-dimensional biomedical image data may have one spatial dimension and one temporal dimension, and / or may correspond to one-dimensional data in space for different points in time. As an example, two-dimensional biomedical image data may be or may comprise at least one OCT A-scan at different points in time.

[0024] In some embodiments, three-dimensional biomedical image data may have three spatial dimensions. As an example, three-dimensional biomedical image data may be or may comprise at least one volumetric OCT-scan. Alternatively or additionally, in some embodiments, three-dimensional biomedical image data may have two spatial dimension and one temporal dimension, and / or may correspond to two-dimensional data in space for different points in time. As an example, three-dimensional biomedical image data may be or may comprise at least one OCT B-scan at different points in time.

[0025] For instance, a dimension higher than three may include time as further dimension. For instance, a four-dimensional biomedical image data, e.g. an OCT-scan, may comprise multiple datasets, images and / or scans in three spatial dimensions at different points in time. As an example, four-dimensional biomedical image data may be or may comprise at least one volumetric OCT-scan at different points in time.

[0026] In some embodiments, e.g. when an elevated ERM is to be identified and / or an ERM peeling is to be carried out, it may be provided that the biomedical image data may be two-dimensional and / or comprise one or more OCT B-scans. However, it maybe provided that when an elevated ERM is to be identified and / or an ERM peeling is to be carried out, the dimension of the biomedical image data may be different, e.g. three-dimensional, without necessarily being limited thereto.

[0027] The dimensions of the biomedical image data are not necessarily limited to these examples. It should be noted that the method can be easily applied to all kinds of formats, e.g. including but not limited to OCT A-scans, OCT B-scans, volumetric OCT and 4D OCT.

[0028] Similarly, it may be provided that when the biomedical image data comprises one or more of a CT-scan, a PET scan, a SPECT scan, and an MRI scan, the dimension of the biomedical image data may depend on the use case and / or on the application, and / or on the type or kind of the biomedical image data. The dimension of the biomedical image data may be one-dimensional, two-dimensional, three-dimensional, four-dimensional and / or higher dimensional.

[0029] In some embodiments, a CT-scan may be a volumetric CT-scan and / or may be three- dimensional.

[0030] In some embodiments, the biomedical image data may comprise data, images and / or scans, at least two or more of which may have different dimensions.

[0031] In some embodiments, the method may be used to transform structural patterns of the ERM and the retinal tissue into discernible auditory textures. In some embodiments, the method may take an OCT A-scan as input and assign a region of interest within a local attenuation map to an acoustic model capable of producing sound grains. In some embodiments, the method may take an OCT A-scan as input, compute a local A-scan attenuation map, and assign a region of interest within the local attenuation map to an acoustic model capable of producing sound grains. A sound output corresponding to the sequence of retina-informed sound signals may be achieved by the concatenation of sound grains generated from individual excitations of the acoustic model for different regions of interest.

[0032] In some embodiments, the system maybe used and / or combined with recent 4D iOCT systems that enable continuous scanning of a surgical site, such as e.g. a retina. It may be provided to use methods known from the art to track a surgical instrument, such as e.g. a microsurgical forceps, and use it as an input device to select a region of interest and / or an A-scan.

[0033] The region of interest may be a part of the biomedical image data. The region of interest may be a cut through the biomedical image data. The region of interest may be an intersection of the biomedical image data. Alternatively or additionally, the region of interest may cover the whole biomedical image data. The region of interest may be one-dimensional or line-like, two-dimensional or plane-shaped, and / or three-dimensional. However, the number of dimensions of the region of interest is not necessarily limited to three. It may be provided that the number of dimensions corresponds to a number of spatial directions in which the region of interest may extend or may be arranged. The number of dimensions of the region of interest may be equal to the number of dimensions of the biomedical image data and / or the biomedical image. The number of dimensions of the region of interest maybe smaller than the number of dimensions of the biomedical image data and / or the biomedical image.

[0034] Multiple regions of interest maybe provided. Different regions of interest may overlap, or may nor overlap. Different regions of interest maybe separated from each other by some distance. Neighboring regions of interest may overlap, or may not overlap. Neighboring regions of interest may be separated from each other by some distance. It may be provided that the biomedical image data, fully or at least partially, may be separated into at least two or more regions of interest.

[0035] When assigning a region of interest to a physical vibration model, the corresponding region of interest maybe associated with a respective physical vibration model, and / or maybe modeled by a respective physical vibration model. Each region of interest may be assigned to a respective physical vibration model, and / or may be associated with or modeled by a respective physical vibration model. Different regions of interest may be represented by different physical vibration models.

[0036] The acoustic model may comprise the physical vibration model. The acoustic model may comprise multiple physical vibration models. The acoustic model may be configured to generate sound output from the vibration of a physical vibration model. The acoustic model may be configured to generate a sound grain from a physical vibration model.

[0037] The physical vibration model may comprise nodes, which when vibrating may vibrate, oscillate and / or be displaced depending on the parameters of the physical vibration model. A node may be or may comprise an oscillating node, which may vibrate, oscillate and / or be displaced. The vibrations of the physical vibration model, and / or of the nodes, can be translated into acoustic properties of the sound output by extracting cumulative signals resulting from vibrations of all nodes of the physical vibration model.

[0038] A node may be or may comprise a fixed node, which may be configured to not move and / or to be not displaced. The physical vibration model may comprise at least two nodes coupled and / or connected together. The physical vibration model may comprise a plurality of nodes coupled and / or connected together. The position and / or the displacement of the respective nodes as a function of time may be given by one or more waves, such as sine waves or cosine waves, and / or by a superposition of two or more waves.

[0039] “Vibrating” or “vibration” may refer to a position and / or to a displacement of one or more nodes around a respective equilibrium position. A “vibration” of a physical vibration model may refer to vibrations, oscillations and / or displacements of nodes of the physical vibration model.

[0040] When vibrating, there may be a force, such as e.g. due to an external force, due to a spring and / or due to a dampener, which may act on a node and cause an acceleration, movement and / or displacement of the node. The displacement of one node may cause a force such as e.g. a spring force acting on another node coupled and / or connected to said node. An external force acting on one node may be propagated by the coupling and / or by the connection of respective nodes. It maybe provided that a “vibration”, “vibration signal” or “vibrating” may refer to the one or more waves, or to a superposition of at least two or more waves, as given e.g. from the position and / or from the displacement of the nodes as function of time.

[0041] A data feature may comprise a physical property, such as e.g. a temperature or a density. A data feature may comprise an intensity, a refractive index, or the like. A data feature may comprise a measurement, a sensor value and / or sensor data. A data feature maybe calculated and / or determined from one or more of another value or variable, measurement, sensor value and / or sensor data.

[0042] It may be provided that a data feature may be associated with a pixel, a voxel, a region and / or a position, of the biomedical image data. It may be provided that the biomedical image data may comprise one or more biomedical images, and data features associated with said one or more biomedical images. For instance, the biomedical image data may comprise one or more OCT scans, such as e.g. one or more OCT B-scans and / or OCT A-scans, and may further comprise temperature data corresponding to a temperature at the different positions of the biomedical image data. A data feature may be or may comprise an “overlay” over a biomedical image.

[0043] Alternatively or additionally, a data feature maybe or may comprise one or more data points, pixels or voxels of a biomedical image. In some embodiments, a data feature may be or may comprise e.g. an intensity. It may be provided that the biomedical image substantially corresponds to the data features. It may be provided that the data feature comprises or corresponds to measurements from which the biomedical image was generated, e.g. intensities or the like.

[0044] When mapping a data feature of the region of interest to a parameter of the physical vibration model, values of the parameters may be chosen and / or determined depending on the data features, and / or depending on values of the data features.

[0045] The mapping may be unsupervised. The mapping may be carried out by a machine learning model, such as e.g. by an artificial neural network or by a deep learning model, but is not necessarily limited thereto. In some embodiments, the mapping may be carried out by one or more mapping functions.

[0046] The mapping between the data features and the parameters may be a one-to-one map, but this is not necessarily the case. The mapping may also be a many-to-one map, for example where a complete subset of data points within a biomedical image, e.g. of a tissue and / or values of physical properties thereof, may be associated to one specific parameter.

[0047] In one or more preferred examples, at least one data feature is isomorphically mapped to at least one parameter. Isometrically mapping may comprise mapping an ordered value range of the at least one of the one or more data features to an ordered value range of at least one of the one or more parameters. For example, if the data feature comprises intensity values, the intensity range covered by said intensity values may be isomorphically mapped to the parameters, such that increasing intensities may be mapped to increasing values of the parameters and decreasing intensities maybe mapped to decreasing values of the parameters or the other way around. By being based on an isomorphism for at least one of the one or more data features, the parameters maybe able to preserve the structure as given by the data features and to reflect it accordingly in the sound output.

[0048] In some embodiments, mapping may include mapping a range of intensity values of the biomedical image data to the ranges of parameters of the physical vibration model. For instance, intensity values may be mapped to parameters such as e.g. a stiffness, a mass, or a damping coefficient. Alternatively or additionally, it may be provided to map a function of the data feature, e.g. a function of the intensity, to the parameters. For example, a function may be or may comprise an average or a maximum value in a suitable window and / or segment. In some embodiments, the average may be a moving average. The vibrations of the physical vibration model may depend on the parameters of the physical vibration model. Thereby, the sound output may depend on the one or more data features. The sound output maybe different for different data features, and / or maybe different for different values of the parameters and / or data features.

[0049] This can be accomplished by collecting vibrations of one or more nodes corresponding to a plurality of vibration signals and by combining the vibration signals into a composite signal, for example as a sum of the individual vibration signals. The individual vibration signals may for example be sine wave signals. The vibration signals can be combined into the composite signal using a sound synthesis method known as additive synthesis, for example as a Fourier series or as an inverse Fourier transform, to cite some implementation examples. Thereby, the vibrations of each node may contribute to the final sound profile, i.e., to the shape of the resulting sound wave, even though the individual contributions may be moderate per se.

[0050] It may be provided that the dimension and / or the shape of the structure of the physical vibration model may correspond to the dimension and / or to the shape of the region of interest. Herein, a “structure” may mean the arrangement of connections and / or couplings between the nodes. However, the structure is not necessarily limited thereto, and the dimension and / or the shape of the structure of the physical vibration model may differ from the dimension and / or from the shape of the region of interest.

[0051] For instance, for a physical vibration model including a mass parameter for a data set corresponding to a biomedical image, the mass parameter corresponding to a given data point, pixel or voxel may take a value proportional to a value of a data feature associated to said data point, pixel or voxel, for example proportional to a value of an intensity or a density associated to said data point, pixel or voxel. In such cases, a mass modelled by the mass parameter of the model tends to vibrate or oscillate at a frequency inversely proportional to said value of the data feature, e.g., of said intensity or density value. Accordingly, if different mass values are used to model data points, pixels or voxels in a region of interest in a biomedical image with varying values of the data feature, e.g., of said intensity or density value, different vibration frequencies will result. For example, the resulting frequency distribution may be e.g. specific of a retina being analyzed and may be further specific when an elevated ERM is present. For example, the resulting frequency distribution may be e.g. specific of a tissue being analyzed and may be specific of a given tissue type, for example, of tumorous or non-tumorous tissue.

[0052] The physics of the physical model corresponding to the sound model is hence correlated, via the data features of the data set, to the physics of a physical system represented by the data set. As a result, the acoustic properties of the sound output encode information about the physical properties of the physical system represented by the data set, which can be explored by interacting with the physical model. The acoustic model may thereby maintain the spatial and geometrical characteristics encoded in the data set and the temporal flow of the interactions.

[0053] The sound output may be or may correspond to an audible signal, i.e., to an acoustic signal perceivable by a person. However, the sound output may also correspond to an electronic audio signal encoding a corresponding audible signal, i.e., an electronic signal configured for being provided to some piece of electronic equipment, for example an amplifier and / or a loudspeaker, and for triggering the generation of a corresponding audible signal. The previously mentioned electronic audio signal maybe an analogue signal or a digital signal.

[0054] At least one of the one or more acoustic properties of the sound output may correspond to one of pitch, pulsing frequency, duty cycle, loudness, and tone colour. “Pitch” may refer herein to a sound frequency and is measurable in Hz. “Pulsing frequency” may refer herein to an intermittency frequency of the acoustic output. It may be provided that the acoustic output, when being audible sound or used for producing audible sound, can be perceived by the human ear as a sequence of sounds regularly repeating with periods corresponding to (the inverse of) the pulsing frequency. For example, a pulsing frequency of 1 Hz may refer to the fact that the acoustic output maybe an intermittent signal alternating active periods of non-zero amplitude with inactive periods of zero amplitude periodically repeating in intervals of 1 s. The alternating active periods of non-zero amplitude and the inactive periods of zero amplitude may have the same duration (for example 0,5 s and 0,5 s, respectively, for a pulsing frequency of 1 s1) or different durations (for example 0,8 s and 0,2 s, respectively, for a pulsing frequency of 1 s1). “Duty cycle” may refer herein to a fraction of one period of an intermittent (pulsing) acoustic output over which the signal is active, i.e., may have a non-zero amplitude. A duty cycle of 0,5 may correspond to a pulsing signal in which the alternating active periods of non-zero amplitude have the same duration as the inactive periods of zero amplitude (for example 0,5 s and 0,5 s, respectively, for a pulsing frequency of 1 Hz). A duty cycle of 1 may correspond to a continuous non-intermittent signal. Notably, it may be provided that the pitch, the pulsing frequency and / or the duty cycle of an auditory signal can be independently and / or simultaneously perceived by the human ear. For a given auditory signal, the human ear can for example identify when a pulsing frequency or a duty cycle of the auditory signal is varying (i.e., is pulsing slower or faster) and can further independently identify, possibly at the same time, whether a pitch of the signal is varying (i.e., is becoming higher or lower). “Tone colour”, also named “timbre”, may refer herein to an acoustic property that distinguishes different types of sound production, such as choir voices, sound resulting from mechanical action with different materials, or different musical instruments. The sound output may contain one or more sound grains. A sound grain may be associated to a region of interest. When playing the sound output, the sound grains maybe played one after the other, depending on some ordering. The sound output may vary over time depending on the data features of regions of interest.

[0055] According to the invention, the sound output is generated using the parameters of the physical vibration model to which the at least one of the one or more data features are mapped. Thus, the method of the first aspect of the invention may result in the generation of a sound output that encodes the data features of the dataset. Accordingly, the method of the invention may allow providing audible information about the data features of the data set. This audible information can convey knowledge about the spatial structure, the time evolution and / or the physical condition of a physical system represented by the data set. Since the information is suitable for being perceived via audition, the perception of such information can easily be combined with the simultaneous perception of further information, in particular via visual perception. This allows enhancing the perception of information by the receptor of the information.

[0056] In one or more preferred examples, the method further comprises the steps: determining a local attenuation map from the provided biomedical image data; and when assigning a region of interest of the biomedical image data, the region of interest is assigned as a region of interest of the local attenuation map, such that when mapping a data feature, a data feature of the region of interest of the local attenuation map is mapped to a parameter of the physical vibration model.

[0057] “Attenuating” may comprise filtering the biomedical image data and / or the region of interest, and / or may comprise applying a filter to the biomedical image data and / or to the region of interest. The dimension and / or the shape of the local attenuation map may correspond to the dimension and / or to the shape of the attenuated obj ect, such as to the dimension and / or to the shape of the biomedical image data and / or to the region of interest. In other words, the local attenuation map is not necessarily two-dimensional, but may also be one-dimensional or linelike, three-dimensional, or higher dimensional.

[0058] In one or more examples, when determining a local attenuation map, a local attenuation of at least one OCT A-scan is performed. Performing a local attenuation of on OCT A-scan may comprise filtering the OCT A-scan.

[0059] Attenuating and / or determining a local attenuation map may lead to more distinctive sound textures. In one or more examples, the biomedical image data is provided in real time. Alternatively or additionally, the sound output is generated in real time. This may allow for an interactive use of the method according to the first aspect of the invention. F or instance, the method may assist peeling an ERM from a retina in real time. However, the method is not necessarily limited thereto.

[0060] In one or more examples, the physical vibration model comprises two or more nodes. It may be provided that the nodes can be coupled in sequence. When the nodes are coupled in sequence, the resulting structure of the physical vibration model maybe substantially line-like, and / or the physical vibration model maybe one-dimensional. When the nodes are coupled in sequence, it may be provided that each node of the physical vibration model may be coupled to at least one other node, and at most to two other nodes. When the nodes are coupled in sequence, it maybe provided that the vibrations are substantially one-dimensional.

[0061] For example, a region of interest may be modelled as a physical vibration model having a string-like connection of nodes and / or where the nodes are coupled in sequence. Each node of the physical vibration model maybe coupled and / or connected to at least one other node, and at most to two other nodes. The resulting structure may be substantially one-dimensional. It may be provided that vibrations of the nodes are substantially in one direction, e.g. along a line and / or along the string.

[0062] In some embodiments, the nodes may be coupled in sequence and / or may have a string-like coupling when the region of interest is or comprises an OCT A-scan. However, the physical vibration model may have a different structure and / or the nodes may be coupled and / or connected differently, when the region of interest is or comprises an OCT A-scan.

[0063] For example, a region of interest may be modelled as a cubical grid of nodes, with each mass representing one data point of the region of interest, for example one voxel, and with neighboring nodes being mutually coupled by a spring-like coupling having a spring constant and / or a damper having a damping coefficient. For instance, the spring constant and / or the damping coefficient may correspond to an average of a value of one physical property represented by the region of interest, for example a density and / or an intensity, over the two mutually coupled data points.

[0064] In some embodiments, the nodes maybe arranged in a cubical grid, when the region of interest is or comprises a CT scan and / or a CT scan volume. However, the physical vibration model may have a different structure and / or the nodes maybe coupled and / or connected differently, when the region of interest is or comprises a CT scan and / or a CT scan volume.

[0065] When the region of interest is substantially one-dimensional and / or line-like, it may be provided that the physical vibration model may have a string-like connection of nodes. When the region of interest is substantially three-dimensional, it may be provided that the physical vibration model may have nodes arranged in a cubical grid. In some embodiments, the structure of the physical vibration model may correspond to the dimension of the region of interest. However, the invention is not necessarily limited thereto.

[0066] In one or more preferred examples, an external force is applied to at least one node of the physical vibration model. Alternatively or additionally, at least one node may be displaced at a reference time, such that a spring force and / or a damping force maybe exerted.

[0067] In one or more examples, the external force is a parameter of the physical vibration model. Alternatively or additionally, the external force may correspond to a virtual instrument. In some embodiments, the virtual instrument may represent a physical instrument such as e.g. a microsurgical forceps. It may be provided that the virtual instrument tracks the physical instrument. It may be provided that the region of interest may coincide with a position of the virtual instrument and / or of the physical instrument, and / or that the region of interest is assigned according to a position of the virtual instrument and / or of the physical instrument, but is not necessarily limited thereto.

[0068] In some examples, a user, e.g. a surgeon, may use the physical instrument, e.g. a microsurgical forceps, to perform some procedure. Alternatively or additionally, in some examples, the virtual instrument may be or may comprise a pointer, such as e.g. but not limited to a mouse pointer. For instance, it may be provided that a user may use the pointer to determine the region of interest. For instance, the biomedical image data may be displayed, and the pointer may select the region of interest in the biomedical image data. However, these examples are not necessarily limiting.

[0069] In some embodiments, the external force may depend on a force exerted by the physical instrument on a body, such as e.g. a tissue or a retina, but is not necessarily limited thereto. In some embodiments, the external force is independent of the region of interest, of the virtual instrument and / or of the physical instrument. In some embodiments, the external force may be time-dependent. In some embodiments, the external force may be predetermined and / or maybe chosen. Thereby, the sound output may vary in real time, depending e.g. on a user and / or on an interaction of the user. Thus, the user and / or the surgeon may receive in real time feedback on their / his / her actions. In an example, the position of an elevated ERM maybe indicated by the sound output. For instance, a position of a microsurgical forceps may be tracked, and when the microsurgical forceps is in a region of an elevated ERM, a distinctive sound output may be generated. Identifying the elevated ERM and / or its position may be facilitated. However, the invention is not necessarily limited to this.

[0070] In one or more preferred examples, at least one node is or comprises a listener node, wherein the sound output of the acoustic model is based on vibrations of the listener node. Thus, the sound output may be based on the displacement of the listener node. As the listener node is coupled to at least one other node, and thereby indirectly to all other nodes, the vibrations of the listener node may depend on the vibrations of all other nodes of the physical vibration system.

[0071] It may be provided that at least one node may be or may comprise an absorption node, and / or may be highly damped. For instance, the damping coefficient of a damper of an absorption node, and / or of a damper coupled between two nodes, may be relatively high compared to other damping coefficients. The absorption node may reduce noise, and / or may contribute to a clearer contrast in the sound output. The absorption node may improve the stability of the physical vibration model.

[0072] In one or more preferred examples, the parameters of the physical vibration model comprise one or more of: one or more spring constants, one or more masses and one or more damping coefficients.

[0073] In one or more preferred examples, the physical vibration model comprises at least one fixed node, the fixed node being configured to have zero displacement when generating the sound output. It may be provided that a fixed node may not oscillate. In some examples, the physical vibration model may comprise two or more fixed nodes. When the structure of the physical vibration model is line-like and / or when the physical vibration model is one-dimensional, it may be provided that the first and last node in line-direction and / or in the one dimension may be or may comprise a fixed node. When the nodes are coupled in sequence, it maybe provided that the first and last node of the sequence may be or may comprise a fixed node.

[0074] In one or more preferred examples, when or before mapping a data feature to a parameter of the physical vibration model, the region of interest is segmented into segments. It may be provided that at least one data feature of each segment may be mapped to at least one parameter of the physical vibration model.

[0075] In one or more preferred examples, a first segment of the segments starts at an occurrence of a distinctive structure. Alternatively or additionally, a first segment of the segments comprises at least partially a distinctive structure. The distinctive structure may comprise an ERM region of the biomedical image data.

[0076] In one or more preferred examples, the region of interest is segmented according to a Fibonacci sequence. In one or more examples, the region of interest is segmented according to an exponential sequence.

[0077] In one or more preferred examples, each segment is represented by one or more nodes of the physical vibration model. In some examples, each segment may be represented by two nodes of the physical vibration model.

[0078] In one or more examples, when mapping a data feature to a parameter of the physical vibration model, a maximum value of the data features of each segment is mapped to a parameter of the physical vibration model.

[0079] In one or more preferred examples, generating the sound output of the acoustic model comprises concatenating sound grains, each sound grain corresponding to a different physical vibration model of a respective different region of interest. When the data features of different regions of interest differ, the respective sound grains may differ. Thereby, the regions of interest maybe differentiated acoustically, and / or their different properties maybe identified.

[0080] The method according to the first aspect may be used to identify ERM elevations, but is not necessarily limited thereto. For instance, the method allows to identify gaps between an elevated ERM and retinal structure. When using the method, the amount of synthetic dye may be reduced and decision-making and surgery outcomes may be improved.

[0081] The method can be applied in a broad range of ophthalmic scenarios, or in other medicinal application.

[0082] At least one, multiple or all steps of the method according to the first aspect of the invention may be computer-implemented. The method according to the first aspect of the invention may further be used, according to another aspect of the invention, as part of a method of training a machine learning algorithm. For this use, the sound output generated by the method according to the first aspect of the invention may be used as part of training input data for training the machine learning algorithm. The auditory information encoded in the sound output generated by the method according to the first aspect of the invention can be treated, in particular along with other data, to enhance and augment the training effect. For example, a machine learning algorithm to be trained based on medical images and / or patient information may further be trained using the sound output as generated by the method according to the first aspect of the invention for datasets representing said medical images as part of the training data. This provides additional information and context to the machine learning algorithm, which can improve the accuracy and performance of the algorithm.

[0083] A further use of the method according to the first aspect of the invention may be part of a method of applying a surgical treatment or a diagnostic treatment to a patient, wherein the method according to the first aspect of the invention may be used for generating a sound output based on an interaction with a data set corresponding to a representation of a body part of the patient.

[0084] Another aspect of the invention relates to a computer program comprising executable instructions which, when executed by a processor, cause the processor to implement the method according to the first aspect of the invention.

[0085] Another aspect of the invention relates to a system for generating a sound output for representing biomedical image data, wherein the system comprises a sound model module configured for generating a sound output according to the method according to the first aspect of the invention.

[0086] It may be provided that the system may further comprise a sound generator for generating a sound signal according to the sound output generated by the sound model module. The sound signal may be an audible sound signal. Alternatively, the sound signal may be an electronic sound signal encoding a corresponding audible sound signal, i.e., an electronic sound signal configured for being provided to some piece of electronic equipment, for example an amplifier and / or a loudspeaker, and for triggering the generation of a corresponding audible sound signal.

[0087] In one or more examples, the system comprises an interaction module configured for obtaining an interaction with at least a part of the biomedical image data. It may be provided that the interaction module may comprise a user input device configured for allowing a user to input an interaction. The user input device may comprise at least one of a touch pad, a touch screen, a keyboard, a mouse, a joystick, a pedal, a camera, and a tracker. The interaction module may be configured to track an instrument, such as a physical instrument, e.g. a microsurgical forceps or the like. In some embodiments, the user input device may be combined with and / or may be contained in an instrument, such as a physical instrument, e.g. in a microsurgical forceps or the like.

[0088] BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Examples of the invention are further described with reference to the following figures:

[0090] Fig. i: shows a flow diagram of an exemplaiy method according to the first aspect of the invention;

[0091] Fig. 2a: shows an exemplaiy physical vibration model;

[0092] Fig. 2b: shows another exemplary physical vibration model;

[0093] Fig. 3: shows a schematic representation of a region of interest and a physical vibration model;

[0094] Fig. 4a: shows an exemplaiy biomedical image;

[0095] Fig. 4b: shows an exemplaiy local attenuation map based on the biomedical image of fig- 4a;

[0096] Fig. 5: shows an exemplaiy biomedical image and a corresponding sound output;

[0097] Fig. 6: shows another exemplary biomedical image and a corresponding sound output;

[0098] Fig. 7: shows a schematic illustration of a system according to another aspect of the invention; and

[0099] Fig. 8: shows another exemplary physical vibration model.

[0100] DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0101] For the purposes of promoting an understanding of the principles of the invention, reference will now be made to embodiments illustrated in the drawings, and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended, such alterations and further modifications in the illustrated apparatus and such further applications of the principles of the invention as illustrated therein being contemplated as would normally occur now or in the future to one skilled in the art to which the invention relates. Figure 1 shows a flow diagram of an exemplary method 200 according to the first aspect of the invention. The method 200 comprises providing biomedical image data at step 202.

[0102] The biomedical image data may comprise one or more biomedical images. The biomedical image data may be or may comprise one or more of at least one of a OCT scan, a PET scan, a CT scan, a SPECT scan, and an MRI scan. It may be provided that the biomedical image data may be or may comprise one or more OCT B-scans 106.

[0103] An OCT B-scan 106 may comprise or may consist of multiple OCT A-scans 104. An OCT B-scan 106 may be substantially two-dimensional. An OCT A-scan 104 may be substantially onedimensional. In some embodiments, an OCT A-scan 104 and / or an OCT B-scan 106 may contain depth information.

[0104] At step 204, a region of interest R of the biomedical image data is assigned to a physical vibration model 116. The physical vibration model 116 maybe part of an acoustic model, and / or an acoustic model may comprise the physical vibration model 116.

[0105] In some embodiments, the region of interest R can be selected and / or determined by a user, e.g. by a surgeon. Alternatively or additionally, the region of interest R can be selected and / or determined according to one or more criteria. For instance, the region of interest R can be selected and / or identified by some model. The region of interest R can be selected, determined and / or identified as a region comprising a distinctive structure.

[0106] In some embodiments, the region of interest R may be determined from the position of an instrument, such as e.g. a microsurgical forceps.

[0107] It may be provided that there is more than one region of interest R, each of which may be assigned to a different physical vibration model 116 of the acoustic model. In some embodiments, different physical vibration models 116 may have similar or identical structures, numbers of nodes N and / or couplings of the nodes N, but differ in at least one parameter. In some embodiments, at least two different physical vibration models 116 may differ in at least one of structure, number of nodes N and coupling of the nodes N.

[0108] In some embodiments, the same physical vibration model 116 maybe used for different regions of interest R, but the parameters of the physical vibration model 116 may be changed and / or updated when or after assigning a region of interest R and / or mapping the data features of the respective region of interest R. The physical vibration model 116 may have at least two nodes N. It may be provided that each node N of the physical vibration model 116 may be coupled to at least one other node N. In some embodiments, at least one or each node N may be coupled to one or two other nodes N.

[0109] A node N maybe or may comprise an oscillating node. An oscillating node may vibrate, and / or maybe displaced with respect to its equilibrium position. It may be provided that at least one node N is a fixed node M. A fixed node M maybe a non-vibrating node, and / or may not change its position or may not be otherwise displaced.

[0110] A coupling between nodes N maybe provided and / or may be modeled by springs K and / or by dampers D. The nodes N may be connected and / or coupled to each other with springs K and / or dampers D. A node N may be modeled and or represented by a mass m. A parameter of the model may be or may comprise a mass m of a node N, a spring constant c and / or a damping coefficient k.

[0111] The physical vibration model 116 may be or may comprise one or more damped harmonic oscillators, and / or may model one or more damped harmonic oscillators.

[0112] A damped harmonic oscillator may be given by a fixed node M and an oscillating node N, wherein the fixed node M and the oscillating node N are coupled and / or connected via a spring K and a damper D. Then, the balance of forces (Newton’s second law) yields

[0113] F(t) = mx + c x + k x (1) where m denotes the mass (of the oscillating node N), c the damping coefficient of the damper D, k the spring constant of the spring K, F(t) a (potentially time-dependent) force acting on the oscillating node N, and x the displacement of the oscillating node N from its equilibrium position, x is the acceleration of the oscillating node N, and x is the velocity of the oscillating node N. The vibrations of the oscillating node N can then be computed from formula (1) by solving the ordinary differential equation (ODE).

[0114] In the physical vibration model 116, additional nodes N may be coupled and / or connected to the oscillating node N of a damped harmonic oscillator. For instance, another oscillating node N having a mass may be coupled and / or connected via a spring and / or damper to the oscillating node N of the damped harmonic oscillator. When a force F is acting on the damped harmonic oscillator, the additional node N coupled and / or connected thereto may vibrate as well. In particular, the respective vibrations of the nodes N may interact and / or may be superposed, depending on the values of the respective mass, spring constant and damping coefficient.

[0115] Further, additional nodes N may be coupled and / or connected. For instance, yet another oscillating node N may be coupled and / or connected to the additional node N. It may be provided that a specific number of nodes N are coupled and / or connected in sequence via respective springs and / or dampers. Each node N may have a mass, each spring may have a spring constant and each damper may have a damping coefficient, which may be parameters of the physical vibration model 116 given by the respective nodes N.

[0116] When additional nodes N are coupled and / or connected, the position and / or the displacement of each node N may be given by a suitably modified formula (1), as known from the general principles of mechanics. For instance, an equation for the balance of forces may be written for each node N. The resulting system of differential equations may be solved and / or computed e.g. numerically.

[0117] However, the physical vibration model 116 is not limited to such containing a damped harmonic oscillator.

[0118] By or when applying a force to one or more of the nodes N, the force may be propagated throughout the physical vibration model 116.

[0119] At step 206, a data feature of the region of interest R is mapped to a parameter of the physical vibration model 116.

[0120] The data feature may comprise a physical property, such as e.g. a temperature or a density. The data feature may comprise an intensity, a refractive index, or the like. The data feature may comprise a measurement, a sensor value and / or sensor data. The data feature may be calculated and / or may be determined from one or more of another value or variable, measurement, sensor value and / or sensor data.

[0121] It may be provided that the data feature may be associated with a pixel, a voxel, a region and / or a position, of the biomedical image data. It may be provided that the biomedical image data may comprise one or more biomedical images, and data features may be associated with said one or more biomedical images. For instance, the biomedical image data may comprise one or more OCT scans, such as e.g. one or more OCT B-scans and / or OCT A-scans, and may further comprise temperature data corresponding to a temperature at the different positions of the biomedical image data. A data feature may be or may comprise an “overlay” over a biomedical image.

[0122] Alternatively or additionally, the data feature maybe or may comprise one or more data points, pixels or voxels of a biomedical image. In some embodiments, a data feature may be or may comprise e.g. an intensity. It may be provided that the biomedical image substantially corresponds to the data features. It may be provided that the data feature comprises or corresponds to measurements from which the biomedical image was generated, e.g. intensities or the like.

[0123] The mapping maybe carried out using one or more mapping functions. In some embodiments, the mapping maybe carried out by a machine learning model, such as e.g. by an artificial neural network or by a deep learning model. However, the mapping model and / or the mapping function are / is not limited thereto.

[0124] It maybe provided that an external force F may act on one or more nodes N. The external force F may be time-dependent. The vibrations of the physical vibration model 116 and / or of the nodes N may be due to the external force F. The amplitude and / or the phase of the vibrations may depend on the external force F.

[0125] The external force F may correspond to a force exerted by a virtual instrument on a node N. The virtual instrument may be a model of a physical instrument, such as a microsurgical forceps. The external force may e.g. correspond to or may model a force exerted by the physical instrument on a retina or a tissue, but is not necessarily limited thereto.

[0126] Different physical vibration models 116 may differ in the number of parameters, and / or in the values of the parameters. Different physical vibration models 116 may differ in the number of nodes N, and / or in the connection and / or in the coupling of the nodes N. For different physical vibration models 116, the respective nodes N may be differently coupled. In some embodiments, the physical vibration models 116 may have the same structure, e.g. the same number of nodes N and their respective arrangement and / or coupling, but may have different parameters.

[0127] At step 208, the sound output of the acoustic model is generated based on vibrations of the physical vibration model 116. For instance, the vibrations of the physical vibration model 116 may be translated by the acoustic model to a sound grain 114 and / or to a sound output 112. For this, e.g. FFT or another suitable method may be used. The sound output 112 maybe or may comprise one or more sound grains 114. The sound output 112 may be or may comprise a concatenation of more than one sound grain 114. The sound output 112 may be referred to as sound texture.

[0128] The sound output 112 and / or the sound grain 114 may be digital.

[0129] The sound output 112 and / or the sound grain 114 may be played as audible sound, thereby enabling a user and / or a surgeon to perceive the sound output 112 and / or the sound grain 114 corresponding to the region of interest.

[0130] Figure 2a shows a damped harmonic oscillator. A node N, which may be an oscillating node N, may be coupled and / or connected via a spring K and / or via a damper D to a fixed node M. The position of the fixed node M may be constant, and / or the fixed node M may not be displaceable. The node N may be displaced from an equilibrium position by a displacement d. The node N may have a mass m. When the position of the node N coincides with its equilibrium position, a force as exerted on the node N by the spring K may vanish. When an external force F (not shown in fig. 2a) acts on the node N, the node N may be displaced, and may vibrate and / or may oscillate according to formula (1) as given above. Alternatively or additionally, the node N may be displaced at a reference time, e.g. at a starting time when solving formula (1), and may oscillate and / or may vibrate according to formula (1) due to the displacement d at the reference time.

[0131] The spring constant k, the mass m and / or the damping coefficient c maybe parameters. It may be provided that the external force F (if present) and / or the displacement at reference time (if present) maybe parameters.

[0132] Figure 2b shows another node N2 coupled and / or connected to the node N and / or to the damped harmonic oscillator of fig. 2a. The other node N2 may be coupled by another spring K2 and / or by another damper D2 to the node N. The other node N2 may be an oscillating node N2, and may vibrate and / or may be displaced. The displacement of the other node N2 relative to its equilibrium position may be denoted by d2 in fig. 2b. The other spring K2 may have a spring constant k2. The other damper D2 may have a damping coefficient C2. The other node N2 may have a mass m2. A force as exerted by the other spring K2 on the node N and on the other node N2 may vanish when both the node N and the other node N2 are in their respective equilibrium position.

[0133] An external force F (not shown in fig. 2b) may act upon the node N, and / or upon the other node N2. Due to the external force F, the node N and the other node N2 may vibrate and / or may be displaced. Alternatively or additionally, the node N and / or the other node N2 may be displaced at a reference time, which may result in the node N and the other node N2 vibrating and / or oscillating.

[0134] The spring constants k, k2, the masses m, m2 and / or the damping coefficients c, C2 may be parameters. It may be provided that the external force F (if present) and / or the displacement d, d2 at reference time (if present) may be parameters.

[0135] It may be provided that further nodes N may be coupled and / or connected accordingly. For instance, further nodes N may be connected and / or coupled in sequence to the other node N2 and / or to the node N.

[0136] A physical vibration model 116 may be given by the structure and / or by the arrangement as shown in fig. 2a and / or fig. 2b, or may comprise the structure and / or the arrangement as shown in fig. 2a and / or fig. 2b. However, the physical vibration model 116 may not be limited thereto.

[0137] While the structure, the coupling and / or the connection of the nodes N and / or the respective physical vibration model 116 may be one-dimensional, other couplings and / or connections of nodes N in different spatial directions and / or dimensions are possible. While the nodes N as shown in figs. 2a and 2b may vibrate substantially in one direction, in some physical vibration models 116 at least one or more nodes N may vibrate in higher dimensions and / or in additional spatial directions.

[0138] Figure 3 shows a schematic representation of a region of interest R as well as an acoustic model comprising a physical vibration model 116.

[0139] In some examples, the region of interest R may be or may comprise an OCT A-scan 104. In some examples, the region of interest R may be or may comprise an attenuated and / or a filtered OCT A-scan 104.

[0140] The physical vibration model 116 maybe one-dimensional. In the physical vibration model 116, the nodes N may be coupled in sequence. It may be provided that the first node N and / or the last node N in the sequence may be or may comprise a fixed node M. It may be provided that the nodes N between the first node N and the last node N in the sequence may be or may comprise oscillating nodes N. The nodes N maybe coupled and / or connected with dampers D and / or with springs Kto each other. A damper D and / or a spring K may couple and / or connect two nodes N in the sequence. Each damper D may have a damping coefficient c. Each spring K may have a spring constant k. Each node N may have a mass m and / or a weight. The respective masses m and / or weights, damping coefficients c and spring constants k may be parameters of the physical vibration model 116. The parameters of the physical vibration model 116 may determine the vibration and / or the displacement of the nodes N.

[0141] The first and / or the last node N of the sequence may be fixed in their position and / or may not be displaced, and the nodes N between the first and the last node N may be displaceable around their respective equilibrium position and / or may vibrate.

[0142] It may be provided that at least one node N may be or may comprise an absorption node A. An absorption node A may be a node N which is coupled by a damper D having a relatively large damping coefficient c to another node N. It may be provided that the absorption node A may oscillate and / or may vibrate less than an oscillating node N. It may be provided that the second to the last node N in the sequence maybe or may correspond to an absorption node A.

[0143] At least one node N maybe or may comprise a listener node L. A listener node L maybe or may comprise an oscillating node N, and / or may vibrate and / or may be displaced. In some embodiments, the second and the third node N may be or may comprise listener nodes. It may be provided that the sound grain 114 and / or the sound output 112 is generated based on the vibration of one or all listener nodes L.

[0144] The region of interest R may correspond to an OCT A-scan 104, or to an attenuated OCT A- scan, but is not necessarily limited thereto. The OCT A-scan 104, and / or the region of interest R, may comprise an intensity I. The intensity I may vary along the region of interest R and / or along the OCT A-scan 104.

[0145] It may be provided that a local attenuation map 108 of the biomedical image and / or of the biomedical image data may be determined. For instance, when the biomedical image data comprises intensities I, a local attenuation may be computed. Here, I(i) may be the intensity at index i, w may be a local window size, and A maybe an axial pixel resolution. In some exemplary embodiments, the window size maybe chosen to be w = 5, but is not necessarily limited thereto. The index i may correspond to pixels, voxels and / or data points. The number of pixels, voxels and / or datapoints may depend on the resolution of the instrument with which e.g. the biomedical image data may have been captured. Attenuating may include filtering, and / or applying a filter. The above function (2) maybe only exemplary, and other suitable functions maybe used.

[0146] It may be provided that OCT A-scans 104 are attenuated, and the attenuation map 108 may be formed from the attenuated OCT A-scans 104. The attenuation map 108 may be or may comprise an attenuated OCT B-scan 106.

[0147] It may be provided that when assigning the region of interest R, the corresponding region of interest R of the local attenuation map 108 is assigned to the physical vibration model 116. Thus, instead of assigning a region of interest R of the original biomedical image or biomedical image data, and / or non-filtered and / or non-attenuated biomedical data, the corresponding filtered and / or attenuated region of interest R may be assigned to the physical vibration model 116. It may be provided that data features of the filtered and / or attenuated region of interest R are mapped to the parameters of the physical vibration model 116.

[0148] In some embodiments, it may be provided to segment the region of interest R into segments S. The segments S may be non-overlapping. Alternatively, at least two segments S may overlap. When the region of interest R is one-dimensional, such as e.g. an OCT A-scan 104, a segment S may be a line segment and / or a portion of the region of interest R. However, the segment does not need to be one-dimensional. Depending on the shape and / or on the dimension of the region of interest R, the shape and / or the dimension of a segment S may differ.

[0149] The segmentation may be carried out according to a Fibonacci sequence. Alternatively, the segmentation maybe carried out according to an exponential sequence. When segmenting, the number of pixels, voxels and / or data points, may be counted, such that the number of pixels, voxels and / or data points may correspond to the respective number of the Fibonacci sequence or the exponential sequence. For instance, when using the Fibonacci sequence, the first segment may contain the first pixel, voxel and / or data point, such that the first segment is formed by a single pixel, voxel and / or data point, the second segment may contain the second pixel, voxel and / or data point, such that the second segment is formed by a single pixel, voxel and / or data point, the third segment may contain the third and fourth pixel, voxel and / or data point, such that the third segment is formed by two pixels, voxels and / or data points, the fourth segment may contain the fifth, sixth and seventh pixel, voxel and / or data point, such that the fourth segment is formed by three pixels, voxels and / or data points, the fifth segment may contain the eight, ninth, tenth, 11thand 12thpixel, voxel and / or data point, such that the fifth segment is formed by five pixels, voxels and / or data points, and so on. Similarly, when using an exponential sequence, the segments may be chosen to correspond to the respective exponential sequence.

[0150] A data point may correspond to a measured value or to measured data, and / or may be determined and / or calculated from a measured value or measured data. A data point may be or may comprise e.g. an intensity I. A data point may be or may comprise a physical property. The data point may be or may comprise a data feature to be mapped onto the parameters. A data feature to be mapped onto the parameters may be given, computed and / or determined from one or more data points. The data points may be or may comprise attenuated and / or filtered intensities of the attenuated and / or of the filtered OCT A-scan 104. A data point may be or may comprise intensities of the OCT A-scan 104.

[0151] The segments S may be non-overlapping. The segments S may be chosen such that there are no gaps between the segments S, i.e. that there are no pixels, voxels and / or data points that are not in any segment S / which are between pixels, voxels and / or data points that are in any segment S. In some embodiments, at least two segments S may be overlapping, such that a pixel, a voxel and / or a data point maybe in at least two segments S.

[0152] The first segment S may start at an occurrence of a distinctive structure. The first segment S may comprise at least partially a distinctive structure. In some embodiments, the distinctive structure may comprise an ERM region. Thereby, parts which may be contained in the region of interest R, but which are of no importance, maybe excluded from the mapping.

[0153] For instance, when the region of interest R is or comprises an OCT A-scan 104 of a retina, a large part of the A-scan 104 may represent the vitreous chamber above the retina, which may be excluded by the segmentation. For instance, the segmentation may start with the first occurrence of a distinctive structure, and / or at an area that may have a great significance as it includes the gaps formed by an ERM too, if such gaps happen to exist. On the other hand, the retinal structures between the internal limiting membrane (ILM) 102 and retinal pigment epithelium (RPE), or below the RPE, have less impact, and may be excluded.

[0154] It may be provided that at least one node N is assigned to a segment S, and / or that a segment S is represented by at least one node N. Each segment S may be represented by at least one oscillating node N. In some embodiments, at least one or each segment S may be represented by two nodes N, preferably by two oscillating nodes N. When mapping the data feature to a parameter, it may be provided that a maximum value V of the data feature, and / or of the data points, pixels and / or voxels, e.g. of the intensity I, in a segment S may be mapped to one or more respective parameters. When mapping the data feature to a parameter, it may be provided that a maximum value V of the data feature, and / or of the data points, pixels and / or voxels, e.g. of the intensity I, in a segment S may be mapped to one or more respective nodes N. For instance, the spring constant k, the damping coefficient c, and / or the mass m of the nodes N representing the segment S may be chosen depending on the maximum value V. Alternatively or additionally, another measure or summary statistic of the data feature, and / or of the data points, pixels and / or voxels, e.g. of the intensity I, may be mapped to the parameter. The measure and / or the summary statistic may e.g. be or may e.g. comprise a mean or a median of the data points, pixels and / or voxels, e.g. of the intensity I, in a respective segment S, without necessarily being limited thereto.

[0155] Figures 4a and 4b show a comparison of an OCT B-scan 106 (fig. 4a) and an attenuation map 108 based on said OCT B-scan 106 (fig. 4b). The attenuated map 108 and / or the attenuated OCT B-scan io6may be formed by attenuating one-dimensional line scans, such as e.g. OCT A- scans 104, according to the above formula (2).

[0156] In some embodiments, the method according to the invention may be employed to facilitate peeling of an ERM too.

[0157] As seen in figs. 4a and 4b, the ILM 102, ERM too and other layers and structures of the retina are retained in the attenuation map 108. In particular, the ILM 102 and the ERM too are more distinctive from the background, and can be more clearly seen. In figure 4a, an OCT B-scan 106 is shown. In some embodiments, the OCT B-scan 106 may be a biomedical image and / or biomedical image data. Alternatively, the biomedical image data may comprise one or more OCT B scans 106. The OCT B-scan 106 maybe or may comprise an OCT B-scan of an eye and / or of a retina.

[0158] An OCT B-scan 106 maybe formed by a plurality of OCT A-scans 104, and / or may comprise a plurality of OCT A-scans 104.

[0159] By filtering and / or attenuating, the sound output 112 and / or the sound grain 114 may be more distinctive and clearer.

[0160] Fig. 5 and fig. 6 show a biomedical image and the corresponding sound output 112. The biomedical image may correspond to an OCT B-Scan 106. In figs. 5 and 6, clearly an elevated ERM too is seen, which is to be peeled. The peeling of an ERM too is a complex procedure wherein a membrane, only a few micrometers thick, that develops on the retinal surface is delicately removed using a microsurgical forceps. Insights regarding small gaps between the ERM too and the retinal tissue are valuable for surgical decision-making, particularly in determining a suitable location to initiate the peeling. Depth- resolved imaging of the retina provided by iOCT enables visualization of this gap and supports decision-making, as seen in the OCT B-scans 106 of figs. 5 and 6.

[0161] The spectrograms 118 show distinctive sound grains 114 as generated by the method according to the first aspect of the invention for regions of interest R (e.g. corresponding OCT A-scans 104) which contain the elevated ERM too. Thus, the position of the elevated ERM too may be identified by means of the sound output 112. The spectrograms 118 may be or may correspond to mel spectrograms. A mel spectrogram may comprise frequencies on a mel scale. A mel scale maybe a transformation of a Hz scale.

[0162] The spectrograms 118 depicted in figs. 5 and 6 illustrate the ability to distinguish between regions featuring elevated ERM too and those with an attached membrane. Furthermore, this differentiation persists even in scenarios where the ERM too is only marginally elevated and minor gaps are visible in the OCT B-scan 106.

[0163] The method and system of the present invention e.g. thus allows that a user, such as a surgeon, does not need to move his / her gaze from the surgical site during surgery for identifying the elevated ERM too, as it may be determined from the sound output 112 and / or from the sound grain 114. Thereby, the cognitive load of the surgeon maybe lowered.

[0164] It may be provided that the sound output 112 corresponds to a sound grain 114 as generated for a position of an instrument, such as a microsurgical forceps used by the user and / or by the surgeon for peeling the membrane. Thereby, the sound output 112 may change and / or may vary depending on the position of the instrument, allowing for an interactive procedure. Further, the amount of synthetic dye on the retina may be reduced, and decision-making and surgery outcomes may be improved.

[0165] To evaluate the efficacy of the method in conveying structural information, a user study involving 15 novice participants was conducted. The users were introduced to the concept by demonstrating the sonification of a simulated OCT B-scan 106, produced using an existing simulation framework. Following a brief training session, the participants were asked to identify the presence and locations of ERM too elevations within 10 randomly ordered OCT B- scans, which contained zero, one, or two gaps. Among 150 trials, in 94% of the cases a correct number of gaps was predicted. The distance error of the selected area, from the total 208 number of gaps, was 12.89ii5.09 pixels, 98.5% being within the area of the elevated ERM 100.

[0166] Further, an interview and an explorative user study with an ophthalmic expert was conducted to evaluate the potential of the method. First, sound output of a diagnostic OCT B-scan 106 was presented, for which the expert was able to easily associate acoustic feedback with elevations in ERM 100. Overall, the clinician found the method to be very effective, as even slight ERM 100 elevations could be identified by use of the sound output 112. Furthermore, a potential user interface was presented in a simulated surgical scenario with integrated ERM 100, where a forceps was moved across the retina and a sound output 112 was generated based on a simulated OCT A-scan 104 in between the forceps tips.

[0167] Qualitative feedback included that such a system could assist surgeons in identifying starting points for grasping, particularly in cases where the ERM 100 covers a larger area of the retina. In addition, the expert suggested that the method could reduce the amount of administered synthetic dye, which was associated with a toxic effect on retinal cells.

[0168] Fig. 7 illustrates an exemplary system 20 according to another aspect of the invention, which is configured for implementing a method 200 for generating a sound output 112 based on vibrations of the physical vibration model 116 according to the first aspect of the invention. The system 20 may be a hardware-based system 20, for example implemented as an independent processor or CPU, or may be a software-based system 20 executable by a processor.

[0169] As represented in fig. 7, a body part of a patient P may be scanned by a sensor device 12, which can for example correspond to a medical imaging device such as CT scanner, an MRI scanner, an OCT scanner or the like. The sensor device 12 may scan a part of the body of the patient P, e.g. a retina or an eye, and may codify the scan results as data. The data may be processed by the CPU 14, which maybe connected to the system 20. The CPU 14 maybe a CPU of the medical imaging device, an external an independent CPU or a CPU of the system 20.

[0170] The system 20 may receive the data from the CPU 14. The data may be or may correspond to the biomedical image data.

[0171] In some embodiments, the data can comprise one or more additional data features, for example a density value or a temperature measured by the sensor device 12 at a corresponding spatial region of the body of the patient P. In some other embodiments, the data may comprise and / or may consist of one or more biomedical images. The system 20 may receive the data from the CPU 14. The system 20 comprises a sound module 24. The system 20 may comprise one or more of an interaction module 22, a sound generator 26 and a user input device 28.

[0172] The interaction module 22 maybe configured for obtaining an interaction. An example of such interaction may be a movement of an instrument, such as e.g. a microsurgical forceps. In this example, the user may determine the region of interest R by moving the instrument accordingly. In some other examples, the user input device 28 can comprise a mouse and the user can determine a region of interest R of a biomedical image by moving the mouse accordingly relative to the biomedical image. The biomedical image may be displayed on a display, and the mouse cursor may be also displayed on the display, preferably overlaying the biomedical image. By correspondingly moving the mouse and thereby guiding the cursor on a screen representing a graphical representation of the dataset, e.g., a biomedical image and / or a graphical representation of a scan obtained by the sensor device 12, the user may choose or select a region of interest R.

[0173] The sound model module 24 is configured for generating a sound output 112 based on the vibrations of the physical vibration model 116 to which the region of interest R is assigned. One or more of the data features of the region of interest R may be mapped to one or more parameters of the physical vibration model 116, and thereby to acoustic properties of the sound output 112. For example, the acoustic model can isomorphically map the intensity I of an OCT A-scan 104 and / or of an OCT B-scan 106 to a pitch of the sound output 112.

[0174] The sound output 112 outputted can be transmitted to the sound generator 26, which can comprise an amplifier and / or a loudspeaker, to generate an audible signal based on the sound output 112. A person listening to such audible signal based on the sound output 112 hence obtains, via auditory perception, information about the data feature for the region of interest R. For instance, an elevated ERM too and / or its position may be identified by the user based on the sound output 112.

[0175] Fig. 8 shows another exemplary physical vibration model 116. The physical vibration model 116 may have multiple nodes N. The nodes N may be arranged such that they form a cube, and / or may be arranged in a cubical grid. The physical vibration model 116 may have a three- dimensional structure. The nodes N may be coupled by springs K and / or by dampers D (not shown in fig. 8). An external force F (not shown in fig. 8) may act upon one or more nodes N. The external force F maybe time-dependent. The nodes N may vibrate due to the external force F. Although preferred exemplary embodiments are shown and specified in detail in the drawings and the preceding specification, these should be viewed as purely exemplary and not as limiting the invention. It is noted in this regard that only the preferred exemplary embodiments are shown and specified, and all variations and modifications should be protected that presently or in the future lie within the scope of protection of the invention as defined in the claims.

[0176] The features as disclosed in the claims, the description and the figures by themselves or in any combination thereof, may be relevant for the realization of the invention.

[0177] List of reference numerals

[0178] 12 sensor device

[0179] 14 CPU

[0180] 20 system

[0181] 22 interaction module

[0182] 24 sound model module

[0183] 26 sound generator

[0184] 28 user input device

[0185] P patient

[0186] R region of interest

[0187] S segment

[0188] N node

[0189] K spring

[0190] D damper

[0191] M fixed node

[0192] A absorption node

[0193] B interaction node

[0194] L listener node

[0195] F external Force

[0196] I intensity

[0197] V maximum k spring constant c damping coefficient m mass d displacement

[0198] 100 ERM

[0199] 102 ILM

[0200] 104 A-Scan

[0201] 106 B-Scan

[0202] 108 local attenuation map

[0203] 110 attenuated A-Scan

[0204] 112 sound output

[0205] 114 sound grain 116 physical vibration model

[0206] 118 spectrogram

Claims

CLAIMS1. A method for representing biomedical image data, the method comprising the steps: providing biomedical image data; assigning a region of interest (R) of the biomedical image data to a physical vibration model (116) of an acoustic model; mapping a data feature of the region of interest (R) to a parameter of the physical vibration model (116); and generating a sound output (112) of the acoustic model based on vibrations of the physical vibration model (116).

2. The method of claim 1, wherein the biomedical image data comprises one or more of an OCT-scan, preferably an OCT A-scan, a CT-scan, a PET scan, a SPECT scan, and a MRI scan.

3. The method of claim 1 or 2, wherein the biomedical image data is or comprises an OCT B-scan (106), and / or wherein the region of interest is or comprises an OCT A- scan (104).

4. The method of any of the preceding claims 1 to 3, the method further comprising the steps: determining a local attenuation map (108) from the provided biomedical image data; and when assigning a region of interest (R) of the biomedical image data, the region of interest (R) is assigned as a region of interest (R) of the local attenuation map (108), such that when mapping a data feature, a data feature of the region of interest (R) of the local attenuation map (108) is mapped to a parameter of the physical vibration model (116).

5. The method of claim 4, wherein when determining the local attenuation map (108), a local attenuation of at least one OCT A-scan (104) is performed.

6. The method of any of the preceding claims, wherein the biomedical image data is provided in real time, and / or wherein the sound output is generated in real time.

7. The method of any of the preceding claims, wherein at least one data feature is isomorphically mapped to at least one parameter.

8. The method of any of the preceding claims, wherein the physical vibration model (116) comprises two or more nodes (N), the nodes (N) being preferably coupled in sequence.

9. The method of claim 8, wherein an external force (F) is applied to at least one node (N) of the physical vibration model (116).

10. The method of claim 9, wherein the external force (F) is a parameter of the physical vibration model (116), and / or wherein the external force (F) corresponds to a virtual instrument.

11. The method of any of the preceding claims 8 to 10, wherein at least one node (N) is or comprises a listener node (L), wherein the sound output (112) of the acoustic model is based on vibrations of the listener node (L).

12. The method of any of the preceding claims, wherein the parameters of the physical vibration model (116) comprise one or more of: one or more spring constants (k), one or more masses (m) and one or more damping coefficients (c).

13. The method of any of the preceding claims, wherein the physical vibration model (116) comprises at least one fixed node (M), preferably two or more fixed nodes (M), the fixed node (M) being configured to have zero displacement when generating the sound output (112).

14. The method of any of the preceding claims, wherein when or before mapping a data feature to a parameter of the physical vibration model (116), the region of interest (R) is segmented into segments (S), wherein preferably at least one data feature of each segment (S) is mapped to at least one parameter of the physical vibration model (116).

15. The method of claim 14, wherein a first segment of the segments (S) starts at an occurrence of a distinctive structure and / or wherein a first segment of the segments (S) comprises at least partially a distinctive structure, the distinctive structure preferably comprising an ERM region of the biomedical image data.

16. The method of claim 14 or 15, wherein the region of interest (R) is segmented according to a Fibonacci sequence or an exponential sequence.

17. The method of any of the preceding claims 14 to 16, wherein each segment (S) is represented by one or more nodes (N) of the physical vibration model (116), preferably by two nodes (N) of the physical vibration model (116).

18. The method of any of the preceding claims 14 to 17, wherein when mapping a data feature to a parameter of the physical vibration model (116), a maximum value (V) of the data features of each segment (S) is mapped to a parameter of the physical vibration model (116).

19. The method of any of the preceding claims, wherein generating the sound output (112) of the acoustic model comprises concatenating sound grains (114), each sound grain (114) corresponding to a different physical vibration model (116) of a respective different region of interest (R).

20. A computer program comprising executable instructions which, when executed by a processor, cause the processor to implement the method of any of the preceding claims.

21. A system (20) for generating a sound output for representing biomedical image data, wherein the system comprises: a sound model module (24) configured for generating a sound output according to the method of any of the preceding claims 1 to 17; wherein the system preferably further comprises a sound generator (26) for generating a sound signal according to the sound output generated by the sound model module (24).

22. The system of claim 21, wherein the system further comprises an interaction module (22) configured for obtaining an interaction with at least a part of the biomedical image data, the interaction module preferably comprising a user input device (28) configured for allowing a user to input an interaction, wherein the user input device more preferably comprises at least one of a touch pad, a touch screen, a keyboard, a mouse, a joystick, a pedal, a camera, and a tracker.

Images