A method and system for processing ultrasound data

Abstract

The present invention relates to a method for processing ultrasound data using a system arranged for real-time image formation of the ultrasound data, the method comprising the steps of receiving, by a local processing unit, raw channel data obtained by an ultrasound probe from an array of transducer elements comprised in the ultrasound probe, establishing, by the local processing unit, a network communication session with a remote data processing facility; determining, by the remote data processing facility, a set of quality of service metrics for the network communication session between the local processing unit and the remote data processing facility; applying, by the local processing unit, a transmission scheme to the raw channel data for transmission to the remote data processing facility over the established network communication session; transmitting, by the local processing unit, the raw channel data to a remote data processing facility in accordance with the transmission scheme; generating, by the remote data processing facility, the ultrasound images on the basis of the received raw channel data; transmitting, by the remote data processing facility, the generated ultrasound images to one or more local user interfaces for real-time visualization of the ultrasound images obtained by the ultrasound probe; wherein the method further comprises the step of: transmitting, by the remote data processing facility, the set of quality of service metrics to the local unit, and adapting, by the local processing unit, the transmission scheme in accordance with the set of quality of service metrics determined by the remote data processing facility.

Description

[0001] Title: a method and system for processing ultrasound data

[0002] Description:

[0003] TECHNICAL FIELD

[0004] The present invention relates to a method for processing ultrasound data. More in particular, relates the present invention to a method for processing ultrasound data using a system arranged for real-time image formation of the ultrasound data.

[0005] The present invention further relates to a system, arranged for such processing of ultrasound data.

[0006] The present invention also relates to an ultrasound data acquisition device, an ultrasound data processing facility and a computer program product arranged for such processing of ultrasound data.

[0007] BACKGROUND

[0008] Ultrasound imaging is a widely used non-invasive diagnostic technique that utilizes high-frequency sound waves to produce real-time images of internal body structures. Ultrasound imaging, also known as sonography, has become an important tool in medical diagnostics due to its non-ionizing nature (which X-rays, CT scans and nuclear images have), portability, and real-time imaging capabilities. It plays a vital role in various medical specialties, including obstetrics, gynecology, cardiology, radiology, and general abdominal imaging. During an ultrasound examination, an ultrasound probe, also called a transducer, is placed on the patient's skin, and it emits high-frequency sound waves into the body.

[0009] As these sound waves encounter different tissues and structures within the body, they reflect back to the probe. The probe's receiver collects these reflected signals, which are then processed to create visual representations of the internal structures in real-time. The resulting ultrasound images are valuable for diagnosing various medical conditions, guiding medical procedures, and monitoring fetal development during pregnancy.

[0010] Conventional ultrasound imaging systems may comprise several components, including the ultrasound probe, thus including circuitry with ultrasound transducer elements, a processing unit, and a display unit. The ultrasound probe consists of, for example, of an array of piezoelectric crystals that convert electrical signals into mechanical vibrations and vice versa. Other techniques such as CMLIT may however also be used. By varying the electrical currents applied to the crystals, the probe can generate and receive ultrasound waves at specific frequencies.

[0011] The received ultrasound signals contain information about the density and composition of the tissues encountered by the sound waves, allowing for the creation of detailed images. The processing unit receives the ultrasound data from the probe, applies various algorithms and signal processing techniques, and converts the data into visual images. These images are then displayed on a monitor for examination and analysis by medical professionals.

[0012] One significant advancement in this domain involves the continuous evolution of ultrasound probes, specifically their transducer elements. Over the years, ultrasound probe technology has seen a remarkable increase in the number of transducer elements, leading to substantial improvements in resolution and imaging capabilities. By incorporating a higher density of transducer elements, modern ultrasound probes can capture finer details and provide higher image quality, enhancing the accuracy of diagnoses and facilitating more precise medical interventions. This trend towards ever-increasing transducer element numbers has significantly elevated the performance and clinical utility of ultrasound imaging systems, making them indispensable tools in various medical specialties and applications.

[0013] This advancement due to, amongst others, increase in the number of transducer elements in ultrasound probes also presents certain challenges. As the number of elements grows, there is a significant surge in the volume of the data obtained during each imaging session. Processing and converting this vast amount of data into meaningful, high-quality images demand substantial computational resources and can lead to challenges in real-time imaging. The sheer complexity of data handling and processing places higher demands on the processing unit, requiring more powerful hardware and longer processing times.

[0014] Moreover, there is a trend towards high frame rate ultrasound systems which is opening unvaluable opportunities for quantitative ultrasound (e.g. shear wave imaging), but at the cost of a higher channel data rate (i.e. , more data per seconds). This also places a higher burden and higher demand on the processing unit.

[0015] To this end, conventional ultrasound systems are stand-alone units where the image formation occurs on-site. These systems are equipped with powerful processing units that can cope with these high demands in resources.

[0016] Another disadvantage of conventional systems is that they are designed for peak computational load. Most of the time, the system is not hitting that limit. The system has to be over-specified and over-priced to deal with the spurious moments of high load.

[0017] There are both bulky and portable conventional systems, the latter functioning without bulky consoles, but high image quality and advanced clinical applications make these systems bulky as they are high demanding on (computational) resources. As such, conventional ultrasound systems, both bulky and handheld, have several limitations, such as the deployment of advanced image formation methods in such conventional systems and especially in portable or wearable devices, forcing to use bulky console machines unfeasible in circumstances where manoeuvrability and portability are essential. Due to the high demand on resources, conventional systems are often tailor-made and specifically configured to cope with these demands. This however significantly limits the flexibility of these systems and any update or upgrade such as implementing novel image formation techniques or algorithms, may require upgrading the compute resources of the ultrasound machine, thus replacing costly hardware or if even possible, scale its resources which all in all makes these conventional systems poorly flexible.

[0018] The overall consequence is that advanced methods are hardly deployed in portable or wearable ultrasound devices, which, on the other side, are advisable in several circumstances where manoeuvrability and portability are key.

[0019] As such, there is a need to overcome at least some of these limitations and drawbacks with a method and system which can be embodied in a portable and hand-held ultrasound system but is more flexible but does not compromise the inherent benefits of ultrasound imaging systems for use in real-time applications, for example where hand-eye coordination is mandatory.

[0020] SUMMARY

[0021] It is an object of the present invention to address the above-described drawback. The present invention, in at least some of the aspects disclosed herein or in particular embodiments or examples thereof therefore aim to be embodied in a portable and hand-held ultrasound system which is more flexible but does not compromise the inherent benefits of ultrasound imaging systems for use in real-time applications, for example where hand-eye coordination is mandatory.

[0022] In accordance therewith, in a first aspect, there is provided, a method for processing ultrasound data using a system arranged for real-time image formation of the ultrasound data, the method comprising the steps of: receiving, by a local processing unit, raw channel data obtained by an ultrasound probe from an array of transducer elements comprised in the ultrasound probe; establishing, by the local processing unit, a network communication session with a remote data processing facility; determining, by the remote data processing facility, a set of quality of service metrics for the network communication session between the local processing unit and the remote data processing facility; applying, by the local processing unit, a transmission scheme to the raw channel data for transmission to the remote data processing facility over the established network communication session; transmitting, by the local processing unit, the raw channel data to a remote data processing facility in accordance with the transmission scheme; generating, by the remote data processing facility, the ultrasound images on the basis of the received raw channel data; transmitting, by the remote data processing facility, the generated ultrasound images to one or more local user interfaces for real-time visualization of the ultrasound images obtained by the ultrasound probe; wherein the method further comprises the step of: transmitting, by the remote data processing facility, the set of quality of service metrics to the local unit, and adapting, by the local processing unit, the transmission scheme in accordance with the set of quality of service metrics determined by the remote data processing facility.

[0023] What is proposed is a method for processing ultrasound data using a system arranged for real-time image formation of the ultrasound data. With this method and corresponding system and other aspects of the present invention, the limitations or at least to some extent, of the conventional processing of ultrasound data are resolved. It enables portable, wearable, hand-held ultrasound systems which are more flexible without compromising the inherent benefits of ultrasound imaging systems for use in real-time applications, for example where hand-eye coordination is mandatory.

[0024] Recent advancements in image formation methods (such as digital beamforming) and the introduction of deep learning-based signal processing methods, are leading to unprecedented image quality, at the cost of a high computational footprint. Since these methods should be deployed on the data capture device, e.g. the probe or the local processing unit attached to the probe, due to its stand-alone architecture. The increase in required compute power imposes strict hardware requirements on the ultrasound device (e.g., large size and power consumption). The overall consequence is that advanced methods are hardly deployed in portable or wearable ultrasound devices, which, on the other side, are advisable in several circumstances where manoeuvrability and portability are key.

[0025] It was in insight of the inventors to apply distributed image formation techniques in which the image formation is separated from the data capture device. Conventional ultrasound image devices are either equipped for advanced image formation algorithms which do provide high image quality, but require high computational power and are thus embodied in poorly flexible, bulky machines, or are equipped for simple image formation methods, to be embodied in portable devices and for example use basic beamforming implemented on the hand-held probe.

[0026] Moreover, the inventors had the insight to enable ultra-low latency, broad bandwidth medical data communication between an on-premise raw data collection system and a remote data processor.

[0027] In the first aspect, there is provided an ultrasound data using a system arranged for real-time image formation of the ultrasound data, the method comprising at least the following steps, which can be applied in the proposed, or in a different order:

[0028] The system comprises a local processing unit, which may be a unit embodied as a device separate of, and attached to the probe, for example in a retrofitted manner, or may be embodied in the ultrasound probe as a ultrasound capture device. The local processing unit receives the raw channel data from the ultrasound probe and in the embodiment of the processing unit to be embodied into the housing of the probe, the channel data is received from the receive circuits, which in turns receive the signals from the transducer elements

[0029] Hence the raw channel data is the data obtained from the array of transducer elements and in accordance with the present invention, raw channel data may refer to the data obtained from a single element, or from a group of elements, e.g. after these elements are multiplexed or beamformed in a (simple) first stage beamforming. In accordance with the present invention, raw channel data at least means that it contains the original data of the transducer element(s), whereas conventionally, such raw data is not transmitted but (pre)processing is applied first prior to transmission. It is expressed that the term raw channel data may also include channel data before or after 1st stage beamforming in a probe, or filtered, or pre- processed, digital signal. Moreover, it may also include data of one or several transducer elements when amplified by a LNA, TGC, and converted to digital with an ADC.

[0030] The local processing unit establishes a network communication session with a remote data processing facility.

[0031] In accordance with the present invention, remote data processing facility may be defined as a centralized or distributed computing system, typically located at a different physical location from the local processing unit or the ultrasound probe. This facility is designed to handle computationally intensive tasks, such as advanced image processing, data analysis, and storage, which may require significant computational power and storage capabilities.

[0032] In the context of the present invention, the remote data processing facility enables processing the raw ultrasound data received from the local processing unit. The facility may utilize advanced algorithms and image processing techniques to convert the raw channel data into meaningful ultrasound images. By offloading these resource-intensive tasks to a dedicated remote facility, the local processing unit can be configured and optimized for its task of data acquisition, optionally data preprocessing, and data transmission arranged for real-time imaging applications.

[0033] The remote data processing facility communicates with the local processing unit through a network, allowing for efficient data exchange. The facility receives the raw channel data transmitted by the local processing unit, performs the necessary image reconstruction and may perform image enhancement, and may perform post-processing or quantitative analysis for the extraction of clinical features / indices, and then sends the processed ultrasound images back to a local unit for real-time visualization on local user interfaces. With the remote data processing facility several advantages arise, among other things, computational power as the facility can host high-performance servers and specialized hardware, enabling complex and computationally intensive image processing tasks that may not be feasible to perform locally on the ultrasound probe or the local processing unit or would require extreme bulky hardware. Such a remote processing facility may also be staffed with experts in image processing and diagnostic interpretation, allowing for centralized expertise and consistency in analysis across multiple locations.

[0034] And very importantly, it significantly increases the flexibility and scalability as the implementation of new image processing technologies, algorithms, increase (or even decrease) of usage of the system (resources of the centralized remote facility could be shared between several local data processing units and thus data capturing devices) can be employed more easily and efficiently through such a centralized processing facility than in a conventional local, bulky, system with fixed hardware and corresponding constraints.

[0035] The remote facility can also serve as a centralized storage repository for ultrasound data, enabling secure and efficient data management and it enables realtime collaboration as multiple users can access and interact with the same data simultaneously, facilitating real-time collaboration and consultation by medical staff, even if they are present in different locations.

[0036] The remote facility also enables configuration, upgrading and allowing improvements to the algorithm as for example the data obtained by the remote facility may be used as input for learning algorithms.

[0037] In the context of the present invention, remotely is defined as not directly attached to, and located adjacent to the local processing unit. Preferably, remotely may be considered as located in a centralized data-centre or the like. The data-centre may however also be located near the local processing unit and thus the capturing device, for example in a centralized server room within the facility in which the capturing device is used. This may for example be a hospital in which the capturing device is used directly in the treatment room or operating room of the hospital, whereas the remote processing facility is located in the server area of the hospital, the data may be communicated via private, secure, access-limited data networks.

[0038] In any of the examples and aspects of the invention, the data may be considered data that is communicated via private, secure, access-limited data networks as part of the definition of a remote facility for processing data.

[0039] The raw channel data is thus transmitted, either wired or wirelessly, over the network communication session to the remote data processing facility. Sending all available raw transducer data in uncompressed form over the network places very high demands on the network, which cannot or hardly be guaranteed. This can compromise stability of the network and thus the system, compromise real-time properties of the system and can lead to high costs for setting up and maintaining such a network.

[0040] It is therefore a challenge to, on the one hand, keep real-time requirements acceptable, whereas on the other hand, enabling maximum bandwidth usage, so that the best possible transfer of data can take place in order to generate high-quality images in which the system is capable of displaying these images in realtime on a local user device. The system requirements are primarily derived from the real-time use, which - given that the system includes data transferring over the network - translate into a requirements on the end-to-end data transferring delay. The latter is dependent on the network throughput and may be determined by measuring such.

[0041] To this end, the remote data processing facility determines a set of quality of service metrics for the network communication session between the local processing unit and the remote data processing facility as these may impact data transfer rate, latency, and reliability of the network.

[0042] The set of quality of service metrics, once determined, are applied by the local processing unit, which receives these directly or as input parameters, to apply a transmission scheme to the raw channel data for transmission to the remote data processing facility over the established network communication session. In another example, the remote processing facility computes one or more of the transmission scheme, the acquisition settings and the reconstruction settings. It sends to the local processing unit the outcomes and the local processing unit adjusts the settings accordingly. In this way, there is no heavy task completed by the local processing unit to search for the best transmit scheme given the set of quality of service metrics.

[0043] The local processing unit transmits the raw channel data to the remote data processing facility in accordance with this transmission scheme, or put differently, the raw channel data is transmitted with this transmission scheme.

[0044] The transmission scheme, in the context of data communication and networking and in the context of the present invention, may refer to a set of rules, protocols, or methods that dictate how data is efficiently and reliably transmitted from one point to another within a communication system, i.e. between the local processing unit and the remote processing facility. It may encompass various aspects of data transfer, including data encoding and modulation, but also may encompass error detection, error correction, and flow control mechanisms, and selection of network profile / slide (like the profile “LIRLLC” for ultra-low latency or “EMBB” for high bandwidth) that determines speed, latency and reliability. The transmission scheme may be optimized for data transfer performance, minimize data loss or corruption during transmission, ensure data integrity, and adapt to varying network conditions such as bandwidth availability, latency, and congestion. Such transmission scheme enables efficient and robust data communication, especially enabling real-time applications like ultrasound imaging where timely and accurate data exchange is essential for generating high-quality images.

[0045] When the raw channel data is transferred accordingly, and thus in a specific and optimal manner, only then will the ultrasound images be reconstructed and generated at and by the remote data processing facility. It is thus emphasized that the images are only generated after transmission to the remote data processing facility. When the images are generated, they are ready to be send to one or more user interfaces, e.g., displays. In an example, the user interface may be integrated into the data capturing device. At least one of these interfaces is typically locally present for real-time application of an ultrasound imaging system. Some user interfaces may however also be remote, i.e. remote from the remote data processing facility and remote from the data capturing device. An example of such embodiment would be to allow further medical staff to also be presented with the ultrasound images, which further medical staff is not located on the same location as where the data is obtained by the probe or capturing device.

[0046] The transmission scheme is not static but dynamic or adaptable, meaning that the set of quality of service metrics may vary depending on instantaneous network conditions but also on user input, compression parameters, acquisition parameters and / or reconstruction parameters.

[0047] To this end, an effective and efficient trade-off is achieved between delay- constrained communication on the one hand, and image quality on the other.

[0048] As such, the present invention proposes a distributed architecture for ultrasound imaging system, whereas contrary to traditional commercial ultrasound machines, the present invention proposes to live stream digital raw channel data (before any processing) over the network to a remote data processing facility, where image formation is executed. The reconstructed images are then transferred via network to one or more user interfaces for visualization.

[0049] In the context of distributed ultrasound systems, real-time live streaming of ultrasound raw channel data to the remote processing facility and formed image data from it under limited and potentially time-varying network capacity circumstances is assured while always also ensuring the highest image quality through adaptive acquisition, compression, decompression and image reconstruction schemes.

[0050] Image quality, in the context of the present invention, may be used as a broad term for any desirable metric. Depending on the application, image quality might be computed using traditional metrics, like generalized contrast-to-noise ratio, or specific metrics computed on features extracted from the reconstructed images using task-driven algorithms. For example, one image could have a low image quality according to conventional metrics, but could work very well for estimating a certain feature of interest.

[0051] The method and system of the present invention may allow: continuous monitoring of the instantaneous network capacity; may be employed based on automatically extracted information from the reconstructed images (e.g., quality of the reconstructed images); may be based on parameters set by a user on a user interface.

[0052] It also may allow changing: the compression factor (hence the encoder’s and decoder’s parameters) of a flexible lossy data compression scheme for raw ultrasound channel data and potentially also another compression scheme for reconstructed images, and / or one or multiple ultrasound acquisition / reconstruction run-time parameters in a way that ensures real-time and low latency performance while maximizing one or multiple quality aspects of the reconstructed image rendered on a display.

[0053] It however also allows for: asynchronous data acquisition, data transferring, and data processing, to support real time and low latency performance. multi-threaded raw channel data transmission to support real time and low latency performance. a fallback solution with local processing facility implemented in the probe (i.e., data capture device) to be used if the network drops or the network is so congested that the rendered image is degraded (i.e., quality below what is achievable with the local basic processing). image (and potentially also other clinical indices extracted with postprocessing) broadcasting to multiple user interfaces and direct remote control of runtime acquisition and reconstruction parameters via the same interface.

[0054] In an example, the remote data processing facility transmits the transmission scheme and the acquisitions settings, which are selected in accordance with the set of quality of service metrics, to the local processing unit for adapting, by the local processing unit, the transmission scheme and the acquisitions settings in accordance therewith, and wherein the remote data processing facility preferably computes the transmission scheme according to the set of quality of service metrics, and jointly adapts acquisition settings and reconstruction settings in accordance with the determined set of quality of service metrics.

[0055] Either the remote data processing facility may determine the set of quality of service metrics and computes the transmission scheme, or determine the set of quality of service metrics and let the local processing unit compute the transmission scheme on the basis thereof. Preferably, however, the remote data processing unit computes the transmission scheme (and preferably the acquisition settings, and the reconstruction parameters). The (heavy) task of choosing the ‘best’ transmit scheme (and acquisition settings) according to the metrics may be left to the local processing unit or remote data processing facility.

[0056] Hence, there are two alternatives, wherein the remote processing facility transmits the set of quality metrics to the local processing unit. Then, the local processing unit itself decide according to these metrics the ‘best’ transmit scheme, or wherein the remote processing facility transmits the transmission scheme (and the acquisition settings) which is selected by the remote stream & settings controller in agreement with the set of quality metrics (and other inputs) to the local processing unit. Then, the local processing unit has only to adjust the settings without requiring any additional heavy search-task.

[0057] Preferably, the transmission scheme is computed along with the best or preferred acquisition settings and reconstruction settings to satisfy a trade-off between real-time requirements and maximum quality (where quality is described accurately in description) given the inputs (i.e., the set of quality of service metrics, the user’s inputs, the computed image quality / features).

[0058] In an example, the method further comprises the step of: adapting, by the local processing unit, acquisition settings in accordance with the determined set of quality of service metrics.

[0059] In an example, the method further comprises the step of: adapting, by the remote data processing facility, reconstruction settings in accordance with the determined set of quality of service metrics.

[0060] In an example, the local processing unit is integrated into the ultrasound probe, and the ultrasound probe preferably being a portable handheld ultrasound probe.

[0061] The local processing unit is preferably integrated in the probe but may alternatively also be configured as and physically separated from the probe.

[0062] In an example, the local processing unit is comprised in a raw data acquisition device, connected locally to the ultrasound probe.

[0063] In an example, the transmission scheme comprises a level of compression of the raw channel data, and preferably a level of lossy compression of the raw channel data, which level of compression, and preferably lossy compression, is defined by the set of quality of service metrics for the network communication session.

[0064] In an example, the remote data processing facility computes application specific metrics from raw channel data or reconstructed images and uses them along with the quality of service metrics for the network communications session for adapting one or more transmission scheme, acquisitions settings, and reconstruction settings.. With the example, probe-specific settings are made available at the remote side, allowing the remote data processing facility, i.e. a hardware and / or software controller, to define admissible acquisition settings.

[0065] In an example, the raw channel data is analog channel data which is digitized by an analog to digital converter prior to transmission to the remote data processing facility.

[0066] In an example, upon generating, by the remote data processing facility, the ultrasound images on the basis of the received raw channel data, features are extracted from the generated ultrasound images.

[0067] In an example, the local processing unit applies a separate transmission scheme for the features extracted by the remote data processing facility for transmission to the remote data processing facility over the established network communication session.

[0068] In an example, the remote data processing facility provides a user interface for configuration of a set of user input parameters, which user input parameters are input to the remote data processing facility and may comprise one or more of acquisition mode, time-gain compensation, a compression scheme, compression level, compression algorithm, maximum compression level, and minimum compression level.

[0069] Some configurations, e.g. user input parameters may be more clear and others may be more obscure for a typical user, e.g. a clinician, as he or she may not correctly interpret the implication of switching between compression algorithms, although other user such as administrators may be able to configure such settings for a clinician. As such, some user parameters may be made available according to certain user roles having corresponding credentials relating to general hardware and IT-related configuration and others to use-related configurations. Typically, the user may be able to set a level of compression or level of quality whereas the admin may configure the type of compression and minimum and maximum ranges made available for the user.

[0070] In an example, wherein the transmission scheme and / or the acquisition settings and / or the reconstruction settings are tuned in accordance with a set of quality service metrics for the network communication session and / or image content (environmental changes, patient-to-patient variability) and / or users' inputs and / or sensors data (e.g., accelerometer).. Hence, in an example, the set of quality of service metrics are related to the network, and the environmental changes and patient-to- patient variability changes to the image content. Hence, the transmission scheme is, in the example, defined in accordance to all three.

[0071] The environmental changes and patient-to-patient variability are observed in the reconstructed images. The remote data processing facility, e.g. controller, extracts features related to the environmental changes and / or variability from these images and use them (along with the network info, and user's inputs) to determine the optimal transmission scheme, the acquisition settings, the reconstruction parameters. Other variables may also be taken into consideration, and used to determine the optimal transmission scheme, i.e. sensor data such as accelerometers. The skilled person will appreciate which other sensor data may be applicable.

[0072] In an example, the remote data processing facility is arranged for storing the raw data and the generated ultrasound images in a local or remote storage facility, for offline accessibility of the ultrasound images by the one or more local user interfaces from the storage facility.

[0073] The remote data processing facility can be arranged to storing the channel data, which can be the raw channel data, the original unprocessed channel data, pre- processed channel data or compressed channel data.

[0074] In an example, the method further comprises the steps of: determining, by the local processing unit, from the set of quality of service metrics for the network communication session a maximum network transmission bandwidth; and comparing, by the local processing unit, the determined network transmission bandwidth with a predefined threshold; generating, by the local processing unit, locally generated ultrasound images from the obtained raw channel data when determining the network transmission bandwidth to drop below the threshold; transmitting, by the local processing unit, the generated ultrasound images to the one or more local user interfaces for real-time visualization of the ultrasound images obtained by the ultrasound probe.

[0075] In an example, after generating, by the remote data processing facility, the ultrasound images on the basis of the received raw channel data and prior to transmitting, by the remote data processing facility, the generated ultrasound images to the one or more local user interfaces for real-time visualization of the ultrasound images obtained by the ultrasound probe, the method further comprises: applying, by the remote data processing facility, an image transmission scheme to ultrasound images for transmission to the one or more local user interfaces, and wherein the image transmission scheme comprises one or more image transmission scheme variables defining a level of compression applied to the ultrasound images prior to transmission.

[0076] In a further example, the remote data processing facility also applies the transmission scheme other clinical indices which may be extracted from images via post-processing.

[0077] In an example, the transmission scheme comprises the use of transform coding, wherein the parameters of the transform coding method are defined according to the quality of service metrics to obtain a specific rate. The local processing unit may adapt the transmission scheme according to the set of quality of service metrics, it may select one among a set of neural transform coding methods previously trained to work with a specific network bandwidth. In an example, the local processing unit is also arranged for generating the ultrasound images on the basis of the raw channel data, and generates said ultrasound images when predefined network conditions are met, wherein the predefined network conditions comprise one or more of: a non-established network connection session with the remote data processing facility, a drop of network bandwidth below a predefined threshold, exceeding latency of the network above a predefined threshold, determined network instability, for example determined by an exceeding number of dropped network packages per time period.

[0078] As indicated, with conventional ultrasound imaging there are trade-offs between image quality, bandwidth, and latency. Image quality is about the clarity and amount of detail in the formed images. Image quality benefits from a large range of used frequencies (hence a large bandwidth) and from a large number of transmitted ultrasound waves with a large number of corresponding responses from which an image is formed by an advanced and compute-intensive image formation algorithm. Hence, for a high image quality a large amount of data per image is beneficial.

[0079] Latency refers to the delay between the transmission of ultrasound waves and the display of the corresponding formed image on a screen. For real-time imaging a low latency is important as the physician does not want to experience a lag between hand manipulation of the ultrasound probe (and of instruments for minimally invasive procedures which may appear in the ultrasound field of view). For a low latency a low amount of data per image is beneficial.

[0080] Different clinical applications may require a different trade-off between image quality, bandwidth and latency. Also, within a single clinical application a different trade-off may be optimal at different moments of time within the workflow. For example, when the physician is navigating by moving the ultrasound probe to find the optimal view of the anatomy of interest, a low latency (a small lag between hand motion and image on the screen) is needed. Once the probe is in position and is no longer moved, the physician is interested in a high image quality and is not hindered by a small lag if there is no hand motion. According to the current state of the art of ultrasound imaging the tradeoff between image quality, bandwidth, and latency is made within the ultrasound apparatus depending on the user interface settings, and is kept fixed as long as the user interface settings are not modified.

[0081] According to certain aspects and examples of the present disclosure, the trade- off between image quality, bandwidth, and latency, is made automatically by an algorithm, depending on detected user probe movement. The proposed method, system, and devices can in principle be used in ultrasound architectures where all processing happens locally inside the probe or inside the ultrasound console, but is most useful in a distributed ultrasound architecture according where there is a network between devices (probe and display) and computing.

[0082] According to an aspect of the present disclosure, the trade-off between image quality, bandwidth, and latency is made automatically both at the capture side for the communication of channel data from the probe to the computing unit, and at the display side for the communication of image data from the computing unit to a screen.

[0083] According to an aspect of the present disclosure, means are implemented to detect probe movement to discriminate between “navigation mode” and “imaging mode” during the workflow of a procedure. This information is then used along with the quality of service metrics to adjust the transmission scheme and / or acquisition settings and / or reconstruction settings. A good way to achieve this is by using an accelerometer inside the probe in combination with an algorithm that discriminates between the situation where the physician is moving the probe by hand and the situation where the physician keeps the probe in place.

[0084] Alternatively, probe movement can be detected directly from the acquired ultrasound images by an image processing algorithm. According to the aspect, one or multiple of the following system settings may be adjusted automatically depending on being in “navigation mode” or in “imaging mode”:

[0085] • Ultrasound driving scheme (the amount of transmitted ultrasound waves and the ultrasound wave parameters like the shape of the excitation pulse, the transmit apodization weights, the center frequency, and the focus point).

[0086] • Compression factor of the (lossless or lossy) compression algorithm that compresses the ultrasound channel data to vary the used bandwidth and / or latency.

[0087] • Compression factor of the (potentially lossy) compression algorithm that compresses the images to vary the used bandwidth and / or latency.

[0088] • The image formation algorithm (which may be designed to make a trade-off between a high computational complexity for high image quality versus a low latency.

[0089] • Network slicing settings for 5G or 6G networks (like the profile “URLLC” for ultra-low latency or “EMBB” for high bandwidth) whereby speed, latency and reliability can be adjusted (with different cost settings by network operators who make more money for the more demanding profiles).

[0090] With closed-loop imaging the ultrasound transmit parameters are varied between transmit events. In a distributed architecture the image formation processing operating on received channel data requires knowledge of these transmit parameters. Because in a networked environment synchronization between probe and remote processing facility may vary, the computing algorithm in the processing facility may mistakenly use a wrong set ultrasound transmit parameters for image formation which actually belong to a different transmit event than what the algorithm in the processing facility believes.

[0091] In an example, the ultrasound transmit parameters are transmitted together with the ultrasound channel data for each ultrasound transmit event, from probe to the remote processing facility. Alternatively, to save bandwidth for transmitting many parameters, a unique identifier that belongs to a specific set of ultrasound transmit parameters is sent together with the ultrasound channel data for each ultrasound transmit event, from probe to the processing facility.

[0092] In an example, the remote data processing facility is further arranged for allowing remote users through a user interface to some or all control device settings, comprising one or more of acquisition parameters and reconstruction parameters.

[0093] In this embodiment, the ultrasound system is configured to enable remote users to have full control over the device settings, including both acquisition and reconstruction parameters. This functionality may be facilitated by the remote data processing facility, which may provide a user interface accessible by remote users. Through this interface, users can adjust acquisition settings such as mode, gain, and depth, as well as reconstruction parameters including image processing algorithms and display settings. This feature enhances the flexibility and usability of the system, allowing for real-time adjustments and optimizations based on the specific requirements of the ultrasound examination, even when the operator is not physically present with the ultrasound probe.

[0094] In a second aspect, there is provided an ultrasound image system for realtime image formation of ultrasound data, the system comprising: an ultrasound probe arranged for obtaining raw channel data from an array of transducer elements comprised in the ultrasound probe; a local processing unit, arranged for receiving the raw channel data from the ultrasound probe and for establishing a network communication session with a remote data processing facility, the local processing unit further being arranged for applying a transmission scheme to the raw channel data for transmission to the remote data processing facility over the established network communication session, and for transmitting the raw channel data to, a remote data processing facility, arranged for determining a set of quality of service metrics for the network communication session between the local processing unit and the remote data processing facility, and transmitting, the set of quality of service metrics to the local processing unit, and receiving the raw channel data from the local processing unit, and arranged for generating the ultrasound images the basis of the received raw channel data, and transmitting the generated ultrasound images to, one or more local user interfaces, arranged for receiving the generated ultrasound images from the remote data processing facility, for real-time visualization of the ultrasound images obtained by the ultrasound probe; wherein the local processing unit is further arranged for adapting the transmission scheme in accordance with the set of quality of service metrics determined by the remote data processing facility.

[0095] In a third aspect, there is provided an ultrasound data acquisition device, arranged for use in an ultrasound image system according to the first aspect, wherein the ultrasound data acquisition device comprises a local processing unit and is connected to the ultrasound probe of an ultrasound image system, wherein the ultrasound data acquisition device is arranged for receiving the raw channel data from the ultrasound probe and for establishing a network communication session with a remote data processing facility, the local processing unit further being arranged for applying a transmission scheme to the raw channel data for transmission to the remote data processing facility over the established network communication session, and for transmitting the raw channel data to a remote data processing facility according to the first aspect, and wherein the local processing unit is further arranged for adapting the transmission scheme in accordance with the set of quality of service metrics determined by the remote data processing facility.

[0096] In a fourth aspect, there is provided an ultrasound data processing facility, arranged for use in an ultrasound image system according to the first aspect, arranged for determining a set of quality of service metrics for a network communication session between a local processing unit of the ultrasound image system and the ultrasound data processing facility located remotely, and arranged for transmitting, a set of quality of service metrics to the local processing unit, for the local processing unit to adapt a transmission scheme in accordance with the set of quality of service metrics determined by the remote data processing facility, and receiving raw channel data from the local processing unit, and arranged for generating ultrasound images the basis of the received raw channel data, and transmitting the generated ultrasound images to one or more local user interfaces, arranged for receiving the generated ultrasound images from the remote data processing facility, for real-time visualization of the ultrasound images obtained by the ultrasound probe.

[0097] The remote processing facility may transmit the metrics back to the local processing unit, or in an alternative example, directly the already computed settings, e.g. transmission scheme and the acquisition settings. In the first, the local processing unit may have quite a large computational task (explore all the possible settings configuration and select the one that -according to the quality metrics - satisfies the requirements) and in the latter, this is done at the remote end.

[0098] In a fifth aspect, there is provided a computer program product embodied on a non-transitory computer-readable medium, comprising a set of instructions executable by a processor, the computer program product adapted to perform the method for processing ultrasound data in accordance with the first aspect.

[0099] In another aspect, the method may be applied to other data than the ultrasound data from which ultrasound images are generated. Hence, the data may relate to clinical data or (raw) sensor data in general, in which aspect the local processing unit receives clinical data from a data acquisition device. The local processing unit then establishes a network communication session with a remote data processing facility to allow transmission of the data to the remote data processing facility. Prior to, or shortly after transmission of the data, a set of quality service metrics for the network communication session between the local processing unit and the remote data processing facility are determined such that these can be taken into account and applied on a transmission scheme for transmission of the data to the remote data processing facility, where the relevant clinical information may be extracted from or generated out of the raw clinical data obtained locally by the local data acquisition device. The generated relevant clinical information is then made available for display, interpretation or further processing by a further device, e.g. one or more local user interfaces, such that real-time display, real-time interpretation or further processing is enabled by adapting the transmission scheme, preferably in a dynamic manner, in accordance with the set of quality of service metrics determined by the remote data processing facility.

[0100] Accordingly, a method is provide similar to the first aspect of the present invention, but related to other data than ultrasound image data, which data in its raw, uncompressed form is large and challenging to transmit over a network to a remote location, and which may relate to clinical data (i.e. (raw) sensor data) such as medical images like X-rays, CT scans, MRI scans, digital pathology slides, etc. The proposed controller or processing unit is arranged to adapt - not only the compression - but also the acquisition and reconstruction settings in a jointly manner so as to ensure the realtime requirements while maximizing the quality, which is applied to the raw digitized channel data of ultrasound element or elements, and may thus also be applied to other data such as (raw) sensor data, e.g. relating to clinical data as the above mentioned examples.

[0101] The skilled person will appreciate that the effect, advantages and examples discussed and disclosed in relation to the first aspect of the present invention are also applicable to this method for general clinical data.

[0102] It will be understood that the above-described embodiments are not meant to limit the invention, whose scope is solely determined by the claims. In particular, the skilled person will appreciate that various individual or combined features selected from various embodiments within this description (above and below) may be added to other embodiments as long as there is no technical hindrance to doing so, without departing from the scope of the invention as defined by the claims.

[0103] BRIEF DESCRIPTION OF THE DRAWINGS

[0104] The embodiments described herein will be more fully understood with the help of the detailed description below and with reference to the appended drawings, in which: Fig. 1 illustrates steps of the method according to an aspect of the present invention;

[0105] Fig. 2 illustrates the transmission scheme and the acquisition and reconstruction settings joint control, in accordance with an aspect of the present invention;

[0106] Fig. 3 illustrates the framework according to another aspect of the present invention;

[0107] Fig. 4 illustrates a data capture device according to another aspect of the present invention;

[0108] Fig. 5 illustrates a data processing facility according to another aspect of the present invention;

[0109] Fig. 6 illustrates an electronic device with a user interface for displaying the generated images, in according to another aspect of the present invention;

[0110] Fig. 7 illustrates a user interface according to another aspect of the present invention;

[0111] Fig. 8 illustrates a flow diagram according to another aspect of the present invention.

[0112] DETAILED DESCRIPTION

[0113] In Fig. 1 steps are illustrated of a method 100 of processing ultrasound data using a system arranged for real-time image formation of the ultrasound data. The method 100 comprises several steps in accordance with the first aspect of the present invention.

[0114] First, a local processing unit, receives 110 raw channel data obtained by an ultrasound probe from an array of transducer elements comprised in the ultrasound probe.

[0115] The local processing unit then establishes 120 a network communication session with a remote data processing facility, e.g. a cloud based server environment. As indicated, the steps may be followed in a different order, for example, establishing 120 the network communication session may be performed prior to obtaining 110 the raw channel data.

[0116] Once the network communication session is established a transmission scheme is applied 130 on the raw channel data before or view of transmission 140 therefor to the remote data processing facility.

[0117] The network communication session may have a fluctuating network capacity and the remote data processing facility, or alternatively but not preferably, the local processing unit, determines 150 a set of quality of service metrics over time for this network communication session. Step 150 of determining the set of quality of service metrics may also occur in parallel to other operations. For example, during the time it takes to complete the transmission 140 the network condition could be determine, but determining the metrics 150 may not occur continuously (i.e. , not every time that a raw channel data is transferred).

[0118] As a fallback solution the local processing unit, e.g. data capture device, may in addition be used to reconstruct images if the network drops or the network is so congested that the rendered image quality is degraded, i.e., quality below what is achievable with the local basic processing. In such situations, the local processing unit may take over from the remote data processing facility and may halt transmission but generate the images locally, i.e. in a more rudimentary form, in accordance with the capabilities of the local processing unit.

[0119] In the fallback scenario, the local processing unit operates stand-alone without the remote facility but may still apply algorithms which are maintained and updated by the remote data processing facility, e.g. when the network has reestablished.

[0120] Once the raw channel data is transmitted 140 and the metrics are determined 150, of which it is emphasised that these do not exclude user inputs, the determined metrics are transmitted 160 to the local data processing facility. On the basis of the transmitted metrics it is determined which adaption of the scheme is required for the local processing unit, once these instructions are received from, may adapting 170 the transmission scheme accordingly. Step 160 may, however, also follow step 150, but may occur in parallel to step 140, as these are two different ongoing processes. Now the updated transmission scheme applies and is used to transmit 180 the raw channel data in accordance with this adapted and update transmission scheme, such that the remote data processing facility may generate 190 the images out of the received raw channel data received through the adapted scheme. The images may then be stored locally, remotely, elsewhere and / or be sent to a local electronic device as illustrated in more detail in Fig. 6, for display through a user interface as illustrated in Fig. 7.

[0121] In example illustrated in Fig. 1 , the step of determining the set of quality of service metrics for the network communication session takes place after initial transmission of the raw channel data. Hence, first the raw channel data is transmitted in accordance with an initial transmission scheme, and once the metrics are determined, the scheme is updated / amended accordingly, for the remaining transmission of the raw channel data, until a new set of service metrics is computed and the transmission scheme is adapted again. In another example, there however may not be an initial transmission scheme, but the transmission scheme is optimized prior to any transmission of the raw channel data. Hence, first the network is established, then the metrics determined, transmission scheme optimized (i.e. adapted) accordingly, and only then the raw channel data is transmitted. It is emphasized that the present invention convers both examples.

[0122] Fig. 2 illustrates the stream and settings controller, which is a component of the remote processing facility (i.e., the remote processing facility also presents a beamformer, a post-processor, etc) to jointly adapt the transmission scheme, and / or the acquisition parameters, and / or the reconstruction parameters, in accordance with the set of quality metrics for the network communication session, the users’ inputs and additional quality metrics computed from the reconstructed images.

[0123] Fig. 3, shows the framework 200 for distributed ultrasound image formation in real-time. The framework 200 includes an ultrasound data capture device 210, a data processing facility 220, and one or more electronic devices 230 used as user interfaces.

[0124] According to an exemplary embodiment of the present invention, the system operates as follows. The ultrasound data capture device acquires raw, high- bandwidth, ultrasound channel data; after digitization, the data is transferred over the network to a data processing facility. At the facility, the raw data is processed to reconstruct diagnostic images. The reconstructed images may be further analysed to extract relevant clinical information (e.g., blood flow). The results of the image diagnostic application executed by the facility may be stored at the facility, and / or transmitted to the electronic device for rendering.

[0125] According to an exemplary aspect of the present invention, there is provided a data processing facility (Fig.5), or server, preferably including: an input / output module

[0300] , an input / output unit

[0301] , and one or more processing unit

[0302] preferably comprising multiple processing cores (e.g., graphical processing unit). The former input / output unit

[0300] is configured to enable a two-way communication with the data capture device. The second input / output unit

[0301] is configured to enable a two-way communication with one or more electronic devices. The processing unit

[0126]

[0302] is responsible for the reconstruction of ultrasound diagnostic images and further post-processing.

[0127] The data processing facility may further include multiple input / output units to enable multi-threaded transmission, as opposed to single-thread mode. This is expected to speed up the transferring hence supporting real-time system functioning.

[0128] The processing facility may preferably further comprise a data storage unit

[0129]

[0303] that stores the data transmitted from the data capture device to the data processing facility over the network, and another data storage unit

[0304] that stores the reconstructed image and post-processed data. The data storage unit

[0303] decouples the transferring from the processing. The data storage unit

[0304] decouples processing from the image rendering. Both data storage units enable transferring, processing, and rendering to execute asynchronously. Asynchronous programming facilitates the implementation of strategies for real-time functioning.

[0130] The processing unit

[0302] may preferably further comprise a stream & settings controller

[0305] , which computes at any time the optimal acquisition and reconstruction scheme and / or compression strategy. Over time, the controller proposes the schemes that maximize the image quality provided the constraints on the raw channel data rate imposed by the network environment. The outcomes of this analysis may be returned to the data capture device and used to program: the transmitter

[0100] , the analog-front-end

[0102] , and a potential encoder

[0109] , The outcomes to the controller may be used also to program the potential raw channel data decoder deployed at the server side

[0307] , The controller

[0305] may be responsible also for defining another encoder

[0308] to compress the reconstructed images (in addition to the compression of the raw channel data) and ensure that also the reconstructed images arrive at the electronic device within a predefined maximum latency (when the images are rendered via network). In this configuration, the controller should also return the commands to program a potential decoder at the electronic device.

[0131] In an exemplary embodiment of the current invention, as shown in Fig.8, the network conditions might be estimated considering the transferring rate (i.e. , the amount of raw channel data sent by the data capture device over the network and received by the server over time).

[0132] The data processing facility may be public or private, and placed close to the ultrasound data capture device (e.g., hospital data centre) or at a large distance (e.g., public cloud).

[0133] According to another exemplary aspect of the present invention, there is provided a data capture device, as illustrated in Fig.4, preferably including: a transmitter

[0100] featuring a pulser that generates a pulse signal, a transmit beamformer, and transmit / receive switch

[0101] ; a transducer

[0102] that converts the pulse signal generated by the pulser to ultrasonic waves, and converts received waves to an electrical signal; an analog front-end

[0103] featuring an amplifier and an analog / digital (A / D) converter that digitizes the electrical signal; a input / output unit

[0104] that directly connects to the network; and a (preferably low-power) processing unit

[0105] ; and a power supply module

[0106] , The input / output unit

[0104] (communication module) is configured to enable a two-way communication with the data processing facility. The connection is used to transfer raw ultrasound channel data to the data processing facility and to receive from the data processing facility commands about the acquisition scheme and a potential raw channel data compression stage. Commands about the acquisition scheme (i.e., how to program the transmitter

[0100] , the analog-front-end

[0103] ) and a potential compression stage

[0109] are communicated by the data processing facility to the data capture device over the network. The data processing facility itself estimates these inputs with the controller

[0305] which combines info gathered from the user (manually set through the user interface and transferred from the electronic device to the processing facility), from the automatic analysis of the reconstructed images, and from the estimates of the network conditions.

[0134] The ultrasound data capture device may further include multiple input / output units

[0104] to enable multi-threaded transmission, as opposed to singlethread mode. This is expected to speed up the transferring hence supporting real-time system functioning.

[0135] The processing unit

[0105] is used to program the transmitter

[0100] and the analog-front-end

[0103] and to execute data compression (if any).

[0136] The data capture device may preferably further include a storage

[0107] to store the acquired data; such a storage unit

[0107] decouples the acquisition from the transferring making it possible to program them asynchronously. Additionally, the storage unit is needed to allow data compression before transmission according to network environment.

[0137] The ultrasound data capture device may be a cart-type, a portable-type, or a wearable-type ultrasound diagnosis apparatus. According to another exemplary aspect of the present invention, there is provided an electronic device, as illustrated in Fig.6, preferably including: an output unit

[0200] , an input unit

[0201] , and an input / output unit

[0202] , The output unit

[0200] is used to render reconstructed images and additional relevant information extracted with post-processing to a display. The input unit

[0201] is used to receive user commands about the acquisition scheme and mode, and may include keyboard devices or pointing devices. The input / output unit

[0202] connects to the network and it is configured to communicate with the data processing facility to transfer the users’ inputs about the image settings and mode, and receive the data processing outcomes to be rendered via display.

[0138] The reconstructed images and processed data may be transferred simultaneously to multiple electronic devices (i.e., user interfaces) to enable the visualization from multiple users. To this end, in an exemplary embodiment of the current invention the user interface may be implemented as a web application, as illustrated in Fig. 7, accessible after authentication by multiple electronic devices connected to internet. To give an example, in the context of wearable ultrasound devices, this multi-users feature may be used to enable image rendering to both a doctor and a patient. In this example, the user interface of the doctor may differ from the one for the patient (e.g., doctor may be allowed to change parameters that the patient cannot change). This personalized interface feature could be enabled through authentication levels.

[0139] The electronic device used as user interface may be placed locally (where the data capture device is placed) and / or remotely for expert or non-expert consulting. Consulting can take place through another external system (e.g., telemedicine used to allow the communication between a remote clinician and the clinician located at the patient site who is performing the medical exam or surgery) or “direct” consulting if the remote user uses the user interface to modify directly the run-time parameters (both reconstruction and acquisition parameters).

[0140] When reconstruction parameters are changed, the info is transferred to the data processing facility where reconstruction and processing occur. When acquisition parameters are changed, the info is transferred to the data capture device. The communication of users’ commands about acquisition run-time parameters to the data capture device can occur in two ways: the electronic device may communicate directly with the data capture device or, alternatively, the info may flow from the electronic device to the data processing facility and then to the data capture device. If the electronic device is placed locally and it communicates directly with the data capture device, the electronic device may be connected to the data capture device via cable or through the network. If the user interface is placed remotely, the communication is always via network.

[0141] According to an exemplary embodiment of the current invention, the system may be used also in disconnected environments or when the network is so congested that the data shall be compressed extensively completely degrading the reconstructed image quality (e.g., below a certain threshold). To avoid discontinuity in the rendered video and ensure that data is always displayed, a basic processing method may be deployed on the processing unit

[0105] or on a different processing unit that can be connected to the data capture device when needed. In this configuration, the controller

[0305] can decide, depending on the network environment, to process the raw data on the data capture device, and therefore to avoid transferring data to the remote processing facility, or cloud. Additionally, if the electronic device is connected to the data capture device via cable, locally reconstructed images can be streamed via cable and do not need to rely on the network. Importantly, if the network is completely unavailable the controller

[0305] cannot communicate with the data capture device, not even to give the command of starting a local processing. Hence, another controller

[0106] may be also present on the data capture device which is responsible for switching from the remote to the local processing when no info is received from the processing facility for more than a certain timeout value.

[0142] According to an exemplary embodiment of the current invention, ultrasound raw channel data could be compressed using a lossy compression strategy based on transform coding. Transform coding typically includes a first stage used to transform the data from the original space to a latent space, and a second stage where the latent space is entropy coded into a bitstream. The advantage of transform coding is that multiple encoders (including transforms and entropy models) can be designed to target different bandwidths, and, during use, the model which is expected to return the rate closest to the network throughput can be selected. In particular, transform coding leveraging neural networks as transforms may be used, as neural networks offer the opportunity to jointly optimize (in an end-to-end fashion) the transforms’ parameters with the density model used by the codec for entropy coding, hence obtaining better rate-distortion performance. In particular for neural transform coding, changing the encoder (to target different bandwidth) is typically a matter of changing the weights of the neural network. These may be all stored locally (with the remote controller pointing to the right set of weights), or remote where the remote controller communicates the full set of weights to the local encoder.

[0143] According to an exemplary embodiment of the current invention, the proposed system might be used in cardiac applications. Cardiac applications could benefit extensively from this invention since advanced image formation methods (with large computational footprint hence requiring the large compute power offered by cloud) can improve dramatically the quality of cardiac ultrasound images (e.g., remove haze-like noise). Additionally, cardiac applications require high frame rate to follow the heart movement, hence strategies implemented to ensure that a pre-defined end-to- end latency is maintained are key to ensure the proper functioning in this application. Ultimately, adaptive imaging (combined with the afore mentioned advanced image formation methods) is expected to improve even more the image quality especially in cardiac applications where several parameters need to be adjusted according to environmental changes and patient-to-patient variability detected from the automatic analysis of the reconstructed images. A few representative examples of run-time parameters that might be optimally tuned during cardiac ultrasound with the proposed system are listed in Tab.1 below.

[0144] Table 1 - Examples of acquisition and reconstruction parameters that might need to be updated during cardiac examinations given the associated environmental change, patient-to-patient variability, and network condition. Environmental changes and patient-to-patient variability are detected from the automatic analysis of the reconstructed images. Instead, network condition is measured by the continuous inspection of the channel bandwidth.

Claims

CLAIMS1. A method for processing ultrasound data using a system arranged for realtime image formation of the ultrasound data, the method comprising the steps of: receiving, by a local processing unit, raw channel data obtained by an ultrasound probe from an array of transducer elements comprised in the ultrasound probe; establishing, by the local processing unit, a network communication session with a remote data processing facility; determining, by the remote data processing facility, a set of quality of service metrics for the network communication session between the local processing unit and the remote data processing facility; applying, by the local processing unit, a transmission scheme to the raw channel data for transmission to the remote data processing facility over the established network communication session; transmitting, by the local processing unit, the raw channel data to a remote data processing facility in accordance with the transmission scheme; generating, by the remote data processing facility, the ultrasound images on the basis of the received raw channel data; transmitting, by the remote data processing facility, the generated ultrasound images to one or more local user interfaces for real-time visualization of the ultrasound images obtained by the ultrasound probe; wherein the method further comprises the step of: transmitting, by the remote data processing facility, the set of quality of service metrics to the local unit, and adapting, by the local processing unit, the transmission scheme in accordance with the set of quality of service metrics determined by the remote data processing facility.

2. The method for processing ultrasound data in accordance with claim 1 , wherein the remote data processing facility transmits the transmission scheme and the acquisitions settings, which are selected in accordance with the set of quality of service metrics, to the local processing unit for adapting, by the local processing unit, thetransmission scheme and the acquisitions settings in accordance therewith, and wherein the remote data processing facility preferably computes the transmission scheme according to the set of quality of service metrics, and jointly adapts acquisition settings and reconstruction settings in accordance with the determined set of quality of service metrics.

3. The method for processing ultrasound data in accordance with claim 1 , or 2, the method further comprising the step of: adapting, by the local processing unit acquisition settings in accordance with the determined set of quality of service metrics.

4. The method for processing ultrasound data in accordance with claim 1 ,2 or 3, the method further comprising the step of: adapting, by the remote data processing facility, reconstruction settings in accordance with the determined set of quality of service metrics.

5. The method for processing ultrasound data in accordance with any of the previous claims, wherein the local processing unit is integrated into the ultrasound probe, and the ultrasound probe preferably being a portable handheld ultrasound probe.

6. The method for processing ultrasound data in accordance with any of the previous claims, wherein the local processing unit is comprised in a raw data acquisition device, connected locally to the ultrasound probe.

7. The method for processing ultrasound data in accordance with any of the previous claims, wherein the transmission scheme comprises a level of compression of the raw channel data, and preferably a level of lossy compression of the raw channel data, which level of compression, and preferably lossy compression, is defined by the set of quality of service metrics for the network communication session.

8. The method for processing ultrasound data in accordance with any of the previous claims, wherein the remote data processing facility computes application specific metrics from raw channel data or reconstructed images and uses them along with the quality ofservice metrics for the network communications session for adapting one or more transmission scheme, acquisitions settings and reconstruction settings..

8. The method for processing ultrasound data in accordance with any of the previous claims, wherein the raw channel data is analog channel data which is digitized by an analog to digital converter prior to transmission to the remote data processing facility.

9. The method for processing ultrasound data in accordance with any of the previous claims, wherein upon generating, by the remote data processing facility, the ultrasound images on the basis of the received raw channel data, features are extracted from the generated ultrasound images.

10. The method for processing ultrasound data in accordance with any of the previous claims, wherein the local processing unit applies a separate transmission scheme for the features extracted by the remote data processing facility for transmission to the remote data processing facility over the established network communication session.

11. The method for processing ultrasound data in accordance with any of the previous claims, wherein the remote data processing facility provides a user interface for configuration of a set of user input parameters, which user input parameters are input to the remote data processing facility and may comprise one or more of acquisition mode, time-gain compensation, a compression scheme, compression level, compression algorithm, maximum compression level, and minimum compression level.

12. The method for processing ultrasound data in accordance with any of the previous claims, wherein the transmission scheme and / or the acquisition settings and / or the reconstruction settings are tuned in accordance with a set of quality service metrics for the network communication session and / or image content (environmental changes, patient-to-patient variability) and / or users' inputs and / or sensors data (e.g., accelerometer)..

13. The method for processing ultrasound data in accordance with any of the previous claims, wherein the remote data processing facility is arranged for storing the raw data and the generated ultrasound images in a local or remote storage facility, for offline accessibility of the ultrasound images by the one or more local user interfaces from the storage facility.

14. The method for processing ultrasound data in accordance with any of the previous claims, further comprising the step of: determining, by the local processing unit, from the set of quality of service metrics for the network communication session a maximum network transmission bandwidth; and comparing, by the local processing unit, the determined network transmission bandwidth with a predefined threshold; generating, by the local processing unit, locally generated ultrasound images from the obtained raw channel data when determining the network transmission bandwidth to drop below the threshold; transmitting, by the local processing unit, the generated ultrasound images to the one or more local user interfaces for real-time visualization of the ultrasound images obtained by the ultrasound probe.

15. The method for processing ultrasound data in accordance with any of the previous claims, wherein the after generating, by the remote data processing facility, the ultrasound images on the basis of the received raw channel data and prior to transmitting, by the remote data processing facility, the generated ultrasound images to the one or more local user interfaces for real-time visualization of the ultrasound images obtained by the ultrasound probe, the method further comprises: applying, by the remote data processing facility, an image transmission scheme to ultrasound images for transmission to the one or more local user interfaces, and wherein the image transmission scheme comprises one or more image transmission scheme variables defining a level of compression applied to the ultrasound images prior to transmission.

16. The method for processing ultrasound data in accordance with any of the previous claims, wherein the local processing unit is also arranged for generating the ultrasound images on the basis of the raw channel data, and generates said ultrasound images when predefined network conditions are met, wherein the predefined network conditions comprise one or more of: a non-established network connection session with the remote data processing facility, a drop of network bandwidth below a predefined threshold, exceeding latency of the network above a predefined threshold, determined network instability, for example determined by an exceeding number of dropped network packages per time period.

17. The method for processing ultrasound data in accordance with any of the previous claims, the method further comprising the steps of: determining probe movement, wherein the determining discriminates between a navigation mode and an image mode; using the determined info along with the quality of service metrics to adjust the transmission scheme (e.g., level of lossy compression or network slicing settings / profile), and / or acquisition settings and / or reconstruction settings.

18. The method for processing ultrasound data according to claim 17, wherein the determining is achieved by obtaining movement data from an accelerometer comprised in a housing of the probe to detect movement.

19. The method for processing ultrasound data according to claim 17, wherein the determining is achieved by processing the ultrasound images with an image processing algorithm to detect movement.

20. The method for processing ultrasound data in accordance with any of the previous claims, wherein ultrasound parameters are transmitted to the remote data processing facility together with data for each ultrasound transmit event.

21. The method for processing ultrasound data in accordance with any of the previous claims, wherein a unique identifier is sent together with the channel data foreach ultrasound transmit event, wherein the unique identifier corresponds to a specific set of ultrasound transmit parameters.

22. The method for processing ultrasound data in accordance with any of the previous claims, wherein the transmission scheme comprises the use of non-linear neural transform coding, where the parameters of the transform coding method are defined according to the quality of service metrics to obtain a specific rate.

23. The method for processing ultrasound data in accordance with any of the previous claims, wherein the remote data processing facility is further arranged for allowing remote users through a user interface to control device settings, comprising one or more of acquisition parameters and reconstruction parameters.

24. An ultrasound image system for real-time image formation of ultrasound data, the system comprising: an ultrasound probe arranged for obtaining raw channel data from an array of transducer elements comprised in the ultrasound probe; a local processing unit, arranged for receiving the raw channel data from the ultrasound probe and for establishing a network communication session with a remote data processing facility, the local processing unit further being arranged for applying a transmission scheme to the raw channel data for transmission to the remote data processing facility over the established network communication session, and for transmitting the raw channel data to, a remote data processing facility, arranged for determining a set of quality of service metrics for the network communication session between the local processing unit and the remote data processing facility, and transmitting, the set of quality of service metrics to the local processing unit, and receiving the raw channel data from the local processing unit, and arranged for generating the ultrasound images the basis of the received raw channel data, and transmitting the generated ultrasound images to, one or more local user interfaces, arranged for receiving the generated ultrasound images from the remote data processing facility, for real-time visualization of the ultrasound images obtained by the ultrasound probe;wherein the local processing unit is further arranged for adapting the transmission scheme in accordance with the set of quality of service metrics determined by the remote data processing facility.

25. An ultrasound data acquisition device, arranged for use in an ultrasound image system according to claim 24, wherein the ultrasound data acquisition device comprises a local processing unit and is connected to the ultrasound probe of the an ultrasound image system, wherein the ultrasound data acquisition device is arranged for receiving the raw channel data from the ultrasound probe and for establishing a network communication session with a remote data processing facility, the local processing unit further being arranged for applying a transmission scheme to the raw channel data for transmission to the remote data processing facility over the established network communication session, and for transmitting the raw channel data to a remote data processing facility according to claim 25, and wherein the local processing unit is further arranged for adapting the transmission scheme in accordance with the set of quality of service metrics determined by the remote data processing facility.

26. An ultrasound data processing facility, arranged for use in an ultrasound image system according to claim 24, arranged for determining a set of quality of service metrics for a network communication session between a local processing unit of the ultrasound image system and the ultrasound data processing facility located remotely, and arranged for transmitting, a set of quality of service metrics to the local processing unit, for the local processing unit to adapt a transmission scheme in accordance with the set of quality of service metrics determined by the remote data processing facility, and receiving raw channel data from the local processing unit, and arranged for generating ultrasound images the basis of the received raw channel data, and transmitting the generated ultrasound images to one or more local user interfaces, arranged for receiving the generated ultrasound images from the remote data processing facility, for real-time visualization of the ultrasound images obtained by the ultrasound probe.

27. A computer program product embodied on a non-transitory computer- readable medium, comprising a set of instructions executable by a processor, the computer program product adapted to perform the method for processing ultrasound data in accordance with any of the previous claims 1-23.

Images