Equipment for medical image analysis

Abstract

The invention relates to an apparatus 10 for medical image analysis comprising a camera 20, a processing unit 30 and an output unit 40. The camera is configured to be placed near a system image display of a medical imaging system. The camera is configured to acquire a local image of a system image displayed on the system image display, the system image comprising medical image data of a patient and the local image comprising local image data of the patient's medical image data. The processing unit is configured to determine a plurality of images and imaging parameters, the determination comprising the use of the local image. The processing unit is configured to determine a process decision utilizing the plurality of images and imaging parameters, the process decision for either determining whether the local image data of the medical image data is suitable for further processing or whether to acquire a new local image, determine a new plurality of images and imaging parameters for the new local image, and determine a new process decision utilizing the new plurality of images and imaging parameters. The output unit is configured to output the image data.

Description

[Technical field]

[0001] The present invention relates to an apparatus for medical image analysis, a method for medical image analysis, as well as a computer program element and a computer readable medium. [Background technology]

[0002] In recent years, AI-based image analysis has shown promising results in a wide range of applications, from computer vision tasks to medical image analysis.

[0003] EP 3758015 A1 describes an imaging system comprising a medical imaging device. The medical imaging device comprises a detection unit for acquiring a first image of a patient in an imaging session and a display unit for displaying the first image on a screen. The system further comprises a mobile image processing device, separate from the medical imaging device. The mobile processing device comprises an interface for receiving a representation of the first image and an image analysis unit configured to analyze the representation and to calculate medical decision support information during the imaging session based on the analysis. The decision support information is displayed on an on-board display device of the mobile processing device.

[0004] US10790056B1 describes a system for distributing one or more examination results, each of which is associated with only one person and has a sequence of digital images produced by an imaging modality. The system includes a synchronization application configured to run within a local area network and in data communication with the imaging modalities and / or computing devices configured to display the images produced by each of the imaging modalities. The system also includes a server external to the local area network and adapted to data communication with the synchronization application, and a client-side viewing application installed on one or more of the computing devices. The client-side viewing application is configured to retrieve the examination results including unrendered data representing the sequence of digital images, locally render the unrendered data, and enable a user to manipulate the digital images. Summary of the Invention [Problem to be solved by the invention]

[0005] However, because medical devices such as X-ray systems and diagnostic applications are often used in secure environments (e.g., without internet access), AI-based image analysis software can become outdated as frequent updates are not possible due to quality and regulatory requirements.

[0006] This problem needs to be solved. [Means for solving the problem]

[0007] It would be advantageous to have an improved means for analysing medical images. The object of the invention is solved by the subject matter of the independent claims, further embodiments are incorporated in the dependent claims. It is noted that the below described aspects and examples of the invention apply to an apparatus for medical image analysis, a method for medical image analysis, as well as a computer program element and a computer readable medium.

[0008] In a first aspect, there is provided an apparatus for medical image analysis, the apparatus comprising: Camera and A processing unit; Output unit and An apparatus is provided comprising:

[0009] The camera is configured to be positioned proximate to a system image display of the medical imaging system. The camera is configured to acquire a local image of a system image displayed on the system image display, the system image including medical image data of a patient, and the local image including local image data of the medical image data of the patient. The processing unit is configured to determine a plurality of images and imaging parameters, the determining including utilizing the local image. The processing unit is configured to determine a process decision utilizing the plurality of images and the imaging parameters, the process decision including utilizing the plurality of images and the imaging parameters. Determining whether local image data of the medical image data is suitable for further processing; or determining whether to acquire a new local image, determining new images and imaging parameters for the new local image, and determining a new process decision utilizing the new images and imaging parameters; The purpose of the present invention is to carry out any of the following: The output unit is configured to output the image data.

[0010] In one example, if the local image data of the medical image data is determined to be suitable for further processing, the processing unit is configured to implement a machine learning algorithm to generate locally processed image data, the generating including utilizing the local image data of the medical image data of the patient. The output unit is configured to output the locally processed image data.

[0011] Thus, diagnosis of conditions that a patient may have captured in medical images from a medical imaging device or system, such as an X-ray system or MRI, is aided using a screen capture of the display of the medical imaging device or system.

[0012] Prior to implementation of a machine learning algorithm such as a trained neural network, certain imaging parameters such as the distance between the smartphone and the medical imaging device, the actual display settings on the display of the medical imaging device, and image quality are utilized to determine whether the machine learning algorithm is operable on an image acquired by the smartphone or whether the smartphone should reposition the smartphone for acquisition of a new image and / or adjust the display of the medical imaging device to acquire a more suitable image that is subsequently analyzed by the machine learning algorithm operating on the smartphone.

[0013] In other words, before applying the neural network for disease detection, the acquisition conditions of the smartphone camera acquisition of the image of the medical image display can be detected, the expected image quality of the smartphone camera image can be estimated, and the user can be advised to change the acquisition settings and retake the image, and then confidence level or certainty index information related to the detected disease according to the image quality can be displayed, or the neural network analysis can be operated on the smartphone image, and the acquisition conditions can be incorporated into the neural network analysis, thus improving the image quality and achieving AI disease detection.

[0014] It should be noted that the system image display of a medical imaging system need not be attached to the medical imaging unit, but may be separate from the medical imaging unit and may in fact be part of the PACS display. Thus, the system image display need not show a live image of the patient, but may show a stored image of the patient. However, the medical image display may show a current image of the patient that was acquired relatively recently. This explains what is meant by "medical imaging system" and "system image display."

[0015] In this way, diagnostic accuracy can be improved and specific machine learning algorithms can be utilized.

[0016] In one example, generating the locally processed image data includes calculating error information using a plurality of images and imaging parameters.

[0017] In one example, the processing unit is configured to utilize the plurality of images and imaging parameters to calculate at least one certainty indicator or confidence level for the locally processed image data.

[0018] In one example, the processing unit is configured to generate a certainty index or confidence level heatmap overlaid on the locally processed image data.

[0019] Thus, taking into account imaging parameters such as the quality of the image acquired by the smartphone or other imaging device allows a user viewing the processed image to understand whether there is a greater error associated with any determined diagnosis in each part of the image. Thus, for example, if there is glare in a part of the medical imaging system display captured by the smartphone, this part of the processed image may have a greater error associated with its diagnosis than other parts of the image, which allows the user to better interpret the processed image. The same applies to parameters such as the signal-to-noise of the entire image and the distance from the camera to different parts of the medical imaging system display.

[0020] In one example, the implementation of the machine learning algorithm includes utilizing multiple images and imaging parameters.

[0021] In one example, the processing unit is configured to adapt a machine learning algorithm, where the adaptation includes utilizing a plurality of images and imaging parameters.

[0022] Thus, a particular machine learning algorithm, such as a neural network, can be selected based on the image and imaging parameters, thereby allowing the best or most appropriate machine learning algorithm to be utilized. Furthermore, the selected or default machine learning algorithm can be adapted based on the image and imaging parameters so as to be optimized for analysis of image data acquired on a smartphone or other comparable device.

[0023] In one example, the processing unit is configured to utilize the plurality of images and the imaging parameters to determine a new distance between the camera and the system image display for acquisition of a new local image, and the output unit is configured to output information related to the new distance.

[0024] In one example, the processing unit is configured to utilize the plurality of images and the imaging parameters to determine a new angle between the camera's viewing axis and an axis perpendicular to the system image display for acquisition of a new local image, and the output unit is configured to output information related to the new angle.

[0025] In this way, a device such as a smartphone can use knowledge of the display screen size of the medical imaging system to determine the distance to the display screen, for example, as determined from the outer extent of the display screen. This can also be used to determine the angle between the camera and the display screen. A smartphone or other device can have more than one camera and can use triangulation or other standard distance determination methodologies employed by smartphones to determine the distance to the display screen. The processing unit can then determine that these parameters are not optimal for the image acquired by the smartphone and present information to the user, such as moving the smartphone closer to the medical system display and / or changing the angle, such as pointing it more directly at the display, so that a better image can be acquired by the smartphone.

[0026] In one example, the processing unit is configured to utilize the plurality of images and imaging parameters to determine at least one new display setting for a system image display for acquisition of a new local image, and the output unit is configured to output information related to the new display setting.

[0027] Thus, the processing unit may implement, for example, image processing software with character and icon identification capabilities, which may allow for determining display settings such as resolution, contrast, brightness, etc. of the medical system display. The smartphone or other similar device may then provide the user with information to adjust the display settings on the medical system display so that a better and more suitable image of the image data being presented on the medical image display may be obtained by the smartphone.

[0028] In one example, the processing unit is configured to utilize the plurality of images and imaging parameters to determine changes in lighting conditions within a room in which the system image display is located, and the output unit is configured to output information related to the changes in lighting conditions.

[0029] From the smartphone image of the medical image display, or other device image of the medical image display, image processing can determine if there is glare on the display screen, caused for example by lighting or sunlight coming through a window. A determination can then be made that this is leading to sub-optimal image data acquisition by the smartphone, and information can be relayed to the user to adjust the lighting conditions in the room in which the medical image display is located.

[0030] In one example, the multiple image and imaging parameters include two or more of a determined distance between the camera and the system image display, a determined angle between the camera's viewing axis and an axis perpendicular to the system image display, a determined signal-to-noise at one or more locations in the local image, a determined one or more lighting conditions in a room in which the system image display is located, and a determined display setting for the system image display.

[0031] In a second aspect, there is provided a method for medical image analysis, the method comprising: a) positioning a camera of the device near a system image display of a medical imaging system; b) acquiring, by a camera, a local image of a system image displayed on a system image display, the system image including medical image data of a patient, and the local image including local image data of the medical image data of the patient; c) determining, by a processing unit of the device, a plurality of images and imaging parameters, the determining comprising utilizing the local images; d) determining, by a processing unit, a process decision, the determining step comprising utilizing the plurality of images and imaging parameters, the process decision comprising: Determining whether local image data of the medical image data is suitable for further processing; or determining whether to acquire a new local image, determining new images and imaging parameters for the new local image, and determining a new process decision utilizing the new images and imaging parameters; determining whether or not the determination is successful; f) outputting the image data by an output unit of the device; A method is provided comprising:

[0032] In one example, if the local image data of the medical image data is determined to be suitable for further processing, the method includes: e) generating locally processed image data by implementing a machine learning algorithm by a processing unit to generate locally processed image data, the generating step comprising utilizing local image data of the patient's medical image data; Step f) comprises outputting the locally processed image data by an output unit.

[0033] According to another aspect, there is provided a computer program element for controlling one or more of the apparatuses as described above, which when executed by a processing unit is adapted to perform the method as described above.

[0034] According to another aspect, there is provided a computer readable medium having stored thereon a computer element as previously described.

[0035] The computer program element may for example be a software program, but may also be an FPGA, a PLD or any other suitable digital means.

[0036] Advantageously, benefits provided by any of the above aspects are equally applicable to all of the other aspects, and vice versa.

[0037] The above aspects and examples will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0038] Exemplary embodiments are described below with reference to the following drawings: [Brief description of the drawings]

[0039] [Figure 1] FIG. 1 shows a schematic setup of an example of an apparatus for medical image analysis. [Diagram 2] FIG. 1 illustrates a method for medical image analysis. [Diagram 3] FIG. 1 illustrates a smartphone capturing an image of a medical image shown on a display of a medical imaging system / device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0040] FIG. 1 shows a schematic example of an apparatus 10 for medical image analysis. The apparatus comprises a camera 20, a processing unit 30 and an output unit 40. The camera is configured to be placed near a system image display of a medical imaging system. The camera is configured to acquire a local image of a system image displayed on the system image display. The system image comprises medical image data of a patient and the local image comprises local image data of the medical image data of the patient. The processing unit is configured to determine a plurality of images and imaging parameters, the determination comprising utilization of the local image. The processing unit is configured to determine a process decision utilizing the plurality of images and the imaging parameters, the process decision comprising utilization of the local image. Determining whether local image data of the medical image data is suitable for further processing; or determining whether to acquire a new local image, determining new images and imaging parameters for the new local image, and determining a new process decision utilizing the new images and imaging parameters; The purpose of the present invention is to perform one of the following:

[0041] The output unit is then configured to output the image data.

[0042] According to an example, if the local image data of the medical image data is determined to be suitable for further processing, the processing unit is configured to implement a machine learning algorithm to generate locally processed image data. In that case, generating the locally processed image data may include utilizing the local image data of the patient's medical image data. In that case, the output image data output by the output unit may include the locally processed image data.

[0043] In one example, the locally processed image data is an indication of a disease classification.

[0044] In one example, the locally processed image data includes an indication of a disease classification.

[0045] In one example, the locally processed image data includes a diagnosis.

[0046] Thus, for example, an indication may be presented that a lung nodule is present, an indication of cancer, or another medical condition, or an indication of a fracture or other condition that may be indicated based on medical imaging. This indication may be in the form of a simple textual yes / no, which may have an associated confidence level or certainty indicator. It may also be in the form of an image corresponding to the local image, with a disease / condition classification superimposed on the image, indicating the location of the determined classification.

[0047] In one example, the device is a smartphone.

[0048] In one example, the processing unit may be at a location separate from the computer. For example, a camera may capture images that are sent over a network to the processing unit, which may reside in the cloud. The processed information may then be sent back to the operator of the camera. This means that the decision to capture a new image or continue with the image just captured can be made locally or remotely. Also, the processing of the image data itself to provide a medical analysis can also be performed remotely or locally, with information being sent back to the operator as appropriate.

[0049] In one example, the device is configured to download updates to the machine learning algorithm over a network and update the machine learning algorithm based on the updates.

[0050] In one example, the device is configured to download a new machine learning algorithm over a network and replace the old machine learning algorithm with the new machine learning algorithm, such that the machine learning algorithm utilized by the device becomes the new machine learning algorithm.

[0051] In one example, the output unit comprises a display screen of the device.

[0052] In one example, generating the locally processed image data includes utilizing a plurality of images and imaging parameters.

[0053] In one example, the plurality of image and imaging parameters includes display parameters associated with the system image on a system image display, such as contrast or grayscale level or dynamic range.

[0054] In one example, the processing unit is configured to implement image processing algorithms to determine the display parameters.

[0055] Thus, the processing unit can determine image and imaging parameters associated with how the image is displayed from markers associated with the image, such as written grayscale values ​​or grayscales shown in bar format. In this way, if it is determined that the local image is not satisfactory for further processing or could be improved, information can be provided to the operator to retake the image with the camera, increasing the dynamic range and / or contrast of what is being displayed on the system display.

[0056] In one example, the processing unit is configured to implement a neural network algorithm to determine the display parameters.

[0057] Thus, a series of images with different contrast and dynamic range levels can be acquired and used to train the neural network, providing ground truth information regarding the contrast levels / dynamic range for the training images. Then, when an operator acquires an image of what is being presented on the display of the medical image unit, a decision can be made based on the content in the image itself as to whether the image could be improved by adjusting the output settings of the medical image display unit.

[0058] According to one example, the processing unit is configured to utilize the plurality of images and imaging parameters to calculate at least one certainty indicator or confidence level for the locally processed image data.

[0059] According to one example, the processing unit is configured to utilize the multiple images and imaging parameters to calculate multiple certainty indices or confidence levels for multiple locations of the locally processed image data.

[0060] Thus, a confidence level regarding the disease classification can be provided, which allows the operator to better interpret and understand the analysis of the images. Those skilled in the art will also appreciate that the "certainty index" can also be interpreted as an "uncertainty index", in that the certainty index that a diagnosis is correct implicitly provides an uncertainty index related to whether the diagnosis is correct.

[0061] According to one example, the processing unit is configured to generate a certainty index or confidence level heatmap overlaid on the locally processed image data.

[0062] According to one example, the implementation of the machine learning algorithm includes utilizing multiple images and imaging parameters.

[0063] In one example, implementing the machine learning algorithm includes selecting a machine learning algorithm from a plurality of machine learning algorithms, where the selection includes utilizing a plurality of images and imaging parameters.

[0064] According to one example, the processing unit is configured to adapt a machine learning algorithm, the adaptation including utilizing a plurality of images and imaging parameters.

[0065] According to one example, the processing unit is configured to utilize the plurality of images and the imaging parameters to determine a new distance between the camera and the system image display for acquisition of a new local image, and the output unit is configured to output information related to the new distance.

[0066] According to one example, the processing unit is configured to utilize the plurality of images and the imaging parameters to determine a new angle between the camera's viewing axis and an axis perpendicular to the system image display for acquisition of a new local image, and the output unit is configured to output information related to the new angle.

[0067] According to one example, the processing unit is configured to utilize the plurality of images and the imaging parameters to determine at least one new display setting for the system image display for the acquisition of the new local image. The output unit is configured to output information related to the new display setting.

[0068] According to one example, the processing unit is configured to utilize the plurality of images and the imaging parameters to determine a change in lighting conditions within a room in which the system image display is located. The output unit is configured to output information related to the change in lighting conditions.

[0069] According to one example, the multiple image and imaging parameters include two or more of a determined distance between the camera and the system image display, a determined angle between the camera's viewing axis and an axis perpendicular to the system image display, a determined signal to noise at one or more locations in the local image, a determined one or more lighting conditions in a room in which the system image display is located, and a determined display setting for the system image display.

[0070] In one example, the machine learning algorithm is a trained neural network.

[0071] In one example, a neural network is trained using multiple medical images of patients and associated ground truth information about condition and disease diagnoses made by medical professionals.

[0072] In one example, the neural network is trained using the image and imaging parameters associated with the images used for training. Thus, the neural network can be trained based on images acquired by a medical image camera that are presented on a screen together with the image and imaging parameters. The neural network can then operate more effectively when analyzing new images. The generation of the locally processed data for the new images can then utilize the local image data of the patient's medical image data, as well as the multiple image and imaging parameters associated with the acquisition of the local images.

[0073] Those skilled in the art will also appreciate that an overall system may be formed or provided that includes the apparatus described above with respect to FIG. 1, and also includes a medical imaging system.

[0074] 2 shows a method 100 for medical image analysis, where step e) is optional. The method comprises: In a positioning step 110, also referred to as step a), positioning a camera of the device near a system image display of a medical imaging system; acquiring, in an acquiring step 120, also referred to as step b), by a camera, a local image of the system image displayed on the system image display, where the system image comprises medical image data of the patient and the local image comprises local image data of the medical image data of the patient; determining, by a processing unit of the device, a plurality of images and imaging parameters in a determining step 130, also referred to as step c), where the determining step comprises utilizing a local image; In a determining step 140, also referred to as step d), determining a process decision by a processing unit, the determining step comprising utilizing a plurality of images and imaging parameters, the process decision being: Determining whether local image data of the medical image data is suitable for further processing; or determining whether to acquire a new local image, determining new images and imaging parameters for the new local image, and determining a new process decision utilizing the new images and imaging parameters; determining whether or not the determination is successful; outputting the image data by an output unit of the device in an output step 160, also referred to as step f); has.

[0075] According to one example, if it is determined that the local image data of the medical image data is suitable for further processing, the method comprises: In a generating step 150, also referred to as step e), the method comprises generating locally processed image data by implementing a machine learning algorithm by a processing unit to generate locally processed image data, the generating step comprising utilizing local image data of medical image data of a patient; Step f) comprises outputting the locally processed image data by an output unit.

[0076] In one example, the device is a smartphone.

[0077] In one example, the method includes downloading an update to the machine learning algorithm to the device over a network and updating the machine learning algorithm based on the update.

[0078] In one example, the method includes downloading a new machine learning algorithm to the device over a network and replacing the old machine learning algorithm with the new machine learning algorithm, such that the machine learning algorithm utilized by the device becomes the new machine learning algorithm.

[0079] In one example, the output unit comprises a display screen of the device.

[0080] In one example, generating the locally processed image data includes calculating error information using the plurality of images and imaging parameters.

[0081] In one example, the method comprises calculating, by the processing unit, at least one confidence level or certainty indicator for the locally processed image data utilizing the plurality of images and imaging parameters.

[0082] In one example, the method comprises calculating, by the processing unit, a plurality of confidence levels or certainty indices for a plurality of locations of the locally processed image data utilizing a plurality of images and imaging parameters.

[0083] In one example, the method comprises generating, by the processing unit, a confidence level or certainty index heatmap overlaid on the locally processed image data.

[0084] In one example, implementing the machine learning algorithm includes utilizing a plurality of images and imaging parameters.

[0085] In one example, implementing the machine learning algorithm comprises selecting the machine learning algorithm from a plurality of machine learning algorithms, the selecting comprising utilizing a plurality of images and imaging parameters.

[0086] In one example, the method includes adapting, by the processing unit, a machine learning algorithm, the adapting comprising utilizing a plurality of images and imaging parameters.

[0087] In one example, the method includes determining, by a processing unit, a new distance between the camera and the system image display for acquisition of a new local image using the multiple images and imaging parameters, and outputting, by an output unit, information related to the new distance.

[0088] In one example, the method includes determining, by a processing unit, a new angle between the camera's viewing axis and an axis perpendicular to the system image display for acquisition of a new local image using the plurality of images and imaging parameters, and outputting, by an output unit, information related to the new angle.

[0089] In one example, the method includes determining, by a processing unit, at least one new display setting for a system image display for acquisition of a new local image using the plurality of images and imaging parameters, and outputting, by an output unit, information related to the new display setting.

[0090] In one example, the method includes determining, by a processing unit, a change in lighting conditions in a room in which the system image display is located using the plurality of images and imaging parameters, and outputting, by an output unit, information related to the change in lighting conditions.

[0091] In one example, the multiple image and imaging parameters include two or more of a determined distance between the camera and the system image display, a determined angle between the camera's viewing axis and an axis perpendicular to the system image display, a determined signal-to-noise at one or more locations in the local image, a determined one or more lighting conditions in a room in which the system image display is located, and a determined display setting for the system image display.

[0092] In one example, the machine learning algorithm is a trained neural network.

[0093] Thus, the medical imaging unit / system operates as usual, acquiring medical images, also called system images, that are displayed on the display of the medical imaging unit / system. However, the image analysis software provided to the medical imaging unit / system may become outdated and not exhibit the functionality available in the latest software. To mitigate this, the smartphone acquires an image of what is displayed on the unit / system display and analyzes this image with the latest AI algorithms to perform a gold standard diagnosis. However, before the AI ​​operates on the image, a decision is made as to whether the smartphone is suitable for such AI processing, and if not, feedback is given to the user on how to acquire a better and more suitable image of what is displayed on the unit / system display. This is facilitated by constantly improving smartphone hardware (i.e. camera resolution, computing power) and the increasing availability of high speed internet connections, which allows the adoption of (always up-to-date) AI applications in the daily routine. However, it should be noted that the smartphone could simply acquire the image and send it to another location, such as the cloud, where processing takes place. Thus, in this context, a "device" may be a single unit consisting of a camera, a processing unit, and an output unit, or may be formed from separate units in completely separate locations. However, for the sake of simplicity, the following description focuses on a device in the form of a smartphone, but as mentioned above, the description is not limited to that particular embodiment, which merely forms one example of one form of device.

[0094] Therefore, the use of smartphones offers an interesting alternative to bring AI technologies into clinical environments (especially in developing countries), where measured viewing parameters of smartphone camera images and derived information about the unit / system display can be incorporated into AI-based image classification.

[0095] It should be noted that the camera can capture an image of what is being presented on the unit / system display and transmit this over the network to a processor which performs the decision that the image is OK for further processing or that a new image needs to be captured, as well as performing further processing. Such a processor could reside, for example, in the cloud.

[0096] The apparatus for medical image analysis and the method for medical image analysis will now be described in detail in particular detail, and for this purpose a smartphone will be described, as made clear above, by way of example only, and reference will be made to FIG. 3.

[0097] FIG. 3 shows an example of how a smartphone acquires a local image of an image displayed on a display or monitor of a medical image acquisition unit or system, which in this case is an X-ray unit, but may also be an MRI unit or a PET unit or any other medical imaging unit. The smartphone then processes the image and, if necessary, gives feedback on how to take a new, better image, which may be repositioning the smartphone, changing settings on the monitor, and / or changing the lighting conditions in the room. The newly acquired image, or the original image if it was already suitable for further processing, is then provided to a trained neural network running on the smartphone, which may be frequently updated through downloaded updates. The neural network may also utilize the acquired acquisition parameters to operate more effectively, and indeed a particular neural network may be selected from multiple neural networks stored on the smartphone based on the acquisition parameters.

[0098] With continued reference to FIG. 3, the detailed workflow will now be described.

[0099] As shown in Figure 3, the smartphone is positioned relative to the medical unit display / monitor and captures a local image of the displayed system image. With regard to smartphone image capture, several different image capture parameters are detected and utilized for disease detection. These include, but are not limited to:

[0100] Smartphone distance and acquisition angle: A strong angle of the smartphone relative to the medical image display will cause image distortion, and a large distance will cause information loss. This can be measured, for example, through the display frame, the characters on the display, etc.

[0101] Overall image quality. This can be determined from the quality of the display and smartphone camera. This can be measured by overall image quality parameters such as sharpness, noise, signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), etc.

[0102] The level and window of the image to be displayed. This can be estimated via the typical grey value distribution in the image or derived from the characters on the display. This can be determined for example by image processing software which gets this information from the monitor settings shown on the unit / system display.

[0103] The lighting in the room and the reflected spots on the display. This can be determined from the overall brightness of the image and the bright spots within the image.

[0104] It should be noted that the settings for the unit / system display with respect to overall image quality and dynamic range / contrast display may be determined by a trained neural network. Thus, many images with ground truth information of monitor settings such as grayscale settings and / or contrast levels may be used to train the NN. Once an image of the unit / system is acquired, a display modality decision for the unit / system display may be made. If a decision is made that the acquired image is not suitable for further processing, one form of feedback may be, for example, for the operator to increase the dynamic range / contrast settings for the unit / system display, if appropriate.

[0105] The determined parameters are then used for three different purposes:

[0106] Advice is given to smartphone users on whether and how to optimize image capture with their smartphone and how to minimize loss in image quality (e.g. by showing positioning instructions in the interface / indicating the current image quality level).

[0107] When displaying the neural network results for disease detection on the smartphone, the achieved results can be corrected with the corresponding accuracy / uncertainty values. In case of regional or local image quality degradation, the neural network heat map can be correlated with the image quality map and the two are visualized for each disease or a disease certainty value is calculated.

[0108] Finally, image quality adapted neural networks can be applied, which can take into account measured image acquisition settings / quality parameters or these parameters can be incorporated into the training and inference of the network.

[0109] It can therefore be expected that using a smartphone to capture images on a medical image display for applications such as chest X-ray analysis can be problematic due to the expected loss of image quality, which is an inherent limitation in such scenarios, combined with lost information resulting from non-optimal display of diagnostic X-ray images on a monitor or of the medical imaging unit itself, e.g., due to the selection of certain window level settings, resizing, contrast, brightness, etc. It is also expected that environmental conditions such as lighting, smartphone positioning, and camera characteristics can affect the images undergoing AI-based analysis, resulting in a significant degradation of expected image quality and AI performance. However, the present system and method address this through detection of smartphone camera acquisition conditions prior to application of a neural network for disease detection. The expected image quality of the camera image is estimated and the user is advised to change the acquisition settings and retake the image, or when an AI analysis is performed, error bars related to the detected disease are presented, e.g., according to the image quality at different image locations, and the acquisition conditions are also incorporated into the AI ​​neural network analysis to provide a more optimized AI analysis.

[0110] In another exemplary embodiment, a computer program or a computer program element is provided, characterized in that it is arranged for carrying out, on a suitable system, the steps of the method according to one of the previous embodiments.

[0111] Thus, the computer program element is stored on a computing unit, which is also part of an embodiment. The computing unit is configured to execute or cause the execution of the steps of the above-mentioned method. Moreover, it is configured to operate the components of the above-mentioned device and / or system. The computing unit may be configured to operate automatically and / or to execute the instructions of a user. The computer program is loaded into the working memory of a data processor. The data processor is thus equipped to execute a method according to one of the above-mentioned embodiments.

[0112] This exemplary embodiment of the invention encompasses both computer programs that use the invention from the beginning and computer programs that, through updates, turn existing programs into programs that use the invention.

[0113] Moreover, the computer program element is capable of providing all the steps necessary to fulfill the procedures of the exemplary embodiments of the methods described above.

[0114] According to a further exemplary embodiment of the present invention, a computer readable medium such as a CD-ROM, a USB stick or the like is presented, which has stored thereon the computer program elements described by the previous sections.

[0115] The computer program may be stored and / or distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0116] However, the computer program may also be presented via a network such as the World Wide Web and downloaded from such a network into the working memory of a data processor. According to a further exemplary embodiment of the invention, a medium is provided for making available for downloading a computer program element, which computer program element is arranged to perform a method according to one of the aforementioned embodiments of the invention.

[0117] It should be noted that the embodiments of the present invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method type claims, and other embodiments are described with reference to device type claims. However, those skilled in the art will infer from the above and following descriptions that, unless otherwise stated, any combination of features belonging to one type of subject matter, as well as any combination between features related to different subject matters, are also considered to be disclosed together with the present application. However, all features can be combined to produce a synergistic effect that is greater than the mere sum of the features.

[0118] While the invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description are to be considered as illustrative and exemplary and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

[0119] In the claims, the term "comprising" does not exclude other elements or steps, and the singular does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. 1. An apparatus for medical image analysis, comprising: Located near a system image display of a medical imaging system; a camera for acquiring a local image of a system image displayed on the system image display, the system image including medical image data of a patient, and the local image including local image data of the medical image data of the patient; The camera; determining a plurality of images and imaging parameters, said determining including utilizing the local image, said plurality of images and imaging parameters including a determined distance between the camera and the system image display, and / or a determined angle between a line of sight of the camera and an axis perpendicular to the system image display; and determining a process decision utilizing the plurality of images and imaging parameters. The process decision: determining whether the local image data of the medical image data is suitable for further processing; or determining whether to acquire a new local image, determine new images and imaging parameters for the new local image, and determine a new process decision using the new images and imaging parameters, wherein the processing unit uses the new images and imaging parameters to determine a new distance between the camera and the system image display for acquisition of the new local image, and / or the processing unit uses the new images and imaging parameters to determine a new angle between a viewing axis of the camera and an axis perpendicular to the system image display for acquisition of the new local image. The purpose of the present invention is to the processing unit; and outputting image data, information related to the new distance, and / or information related to the new angle; Output unit and An apparatus comprising:

2. If the local image data of the medical image data is determined to be suitable for further processing, the processing unit implements a machine learning algorithm to generate locally processed image data, the generating including utilizing the local image data of the medical image data of the patient; The apparatus of claim 1 , wherein the output unit outputs the locally processed image data.

3. The apparatus of claim 2 , wherein the generating the locally processed image data comprises calculating error information utilizing the plurality of images and imaging parameters.

4. The apparatus of claim 3 , wherein the processing unit utilizes the plurality of images and imaging parameters to calculate at least one certainty indicator or confidence level for the locally processed image data.

5. The apparatus of claim 3 or 4, wherein the processing unit is adapted to generate a certainty index or confidence level heatmap overlaid on the locally processed image data.

6. The apparatus of claim 2 , wherein the implementation of the machine learning algorithm comprises utilising the plurality of images and imaging parameters.

7. The apparatus of claim 6 , wherein the processing unit adapts the machine learning algorithm, the adapting comprising utilization of the plurality of images and imaging parameters.

8. 8. The apparatus of claim 1, wherein the processing unit utilizes the plurality of images and imaging parameters to determine at least one new display setting for the system image display for acquisition of the new local image, and the output unit outputs information related to the new display setting.

9. 9. The apparatus of claim 1 , wherein the processing unit utilises the plurality of images and imaging parameters to determine changes in lighting conditions in a room in which the system image display is located, and the output unit outputs information relating to the changes in lighting conditions.

10. 10. The apparatus of claim 1, wherein the plurality of image and imaging parameters include two or more of a determined signal to noise at one or more locations within the local image, a determined one or more lighting conditions within a room in which the system image display is located, and a determined display setting for the system image display.

11. 1. A method for medical image analysis, the method comprising: a) positioning a camera of the device near a system image display of a medical imaging system; b) acquiring, by said camera, a local image of a system image displayed on said system image display, said system image including medical image data of a patient, said local image including local image data of said medical image data of said patient; c) determining, by a processing unit of the device, a plurality of images and imaging parameters, the determining step comprising utilizing the local image, the plurality of images and imaging parameters including a determined distance between the camera and the system image display, and / or a determined angle between a viewing axis of the camera and an axis perpendicular to the system image display; d) determining, by the processing unit, a process decision, the determining process decision comprising utilizing the plurality of images and imaging parameters, the process decision comprising: determining whether the local image data of the medical image data is suitable for further processing; or Obtaining a new local image; determining a new plurality of image and imaging parameters for the new local image; determining new process decisions utilizing the new plurality of images and imaging parameters; and determining, by the processing unit, whether to use the new plurality of images and imaging parameters to determine a new distance between the camera and the system image display for acquisition of the new local image, and / or determining, by the processing unit, a new angle between a viewing axis of the camera and an axis perpendicular to the system image display for acquisition of the new local image, using the new plurality of images and imaging parameters; f) outputting image data by an output unit of the device, outputting information related to the new distance by said output unit, and / or outputting information related to the new angle by said output unit. The method comprising:

12. If it is determined that the local image data of the medical image data is suitable for further processing, the method further comprises: e) generating locally processed image data by implementing a machine learning algorithm by the processing unit to generate locally processed image data, the generating step comprising utilizing the local image data of the medical image data of the patient; The method of claim 11 , wherein said step f) comprises outputting, by said output unit, said locally processed image data.

13. A computer program for controlling a system according to any one of claims 1 to 10, which, when executed by a processor, performs the method according to claim 11 or 12.

Images