Device for performing a procedure for the evaluation of skin lesions using artificial intelligence

Abstract

The present invention relates to a method for representing at least one image of a skin lesion and associated information to aid in the characterization of the skin lesion. This method comprises the following steps: an image, in particular an extreme close-up, of a skin lesion (13) is detected in an area of ​​skin to be examined by optical detection means (2) designed for this purpose, in particular a videodermatoscope, and image data based on it is provided; the skin lesion is analyzed by electronically processing the provided image data by means of an artificial neural network designed to detect and / or classify skin lesions; and at least one image (12) of the detected skin lesion (13) and information (14, 15, 16) associated with it is generated based on the analysis by the artificial neural network;where the information (14, 15, 16) associated with the image (12) comprises a reproduction of a detected predefined class of skin lesion (14) and / or a preferably numerical associated risk value (15, 16) of the skin lesion.;

Description

[0001] The invention relates to a device for generating images to assist in the characterization of skin lesions on the human body. In particular, the invention relates to a device for evaluating skin lesions or generating images of skin lesions using artificial intelligence.

[0002] A known technique for detecting skin lesions, i.e., skin changes and damage, is dermoscopy, also known as epiluminescence microscopy. This non-invasive examination method allows the skin areas to be analyzed under a microscope using polarized light illumination, reaching into deeper skin layers. The treating physician makes an assessment and diagnosis based on a visual examination of the respective skin lesion. This can then be confirmed with an additional histological examination of the skin area, which, however, requires a surgical procedure to take a tissue sample.

[0003] In US 2018 / 061 046 A1, a dermatoscopic lesion area is identified by: obtaining a dermatoscopic image and running an image classifier with a neural convolutional network on the dermatoscopic image to obtain pixel-wise lesion prediction values. Segmenting the dermatoscopic image into superpixels and calculating the average of the pixel-wise prediction values ​​for the pixels within that superpixel for each superpixel.

[0004] From US 2018 / 189 949 A1, a method for monitoring skin abnormalities on a part of a patient's skin is known, comprising the steps of receiving initial image data from an image acquisition device and identifying an initial skin mask corresponding to the skin part.

[0005] US 2014 / 313 303 A1 establishes that the evolution of a skin condition over time can be helpful in its assessment. To illustrate: A user takes pictures of their skin with a smartphone at various times. The images are then registered together, color-corrected, and presented to the user (or a physician) for review, for example, in chronological order or as one image superimposed on another. During image registration, nevi, hair follicles, wrinkles, pores, and pigmented areas can be used as key points.

[0006] Systems and methods for non-invasive clinical imaging and non-invasive imaging of subdermal blood flow, diffuse reflectance spectroscopy and computer-aided diagnosis are known from WO 2018 / 160 963 A1.

[0007] It is also known to take an image of the respective skin lesion, for example, using a videodermatoscope, and to display it enlarged and / or at least partially altered, for example, by highlighting specific spectral ranges, in appropriate output media for evaluation by the treating physician. However, with an increase in the number of images, the time required for a detailed assessment also increases. Particularly when evaluating a large number of images of different or identical skin lesions, for example, as part of follow-up examinations to monitor changes in a particular skin lesion, a reliable and time-efficient characterization of all images by the treating physician is no longer possible.

[0008] The present invention aims to overcome or at least significantly mitigate the aforementioned disadvantages of the prior art. In particular, it seeks to provide an optimized method for detecting and supporting the assessment of a skin lesion, enabling the treating physician to efficiently and reliably identify malignant skin tissue. This objective is achieved by the subject matter of the independent claim. The dependent claims represent advantageous embodiments of the present invention. Furthermore, the invention addresses other problems or proposes solutions to other problems, as will be evident from the following description.

[0009] The invention relates to a device according to claim 1.

[0010] The method for which this device is designed, but which is not claimed as such, provides a pre-characterization of a skin lesion to be examined, enabling the treating physician to perform an efficient and safe analysis of the respective skin lesion, or significantly simplifying this analysis for the physician. In particular, the simultaneous display of an image of the respective skin lesion in combination with information regarding a class of skin lesion recognized by the artificial neural network and / or an associated risk value supports an efficient and safe assessment by the treating physician, especially when examining a large number of skin lesions.

[0011] The method is preferably carried out, at least partially, using a suitably designed software program. This program can be stored on and / or executed on a storage medium or data carrier, for example, contained in an analysis unit described in more detail below.

[0012] The artificial neural network is preferably designed to recognize predefined classes or types of skin lesions. In particular, the artificial neural network is designed to recognize and / or differentiate at least between non-melanocytic and melanocytic skin lesions. The artificial neural network is preferably designed to recognize at least a plurality of the following types of skin lesions as classes: pigmented nevus (melanocytic nevus), dermatofibroma, malignant melanoma, actinic keratosis and Bowen's disease, basal cell carcinoma (basalioma), seborrheic keratosis, solar lentigo, angioma, and / or squamous cell carcinoma.

[0013] In a preferred embodiment, in addition to and / or based on the analysis of the artificial neural network, the identified classes or types of skin lesions can be output or displayed according to a preferably graded, determined probability. For example, the artificial neural network can output not only the most probable class or type of skin lesion, but also the two or three classes or types with the determined subsequent probabilities. In particular, the artificial neural network can output at least two, preferably three, or optionally more of the most probable classes or types of the skin lesion to be analyzed. Such output or display could, for example, be: basal cell carcinoma, squamous cell carcinoma, angioma, wherein the individual classes or types are preferably output in descending order of probability.Such an output or reproduction allows, in particular, a more efficient analysis of skin lesions that do not appear to be clearly distinguishable from one another and can be more efficiently narrowed down by the information provided.

[0014] In a preferred embodiment, the artificial neural network is designed to recognize predefined risk classes, particularly with regard to the malignancy of the skin lesion. Each risk class can reflect a specific stage of progression of a skin lesion. For example, the artificial neural network can be configured to recognize at least two, preferably at least three, different stages of progression and thus corresponding risk classes of a given skin lesion. These can be distinguished, for example, as low, medium, and high risk classes. A higher risk class can encompass stages of progression of skin lesions that are considered more risky for humans or that require timely treatment and / or surgical intervention.Furthermore, the artificial neural network is preferably designed to distinguish between a multitude of different stages of progression of a skin lesion type. The respective assignment to corresponding risk classes can be performed by the artificial neural network itself and / or by a subsequent calculation.

[0015] In a further preferred embodiment, each risk class can also comprise several skin lesion types that are distinguishable or recognizable by the artificial neural network. A higher risk class can include skin lesion types that are considered more risky for humans. These are, in particular, classes that require timely treatment and / or surgical intervention. Less risky types, especially skin lesion types that do not require timely treatment and / or surgical intervention, can be classified as less risky and thus assigned to a lower risk class.

[0016] The classification of a detected skin lesion type into a lower or higher risk class can be performed using the artificial neural network and / or in a further processing or calculation step of the procedure. For example, several risk classes and these comprehensive skin lesion types can be stored in a predefined and / or adaptable lookup table. After a specific skin lesion type is detected and / or its progression is tracked by the artificial neural network, the corresponding risk class can be determined and / or calculated and then output.

[0017] In a preferred embodiment, based on a respective detected or calculated risk class of skin lesion type and / or based on a respective detected progression level of the skin lesion, a preferably numerical risk value for the respective skin lesion is output and / or calculated. The numerical value preferably lies between 0.1 and 1. A value between 0.1 and 0.2 can be defined as low, a value between 0.2 and 0.49 as medium, and a value between 0.5 and 1.0 as high risk. The calculation of the risk value can be performed using the artificial neural network and / or in a further processing or calculation step of the method.

[0018] In a preferred embodiment, the output or display of at least one image of the detected skin lesion, the analysis of the skin lesion, and / or the display of the information associated with the image are performed in real time. Furthermore, the image of the detected skin lesion is preferably a live or video image of the skin lesion detected or recorded by the detection means. This image can be captured or recorded, for example, using a video dermatoscope and displayed by means of associated output means, such as a display or monitor of an analysis unit connected to the detection means, such as a computer, PC, tablet, or smartphone. By providing the information in real time, i.e.,Without significant time delay, in addition to simplified positioning of the detection devices on the respective skin lesion, a significantly simplified characterization of the lesion by the treating physician is enabled due to the information being provided preferably instantly and related to the lesion shown.

[0019] The image data provided by the acquisition equipment comprises numerous individual images of the skin lesion under examination. These can be provided, for example, as part of a continuous video stream from a videodermatoscope. Each individual image is analyzed separately using the artificial neural network. Based on this analysis, the artificial neural network can then identify and / or classify the skin lesion. The data and information obtained in this process can then be combined to generate an overall assessment result, which is then displayed as corresponding to the image of the skin lesion. In particular, an average of previously acquired individual results or classifications of the skin lesion can be calculated and subsequently displayed.

[0020] The artificial neural network is preferably a known convolutional neural network (CNN). The artificial neural network preferably has at least one hidden layer, more preferably between 1 and 100, and most preferably between 1 and 20 hidden layers. In a preferred embodiment, the artificial neural network has between 2 and 10,000, and preferably between 2 and 1,000, neurons per layer.

[0021] The artificial neural network is preferably trained to recognize a predefined classification based on knowledge acquired through supervised learning. In this process, the artificial neural network is provided with a large number of skin lesions of different types, stages, and / or progressions, corresponding to a specific diagnosis, preferably as image data, for training purposes. Such training can be verified in a subsequent validation process with regard to the recognition accuracy of the trained artificial neural network. Furthermore, an artificial neural network already trained with a large dataset can be used via transfer learning (a technique known per se) and adapted to the specific application with minor parameter changes.Training and validating the artificial neural network can be done using Python TensorFlow and / or Python Keras, for example. Image processing, provisioning, and / or mapping can be performed using OpenCV2.4.

[0022] The artificial neural network can also be trained to further improve previously acquired knowledge during the ongoing analysis of skin lesions from the supplied image data. This means that the artificial neural network is preferably self-learning and continuously expands and improves its knowledge during its ongoing use in the analysis of skin lesions. For example, information provided by the treating physician regarding a specific diagnosis for a recorded skin lesion can be taken into account.

[0023] According to the invention, the device is designed to capture an overview image of a human body region containing a multitude of skin lesions, preferably a so-called "clinical image," and / or to automatically assign a detailed image of a skin lesion to a corresponding skin lesion in a captured overview image, preferably by means of electronic data processing. The overview image can, for example, be a view or representation of a human body part or region, such as a human back view. A detailed image, as used here, is understood to be a capture of a single skin lesion, preferably from close proximity to the skin surface.

[0024] The overview image is captured using the available equipment. This equipment may include not only equipment for individual detail shots but also additional equipment, such as a preferably high-resolution digital photo or video camera for capturing the overview image. Assigning a captured detail shot to its corresponding overview image can be done manually using an appropriate input device or automatically via electronic image processing. In particular, this can be achieved using a comparison algorithm that compares the respective images and, upon detecting a match in the image data, makes the corresponding assignment. For this purpose, a feature recognition algorithm based on an OpenCV library, for example, can be used.

[0025] According to the invention, the device is designed to compare a newly acquired image, in particular a detailed image, of a skin lesion with previously acquired images. Based on this comparison, the image is then assigned as a follow-up image to an existing image or a new image is created as the initial image of a skin lesion.

[0026] For classification purposes, each image can be compared with existing detailed images and a corresponding overview image. For example, a suitable algorithm compares a captured detailed image with existing detailed images in the patient's database. If the image analysis detects matches with existing images, this new image can be marked as a follow-up image and / or assigned to the lesion in the overview image as a new image.

[0027] According to the invention, the device is configured to perform an analysis of one or more skin lesions by electronically processing a captured overview image using an artificial neural network to detect and / or classify the respective skin lesion. In particular, the artificial neural network can be configured to detect and / or classify a multitude of skin lesions in a captured overview image. The analysis of the skin lesions in an overview image can be performed in parallel or in the background to the analysis of a detailed image of a skin lesion.

[0028] According to the invention, the device is designed to output information if a detailed image of a skin lesion has not yet been acquired from an associated overview image, particularly if a predefined classification and / or risk value or risk factor has been determined based on the analysis of the overview image by the artificial neural network. For example, a warning message or a graphic highlight of a skin lesion can be displayed on a representation of the overview image. Based on this, the treating physician can acquire a detailed image of the respective skin lesion for a more precise assessment.

[0029] In a preferred embodiment, the device is designed to perform a preferably regular check of the recency of each detailed image of a skin lesion relative to its associated overview image. If a predefined, preferably absolute, time value for a detailed image is exceeded, for example, for predefined months or years, and / or if there are significant deviations between the recency or acquisition values ​​of different detailed images, for example, from a respective stored acquisition date, information, such as a warning message or a graphic highlight of a skin lesion, can then be displayed on a representation of the overview image. This can include references to detailed images that are older than previously recorded and / or have not been recently taken or updated by the treating physician.This can be done in particular for skin lesions for which a predefined classification and / or a predefined risk factor has been determined based on the analysis of the overview image by the artificial neural network.

[0030] In a preferred embodiment, the artificial neural network is designed to further improve previously learned knowledge in the analysis of skin lesions in a respective overview image from the supplied image data.

[0031] This is preferably done continuously and can be performed in parallel or in the background during the analysis of a detailed image of a skin lesion. Information provided by the treating physician regarding a specific diagnosis of a recorded skin lesion can be taken into account.

[0032] In a preferred embodiment, the captured images of skin lesions are stored in a storage unit, for example, an internal or external storage unit of an analysis unit. These images can then be analyzed using the artificial neural network as part of a preferably periodic analysis. Even older images can be analyzed using new insights from the artificial neural network. If a skin lesion classification deviates from the identified classifications and associated information or diagnoses, a corresponding message can be generated.

[0033] The respective images, representations and / or information can be output using output devices such as a display or monitor of an analysis unit connected to the acquisition devices, such as a computer, PC, tablet or smartphone.

[0034] The analysis unit can be, for example, a computer such as a PC, tablet, or smartphone, or it can include such a computer or devices. The analysis unit preferably includes at least one internal or external storage unit. The artificial neural network and the data required for its operation can be stored on this storage unit in a manner known per se. The analysis unit is also preferably configured to store and execute a software program. This program can preferably be configured to carry out the method. The analysis unit can also have at least one interface for connecting the data acquisition devices and / or external or additional output devices. Furthermore, the analysis unit can have a communication interface for connecting to an external data server and / or the internet.The analysis unit may also be trained to perform electronic processing at least partially with the aid of and / or based on information provided by external servers and / or database resources.

[0035] To avoid repetition, reference is made to the above description of the method. In particular, the features of the method described above shall also be deemed disclosed and claimable for the device according to the invention, and vice versa.

[0036] Further advantages, features and details of the invention will become apparent from the following description of preferred embodiments and from the drawings, which show: Fig. 1 a schematic representation of a preferred embodiment of the device according to the invention; Fig. 2 a flowchart of a preferred embodiment of the method; Fig. 3 a flowchart of a preferred embodiment of the assignment of a captured detail image to a captured overview image; Fig. 4a a preferred representation in the output according to the invention of an image of a skin lesion with associated information; Fig. 4b a preferred representation in the inventive output of the image of a skin lesion and its assignment to an overview image; and Fig. 4c a preferred representation in the output according to the invention of a plurality of images of captured skin lesions and the respective associated information.

[0037] Fig. 1 Figure 1 shows a preferred embodiment of a device according to the invention. The device comprises optical detection means 2, which are designed to capture an image, in particular a detailed image, of a skin lesion 13 of a patient P. The detection means 2 preferably provide digital image data or a signal representing such data based on the captured image. The detection means 2 preferably comprise a videodermatoscope known per se.

[0038] This device can be operated in micro-photography mode to capture detailed images of a skin lesion. The acquisition means 2 can also include a preferably high-resolution digital image or video camera 3. This camera can be designed to capture detailed images and / or an overview image of a patient's skin area.

[0039] The device also includes an analysis unit 1 for the electronic processing of the provided image data using an artificial neural network. The analysis unit comprises a processor and / or storage unit 7. The artificial neural network can be stored on this unit or executed on it to analyze the image data. The artificial neural network is designed and configured for the detection and / or classification of skin lesions. The artificial neural network can access data stored in the storage unit 7 and / or an external server or storage unit 5. This external server or storage unit 5 can be connected to the processor and / or storage unit 7 via a communication interface 6 of the analysis unit 1. The communication interface 6 can also be configured to connect the acquisition devices 2 and 3 to the analysis unit 1.The communication interface 6 can enable wireless and / or wired communication with the acquisition devices 2,3 and / or the external server or an external storage unit 5.

[0040] The analysis unit 1 is preferably configured to include or provide software particularly suitable for carrying out the method. This software can be stored on and / or executable from the processor and / or memory unit 7. The analysis unit 1 also preferably has a user interface for controlling the analysis unit 1 and / or any software running on it. This interface can include input devices known per se, such as a keyboard, mouse, and / or a touchscreen.

[0041] The device also includes output means 4, which are wirelessly or wired connected to, or encompassed by, the analysis unit 1. The output means 4 are preferably configured for the graphical display of information. In particular, the output means 4 may include a screen and / or a touch display. The output means 4 may also be configured to provide acoustic signals or warnings. The output means are particularly configured to provide an image of a detected skin lesion with at least one associated piece of information based on the analysis by the artificial neural network. The analysis unit 1 is preferably configured to provide the output of the at least one image of the detected skin lesion, the analysis of the skin lesion, and / or the display of the information associated with the image in real time.

[0042] Fig. 2 Figure 1 shows a flowchart of a preferred embodiment of the method. In a first step S1, a detailed image of a skin lesion is acquired using the acquisition devices 2. These then provide a single image or a video image containing multiple images (S2). In a next step S3, quality control is performed, in particular to determine whether the image quality of the acquired image, for example with regard to image sharpness and illumination, is sufficient for evaluation by the artificial neural network. If the image quality is insufficient, for example due to image blurring, steps S1-S3 are repeated. If the image quality is sufficient, the associated image data is electronically processed as part of the analysis by the artificial neural network. This may include a first pre-characterization step S4.This process first determines whether or not a skin lesion is present. If it is not a skin lesion, a corresponding output can be generated and / or steps S1 to S4 can be repeated.

[0043] In a further step, S5, the artificial neural network then performs a precise detection and / or classification of the skin lesion. This can involve, for example, identifying a specific class or type of skin lesion. The artificial neural network can also detect the specific progression of the skin lesion. It then provides an output signal corresponding to the detection or classification for further processing. Furthermore, the artificial neural network can be configured to determine or calculate a risk factor based on the detected / classified skin lesion. If the skin lesion is assessed as suspicious during the analysis, a corresponding output can be provided to a user.

[0044] Alternatively, the determination or calculation of the risk factor can be performed in a subsequent calculation step. Based on the provided data regarding the type and / or progression of the skin lesion, this step can assign the lesion to a predefined risk factor or classify it within a predefined range of values. A suitable software algorithm can be provided for this purpose.

[0045] In a further step S6, an image of the recorded skin lesion is then output along with at least one additional piece of information, based on the data provided by the artificial neural network. This can be at least one pre-characterization parameter or classification information 14 and / or at least one determined risk factor 15,16.

[0046] The process for analyzing each image of the skin lesion is preferably performed in real time. This process can be carried out, in particular, by a suitable software program running on the analysis unit described above. The results of the aforementioned steps can be graphically displayed to a user using output devices, such as a display and / or output unit.

[0047] Fig. 3 Figure 1 shows a flowchart of a preferred embodiment for assigning a captured detail image to a captured overview image. One or more overview images can be provided by the acquisition means 3 and preferably show a larger area of ​​skin or body of a patient, such as the upper body in a posterior or frontal view. The overview image can also be a full-body image in a posterior or frontal view. The overview image preferably shows a multitude of skin lesions and enables the assignment of captured detail images for the purpose of improved retrieval, for example, during repeat examinations.

[0048] In steps S1' to S3', detailed images of a skin lesion are captured, a corresponding image or video image is generated, and a subsequent quality control check is performed, particularly regarding image sharpness and illumination, analogous to the previously described steps S1 to S3. In a subsequent step S7, the provided image data of the detailed image is compared with the image data of at least one previously captured overview image. A corresponding, and known, algorithm compares the image data and, if a match is found, automatically assigns the individual image to the overview image. If no match is found, the image can be saved as a new capture and / or another detailed image (S1'-S3') can be captured. The corresponding assignment then takes place in a further step S8.The assignment can be displayed directly or presented to the user as a suggestion for explicit confirmation. In a further step (S9), the result of the assignment can then be displayed. For example, the detailed view can be positioned at the appropriate location within the overview image, and / or a marker can be placed in the overview image with a link to the detailed view. A user can then, for example, open the assigned detailed view by clicking on the corresponding location within the overview image.

[0049] Fig. 4a-4c Figure 1 shows a preferred representation using output means 4 of a device according to the invention or corresponding screenshots of a graphical interface 9 of a software program designed to execute the method.

[0050] The output or graphical interface 9 comprises an image 12 of the recorded skin lesion 13. Preferably, a live image, i.e., a real-time image from a video recording, for example, captured with a videodermatoscope, is displayed. The output or graphical interface 9 preferably includes navigation and / or control means 10 with which, for example, different views and / or magnification levels of the image 12 can be selected. The output or graphical interface 9 also preferably includes information about the examined patient 11. Furthermore, the output or graphical interface 9 includes one or more pieces of information 14, 15, 16, which are assigned to the respective depicted skin lesion 13 and which are based on the recognition and / or classification of the artificial neural network. The information 14, 15, 16 is preferably provided at least partially in real time.

[0051] The output information can include, in particular, (pre-)classification information. This can include an indicator or parameter representation 14a, which indicates whether the analyzed skin anomaly or the recorded skin area is a skin lesion 13. Furthermore, the classification information can include an indicator or parameter representation 14b, 14c, which indicates whether the analyzed skin lesion is melanocytic or non-melanocytic. The output information can also include the respective recognized class or type of skin lesion to be analyzed. For example, it can indicate whether the skin lesion is a melanoma, nevus, basal cell carcinoma, etc. The information can also include two or three of the classes or types of skin lesion to be analyzed that were recognized with the highest probability by the artificial neural network, preferably in descending order of probability.

[0052] The output information can also include a representation of a risk factor 15, which indicates how health-relevant or risky the analyzed skin lesion is classified. This can be output in numerical and / or graphical form 15a. Alternatively or additionally, a corresponding graphical representation can be provided on a predefined comparison scale 15b. This scale can be divided into at least low, medium, and high risk. The output of the aforementioned information preferably occurs in real time based on the recorded skin lesion and the analysis of the artificial neural network.

[0053] The information provided may additionally include a mean value and / or an average risk factor in numerical and / or graphical form. This may be based on several individual analyses of a recorded skin lesion, for example, if several individual images of a specific skin lesion are recorded and analyzed.

[0054] Fig. 4b shows the assignment of a newly recorded detailed image 12 to an already recorded overview image 17. This is done as before with reference to Fig. 3 As described, an automatic assignment of the detailed image 12 to an overview image 17 of patient P is performed by means of a corresponding image comparison. The result of the assignment can be displayed in a separate window or area of ​​the graphical user interface. In particular, a corresponding match can be highlighted from a large number of lesions 19a,... 19n detected in an overview image 17 (19a). In addition, the corresponding position on the body or in the overview image 17 can be displayed by means of an indicator 18. The result of the automatic assignment can preferably be adjusted manually by the user, for example, if the automatic assignment is incorrect.

[0055] Adjacent to a preferably live representation of the captured skin lesion 13 in Figure 12, a reference image or the last detailed image 12' captured for the corresponding lesion in the overview image can be displayed. The newly captured detailed image 12 can be recognized as a repeat or follow-up image and saved accordingly. This allows image history data for a respective lesion or position in an overview image to be recorded.

[0056] Fig. 4c Figure 1 shows a preferred representation in the output of a plurality of images of captured skin lesions and their respective associated information according to the invention. A lesion history can be displayed for each captured skin lesion, consisting of a plurality of detailed images 21a,...,21n, which were captured at intervals, for example, during individual examinations. For each detailed image, associated information such as a risk factor can also be displayed, and / or an assignment to an overview image 17 can be provided by means of a position indicator 20. The display can also include a subdivision of the respective lesions into risk classes such as high, medium, or low. The lesions for which a high risk factor has been determined (22a,...2n) can be displayed separately or highlighted for a user. The other risk classes, such as medium (23a,...,23n) and low (24a,...,24n) can also be represented.

[0057] This method of presentation allows a treating physician to regularly and closely examine particularly risky lesions. Furthermore, the artificial neural network can be configured to analyze the respective skin lesions in a patient's existing image database, either concurrently or in the background, during the examination of detailed images. This can be done based on the captured overview images. Changes in lesions can be detected and / or verified in these overview images. Additionally, the system can check whether the user is continuing to examine a particular lesion using epiluminescence microscopy, i.e., by capturing detailed images. If not, the system can be configured to alert the user to potential abnormalities and / or recommend including a suspicious lesion in the epiluminescence microscopy examination.The embodiments described above are merely exemplary, and the invention is by no means limited to the embodiments shown in the figures, but only to the scope of protection of the claims. Bezugszeichenliste

[0058] 1 Analysis unit 2 Data acquisition device 3 Data acquisition device for overview image 4 Output device 5 External server / data storage 6 Communication interface 7 Processor / memory unit 8 User interface 9 Graphical user interface 10 Navigation / control device 11 Patient information 12,12' Image of skin lesion (detailed image) 13,13' Skin lesion 14a-c (Pre-)classification information 15a Numerical representation of risk value 15b Graphical representation of risk value 16 Average risk value 17 Image of overview image (overview image) 18 Automatic assignment indicator 19a Display of assigned skin lesion 19b,...,n Display of alternatively assignable skin lesions 20 Position indicator 21a,...,n Lesion history 22a,...,n Lesion overview high risk 23a,..., n Lesion overview medium risk 24a,...Lesion overview low risk Patient / Skin area S1,S1`Capture S2,S2'Image / Video image S3,S3`Quality control S4Pre-characterization step S5Recognition and / or classification S6Output assessment S7Image comparison S8Image matching S9Output matching result.

Claims

1. A device for carrying out a method for displaying at least one image of a skin lesion and associated information to assist in characterizing the skin lesion, the device comprising: optical capturing means (2), in particular a video dermatoscope (2) configured to capture a picture, in particular a close-up picture, of a skin lesion (13) in an area of skin to be examined, and to provide image data based thereon, an analyzing unit (1) for electronically processing the provided image data by means of an artificial neural network configured to identify and / or classify skin lesions, and output means (4) configured to output at least one image (12) of the captured skin lesion (13) and information (14, 15, 16) associated with it based on the analysis by means of the artificial neural network, wherein the information (14, 15, 16) associated with the image (12) comprises a rendition of an identified predefined class of the skin lesion (14) and / or an associated numerical risk value (15, 16) of the skin lesion, the image data comprising at least two, preferably a plurality of individual images of the skin lesion (13), the analyzing unit (1) being configured to analyze each of the individual images of the skin lesion (13) by means of the artificial neural network, and the device being configured to calculate an overall evaluation result (16) of the individual images with respect to an identification and / or classification in order to output the information associated with the image, the artificial neural network being configured to identify predefined risk classes, in particular with respect to a malignity of the skin lesion (13), and / or an analysis of the skin lesion (13) by the analyzing unit (1) comprising calculating a risk value (15, 16) based on an identified risk class of the skin lesion, the device being further configured to capture an overview picture (17) of a human body region comprising a plurality of skin lesions, and / or automatically link a close-up picture (12) of a skin lesion with a corresponding skin lesion in a captured overview picture (17), preferably by electronic data processing, and the device being configured to compare a newly captured picture of a skin lesion (12) with previously captured pictures (12') and, based thereon, link the picture as a follow-up picture or newly file the picture as a first picture of a skin lesion, the device being configured to analyze one or more skin lesions (13) by electronically processing the captured overview picture (17) by means of the artificial neural network in order to identify and / or classify the respective skin lesion, and if a close-up picture has not been captured yet of a respective skin lesion, the device being configured to display information indicating for which skin lesion a predefined classification and / or a predefined risk value has been determined based on the analysis of the overview picture by the artificial neural network.

2. The device according to claim 1, wherein the artificial neural network is configured to identify predefined classes of skin lesions, in particular non-melanocytic and melanocytic skin lesions, and / or the classes melanocytic nevus, dermatofibroma, malignant melanoma, actinic keratosis and Bowen's disease, basal-cell carcinoma (basalioma), seborrheic keratosis, solar lentigo, angioma, and / or squamous cell carcinoma.

3. The device according to any one of the preceding claims, wherein the outputting of the at least one image (12) of the captured skin lesion (13), the analysis of the skin lesion, and / or the displaying of the information (14, 15, 16) associated with the image takes place in real time.

4. The device according to any one of the preceding claims, wherein the artificial neural network is configured to identify a predefined classification based on knowledge preferably taught by supervised learning, and / or wherein the artificial neural network is configured to further improve previously taught knowledge while analyzing the skin lesion from the supplied image data.

5. The device according to any one of the preceding claims, wherein the artificial neural network is a convolutional neural network (CNN), and / or wherein the artificial neural network has at least one hidden layer.

6. The device according to claim 1, wherein the device is configured to check a currentness of a respective close-up picture belonging to an overview picture (17) and output information if a predefined time value has been exceeded and / or in the event of deviations from currentness values of close-up pictures (12, 12'), in particular for skin lesions for which a predefined classification and / or a predefined risk value has been determined based on the analysis of the overview picture by the artificial neural network.

7. The device according to any one of claims 1 to 6, wherein the artificial neural network is configured to further improve previously taught knowledge during the analysis of the skin lesions in the overview picture (17) from the supplied image data.

8. The device according to any one of the preceding claims, wherein the device comprises a memory unit (7) in which the captured pictures are stored, and the device is configured to analyze the stored pictures, preferably periodically, by means of the artificial neural network.

9. The device according to any one of the preceding claims, wherein the analyzing unit (1) comprises a computer, a PC, a tablet or a smartphone, and the output means (4) comprise a display or a monitor.

Images