Method for detecting at least one fluorescence pattern on an immunofluorescence image of a biological cell substrate

Description

[0001] The invention relates to a method for digital image processing suitable for detecting at least one fluorescence pattern on an immunofluorescence image of a biological cell substrate.

[0002] For the purpose of obtaining information for a diagnostic question, biological substrates, such as a cell substrate, can be processed by means of so-called immunofluorescence and then the resulting immunofluorescence images of the substrate can be analyzed.

[0003] Indirect immunofluorescence microscopy is an in vitro test for the detection of circulating human antibodies against specific antigens in order to answer or assess a diagnostic question. Such antigens are present, for example, in certain areas of cell substrates, particularly the skin layers of primates. The substrate is therefore a cell substrate that is incubated with a patient sample in the form of blood or diluted blood, or blood serum or diluted blood serum. The patient sample therefore potentially contains certain antibodies that may indicate the presence of a disease in the patient. Such primary or specific antibodies can then bind antigens of the substrate.Such bound primary antibodies can then be labeled by allowing secondary antibodies, preferably anti-human antibodies, to bind to the bound primary antibodies in a further incubation step. These secondary antibodies can then be visualized later by labeling the secondary antibodies with a fluorescent dye. Such a fluorescent dye is preferably a green fluorescent dye, in particular the fluorescent dye FITC. Such binding of a primary antibody together with a fluorescently labeled secondary antibody can then be visualized later by irradiating the substrate with excitation light of a specific wavelength, thus exciting the bound fluorescent dyes to emit fluorescent radiation.

[0004] A biological cell substrate can in particular be a cell smear from cell lines, a colony of cell lines and / or biological tissue with biological tissue cells, in particular from a tissue section.

[0005] Depending on the fluorescence pattern on the respective substrate, the detected autoantibodies can be assigned to disease groups. The task therefore arises of detecting one or more fluorescence pattern types in a fluorescence image from an indirect immunofluorescence microscopy using digital image processing during immunofluorescence microscopy on a cell substrate incubated in the prescribed manner.

[0006] Particularly when considering diseases such as bullous autoimmune dermatoses that affect the human skin, immunofluorescence image analysis can provide information as to whether any bullous autoimmune dermatosis is present at all and, preferably, also a differentiation as to which type or subgroup of bullous autoimmune dermatoses is present.

[0007] For this purpose, sections of the esophagus of a monkey and / or so-called salt-split skin substrates are sometimes subjected to immunofluorescence as biological cell substrates.

[0008] A so-called salt-split skin is a biological cell substrate in the form of a skin substrate from a primate, preferably a monkey. In such a skin substrate, the epidermis is separated from the dermis by injecting a saline solution, particularly a 1 molar sodium chloride solution.

[0009] Document US2019 / 0371425A1 discloses a classification system and a classification method related to the detection of autoantibodies using immunofluorescence. It discloses an input unit into which multiple cell immunofluorescence images are input, and wherein a processor applies a plurality of convolution neural networks to perform a classification of the images.

[0010] Document EP4016082A1 discloses a method and a device for detecting a presence of a fluorescence pattern type on an organ section by means of immunofluorescence microscopy, wherein a partial area of the fluorescence image is determined which is relevant for a formation of the fluorescence pattern type to be detected.

[0011] Document D3 (Jiang Guan-Ting et al: "Automatic HEp-2 cell segmentation in indirect immunofluorescence images using deep learning", Spie smart structures and materials + nondestructive evaluation and health monitoring, 2005, San Diego, California, United States, Spie, US, Vol. 11792, April 20, 2021 (2021-04-20), pages 1179205-1179205, XP060140805, ISSN: 0277-786X, DOI: 10.1117 / 12.2590426, ISBN: 978-1-5106-4548-6) discloses an automatic segmentation of indirect immunofluorescence images using deep learning, in particular convolutional neural networks, whereby the images represent so-called HEp-2 cells.

[0012] The object of the present invention is to analyze an immunofluorescence image of a biological cell substrate in an automated manner using at least one neural network to determine whether at least one fluorescence pattern is present in the immunofluorescence image of the cell substrate.

[0013] The object of the invention is achieved by the method according to the invention. According to the invention, an immunofluorescence image is first provided, which represents the staining of a biological cell substrate by a fluorescent dye.

[0014] Furthermore, according to the invention, segmentation information is determined, which comprises at least a first and a second segmentation region. The respective segmentation regions each represent a respective cell substrate region of the cell substrate. This segmentation information is determined by segmenting the immunofluorescence image using a first neural network.

[0015] Furthermore, a boundary region representing a transition from the first cell substrate region to the second cell substrate region in the fluorescence image is determined based on the segmentation information. Furthermore, a plurality of partial images of the immunofluorescence image are selected along the determined boundary region. Furthermore, a confidence measure of the presence of the fluorescence pattern is determined based on the plurality of partial images using a second neural network.

[0016] In order to illustrate possible advantages of the method according to the invention, various aspects of this method according to the invention will now be discussed in more detail.

[0017] The fundamental question of analyzing an immunofluorescence image for the presence of an expected fluorescence pattern is well known in immunofluorescence. Since the expected fluorescence patterns vary depending on the diagnostic question, an image processing method for an immunofluorescence image is particularly powerful when it is specifically adapted to the expected fluorescence patterns for analysis using a neural network. Simply processing a large, entire immunofluorescence image using a neural network, which considers all image information of the entire immunofluorescence image, requires a highly complex neural network that must perform a particularly large number of computational operations.Furthermore, such a neural network with a high complexity and many degrees of freedom is particularly difficult to train, since a large amount of training data is required to achieve generalization of the neural network.

[0018] Therefore, the proposed method according to the invention explicitly deviates from such an approach of an overall analysis of an entire immunofluorescence image using a neural network in order to be particularly efficient.

[0019] An exemplary immunofluorescence image FB of a cell substrate, here preferably in the form of a salt-split skin, is shown in the Figure 1 shown.

[0020] The immunofluorescence image FB shows a cell substrate area ED for the cell substrate, which represents an epidermis. Furthermore, the immunofluorescence image FB shows an area DE, which represents a dermis. In salt-split skin, a so-called intermediate area or blister is clearly visible as the substrate area BL. The first cell substrate area is therefore preferably the epidermis, while the second cell substrate area is preferably the space or blister between the dermis and epidermis.

[0021] The Figure 1 also shows a so-called background area or background BG in the fluorescence image FB.

[0022] A transition or border area between the epidermis ED and the bladder BL is preferably also referred to as the bladder roof BD. In the immunofluorescence image of the Figure 1 Fluorescence is present along this bubble roof BD as a border area.

[0023] Furthermore, a border area BB is formed as the so-called blister floor, which is a border area between the blister BL (or the interspace) and the dermis DE. In the example of the fluorescence image FB of the Figure 1 There is no fluorescence along the bladder floor.

[0024] If fluorescence is present along at least one of the border areas, i.e., along only the blister roof, only the blister base, or both the blister roof and the blister base, this is an indication of a principal pemphigoid disease. Therefore, if the presence of a fluorescence pattern is detected in a border area according to the invention, this is an indication of a principal pemphigoid disease.

[0025] If the presence of a fluorescence pattern is detected, preferably in a specific border area, which is preferably the blister roof BD, information can be obtained for the question of possible candidate diseases, allowing certain groups of diseases to be ruled out. Therefore, information as to whether fluorescence is present along the border area or the blister roof BD is advantageous for a clinician. Thus, if the previously determined border area between certain cell substrate areas, such as the epidermis ED and the interspace or blister BL, is analyzed and the presence of a fluorescence pattern is detected, a specific subgroup of bullous autoimmune dermatoses can possibly be considered, such as bullous pemphigoid, which is the most common bullous autoimmune dermatosis in Europe.In other words, if fluorescence is present along the so-called blister roof (BD), this may be an indication of two specific target antigens, which in turn may be an indication of bullous pemphigoid.

[0026] If the presence of a fluorescence pattern is detected preferably in a certain border area, which is preferably the blister base BB, this may be an indication of rarer pemphigoid diseases outside the group of bullous pemphigoid, such as epidermolysis bullosa acquisita.

[0027] Preferably, the confidence measure is also output.

[0028] Preferably, the confidence measure is determined in such a way that firstly a respective partial image confidence measure is determined for a respective partial image by means of the second neural network and then the confidence measure is determined on the basis of the partial image confidence measures.

[0029] Preferably, the plurality of partial image regions are randomly selected from the immunofluorescence image along the border region.

[0030] Preferably, the second neural network further determines a respective brightness value for a respective partial image and then determines an overall brightness value on the basis of these respective brightness values.

[0031] Preferably, the segmentation information is further determined in such a way that the segmentation information comprises the first, the second, and also a third segmentation region. The third segmentation region preferably represents a third cell substrate region. Here, too, the segmentation information is determined by segmenting the immunofluorescence image using the first neural network. Preferably, the plurality of partial images (TB1, ..., TBX) are partial images of a first type. Furthermore, preferably, a second boundary region, which represents a second transition from the second cell substrate to the third cell substrate, is determined on the basis of the segmentation information. Preferably, furthermore, a plurality of partial images of a second type are selected from the immunofluorescence image along the second boundary region.Preferably, a second confidence measure of a presence of a second fluorescence pattern is determined on the basis of the plurality of partial images of the second type by means of a third neural network.

[0032] Furthermore, a computer program product is proposed comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method for digital image processing.

[0033] A data carrier signal which transmits the computer program product is also proposed.

[0034] Furthermore, a device for detecting at least one fluorescence pattern on an immunofluorescence image of a biological cell substrate is proposed, comprising.

[0035] The device further comprises at least one computing unit which is designed to carry out the following steps: determining segmentation information comprising at least a first and a second segmentation region, wherein the segmentation regions each represent a respective cell substrate region, by segmenting the immunofluorescence image using a first neural network, determining a boundary region which represents a transition from the first cell substrate region to the second cell substrate region in the fluorescence image on the basis of the segmentation information, selecting a plurality of partial images from the immunofluorescence image along the boundary region, determining a confidence measure of a presence of the fluorescence pattern on the basis of the plurality of partial images by means of a second neural network.

[0036] Furthermore, a computing unit is disclosed which is designed to carry out the following steps during digital image processing: receiving an immunofluorescence image which represents a staining of a biological cell substrate by a fluorescent dye, determining segmentation information comprising at least a first and a second segmentation region, wherein the segmentation regions each represent a respective cell substrate region, by segmenting the immunofluorescence image using a first neural network, determining a boundary region which represents a transition from the first cell substrate region to the second cell substrate region in the fluorescence image, based on the segmentation information, selecting a plurality of partial images from the immunofluorescence image along the boundary region,Determining a confidence measure of the presence of the fluorescence pattern based on the plurality of partial images using a second neural network.

[0037] Furthermore, a data network device is proposed, comprising at least one data interface for receiving a fluorescence image representing a coloration of a cell substrate by a fluorescent dye, and further comprising at least one computing unit which is designed to carry out the following steps during digital image processing: determining segmentation information comprising at least a first and a second segmentation region, wherein the segmentation regions each represent a respective cell substrate region, by segmenting the immunofluorescence image using a first neural network, determining a boundary region representing a transition from the first cell substrate region to the second cell substrate region in the fluorescence image on the basis of the segmentation information, selecting a plurality of partial images from the immunofluorescence image along the boundary region,Determining a confidence measure of the presence of the fluorescence pattern based on the plurality of partial images using a second neural network.

[0038] The invention will be explained in more detail below using specific embodiments without limiting the general inventive concept and the figures. In the figures: Figure 1 a complete fluorescence image, Figure 2a another fluorescence image, Figure 2b segmentation information, Figure 2c border areas, Figure 3 partial images of a fluorescence value selected along a border area, Figure 4a shows steps of a preferred embodiment of a method disclosed herein, Figure 4b preferred substeps for determining a confidence measure according to a preferred embodiment, Figure 5Steps for determining partial image confidence measures and partial image brightness measures as well as a confidence measure and a brightness measure according to a preferred embodiment of the invention, Figures 6a, 6b, 6c preferably carried out steps a preferred embodiment of the disclosed method, Figure 7 an example structure of a second neural network, Figure 8 a preferred embodiment of a device according to the invention, Figure 9 a preferred embodiment of a computing unit, Figure 10 a preferred embodiment of a data network device according to the invention, Figure 11 a preferred embodiment of a computer program product according to the invention as well as a proposed data signal, Figure 12 Experimental results.

[0039] The exemplary embodiments described herein are not to be understood as limiting the invention. Rather, additions and modifications are also entirely possible within the scope of the present disclosure, in particular those that are apparent to a person skilled in the art with regard to solving the problem, for example, through the combination or modification of individual features or method steps described in the general or specific description, as well as those contained in the claims and / or the drawings, and that lead to a new subject matter or to new method steps or method step sequences through combinable features.

[0040] As previously stated, the Figure 1a fluorescence image FB, which can be analyzed using the method according to the invention. The fluorescence image FB shows fluorescence of a cell substrate. The cell substrate is preferably a skin substrate of a primate. In particular, the skin substrate comprises a dermis and an epidermis. The substrate region ED shows the epidermis region, while the substrate region DE shows the dermis region. The space between the epidermis and dermis can be seen as a space or bubble BL. The boundary region BD, as the bubble roof between the epidermis and the space or bubble BL, shows fluorescence in this example.

[0041] The border area of the so-called blister floor BD between the interspace or blister BL and the dermis DE shows no fluorescence in this example.

[0042] The Figure 4ashows a preferred sequence of steps for carrying out a method disclosed herein. In step S1, the cell substrate is incubated with a liquid patient sample potentially containing primary antibodies, as well as with secondary antibodies labeled with a fluorescent dye, further irradiated with excitation radiation, and further acquired with an immunofluorescence image.

[0043] The following steps S2 to S5 are steps of a digital image processing method, which can be summarized as steps SD. In a step S2, segmentation information SEG is determined using a neural network NN1. This is done based on the fluorescence image FB. An exemplary fluorescence image FB is shown in the Figure 2aThe epidermis area ED, the dermis area DE and the area of the interspace or bubble BL are clearly visible. In step S2, the segmentation information SEG is determined, which is shown as an example in the Figure 2b The segmentation information SEG has a first segmentation region SB1, which represents the cell substrate region of the epidermis ED. Furthermore, the segmentation information SEG has a second segmentation region SB2, which preferably represents the region BL of the interspace or the region of the blister. Preferably, the segmentation information SEG can further have a third segmentation region SB3, which represents a third cell substrate region in the form of the dermis DE. Preferably, the segmentation information SEG further has a segmentation region SB4, which represents a background.

[0044] Because, according to the invention, the segmentation information SEG is determined by a first neural network and the second neural network then determines the confidence level of the presence of the fluorescence pattern on the basis of several partial images of the immunofluorescence image, which are determined using the segmentation information SEG or indirectly on the basis of the segmentation information SEG, the first neural network only needs to be trained to recognize segmentation regions which represent respective cell substrate regions and to output this information. Only later does the second neural network then have to determine the confidence level of the presence of the fluorescence pattern on the basis of the selected partial images. Therefore, a single neural network does not have to perform the recognition task of recognizing the substrate regions orRather than performing the segmentation areas and the detection task of determining the confidence level for the presence of the fluorescence pattern simultaneously, these subtasks are distributed among two neural networks in a special way. This makes the method particularly powerful.

[0045] According to the Figure 4a In a step S3, at least one boundary region representing a transition from the first cell substrate region to the second cell substrate region in the fluorescence image is determined based on the segmentation information SEG. This determined and detected boundary region can then be provided as information GB by step S3.

[0046] The Figure 2cshows an exemplary boundary area information GB, which indicates as a boundary area GB1 here preferably a transition from the first cell substrate area in the form of the epidermis ED to the second cell substrate area in the form of the intermediate space or the bubble BL.

[0047] Preferably, in a further step, a second boundary region GB2 can be determined, which represents a second transition from the second cell substrate region in the form of the intermediate space or bubble BL to the third cell substrate region in the form of the dermis DE.

[0048] According to the Figure 4aIn a step S4, several partial images can then be selected along the boundary region, in particular along the first boundary region GB1. Figure 3 shows, as an example, for a section FBA of the fluorescence image SB, the region of the epidermis ED, the region of the interstitial space or blister BL, and the region GB1 as the boundary region. Along the boundary region GB1, partial images, such as partial images TB1, TB2, and TB3, are selected from the fluorescence image FB. Further partial images are indexed here with indices from 0-20 in the image section FBA.

[0049] The confidence measure KM is determined according to the Figure 4a in step S5 using the second neural network NN2.

[0050] Because, according to the invention, the entire fluorescence image FB does not have to be analyzed by the second neural network, but rather only individual partial images selected along the boundary region GB1 are analyzed by the second neural network to determine the confidence measure, the second neural network only needs to be trained for image information provided by partial images of a boundary region, such as the boundary region GB1. Therefore, the second neural network can be trained particularly reliably to determine the confidence measure of the presence of the fluorescence pattern.If a complete fluorescence image (FB) or a complete image section (FBA) were to be fully analyzed by the second neural network, significantly more information would have to be processed by the second neural network, requiring greater degrees of freedom in the training and design of the second neural network. Such an approach would be associated with reduced reliability of the determination of the confidence measure for the presence of the fluorescence pattern.

[0051] In a preferably performed step S6, the confidence measure KM is output. The confidence measure can be a percentage in a range from 0 to 100%, which represents the confidence measure. Alternatively, the confidence measure can preferably be a value from the value range from 0 to 1. According to another preferred embodiment, the confidence measure can be output in the form of "YES" or "NO" information.

[0052] This output of the confidence measure KM can be provided as an output by providing a data element KM. Alternatively, this output can be provided by displaying the confidence measure to the user on a display unit of a computer device.

[0053] The Figure 4bshows a preferred embodiment for carrying out step S5, in which in a first step S5A a respective partial image confidence measure is first determined for a respective partial image by means of the second neural network NN2.

[0054] Subsequently, in a further step S5B, the confidence measure KM is determined based on the partial image confidence measures TBKM.

[0055] The Figure 5 shows an example structure in which partial images TB1, TB2, ..., TBX are analyzed separately, or individually for each partial image, using the second neural network NN2. The results are the respective partial image confidence measures TBKM1, TBKM2, ..., TBKMX. In a further step S5B, the confidence measure KM is then determined based on the partial image confidence measures TBKM.

[0056] Because the second neural network NN2 only processes a single sub-image separately to determine a respective sub-image confidence measure, the second neural network can be trained specifically for analyzing sub-images TB1, TB2, ..., TX. Furthermore, the second neural network NN2 only needs to analyze image information from a single sub-image during a single processing run, rather than considering all image information from all sub-images together in this processing sequence. Therefore, the method is particularly reliable with regard to the final determination of the final confidence measure KM.

[0057] Selecting the drawing files TB1, TB2, ..., TBX along the boundary area GB1, as in the Figure 3 The selection can preferably be performed randomly. In particular, this selection can be performed using a sampling method based on the Poisson disk method.

[0058] Introduction to the determination of the boundary areas: Considering the segmentation information SEG from the Figure 2 The following method can preferably be used to determine the boundary areas GB1, GB2. Preferably, a respective segmentation area SB1, ..., SB3 is expanded in area by means of a dilation. It can then preferably be used to determine the boundary area GB1, as in Figure 2c As shown, an overlap of the dilated region SB1 with at least the dilated region SB2 can be considered. Preferably, the boundary region GB1 is determined by considering an overlap of the dilated region SB1 with at least one of the other dilated regions SB2 or SB3. In particular, a set of pixels of the boundary region GB1 can be described using a dilation operator and the respective pixel sets SB1, SB2, SB3 as GB 1 = DIL SB 1 ∩ DIL SB 2 ∪ DIL SB 3

[0059] It can then preferably be used to determine the limit range GB2, as in Figure 2c As shown, an overlap of the dilated region SB3 with at least the dilated region SB2 can be considered. Preferably, the boundary region GB2 is determined by considering an overlap of the dilated region SB3 with at least one of the other dilated regions SB2 or SB1. In particular, a set of pixels of the boundary region GB2 can be described using a dilation operator and the respective pixel sets SB1, SB2, SB3 as GB 2 = DIL SB 3 ∩ DIL SB 1 ∪ DIL SB 2

[0060] The random selection of the multiple partial image areas is done in the Figure 6a represented by a modified step S4'.

[0061] The advantage of a random-based selection of the sub-image areas is that such a random-based selection of the sub-image areas also introduces a higher variance of the data of the sub-images during training in order to avoid overfitting if the training set is too small.

[0062] According to the Figure 5 The second neural network preferably further determines a respective brightness value or a respective brightness measure HM1, HM2, ... HMX for a respective partial image. This can, as in the Figure 6b shown, in a modified step S5A'.

[0063] It can then be according to the Figure 5 in a modified step S5B', which is also described in the Figure 6b is shown, the brightness value or the total brightness value HM is determined on the basis of the respective brightness values.

[0064] This is advantageous because determining the partial image confidence measures and the information of the respective brightness values in a common neural network NN2 allows a better generalization of the network.

[0065] The Figure 7 shows a preferred structure of the neural network NN2. The neural network NN2 analyzes a respective partial image TB and determines a respective partial image confidence measure TBKMX for each partial image TB. Furthermore, the neural network NN2 preferably determines a respective partial image brightness measure TBHM, as already explained above.

[0066] For this purpose, the neural network NN2 can access a sequence of respective convolutional modules (CM). Each convolutional module (CM) preferably comprises a sequence of a convolutional step (CS), a batch normalization step (BS), another convolutional step (CS), and another batch normalization step (BS), followed by a max pooling step (MAS).

[0067] Several such convolutional modules CM can follow one another.

[0068] The last convolutional module can then be further processed in a processing branch PZ1 using several dense layers DL, also called fully connected layers, and finally the partial image confidence measure TBKMX can be determined using a softmax layer SML.

[0069] Preferably, in a further processing branch PZ2, a sequence of several dense layers DL2 can be carried out in order to determine the partial image brightness measure TBHM.

[0070] Preferably, for each partial image TBX with index 1=x...X, such a partial image confidence measure TBKMX is determined, which is represented as a tuple TBKM x consists of several labels according to TBKM x = f pos TB x , f neg TB x , f uncl TB x , f back TB x with f pos ( TB x ) Probability that the partial image is positive (fluorescence is present) f neg ( TB x) Probability that the partial image is negative (fluorescence not present) f uncle ( TB x ) Probability that the partial image is not assignable f back ( TB x ) Probability that the partial image shows a background is

[0071] If a tuple TBKM x for a partial image TB x has a positive probability as the highest probability value, the partial image TB x classified as positive and assigned to the set P of all positive subimages. If a tuple TBKM x for a partial image TB x has a negative probability as the highest probability value, the partial image TB x classified as negative and assigned to the set N of all negative subimages. If a tuple TBKM x for a partial image TB x as the highest probability value a probability for the class "not assignable", the partial image TB xclassified as not assignable. If a tuple TBKM x for a partial image TB x If the highest probability value is a probability for the class "Background", the partial image TB x classified as background. Partial images of the classes "unassignable" and "background" are not considered further.

[0072] Based on the partial image confidence measures or the tuples TBKM x the partial images TB x , which were classified as positive or negative, the confidence measure KM of the presence of the fluorescence pattern can then be determined as a scalar value y in the range y ∈ 0 1 as y = 1 + y neg − y pos 2 with y neg = ∑ i ∈ N f neg TB i N based on all negative partial images TB i from the set N with i ∈ N and with y pos = ∑ i ∈ N f pos TB i P based on all positive partial images TB i from the set P with i ∈ P .

[0073] Furthermore, to determine the brightness measure HM as the scalar value b, the three partial images TB i with the highest probability of positive f pos ( TB i ) are assigned to a set P3 and considered by calculating their partial image brightness measures TBHM or partial brightness values bi be aggregated according to b = ∑ i ∈ P b i P 3

[0074] The Figure 6c shows a modified sequence of steps of the method according to the invention according to a preferred embodiment. In the second step S2' therein, the segmentation information is determined, which comprises the first, second, and third segmentation regions. Preferably, the plurality of partial images are partial images of a first type.

[0075] By using the previously described step S3 and a further step S31, the first boundary region is determined, as is the second boundary region, which represents a second transition from the second cell substrate region to the third cell substrate region. By performing the previously described step S4 and performing a further step S41, a plurality of partial images of the second type are then also selected from the immunofluorescence image along the second boundary region.

[0076] By carrying out the previously described step S5 as well as the further, additional step S51, a second confidence measure of a presence of a second fluorescence pattern is then determined on the basis of the plurality of partial images of the second type by means of a third neural network NN3.

[0077] The second confidence measure KM2 can preferably be output in a step S61.

[0078] The fact that in this preferred embodiment of the method both a confidence measure of the presence of a fluorescence pattern along the first boundary region and a second confidence measure of a second presence of a second fluorescence pattern along the boundary region are determined and preferably also output, enables advantageous information acquisition for certain cases. The first boundary region GB1 can preferably be referred to as a so-called bubble roof, and the second boundary region can preferably be referred to as a so-called bubble bottom, as also in the Figure 1 explained in detail.

[0079] If the presence of fluorescence patterns or their confidence measures are determined and provided or displayed for both border areas, this information can be evaluated in a special way, preferably by a clinician. If a fluorescence pattern is present only on the blister roof, it is likely bullous pemphigoid. If only the blister base fluoresces, then rarer pemphigoid diseases outside the group of bullous pemphigoids are possible, particularly non-bullous pemphigoid diseases, such as epidermolysis bullosa acquisita.

[0080] If there is fluorescence along the blister base and the blister roof, i.e. along both border areas, then pemphigoid disease is generally a possibility.

[0081] This particular embodiment of the proposed method is therefore helpful to further differentiate between possible primary antibodies or their presence in the patient sample, which may be advantageous for the clinician to differentiate in diagnostics.

[0082] The method proposed herein is therefore, in particular, a method for digital image processing suitable for detecting at least one fluorescence pattern on an immunofluorescence image of a biological cell substrate for obtaining information regarding a possible pemphigoid disease. In a particular embodiment, the method is a method for obtaining information regarding a possible bullous pemphigoid disease. In another particular embodiment, the method is a method for obtaining information regarding a possible non-bullous pemphigoid disease. In yet another particular embodiment, the method is a method for obtaining information regarding a possible non-bullous pemphigoid disease or bullous pemphigoid disease.

[0083] The biological cell substrate is preferably a primate skin substrate. The primate skin substrate is preferably a monkey skin substrate. The primate skin substrate preferably comprises an epidermis and a dermis. The primate skin substrate is preferably a so-called salt-split skin with a blister between the epidermis and dermis.

[0084] Figure 8shows a device V1 by means of which the disclosed method can preferably be carried out. The device V1 can be referred to as a fluorescence microscope. The device V1 has a holder H for a substrate S or a slide with such a substrate S, which has been incubated in the manner described above. Excitation light AL from an excitation light source LQ is guided to the substrate S via an optics O. The resulting fluorescence radiation FL is then retransmitted through the optics O and passes through the dichroic mirror SP1 and an optional optical filter F2. Preferably, the fluorescence radiation FL passes through an optical filter FG, which filters out a green channel. A camera K1 is preferably a monochrome camera, which then records the fluorescence radiation FL in a green channel if an optical filter FG is present.According to an alternative embodiment, the camera K1 is a color camera that does not require the optical filter FG and captures the fluorescence image in the corresponding color channel as a green channel using a Bayer matrix. The camera K1 provides the image information BI or the fluorescence image to a computing unit R, which processes this image information BI. Preferably, the computing unit R can output or provide data DE, such as a fluorescence image and / or confidence measures, via a data interface DS1.

[0085] Figure 9shows a disclosed computing unit which, preferably according to a preferred embodiment, receives a fluorescence image FB as a data signal SI via a data interface DS2. The computing unit R can then determine the previously described information and provide it as a data signal SI3 via a data interface DS3. This can preferably be done via a wired or wireless data network. Particularly preferably, the computing unit R has an output interface AS for outputting the information via an output unit AE. The output unit AE is preferably a display unit for visually displaying the aforementioned information.

[0086] Figure 10shows a data network device DV according to the invention according to a preferred embodiment. The data network device DV receives the fluorescence image FB as a data signal SI1 via a data interface DS4. The data network device DV has a previously described computing unit R and a memory unit MEM. The computing unit R, a memory unit MEM, and the data interface DS4 are preferably connected to one another via an internal data bus IDB.

[0087] The Figure 11 shows an embodiment of a proposed computer program product CPP. The computer program product CPP can have its data signal SI2 received by a computer CO via a data interface DSX.

[0088] Figure 12shows experimental results under the label "Combined" for the cell substrate of salt-split skin for the positive case where a fluorescence pattern is present in at least one of the two boundary regions GB1, particularly the bladder roof, or GB2, particularly the bladder base. This includes the case where fluorescence is present in both boundary regions GB1 and GB2. If fluorescence is not present in any of the boundary regions GB1 and GB2, the case is assessed as negative. Of 32 actually positive fluorescence images, 31 images were recognized as positive and 1 image as negative by the inventive "EPa Classifier" method. Of 77 actually negative fluorescence images, 4 images were recognized as positive and 73 images as negative by the inventive "EPa Classifier" method. Thus, there were a total of 109 images, of which 35 were recognized as positive and 74 as negative. The accuracy was 95.4%. The sensitivity ("PPA") was 96.9%.The specificity ("NPA") was 94.8%.

[0089] Figure 12further shows experimental results under the label "epidermal" for the cell substrate of salt-split skin for the positive case in which a fluorescence pattern is present in the first boundary area GB1, particularly the blister roof. This includes the case in which fluorescence is present in both boundary areas GB1 and GB2. If no fluorescence pattern is present in the boundary area GB1, the case is assessed as negative. Of 26 actually positive fluorescence images, 25 images were identified as positive and 1 image as negative by the inventive "EPa Classifier" method. Of 83 actually negative fluorescence images, 3 images were identified as positive and 80 images as negative by the inventive "EPa Classifier" method. Thus, there were a total of 109 images, of which 28 were identified as positive and 81 as negative. The accuracy was 96.3%. The sensitivity ("PPA") was 96.2%. The specificity ("NPA") was 96.4%.

[0090] Figure 12 further shows experimental results under the label "dermal" for the cell substrate of salt-split skin for the positive case in which a fluorescence pattern is present in the second boundary region GB2, particularly the blister base. This includes the case in which fluorescence is present in both boundary regions GB1 and GB2. Of 8 actually positive fluorescence images, 8 images were identified as positive and 0 images as negative by the inventive "EPa Classifier" method. Of 101 actually negative fluorescence images, 3 images were identified as positive and 98 images as negative by the inventive "EPa Classifier" method. Thus, there were a total of 109 images, of which 11 were identified as positive and 98 as negative. The accuracy was 97.3%. The sensitivity ("PPA") was 100.0%. The specificity ("NPA") was 97.0%.

[0091] Depending on specific implementation requirements, embodiments of the invention may be implemented in hardware or software. The implementation may be performed using a digital storage medium, such as a floppy disk, a DVD, a Blu-ray disc, a CD, a ROM, a PROM, an EPROM, an EEPROM, or a FLASH memory, a hard disk, or other magnetic or optical storage device storing electronically readable control signals that can interact or interact with a programmable hardware component to perform the respective method.

[0092] A programmable hardware component, such as in particular a computing unit, can be formed by a processor, a computer processor (CPU = Central Processing Unit), a graphics processor (GPU = Graphics Processing Unit), a computer, a computer system, an application-specific integrated circuit (ASIC = Application-Specific Integrated Circuit), an integrated circuit (IC = Integrated Circuit), a single-chip system (SOC = System on Chip), a programmable logic element or a field-programmable gate array with a microprocessor (FPGA = Field Programmable Gate Array).

[0093] The digital storage medium can therefore be machine- or computer-readable. Some embodiments thus comprise a data carrier having electronically readable control signals capable of interacting with a programmable computer system or a programmable hardware component such that one of the methods described herein is performed. One embodiment is thus a data carrier (or a digital storage medium or a computer-readable medium) on which the program for performing one of the methods described herein is recorded.

[0094] In general, embodiments of the present invention can be implemented as a program, firmware, computer program, or computer program product with program code or data, wherein the program code or data is effective to perform one of the methods when the program runs on a processor or a programmable hardware component. The program code or data can also be stored, for example, on a machine-readable medium or data carrier. The program code or data can be present, among other things, as source code, machine code, or bytecode, as well as other intermediate code.

[0095] A further embodiment is a data stream, a signal sequence, or a sequence of signals that represents the program for performing one of the methods described herein. The data stream, the signal sequence, or the sequence of signals can be configured, for example, to be transferred via a data communication connection, for example, via the Internet or another network. Thus, embodiments also include signal sequences representing data that are suitable for transmission via a network or data communication connection, wherein the data represents the program.

[0096] A program according to one embodiment can implement one of the methods during its execution, for example, by reading memory locations or writing one or more pieces of data into them, which may trigger switching operations or other processes in transistor structures, amplifier structures, or other electrical, optical, magnetic, or other components operating according to a different functional principle. Accordingly, by reading a memory location, data, values, sensor values, or other information can be acquired, determined, or measured by a program. A program can therefore acquire, determine, or measure quantities, values, measured quantities, and other information by reading one or more memory locations.

Claims

1. Method for digital image processing, comprising - providing an immunofluorescence image (FB), which represents a staining of a biological cell substrate by a fluorescence stain, - determining segmentation information (SEG) comprising at least one first and one second segmentation area (SEG1, SEG2), wherein the segmentation areas each represent a respective cell substrate area (D, BL), via segmentation of the immunofluorescence image (FB) using a first neural network (NN1), - determining a boundary area (GB1), which represents a transition from the first cell substrate area (D) towards the second cell substrate area (BL) in the fluorescence image (FB), on the basis of the segmentation information (SEG), - selecting multiple partial images (TB1, ..., TBX) from the immunofluorescence image (FB) along the boundary area (GB1), - determining a confidence measure (KM) of a presence of the fluorescence pattern on the basis of the multiple partial images (TB1, ..., TBX) via a second neural network (NN2).

2. Method according to Claim 1, furthermore comprising outputting the confidence measure (KM).

3. Method according to Claim 1, furthermore comprising - determining a respective partial image confidence measure (TBKM1, ..., TBKMX) for a respective partial image (TB1, ..., TBX) via the second neural network (NN2) - and determining the confidence measure (KM) on the basis of the partial image confidence measures (TBKMX).

4. Method according to Claim 1, furthermore comprising randomly based selection of the multiple partial images (TB1, ..., TBX) from the immunofluorescence image (FB) along the boundary area (GB1).

5. Method according to Claim 1, furthermore comprising - determining a respective brightness value (TBHM1, ..., TBHMX) for a respective partial image (TB1, ..., TBX) via the second neural network (NN2), - determining an overall brightness value (HM) on the basis of the brightness values.

6. Method according to Claim 1, wherein the multiple partial images (TB1, ..., TBX) are partial images of a first type, furthermore comprising - determining the segmentation information (SEG), comprising at least the first, the second, and furthermore a third segmentation area (SEG1, SEG2, SEG3), which represents a third cell substrate area (DE), via segmentation of the immunofluorescence image (FB) using the first neural network (NN1), - determining a second boundary area (GB2), which represents a second transition from the second cell substrate area (BL) towards the third cell substrate area (DE) in the fluorescence image (FB), on the basis of the segmentation information (SEG), - selecting multiple partial images of a second type from the immunofluorescence image along the second boundary area (GB2), - determining a second confidence measure of a presence of a second fluorescence pattern on the basis of the multiple partial images of the second type via a third neural network (NN3).

7. Computer program product (CPP), comprising commands which, upon the execution of the program by a computer, prompt it to carry out the method for digital image processing according to Claim 1.

8. Data carrier signal (SI2), which transmits the computer program product (CPP) according to Claim 7.

9. Device for detecting at least one fluorescence pattern on an immunofluorescence image (FB) of a biological cell substrate, comprising - a holding device (H) for an object carrier having the cell substrate (S), which was incubated with a patient sample, including the autoantibodies, and furthermore with secondary antibodies, which are each marked using a fluorescence stain, - at least one image capture unit (K1) for capturing a fluorescence image (SG) of the cell substrate (S) and furthermore comprising at least one computing unit (R), which is designed to execute the following steps - determining segmentation information (SEG) comprising at least one first and one second segmentation area (SEG1, SEG2), wherein the segmentation areas each represent a respective cell substrate area, via segmentation of the immunofluorescence image (FB) using a first neural network (NN1), - determining a boundary area (GB1), which represents a transition from the first cell substrate area (D) towards the second cell substrate area (BL) in the fluorescence image (FB) on the basis of the segmentation information (SEG), - selecting multiple partial images (TB1, ..., TBX) from the immunofluorescence image (FB) along the boundary area (GB1), - determining a confidence measure (KM) of a presence of the fluorescence pattern on the basis of the multiple partial images (TB1, ..., TBX) via a second neural network (NN2).

10. Data network device (DV), comprising at least one data interface (DS4) for accepting a fluorescence image (FB), which represents a staining of a cell substrate by a fluorescence stain, and furthermore comprising at least one computing unit (R), which is designed to execute the following steps in the course of digital image processing - determining segmentation information (SEG) comprising at least one first and one second segmentation area (SEG1, SEG2), wherein the segmentation areas each represent a respective cell substrate area, via segmentation of the immunofluorescence image (FB) using a first neural network (NN1), - determining a boundary area (GB1), which represents a transition from the first cell substrate area (D) towards the second cell substrate area (BL) in the fluorescence image (FB), on the basis of the segmentation information (SEG), - selecting multiple partial images (TB1, ..., TBX) from the immunofluorescence image (FB) along the boundary area (GB1), - determining a confidence measure (KM) of a presence of the fluorescence pattern on the basis of the multiple partial images (TB1, ..., TBX) via a second neural network (NN2).