Method for detecting at least one fluorescent pattern on an immunofluorescence image of a biological cell substrate

Abstract

The invention proposes a method for detecting at least one fluorescence pattern on an immunofluorescence image of a biological cell substrate, the method having the steps of incubating a cell substrate with a patient sample in liquid form, the patient sample potentially having a primary antibody and in addition comprising a secondary antibody labeled with a fluorescent dye, irradiating the cell substrate with excitation radiation and detecting the immunofluorescence image; determining segmentation information having at least a first segmentation region and a second segmentation region by means of segmentation of the immunofluorescence image using a first neural network, wherein the segmentation regions each represent a respective cell substrate region; determining a boundary region on the basis of the segmentation information, the boundary region representing a transition in the fluorescence image from the first cell substrate region to the second cell substrate region; selecting a plurality of partial images from the immunofluorescence image along the boundary region; determining a confidence in the presence of the fluorescence pattern on the basis of the plurality of partial images by means of a second neural network.

Description

Technical Field

[0001] The present invention relates to a method for detecting at least one fluorescent pattern on an immunofluorescence image of a biological cell matrix. Background Technology

[0002] For the purpose of obtaining information on diagnostic problems, biological matrices, such as cell matrices, can be treated with immunofluorescence, and the resulting immunofluorescence images of the matrices can then be analyzed.

[0003] Indirect immunofluorescence microscopy is an in vitro assay used to detect circulating human antibodies against specific antigens in order to answer or assess diagnostic questions. Such antigens are present, for example, in specific regions of the cellular matrix, particularly the skin layer of primates. That is, the cellular matrix is ​​used as a substrate, incubated with patient samples in the form of blood or diluted blood or serum or diluted serum. That is, the patient sample potentially contains specific antibodies that can indicate the presence of disease in the patient. Such a primary antibody or specific antibody can then bind to the antigen in the matrix. The bound primary antibody can then be labeled, i.e., in another incubation step, a so-called secondary antibody, preferably an anti-human antibody, binds to the bound primary antibody and is subsequently visible by labeling the secondary antibody with a fluorescent dye. Such a fluorescent dye is preferably a green fluorescent dye, especially the fluorescent dye FITC. This binding of the primary antibody and the fluorescently labeled secondary antibody can then be seen by irradiating the matrix with excitation light of a specific wavelength, thus exciting the bound fluorescent dye to emit fluorescent radiation.

[0004] Biological cell matrix here can be, in particular, cell smears of cell lines, colonies of cell lines, and / or biological tissues, especially those derived from tissue sections, that include biological tissue cells.

[0005] Based on the fluorescence pattern on the corresponding matrix, the detected autoantibodies can be assigned to the disease group. This leads to the task of detecting one or more types of fluorescence patterns in a fluorescent image from an indirect immunofluorescence microscope within a cell matrix cultured in a prescribed manner, using digital image processing during immunofluorescence microscopy.

[0006] Especially when observing diseases such as bullous autoimmune skin diseases involving human skin, immunofluorescence image analysis can be used to obtain information such as whether any bullous autoimmune skin diseases exist and, preferably, what type or group of bullous autoimmune skin diseases are present.

[0007] For this reason, the esophageal segment of apes and / or the so-called salt-cracked skin matrix are sometimes subjected to immunofluorescence as biological cell matrix.

[0008] So-called salt-cracked skin is a biological cellular matrix in the form of a skin matrix, preferably from salt-cracked primates. In such a skin matrix, the epidermis separates from the dermis by injecting a salt solution, particularly a 1-molar sodium chloride solution. Summary of the Invention

[0009] The objective of this invention is to automatically analyze immunofluorescence images of biological cell matrix using at least one neural network, namely, whether at least one fluorescent pattern is present in the immunofluorescence image of the cell matrix.

[0010] The objective of the invention is achieved by the method according to the invention. According to the invention, a cell matrix is ​​first cultured in a liquid patient sample, potentially containing a primary antibody and further comprising a secondary antibody labeled with a fluorescent dye. The cell matrix is ​​then irradiated with excitation radiation, and an immunofluorescence image of the cell matrix is ​​detected. Immunofluorescence image detection is particularly performed in the green channel.

[0011] Furthermore, according to the present invention, segmentation information having at least a first segmentation region and a second segmentation region is determined. The corresponding segmentation regions each represent a corresponding region of the cellular matrix. This segmentation information is determined by means of segmentation in an immunofluorescence image when using a first neural network.

[0012] Furthermore, a boundary region is then determined based on the segmentation information. This boundary region represents the transition from the first cellular matrix region to the second cellular matrix region in the fluorescence image. Multiple partial images of the immunofluorescence image are then selected along the determined boundary region. Finally, a second neural network is used to determine the confidence level of the presence of a fluorescence pattern based on these multiple partial images.

[0013] In order to illustrate the possible advantages of the method according to the invention, different aspects of the method according to the invention will now be discussed more precisely.

[0014] The fundamental problem of analyzing the presence of desired fluorescence patterns in immunofluorescence images is well known in the field of immunofluorescence. Because different desired fluorescence patterns are given according to the diagnostic problem, image processing methods for immunofluorescence images are particularly efficient when they are particularly well-suited for analysis using neural networks to determine the desired fluorescence patterns. While processing large immunofluorescence images using neural networks that consider all image information of the entire image requires highly complex neural networks that must perform a great deal of computation, such networks are particularly difficult to train due to the need for large amounts of training data to achieve generalization.

[0015] Therefore, the proposed method according to the present invention deviates specifically from such measures as total analysis of the entire immunofluorescence image by means of a neural network, in order to be particularly efficient.

[0016] exist Figure 1 An exemplary immunofluorescence image FB is shown here, preferably in the form of salt-cracked skin.

[0017] Immunofluorescence image FB shows the cellular matrix region ED, which represents the epidermis. Furthermore, immunofluorescence image FB shows the region DE, representing the dermis. In the salt-cracked skin, the so-called intermediate region, or blase, is clearly visible as the matrix region BL. That is, the first cellular matrix region is preferably the epidermis, while the second cellular matrix region is preferably in the gap or blase between the dermis and the epidermis.

[0018] Figure 1 Furthermore, the so-called background region, or background BG, is shown in the fluorescence image FB.

[0019] The transition or boundary region between the epidermal ED and the vesicle BL is preferably also called the vesicle top BD. Figure 1 In the immunofluorescence image, fluorescence is present along the top BD of the bubble as a boundary region.

[0020] Furthermore, a boundary region BB is generated as the so-called bubble base, which is the boundary region between the bubble BL (or gap) and the dermis DE. Figure 1 In the example of the fluorescence image FB, no fluorescence is present along the bottom of the bubble.

[0021] The presence of fluorescence along at least one boundary region of the said boundary region (i.e., along only the vesicle top, only the vesicle bottom, or both the vesicle top and bottom) is, in principle, an indication of pemphigoid disease. Therefore, if the presence of a fluorescent pattern is detected in the boundary region according to the invention, it is, in principle, an indication of pemphigoid disease.

[0022] If a fluorescent pattern is detected preferably in a defined boundary region, i.e., the vesicle top BD, information may be obtained regarding the possible diseases under consideration, i.e., a defined group of diseases can be ruled out. Therefore, information regarding the presence or absence of fluorescence along the boundary region, or the vesicle top BD, is beneficial to clinicians. Thus, when analyzing a previously defined boundary region in a defined cellular matrix region, such as the epidermal ED and the interstitial space, or the vesicle BL, and detecting the presence of a fluorescent pattern, a defined group of bullous autoimmune skin diseases, such as bullous pemphigoid, may be considered, which is the most common bullous autoimmune skin disease in Europe. In other words: if fluorescence is present along the so-called vesicle top BD, this can be a clue to two defined target antigens, which again can be a clue to bullous pemphigoid.

[0023] If the presence of a fluorescent pattern is detected preferably in a defined boundary region of the BB at the base of the blister, this could be a clue to a rarer form of bullous pemphigoid disease outside the bullous pemphigoid group, such as acquired epidermolysis bullosa.

[0024] Preferably, confidence levels are also output.

[0025] Preferably, the confidence level is determined in the following manner: first, the confidence level of the corresponding partial image is determined by using a second neural network, and then the confidence level is determined based on the confidence level of the partial image.

[0026] Preferably, the plurality of partial image regions are randomly selected from the immunofluorescence image based on the boundary region.

[0027] Preferably, a second neural network is used to determine the corresponding brightness value for the corresponding portion of the image, and then the total brightness value is determined based on these corresponding brightness values.

[0028] Preferably, the segmentation information is further determined in such a manner that the segmentation information has a first segmentation region, a second segmentation region, and a third segmentation region. The third segmentation region preferably represents a third cell matrix region. Here, the segmentation information is also determined by segmentation of the immunofluorescence image using a first neural network. Preferably, the plurality of partial images (TB1, ..., TBX) are partial images of a first type. Furthermore, preferably, a second boundary region is determined based on the segmentation information, the second boundary region representing a second transition from the second cell matrix to the third cell matrix. Furthermore, preferably, a plurality of second-type partial images are selected from the immunofluorescence image along the second boundary region. Preferably, a second confidence level regarding the presence of a second fluorescent pattern is determined based on the plurality of second-type partial images using a third neural network.

[0029] Furthermore, a method for digital image processing is proposed, comprising the steps of: providing an immunofluorescence image representing biological cell matrix stained with a fluorescent dye; determining segmentation information, using a first neural network, of at least a first segmentation region and a second segmentation region, wherein the segmentation regions each represent a corresponding cell matrix region; determining a boundary region based on the segmentation information, the boundary region representing a transition from the first cell matrix region to the second cell matrix region in the fluorescence image; selecting multiple partial images from the immunofluorescence image along the boundary region; and determining the confidence level of the presence of a fluorescence pattern based on the multiple partial images using a second neural network.

[0030] Furthermore, a computer program product is proposed, the computer program product having instructions that, when the program is executed by a computer, cause the computer to implement the method for digital image processing.

[0031] Furthermore, a data carrier signal is proposed, which transmits the computer program product.

[0032] Furthermore, an apparatus is proposed for detecting at least one fluorescent pattern on an immunofluorescence image of a biological cell matrix.

[0033] The device further includes at least one computing unit configured to perform the following steps: determining segmentation information, using a first neural network, of at least a first segmentation region and a second segmentation region by means of segmentation of an immunofluorescence image, wherein the segmentation regions each represent a corresponding cell matrix region; determining a boundary region based on the segmentation information, the boundary region representing a transition from the first cell matrix region to the second cell matrix region in the fluorescence image; selecting a plurality of partial images from the immunofluorescence image along the boundary region; and determining the confidence level of the presence of a fluorescence pattern based on the plurality of partial images by means of a second neural network.

[0034] Furthermore, a computing unit is proposed, configured to perform the following steps during digital image processing: receiving an immunofluorescence image representing biological cell matrix stained with a fluorescent dye; determining segmentation information of at least a first segmentation region and a second segmentation region by means of segmentation of the immunofluorescence image using a first neural network, wherein the segmentation regions each represent a corresponding cell matrix region; determining a boundary region based on the segmentation information, the boundary region representing a transition from the first cell matrix region to the second cell matrix region in the fluorescence image; selecting multiple partial images from the immunofluorescence image along the boundary region; and determining the confidence level of the presence of a fluorescence pattern based on the multiple partial images using a second neural network.

[0035] Furthermore, a data network device is proposed, having at least one data interface for receiving fluorescence images representing cell matrix stained with fluorescent dyes, and the data network device further having at least one computing unit configured to perform the following steps during digital image processing: determining segmentation information of at least a first segmentation region and a second segmentation region by means of segmentation of an immunofluorescence image using a first neural network, wherein the segmentation regions respectively represent a corresponding cell matrix region; determining a boundary region based on the segmentation information, the boundary region representing a transition from the first cell matrix region to the second cell matrix region in the fluorescence image; selecting multiple partial images from the immunofluorescence image along the boundary region; and determining the confidence level of the presence of a fluorescence pattern based on the multiple partial images using a second neural network. Attached Figure Description

[0036] The invention will now be described in more detail with the aid of accompanying drawings, using specific embodiments without limiting the general inventive concept. In the drawings:

[0037] Figure 1 Show the entire fluorescence image;

[0038] Figure 2a Another fluorescence image is shown;

[0039] Figure 2b Displays segmentation information;

[0040] Figure 2c Show the boundary area;

[0041] Figure 3 A partial image of the fluorescence values ​​selected along the boundary region is shown;

[0042] Figure 4a Some steps of a preferred embodiment of the proposed method are shown;

[0043] Figure 4b Preferred sub-steps for determining confidence levels are shown according to a preferred embodiment;

[0044] Figure 5 The present invention illustrates steps for determining a partial image confidence level and a partial image brightness level, as well as confidence level and brightness level, according to a preferred embodiment of the invention;

[0045] Figure 6a , 6b 6c illustrates the preferred steps to be performed in a preferred embodiment of the proposed method;

[0046] Figure 7 An exemplary structure of the second neural network is shown;

[0047] Figure 8 A preferred embodiment of the apparatus according to the invention is shown;

[0048] Figure 9 A preferred embodiment of the computing unit according to the invention is shown;

[0049] Figure 10 A preferred embodiment of the data network apparatus according to the present invention is shown;

[0050] Figure 11 A preferred embodiment of the computer program product and the proposed data signal according to the present invention is shown;

[0051] Figure 12 The experimental results are shown. Detailed Implementation

[0052] The embodiments described herein should not be construed as limiting the invention. Rather, additions and modifications are entirely possible within the scope of this disclosure, especially such additions and modifications—for example by combining or transforming the various features or method steps described in the general or specific descriptive sections and included in the claims and / or drawings—are deriveable by those skilled in the art for the purpose of such solutions, and by the composable features lead to new technical solutions or new method steps or sequences of method steps.

[0053] As stated earlier, Figure 1 A fluorescence image FB, which can be analyzed using the method according to the invention, is shown. The fluorescence image FB shows the fluorescence of the cellular matrix. The cellular matrix is ​​preferably the skin matrix of a primate. In particular, the skin matrix has a dermis and an epidermis. The matrix region ED shows the region of the epidermis, while the matrix region DE shows the region of the dermis. The gap between the epidermis and dermis, as a gap or vesicle BL, can be seen. The boundary region BD, as the top of a vesicle between the epidermis and the gap or vesicle BL, shows fluorescence in this example.

[0054] In this example, the so-called bubble bottom BD boundary region between the gap or bubble BL and the dermal DE does not show fluorescence.

[0055] Figure 4a The preferred sequence of steps for carrying out the method according to the invention is shown. In step S1, the cell matrix is ​​cultured in a liquid patient sample, which potentially contains a primary antibody and further includes a secondary antibody labeled with a fluorescent dye; the cell matrix is ​​further irradiated with excitation radiation; and an immunofluorescence image is detected.

[0056] The subsequent steps S2 to S5 are steps of the digital image processing method, which can be combined into step SD. In step S2, segmentation information SEG is determined using a neural network NN1. This is based on the fluorescence image FB. Figure 2a An exemplary fluorescence image FB is shown. The areas of epidermal ED, dermal DE, and interstitial or vesicular BL are clearly visible. In step S2, segmentation information SEG is now determined, which is exemplarily shown in… Figure 2b As shown in the diagram, the segmentation information SEG has a first segmentation region SB1 representing the cellular matrix region of the epidermal ED. Furthermore, the segmentation information SEG has a second segmentation region SB2, preferably representing the interstitial region BL or the vesicle region. Preferably, the segmentation information SEG may also have a third segmentation region SB3, which represents a third cellular matrix region in the form of the dermal DE. Preferably, the segmentation information SEG also has a segmentation region SB4 representing the background.

[0057] By determining the segmentation information SEG using a first neural network according to the present invention, and then determining the confidence level of the presence of a fluorescent pattern based on multiple partial images of an immunofluorescence image determined by or indirectly based on the segmentation information SEG, the first neural network only needs to be trained to identify the segmented regions representing the corresponding cellular matrix regions and output them as information. Only then does the second neural network need to determine the confidence level of the presence of the fluorescent pattern based on the selected partial images. Therefore, a single neural network does not need to simultaneously perform the identification tasks of identifying matrix regions or segmented regions and the detection tasks of determining the confidence level of the presence of the fluorescent pattern; instead, these partial tasks are distributed across two neural networks in a specific manner according to the present invention. Therefore, the method is particularly efficient.

[0058] according to Figure 4a In step S3, at least one boundary region is determined based on the segmentation information SEG, the boundary region representing the transition from a first cellular matrix region to a second cellular matrix region in the fluorescence image. This determined and detected boundary region can then be provided as information GB through step S3.

[0059] Figure 2c For this purpose, exemplary boundary region information GB is shown, which here preferably indicates the transition from a first cellular matrix region in the form of epidermal ED to a second cellular matrix region in the form of interstitial space or vesicle BL as boundary region GB1.

[0060] Preferably, a second boundary region GB2 can be determined in another step, the second boundary region representing a second transition from a second cellular matrix region in the form of gaps or vesicles (BL) to a third cellular matrix region in the form of dermal (DE).

[0061] according to Figure 4a Then, in step S4, multiple partial images can be selected along the boundary region, especially along the first boundary region GB1. Figure 3 For this purpose, by way of example, a partial FBA of the fluorescence image SB also shows the region of the epidermal ED, the region of the gap or vesicle BL, and the region GB1 as a boundary region. A portion of the image from the fluorescence image FB is selected along the boundary region GB1, for example, the portion images TB1, TB2, TB3. Other portion images are indicated here with indices 0-20 in the partial FBA of the figure.

[0062] according to Figure 4a In step S5, the confidence level KM is determined when using the second neural network NN2.

[0063] According to the present invention, instead of analyzing the entire fluorescence image FB, only a selected portion of the image along the boundary region GB1 needs to be analyzed by the second neural network to determine the confidence level. The second neural network is trained only on image information given by the boundary region, such as a portion of the image along the boundary region GB1. Therefore, the second neural network can be trained particularly reliably to determine the confidence level of the presence of the fluorescence pattern. If the entire fluorescence image FB or the entire image locality FBA is analyzed by the second neural network, significantly more information must be processed, and therefore a higher degree of freedom must be considered in both the training and design of the second neural network. This approach will result in a decrease in the reliability of the determination of the confidence level of the presence of the fluorescence pattern.

[0064] In a preferred step S6, the confidence level KM is output. The confidence level can be a percentage representation in the range of 0 to 100%, where the percentage represents the confidence level. Alternatively, the confidence level can preferably be a value in the range of 0 to 1. In another preferred embodiment, the confidence level can be output as the information "yes" or "no".

[0065] The confidence level KM can be output by providing data item KM as output. Alternatively, this output can be performed by displaying the confidence level to the user on the display unit of the computer unit.

[0066] Figure 4bA preferred embodiment for implementing step S5 is shown, in which, in the first step S5A, the confidence level of the corresponding partial image is first determined for the corresponding partial image by means of the second neural network NN2.

[0067] Subsequently, in another step S5B, the confidence level KM is determined based on the partial image confidence level TBKM.

[0068] Figure 5 An exemplary structure is shown for this purpose, in which each partial image TB1, TB2, ..., TBX is analyzed separately, either alone or individually, using a second neural network NN2. The result is then the corresponding partial image confidence scores TBKM1, TBKM2, ..., TBKMX. In another step S5B, a confidence score KM is then determined based on the partial image confidence scores TBKM.

[0069] The second neural network NN2 processes each individual partial image separately to determine the confidence level of the corresponding partial image. This second neural network can be specifically trained to analyze partial images TB1, TB2, ..., TX. Furthermore, the second neural network NN2 only needs to analyze the image information of a single partial image during the processing, without considering all image information from all partial images together. Therefore, the method is particularly reliable in determining the final confidence level KM.

[0070] As in Figure 3 As shown in the diagram, it is preferable to select portions of the image TB1, TB2, ..., TBX along the boundary region GB1 based on randomness. In particular, this selection can be performed using a sampling method based on the Poisson disk method.

[0071] Introduction to defining the boundary region:

[0072] Considering the segmentation information SEG from Figure 2, the following method is preferably used to determine the boundary regions GB1 and GB2. Preferably, the corresponding segmentation regions SB1, ..., SB3 are expanded by area using dilation. For example, in... Figure 2c As shown, to determine the boundary region GB1, it is preferable to consider the overlap of the region SB1, which is expanded by dilation, with at least the region SB2, which is expanded by dilation. The boundary region GB1 is preferably determined by considering the overlap of the region SB1, which is expanded by dilation, with at least one of the other regions SB2 or SB3, which are expanded by dilation. In particular, the number of pixels in the boundary region GB1 can be described using dilation operators and the corresponding pixel amounts SB1, SB2, SB3 as follows:

[0073] GB1=(DIL(SB1)∩(DIL(SB2)∪DIL(SB3))

[0074] As in Figure 2c The diagram shows the location, and then, preferably, to determine the boundary region GB2, it is considered that the region SB3, which is expanded by dilation, overlaps with at least the region SB2, which is expanded by dilation. Preferably, the boundary region GB2 is determined by considering that the region SB3, which is expanded by dilation, overlaps with at least one of the other regions SB2 or SB1, which are expanded by dilation. In particular, the number of pixels in the boundary region GB2 can be described as follows using dilation operators and the corresponding pixel amounts SB1, SB2, and SB3:

[0075] GB2=(DIL(SB3)∩(DIL(SB1)∪DIL(SB2))

[0076] exist Figure 6a The modified step S4' represents the random selection of the multiple partial image regions.

[0077] The advantage of random selection of partial image regions is that, during training, such random selection of partial image regions also introduces higher variability in the data of partial images, so as to avoid overfitting when the training amount is too small.

[0078] according to Figure 5 The second neural network preferably determines the corresponding brightness value or brightness level HM1, HM2, ... HMX for the corresponding portion of the image. (For example, also...) Figure 6b As shown in the diagram, this can be done in the modified step S5A'.

[0079] Then you can follow Figure 5 In the modified step S5B', the brightness value or the entire brightness value HM is determined based on the corresponding brightness value. This step also... Figure 6b As shown in the image.

[0080] This is advantageous because the information used to determine partial image confidence and corresponding brightness values ​​in the common neural network NN2 allows the network to generalize better.

[0081] Figure 7 A preferred structure of a neural network NN2 is shown. The neural network NN2 analyzes a corresponding partial image TB and determines a corresponding partial image confidence level TBKMX for that partial image TB. Furthermore, as previously described, the neural network NN2 preferably determines the corresponding partial image brightness level TBHM.

[0082] Therefore, the neural network NN2 can employ a sequence of corresponding convolutional modules CM. Preferably, the corresponding convolutional module CM here has a sequence of convolutional steps CS, batch normalization steps BS, another convolutional step CS, and another batch normalization step BS, followed by a max pooling step MAS.

[0083] Multiple such convolutional modules (CMs) can be sequentially linked.

[0084] The final convolutional module can then be further processed in the processing branch PZ1 using dense layers (DL) also known as fully connected layers, and finally the partial image confidence (TBKMX) is determined using a softmax layer (SML).

[0085] Preferably, multiple dense layers DL2 can be sequenced in another processing branch PZ2 to determine the partial image brightness level TBHM.

[0086] Preferably, for each partial image TBX with indices 1 = x…X, a partial image confidence TBKMX is determined, said partial image confidence as a tuple TBKM. x It consists of multiple tags as follows:

[0087] TBKM x ={f pos (TB x ), f neg (TB x ), f uncl (TB x ), f back (TB x )}

[0088] in,

[0089] f pos (TB x () represents the probability that some images are positive (fluorescence is present).

[0090] f neg (TB x () represents the probability that some images are negative (fluorescence is absent).

[0091] f uncl (TB x ) represents the probability that some images cannot be assigned.

[0092] f back (TB x ) is the probability that part of the image shows the background.

[0093] If used for partial image TB x tuple TBKMx The highest probability value having a positive probability, then the portion of the image TB x The classification is positive and the amount P is assigned to all positive partial images. If TB is used for partial images... x tuple TBKM x The highest probability value is the probability of being negative, then the partial image TB x The amount N is classified as negative and allocated to all negative portions of the image. If used for a portion of the image, TB... x tuple TBKM x The highest probability value has the probability of being used for the class "unassignable", then the partial image TB x The classification is unassignable. If used for a portion of the image (TB). x tuple TBKM x The portion of the image TB has the highest probability value for the class "background". x The classification is set as background. Then, the portions of the image in the "Unassignable" and "Background" categories are disregarded.

[0094] Based on partial images classified as positive or negative (TB) x Partial image confidence or tuple TBKM x Then, the confidence level KM for the existence of the fluorescent pattern can be determined as a scalar value y in the range y∈[0,1].

[0095]

[0096] Among them, all negative partial images TB based on the quantity N i (where i∈N) using

[0097]

[0098] And based on the quantity P, all positive partial images TB i (where i∈P) utilize

[0099]

[0100] Furthermore, in order to determine the brightness level HM as a scalar value b, the value with the highest positive probability f will be used. pos (TB i The three parts of the image TB i The quantity P3 is configured and considered in such a way that the partial image brightness level TBHM or partial brightness value b of the partial image is taken into account. i The following totals:

[0101]

[0102] Figure 6c The diagram illustrates a modified sequence of steps in the method according to the invention, according to a preferred embodiment. In the second step S2', segmentation information is determined, which has a first segmentation region, a second segmentation region, and a third segmentation region. Preferably, the plurality of partial images are partial images of a first type.

[0103] By using the previously described steps S3 and S31, a first boundary region is determined, and a second boundary region is also determined, representing a second transition from the second cellular matrix region to the third cellular matrix region. Following the implementation of the previously described step S4 and S41, multiple second-type partial images are then selected from the immunofluorescence images along the second boundary region.

[0104] In the case of performing the previously described step S5 and the additional step S51, a second confidence level of the presence of the second fluorescent pattern is then determined based on the plurality of partial images of the second type using a third neural network NN3.

[0105] Preferably, the second confidence level KM2 can be output in step S61.

[0106] In this preferred embodiment of the method, not only is the confidence level of the presence of a fluorescent pattern determined along the first boundary region, but also a second confidence level of the presence of a second fluorescent pattern is determined along the boundary region, and preferably, this confidence level can also be output. For the determined cases, advantageous information can be obtained. The first boundary region GB1 is preferably referred to as the so-called bubble top, and the second boundary region is preferably referred to as the so-called bubble bottom, as well as... Figure 1 The explanation is more in-depth.

[0107] If the presence or confidence level of a fluorescent pattern is identified and provided for in two boundary regions, it is preferable that the clinician can evaluate this information in a particular manner. If the fluorescent pattern is present, in principle, only at the top of the vesicle, then bullous pemphigoid may then be involved. If fluorescence is only present at the base of the vesicle, then a rarer pemphigoid disease outside the bullous pemphigoid group should be considered, i.e., especially a non-bullous pemphigoid disease, such as acquired epidermolysis bullosa.

[0108] If fluorescence is present along the base and top of the vesicle, that is, along the two boundary regions, then pemphigoid disease should be considered in principle.

[0109] Therefore, this particular implementation of the proposed method is useful in order to further differentiate between possible primary antibodies or their presence in patient samples, which may be advantageous for clinicians in distinguishing between them in diagnosis.

[0110] That is, the method presented herein is particularly a method for detecting at least one fluorescent pattern on an immunofluorescence image of a biological cell matrix for obtaining information about possible bullous pemphigoid disease. In one particular embodiment, the method is for obtaining information about possible bullous pemphigoid disease. In another particular embodiment, the method is for obtaining information about possible non-bullous pemphigoid disease. In yet another particular embodiment, the method is for obtaining information about possible non-bullous pemphigoid disease or bullous pemphigoid disease.

[0111] The biological cell matrix is ​​preferably a primate skin matrix. The primate skin matrix is ​​preferably a monkey skin matrix. The primate skin matrix preferably has an epidermis and a dermis. The primate skin matrix is ​​preferably a so-called salt-cracked skin comprising vesicles between the epidermis and dermis.

[0112] Figure 8 The apparatus V1 is shown, by means of which the method according to the invention can preferably be implemented. The apparatus V1 may be referred to as a fluorescence microscope. The apparatus V1 has a holding device H for a matrix S or a slide comprising such a matrix S, which is cultured in the manner previously described. Excitation light AL of an excitation source LQ is guided toward the matrix S through an optical system O. The generated fluorescence radiation FL is then emitted back through the optical system O and passes through a dichroic mirror SP1 and an optional filter F2. Preferably, the fluorescence radiation FL is filtered out of the green channel by a filter FG. The camera K1 is preferably a monochrome camera, which then detects the fluorescence radiation FL in the green channel in the presence of the filter FG. In an alternative embodiment, the camera K1 is a color imaging camera, which is sufficient without using the filter FG and detects fluorescence images in the corresponding color channels as the green channel using a Bayer array. The camera K1 provides image information BI or a fluorescence image to a computing unit R, which processes the image information BI. Preferably, the computing unit R can output or provide data DE, such as fluorescence images and / or confidence levels, through the data interface DS1.

[0113] Figure 9 A computing unit according to the invention is shown, wherein the computing unit preferably receives a fluorescence image FB as a data signal SI via a data interface DS2, according to a preferred embodiment. The computing unit R can then determine and provide the aforementioned information as a data signal SI3 via a data interface DS3. Preferably, this can be done via a wired or wireless data network. Particularly preferably, the computing unit R has an output interface AS for outputting information via an output unit AE. The output unit AE is preferably a display unit for optically displaying the aforementioned information.

[0114] Figure 10 A data network device DV according to a preferred embodiment of the invention is shown. The data network device DV receives a fluorescence image FB as a data signal SI1 via a data interface DS4. The data network device DV has the previously described computing unit R and storage unit MEM. The computing unit R, storage unit MEM, and data interface DS4 are preferably interconnected via an internal data bus IDB.

[0115] Figure 11 An embodiment of the proposed computer program product CPP is shown. The computer program product CPP can receive its data signal SI2 through the data interface DSX via the computer CO.

[0116] Figure 12 Under the label "Combined," experimental results for the cellular matrix of salt-cracked skin are shown for positive cases, i.e., the presence of a fluorescent pattern in at least one boundary region GB1, particularly the vesicle top, or GB2, particularly the vesicle bottom, of the two boundary regions. This includes cases where fluorescence is present in both boundary regions GB1 and GB2. If no fluorescence is present in boundary regions GB1 and GB2, the case is evaluated as negative. Of the 32 actually positive fluorescent images, 31 images were identified as positive and 1 image as negative using the "EPa classifier" according to the method of the present invention. Of the 77 actually negative fluorescent images, 4 images were identified as positive and 73 images as negative using the "EPa classifier" according to the method of the present invention. Therefore, in total ("sum"), 109 images are given, of which 35 images were identified as positive and 74 images as negative. The accuracy ("accuracy") is 95.4%. The sensitivity ("PPA") is 96.9%. The specificity (“NPA”) was 94.8%.

[0117] Figure 12Furthermore, under the label "Epidermis," experimental results for positive cases of salt-cracked skin cellular matrix are shown, i.e., the presence of a fluorescent pattern in the first boundary region GB1, particularly at the vesicle tip. This includes cases where fluorescence is present in both boundary regions GB1 and GB2. If no fluorescent pattern is present in boundary region GB1, the case is assessed as negative. Of the 26 actually positive fluorescent images, 25 were identified as positive and 1 as negative using the "EPa classifier" according to the method of the present invention. Of the 83 actually negative fluorescent images, 3 were identified as positive and 80 as negative using the "EPa classifier" according to the method of the present invention. That is, in total ("sum"), 109 images are given, of which 28 were identified as positive and 81 as negative. The accuracy ("accuracy") is 96.3%. The sensitivity ("PPA") is 96.2%. The specificity ("NPA") is 96.4%.

[0118] Figure 12 Furthermore, under the label "Dermis," experimental results for positive cases of cellular matrix in salt-cracked skin are shown, specifically, the presence of a fluorescent pattern in the second boundary region GB2, particularly at the base of the vesicle. This includes cases where fluorescence is present in both boundary regions GB1 and GB2. Of the 8 actually positive fluorescent images, 8 were identified as positive and 0 as negative using the "EPa classifier" according to the method of the present invention. Of the 101 actually negative fluorescent images, 3 were identified as positive and 98 as negative using the "EPa classifier" according to the method of the present invention. In total ("Total"), 109 images were given, of which 11 were identified as positive and 98 as negative. The accuracy ("Accuracy") was 97.3%. The sensitivity ("PPA") was 100.0%. The specificity ("NPA") was 97.0%.

[0119] Embodiments of the invention can be implemented in hardware or software according to the defined execution requirements. The implementation can be carried out using digital storage media, such as floppy disks, DVDs, Blu-ray discs, CDs, ROMs, PROMs, EPROMs, EEPROMs, or flash memory, hard disks, or other magnetic or optical memories, on which electronically readable control signals are stored. These control signals, in conjunction with or potentially in conjunction with programmable hardware components, enable the implementation of the corresponding methods.

[0120] Programmable hardware components, especially computing units, can be formed from processors, computer processors (CPUs), graphics processing units (GPUs), computers, computer systems, application-specific integrated circuits (ASICs), integrated circuits (ICs), systems on chips (SOCs), programmable logic elements, or field-programmable gate arrays (FPGAs) that include microprocessors.

[0121] The digital storage medium can therefore be machine-readable or computer-readable. That is, some embodiments have a data carrier having electronically readable control signals that can interact with a programmable computer system or programmable hardware components to implement a method described herein. One embodiment is therefore a data carrier (or digital storage medium or computer-readable medium) on which a program for implementing one of the methods described herein is recorded.

[0122] Generally, embodiments of the present invention can be implemented as a program, firmware, computer program, or computer program product including program code, or as data, wherein when the program runs on a processor or programmable hardware component, the program code or the data is effectively used to implement one of the methods. The program code or data may also be stored, for example, on a machine-readable medium or data medium. Furthermore, the program code or data may exist as source code, machine code, or bytecode, as well as as other intermediate code.

[0123] Another embodiment is a data stream, a signal sequence, or a sequence of signals that describes a procedure for implementing one of the methods described herein. The data stream, signal sequence, or sequence of signals can be configured, for example, for transmission over a data communication connection, such as the Internet or other networks. This embodiment also represents a signal sequence of data suitable for transmission over a network or data communication connection, wherein the data describes the procedure.

[0124] According to one embodiment, a program can, during its implementation, perform, for example, reading a memory location or writing one or more data into the memory location, thereby causing, if necessary, a switching process or other process in a transistor structure, amplifier structure, or other electrical, optical, magnetic, or otherwise operating component. Accordingly, data, values, sensor values, or other information of the program can be detected, determined, or measured by reading the memory location. The program can therefore detect, determine, or measure various parameters, values, measurement parameters, and other information by reading one or more memory locations.

Claims

1. A method for digital image processing, the method comprising: - Provides immunofluorescence images representing biological cell matrix stained with fluorescent dyes. in, The biological cell matrix is ​​the salt-cracked skin matrix of primates. - Using a first neural network, segmentation information with at least a first segmentation region and a second segmentation region is determined by segmentation of the immunofluorescence image, wherein each segmentation region represents a corresponding cellular matrix region. - Boundary regions are determined based on segmentation information, whereby the boundary regions represent the transition from the first cellular matrix region to the second cellular matrix region in the fluorescence image. The matrix includes a cellular matrix region in the form of an epidermis, another cellular matrix region in the form of a dermis, and yet another cellular matrix region in the form of vesicles. The boundary region is the boundary region between the epidermis and the vesicle, in the form of the vesicle top. Alternatively, the boundary region is the boundary region between the blister and the dermis, in the form of the blister base. - Select multiple partial images from the immunofluorescence images along the boundary region. - The confidence level of the presence of the fluorescent pattern is determined based on the multiple partial images using a second neural network.

2. The method according to claim 1, wherein, The method also has a confidence level output.

3. The method according to claim 1, wherein, The method further has - A second neural network is used to determine the corresponding partial image confidence level for the corresponding partial image. - And determine confidence based on partial image confidence.

4. The method according to claim 1, wherein, The method further includes randomly selecting the plurality of partial images from the immunofluorescence images along the boundary regions.

5. The method according to claim 1, wherein, The method further has - A second neural network is used to determine the corresponding brightness value for the corresponding portion of the image. - Determine the total brightness value based on the brightness value.

6. The method according to claim 1, wherein, The plurality of partial images are partial images of the first type. The method further has - Using a first neural network, segmentation information is determined by segmenting an immunofluorescence image, comprising at least a first segmentation region, a second segmentation region, and a third segmentation region, wherein the third segmentation region represents a third cellular matrix region. - Based on the segmentation information, a second boundary region is determined, which represents a second transition from the second cell matrix region to the third cell matrix region in the fluorescence image. - Select multiple partial images of type II from the immunofluorescence images along the second boundary region. - A second confidence level for the presence of a second fluorescent pattern is determined based on the plurality of partial images of the second type using a third neural network.

7. A computer program product having instructions that, when executed by a computer, cause the computer to perform the method for digital image processing according to claim 1.

8. A digital storage medium on which a computer program product according to claim 7 is stored.

9. An apparatus for detecting at least one fluorescent pattern on an immunofluorescence image of a biological cell matrix, said apparatus comprising: - A device for holding slides containing a cell matrix cultured with a patient sample containing autoantibodies and, in addition, a secondary antibody, wherein the autoantibodies and the secondary antibody are labeled with fluorescent dyes. in, The biological cell matrix is ​​the salt-cracked skin matrix of primates. - At least one image acquisition unit for detecting fluorescence images of the cell matrix, The device further includes at least one computing unit configured to perform the following steps: - Using a first neural network, segmentation information with at least a first segmentation region and a second segmentation region is determined by segmentation of the immunofluorescence image, wherein each segmentation region represents a corresponding cellular matrix region. - Based on the segmentation information, a boundary region is determined, which represents the transition from the first cellular matrix region to the second cellular matrix region in the fluorescence image. The matrix includes a cellular matrix region in the form of an epidermis, another cellular matrix region in the form of a dermis, and yet another cellular matrix region in the form of vesicles. The boundary region is the boundary region between the epidermis and the vesicle, in the form of the vesicle top. Alternatively, the boundary region is the boundary region between the blister and the dermis, in the form of the blister base. - Select multiple partial images from the immunofluorescence images along the boundary region. - The confidence level of the presence of the fluorescent pattern is determined based on the multiple partial images using a second neural network.

10. A data network device, the data network device It has at least one data interface for receiving fluorescence images, which represent biological cell matrix stained with fluorescent dyes. in, The biological cell matrix is ​​the salt-cracked skin matrix of primates. The data network device further includes at least one computing unit configured to perform the following steps during digital image processing: - Using a first neural network, segmentation information with at least a first segmentation region and a second segmentation region is determined by segmentation of the immunofluorescence image, wherein each segmentation region represents a corresponding cellular matrix region. - Boundary regions are determined based on segmentation information, whereby the boundary regions represent the transition from the first cellular matrix region to the second cellular matrix region in the fluorescence image. The matrix includes a cellular matrix region in the form of an epidermis, another cellular matrix region in the form of a dermis, and yet another cellular matrix region in the form of vesicles. The boundary region is the boundary region between the epidermis and the vesicle, in the form of the vesicle top. Alternatively, the boundary region is the boundary region between the blister and the dermis, in the form of the blister base. - Select multiple partial images from the immunofluorescence images along the boundary region. - The confidence level of the presence of the fluorescent pattern is determined based on the multiple partial images using a second neural network.

Images