Fluorescence detection image processing method and device

Abstract

The present invention relates to the field of data processing technology, and in particular to a fluorescence detection image processing method and device. The method comprises the following steps: obtaining a plurality of original samples; staining the original samples to form fluorescence samples; photographing target units in a smear area of ​​the fluorescence sample one by one in a fluorescence environment to form a target fluorescence image; photographing target units in a smear area of ​​the fluorescence sample one by one in a non-fluorescence environment to form a target wet film image; traversing each target wet film image, performing edge smoothing processing on the target wet film image to form an edge texture image of the wet film image; traversing each target fluorescence image, processing the target fluorescence image through a neural network model to obtain a processed fluorescence image; performing edge smoothing processing on the processed fluorescence image to form an edge texture image of the fluorescence image; and training a neural network model based on the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image to form a fluorescence detection image processing neural network model.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a fluorescence detection image processing method and device. Background Art

[0002] Routine gynecological vaginal discharge testing has been steadily increasing its share in the medical testing market, as routine vaginal discharge examination is an important indicator for evaluating the health of the female urogenital tract. There are various methods for routine vaginal discharge testing, the most popular of which include wet mount testing, Gram staining, and fluorescence testing.

[0003] Currently, the most common morphological testing methods, wet mount and Gram stain, both have significant drawbacks. First, wet mount preparation is susceptible to interference and places high demands on the interpreter, making it difficult to accurately detect large numbers of samples. Gram staining, on the other hand, poses a significant challenge: it cannot effectively distinguish certain bacterial communities within clue cells and can damage Trichomonas, which are essential for diagnosing trichomoniasis. Due to these limitations, fluorescent staining has gradually gained market share. However, fluorescence, which relies on excitation light, often produces a halo around the target. This halo can obscure the texture and edges of key targets, such as white blood cells and naked nuclei, leading to errors in subsequent sample interpretation by physicians or artificial intelligence. Therefore, capturing images with sharp edges and clear textures is crucial for fluorescence testing of genital tract microorganisms. Capturing images with clear edges and textures is crucial for gynecological morphological testing using fluorescence, significantly improving accuracy. Summary of the Invention

[0004] The present invention provides a fluorescence detection image processing method and device, which can solve the problem of blurred edges of fluorescence images in related technologies and generate fluorescence images with clear textures and edges.

[0005] In order to solve the technical problem in the background technology, the first aspect of the present invention provides a fluorescence detection image processing method, the fluorescence detection image processing method comprising the following steps:

[0006] Acquire multiple original samples; each of the original samples includes a smear area, and each of the smear areas includes multiple target units of the same size;

[0007] staining the original sample to form a fluorescent sample;

[0008] In a fluorescence environment, target units of the fluorescence sample smear area are photographed one by one within a first photographing window range to form a target fluorescence image group, and all the target fluorescence image groups form a fluorescence training set;

[0009] In a non-fluorescent environment, photographing target units of the fluorescent sample smear area one by one within the first photographing window range to form a target wet mount image group, wherein all the target wet mount image groups form a wet mount training set;

[0010] Traversing each target wet film image in the wet film training set, performing edge smoothing processing on the target wet film image, and forming an edge texture image of the wet film image corresponding to the target wet film image;

[0011] Traversing each target fluorescence image in the fluorescence training set, processing the target fluorescence image through a neural network model, and obtaining a processed fluorescence image corresponding to the target fluorescence image;

[0012] performing edge smoothing processing on the processed fluorescence image to form an edge texture image of the fluorescence image;

[0013] Based on the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image, training the neural network model to form a fluorescence detection image processing neural network model;

[0014] The fluorescence detection image processing neural network model is used to generate fluorescence images with clear textures and edges.

[0015] Among them, the wet film that does not show fluorescent light has the characteristics of clear texture and edges. The edge texture image of the wet film image is first obtained to obtain the clear edge and texture characteristics of the target in the original sample (fluorescent sample). Then, the target fluorescence image is processed by a neural network model to obtain a processed fluorescence image with unknown edge and texture clarity. Then, the edge texture image of the fluorescence image is obtained to obtain the edge and texture characteristics of the processed fluorescence image. The difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image is used to train the neural network model to form a fluorescence detection image processing neural network model. The difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image can be used to train a neural network model for generating fluorescence images with clear texture and edges, thereby solving the problem of blurred edges of the generated fluorescence images.

[0016] Optionally, the step of traversing each target wet film image in the wet film training set, performing edge smoothing on the target wet film image, and forming an edge texture image of a wet film image corresponding to the target wet film image includes:

[0017] Obtaining the original wet film grayscale value of each wet film pixel in a target wet film image;

[0018] Traversing each wet film pixel in the target wet film image, and determining a related wet film pixel and a related wet film pixel grayscale value matrix for each wet film pixel;

[0019] forming first updated wet film grayscale values ​​of the wet film pixels by performing matrix multiplication of a first Gaussian kernel weight matrix and a grayscale value matrix of related wet film pixels; and forming a first updated wet film image by the first updated wet film grayscale values ​​of all wet film pixels in the target wet film image;

[0020] performing matrix multiplication of a second Gaussian kernel weight matrix and a grayscale value matrix of relevant wet film pixels to form second updated wet film grayscale values ​​of the wet film pixels; and forming a second updated wet film image with the second updated wet film grayscale values ​​of all wet film pixels in the target wet film image.

[0021] The first updated wet film image and the second updated wet film image are subtracted and their absolute values ​​are taken to form an edge texture image of the wet film image.

[0022] Among them, unlike the target fluorescence image, the target wet film image has clear boundaries and textures because the target therein does not undergo fluorescence reaction, and no halo with blurred boundaries is produced due to fluorescence. The edge features and texture features of the target in the edge texture image of the wet film image formed by the above steps are highly reliable.

[0023] Optionally, the second Gaussian kernel weight matrix is ​​smaller than the first Gaussian kernel weight matrix.

[0024] By making the second Gaussian kernel weight matrix smaller than the first Gaussian kernel weight matrix, an edge texture image of a wet film with higher definition is obtained.

[0025] Optionally, the step of performing matrix multiplication of the first Gaussian kernel weight matrix and the relevant wet film pixel grayscale value matrix to form the first updated wet film grayscale value of the wet film pixel; and forming the first updated wet film image from the first updated wet film grayscale values ​​of all wet film pixels in the target wet film image comprises:

[0026] Establishing a coordinate system with a wet film pixel point as the center, and determining the coordinate of the relevant wet film pixel point in the coordinate system;

[0027] based on The formula calculates the Gaussian kernel corresponding to the wet film pixel and the Gaussian kernel corresponding to the related wet film pixel to form a first Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ1 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

[0028] Optionally, σ1 is set to 1, and All are set to 0, and the first Gaussian kernel weight matrix is:

[0029]

[0030] Optionally, the step of performing matrix multiplication of the second Gaussian kernel weight matrix and the relevant wet film pixel grayscale value matrix to form the second updated wet film grayscale value of the wet film pixel; and forming the second updated wet film image from the second updated wet film grayscale values ​​of all wet film pixels in the target wet film image comprises:

[0031] based on The formula calculates the Gaussian kernel corresponding to the wet film pixel and the Gaussian kernel corresponding to the related wet film pixel to form a second Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ2 is a variable constant with any value in the range [0,10], and σ2 is greater than σ1 in step S532, and is a variable constant with any value in the range [0,1].

[0032] Optionally, σ2 is taken as 1.6, and All are set to 0, and the second Gaussian kernel weight matrix is:

[0033]

[0034] Optionally, the step of subtracting the first updated wet film image from the second updated wet film image and taking the absolute value to form the edge texture image of the wet film image includes: subtracting the first updated wet film grayscale value and the second updated wet film grayscale value corresponding to all wet film pixels in the first updated wet film image and the second updated wet film image and taking the absolute value.

[0035] Optionally, the step of training the neural network model to form a fluorescence detection image processing neural network model based on the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image includes:

[0036] The set of grayscale values ​​of all wet pixel points in the edge texture image of the wet image is {z 湿1 ,z 湿2 ,z 湿3 …z 湿n}; The set of grayscale values ​​of all fluorescent pixels in the edge texture image of the fluorescence image is {z 荧1 ,z 荧2 ,z荧3 …z 荧n};

[0037] pass Calculating the difference between edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image, and when the loss value is less than 3, training the neural network model to form a fluorescence detection image processing neural network model, wherein the fluorescence detection image processing neural network model can generate a fluorescence image with clear texture and edges;

[0038] Where n is the length of the grayscale value set of the fluorescent pixel point and the grayscale value set of the wet film pixel point, z 荧i is the i-th element in the grayscale value set of the fluorescent pixel, z 湿i is the i-th element of the grayscale value set of the wet film pixel, δ is a variable constant, and the range is any number in [0,10].

[0039] Optionally, in the step of traversing each target fluorescence image in the fluorescence training set, processing the target fluorescence image through a neural network model, and obtaining a processed fluorescence image corresponding to the target fluorescence image, the neural network model includes:

[0040] The convolution layer cluster, transformation layer cluster, hidden layer cluster and upsampling layer cluster are arranged in sequence from the input end to the output end;

[0041] The convolution layer cluster performs local perception and local feature extraction on the input target fluorescence image, and inputs it into the conversion layer cluster after nonlinear change. The conversion layer cluster performs global perception and global feature extraction and then inputs it into the hidden layer. The hidden layer performs mapping and then inputs it into the upsampling layer cluster. The upsampling layer cluster obtains the local perception data and local feature extraction data of the target fluorescence image in the convolution layer cluster and restores it to form a processed fluorescence image.

[0042] Among them, the conversion layer cluster focuses on the extraction of global information, but tends to ignore the image details at low resolution, which will cause great damage to the subsequent upsampling layer cluster to restore the image size, and will cause serious loss of detail information and fail to achieve the expected effect. In this embodiment, by setting a convolution layer cluster before the conversion layer cluster, the target fluorescence image is first subjected to local and detailed feature extraction before the conversion layer cluster is processed, so as to avoid the loss of detail information of the target fluorescence image after the conversion layer cluster is processed. In addition, the upsampling layer cluster is combined with the convolution layer cluster to restore the processed fluorescence image, which can fuse the local detail feature data of the target fluorescence image, make up for the problem of data loss in the conversion layer cluster, and improve the restoration effect of the processed fluorescence image.

[0043] In order to solve the technical problem in the background technology, a second aspect of the present invention provides a fluorescence detection image processing device, the fluorescence detection image processing device includes a processor and a memory, the memory storing executable instructions;

[0044] The processor is configured to read instructions in the memory to execute the fluorescence detection image processing method described in the first aspect of the present invention.

[0045] The fluorescence detection image processing method and device provided by the present invention can train a neural network model for generating fluorescence images with clear textures and edges, thereby solving the problem of blurred edges in the generated fluorescence images. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For non-fluorescence technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0047] Figure 1 A schematic diagram of a fluorescence detection image processing method according to an embodiment of the present invention is shown;

[0048] Figure 2 Schematic diagram of the smear area of ​​an original sample in this embodiment is shown;

[0049] Figure 3 A schematic diagram of a target wet film image in one embodiment is shown;

[0050] Figure 4 Shown Figure 3 The edge texture image of the wet film image formed after the target wet film image in step S5 is processed;

[0051] Figure 5 A schematic diagram showing a wet film pixel and related wet film pixels;

[0052] Figure 6 A schematic diagram of a target fluorescence image in one embodiment is shown;

[0053] Figure 7 Shown Figure 6 The processed fluorescence image formed after the target fluorescence image in step S6 is processed;

[0054] Figure 8 FIG. 1 shows a schematic diagram of a neural network model in step S6 according to an embodiment of the present invention;

[0055] Figure 9 Shown Figure 7 The processed fluorescence image in step S7 is processed to form an edge texture image of the fluorescence image. DETAILED DESCRIPTION

[0056] The following is a clear and complete description of the technical solutions of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0057] In describing the present invention, it should be noted that the descriptions of the orientations or positional relationships indicated by terms such as "center," "up," "down," "left," "right," "vertical," "horizontal," "inside," and "outside" in this embodiment are intended only to explain the relative positional relationships and movement of components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0058] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense. For example, it can be a fixed connection, fixed installation, or a detachable connection, detachable installation, or an integral connection, an integral installation; it can be a mechanical connection or an electrical connection, wherein the electrical connection can include any one or a combination of two of a power drive connection and a communication connection; it can be a direct connection, a direct installation, or an indirect connection or indirect installation through an intermediate medium, or it can be the internal communication of two elements, which can be a wireless connection or a wired connection. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0059] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0060] Figure 1 The following is a flow chart of a fluorescence detection image processing method according to an embodiment of the present invention. Figure 1 As can be seen from FIG, the fluorescence detection image processing method includes the following steps S1 to S8:

[0061] Step S1: Acquire multiple original samples; each of the original samples includes a smear area, and each of the smear areas includes multiple target units of the same size.

[0062] In this embodiment, the original sample is a glass slide with a female leucorrhea sample in the smear area. The smear areas of all original samples are of the same size. The number of original samples ranges from 800 to 2000, preferably 1000. A larger number of original samples is beneficial for improving the accuracy of image processing. The number of target cells in the smear area of ​​each original sample is 80 to 200, preferably 100.

[0063] The target unit in the smear area is the range window of each shooting in steps S3 and S4, not an objective physical unit, and there is no actual boundary between the target units.

[0064] Figure 2 A schematic diagram of the smear area of ​​an original sample in this embodiment is shown.

[0065] from Figure 2 As can be seen from the figure, the size of the smear area 100 of the original sample in this embodiment is 0.4 mm*0.3 mm, and the size of the target unit 110 is 0.004 mm*0.003 mm. Therefore, there are 100 target units 110 in the smear area 100 in this embodiment, namely {M1, M2, M3…, M100}.

[0066] It should be noted that, for the convenience of description Figure 2 The boundary of the target unit 110 is drawn in FIG. 1 , but in reality, the target unit 110 does not have an objective physical boundary. It is only for the convenience of describing the fictitious unit module.

[0067] Step S2: dyeing the original sample to form a fluorescent sample.

[0068] Among them, the original sample can be stained using existing fluorescent staining reagents and fluorescent staining methods.

[0069] Step S3: In a fluorescence environment, target units in the fluorescence sample smear area are photographed one by one within the first photographing window range to form a target fluorescence image group. All target fluorescence image groups form a fluorescence training set.

[0070] In a fluorescent environment, photographing target units of the fluorescent sample smear area one by one within a first photographing window range includes: irradiating the first photographing window with incident light of a specific wavelength, and photographing target units of the fluorescent sample smear area one by one within the first photographing window range.

[0071] The wavelength of the incident light corresponds to the fluorescent dyeing reagent used in step S2, that is, the target dyed by the fluorescent reagent will undergo a fluorescent reaction under the irradiation of the incident light of this wavelength.

[0072] Photographing the fluorescent sample smear area in the fluorescent environment can cause a fluorescent reaction in the fluorescent sample smear area, thereby obtaining a target fluorescent image in which the target emits fluorescence.

[0073] The fluorescence training set includes a plurality of fluorescence samples formed by the original samples; each of the fluorescence samples has a target fluorescence image group, each of the target fluorescence image groups includes a plurality of target fluorescence images, and the target fluorescence images correspond to the target units one by one. Figure 2 Taking the smear area 100 shown in FIG. 1 as an example, a target fluorescence image group of a fluorescence sample corresponding to the smear area 100 is: 荧光1 ,A 荧光2 ,A 荧光3 …,A 荧光100}, the target fluorescence image group includes Figure 2 The target units {M1, M2, M3…, M100} shown in FIG1 correspond to multiple target fluorescence images {A 荧光1 ,A 荧光2 ,A 荧光3 …,A 荧光100}.

[0074] Step S4: In a non-fluorescent environment, target units in the fluorescent sample smear area are photographed one by one within the first photographing window range to form a target wet mount image group. All target wet mount image groups form a wet mount training set.

[0075] The non-fluorescent environment is an environment without any lighting conditions that can cause the smear area of ​​the fluorescent sample to produce a fluorescent reaction.

[0076] In this non-fluorescent environment, the smear area of ​​the fluorescent sample does not undergo a fluorescent reaction, resulting in a target wet mount image in which the target in the smear area does not emit fluorescence.

[0077] The wet film training set includes a plurality of wet film samples formed by the original samples; each of the wet film samples has a target wet film image group, each of the target wet film image groups includes a plurality of target wet film images, and the target wet film images correspond one to one with the target units. Figure 2 Taking the smear area 100 shown in FIG. 1 as an example, a target wet film image group of a wet film sample corresponding to the smear area 100 is: 湿片1 ,A 湿片2 ,A 湿片3 …,A 湿片100}, the target wet film image group includes Figure 2 The target units {M1, M2, M3…, M100} shown in FIG are one-to-one corresponding to multiple target wet film images {A 湿片1 ,A 湿片2 ,A 湿片3 …,A 湿片100}.

[0078] The size of the first shooting window range in step S3 and step S4 is equal to the size of the target unit. Figure 2 Taking the smear area 100 as an example, the range of the first shooting window is equal to the size of the target unit 110, which is 0.004mm*0.003mm.

[0079] Step S5: traverse each target wet film image in the wet film training set, perform edge smoothing on the target wet film image, and form an edge texture image Zwet{z 湿1 ,z 湿2 ,z 湿3 …z 湿n}; where Z 湿n , is the grayscale value of any wet film pixel in the edge texture image of the wet film image, and n is greater than or equal to 1.

[0080] Figure 3 A schematic diagram of a target wet film image in one embodiment is shown. Figure 3 As can be seen from the figure, the target wet mount picture has a target image, and the target image does not have fluorescent light.

[0081] Figure 4 Shown Figure 3 The edge texture image of the wet film image is formed after the target wet film image in step S5 is processed.

[0082] Combine Figure 3 and Figure 4 It can be seen that step S5 extracts Figure 3 The boundary features and texture features of targets such as white blood cells and naked nuclei in the target wet mount image are formed Figure 4 The edge texture image of the wet film is shown.

[0083] Unlike the target fluorescence image, the target wet mount image has clear boundaries and textures because the target therein does not undergo fluorescence reaction, and no halo with blurred boundaries is produced due to fluorescence. The edge features and texture features of the target in the edge texture image of the wet mount image formed in step S5 are highly reliable.

[0084] Exemplarily, the above step S5 may be implemented through the following steps S51 to S55 .

[0085] Step S51: obtaining the original wet film grayscale value of each wet film pixel in a target wet film image;

[0086] Step S52: traverse each wet film pixel in the target wet film image, and determine the related wet film pixel and the related wet film pixel grayscale value matrix of each wet film pixel.

[0087] The related wet film pixels of a wet film pixel have factors that affect the grayscale value of the wet film pixel. Optionally, the related wet film pixels are all other wet film pixels centered on the wet film pixel and surrounding the wet film pixel; and a matrix formed by the grayscale values ​​of all other wet film pixels centered on the wet film pixel and surrounding the wet film pixel is the related wet film pixel grayscale value matrix.

[0088] In this embodiment, all other wet film pixels within a 5*5 range around the wet film pixel point and centered on the wet film pixel point may be selected as relevant wet film pixels.

[0089] Step S53: performing matrix multiplication of the first Gaussian kernel weight matrix and the relevant wet film pixel grayscale value matrix to form the first updated wet film grayscale value of the wet film pixel; the first updated wet film grayscale values ​​of all wet film pixels in the target wet film image form the first updated wet film image.

[0090] Step S54: performing matrix multiplication of the second Gaussian kernel weight matrix and the relevant wet film pixel grayscale value matrix to form the second updated wet film grayscale value of the wet film pixel; the second updated wet film grayscale values ​​of all wet film pixels in the target wet film image form a second updated wet film image.

[0091] The second Gaussian kernel weight matrix is ​​smaller than the first Gaussian kernel weight matrix.

[0092] Step S55: Subtract the first updated wet film image from the second updated wet film image and take the absolute value to form an edge texture image of the wet film image.

[0093] Among them, the subtraction of the absolute value of the first updated wet film image and the second updated wet film image described in step S55 is the subtraction of the absolute value of the first updated wet film grayscale value and the second updated wet film grayscale value corresponding to all wet film pixels in the first updated wet film image and the second updated wet film image.

[0094] Through the above steps S51 to S55 , an edge texture image of a wet film with clear edge features and texture features can be obtained.

[0095] Exemplarily, step S53 may be implemented by following steps S531 to S532:

[0096] Step S531: establishing a coordinate system with a wet film pixel point as the center, and determining the coordinates of the relevant wet film pixel point in the coordinate system.

[0097] Reference Figure 5 , which shows a schematic diagram of a wet film pixel and related wet film pixels.

[0098] from Figure 5 As can be seen from the figure, a coordinate system is established with the wet film pixel point as the center (0,0), and all other wet film pixels within a 5*5 range around the wet film pixel point (0,0) are related wet film pixels, and each related wet film pixel point has corresponding coordinates.

[0099] Step S532: Based on The formula calculates the Gaussian kernel corresponding to the wet film pixel and the Gaussian kernel corresponding to the related wet film pixel to form a first Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ1 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

[0100] In this embodiment, σ1 is set to 1. and All are set to 0.

[0101] by Figure 5 Taking the embodiment shown as an example, when σ1 in the above formula is 1, and When both are 0, the first Gaussian kernel weight matrix calculated through steps S531 to S532 is:

[0102]

[0103] Based on the above steps S531 to S532, step S54 can be implemented by the following steps:

[0104] based on The formula calculates the Gaussian kernel corresponding to the wet film pixel and the Gaussian kernel corresponding to the related wet film pixel to form a second Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ2 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

[0105] In this embodiment, σ2 is set to 1.6. and All are set to 0.

[0106] by Figure 5 Taking the embodiment shown as an example, when σ2 in the above formula is 1.6, and All are set to 0. The second Gaussian kernel weight matrix calculated by the above steps is:

[0107]

[0108] Step S6: traverse each target fluorescence image in the fluorescence training set, process the target fluorescence image through the neural network model, and obtain a processed fluorescence image corresponding to the target fluorescence image.

[0109] Figure 6 A schematic diagram of a target fluorescence image in an embodiment is shown. Figure 6 As can be seen from the figure, the target fluorescence image has a target image, and the target image has bright fluorescence. Due to the bright fluorescence, the edge and texture of the target image are blurred.

[0110] Figure 7 Shown Figure 6 The target fluorescence image in step S6 is processed to form a processed fluorescence image.

[0111] Combine Figure 6 and Figure 7 It can be seen that Figure 6 The target fluorescence image shown is the processed fluorescence image formed after step S6

[0112] Figure 8 The schematic diagram of the neural network model in step S6 is shown in an embodiment of the present invention. Figure 8 It can be seen that the neural network model includes a convolution layer cluster 810, a conversion layer cluster 820, a hidden layer 830 and an upsampling layer cluster 840 arranged in sequence from the input end to the output end. The convolution layer cluster 810 performs local perception and local feature extraction on the input target fluorescence image, and inputs the input into the conversion layer cluster 820 after nonlinear change. Global perception and global feature extraction are performed in the conversion layer cluster 820 and then input into the hidden layer 830. Mapping is performed in the hidden layer 830 and then input into the upsampling layer cluster 840. The upsampling layer cluster 840 obtains the local perception data and local feature extraction data of the target fluorescence image in the convolution layer cluster 810 to restore and form a processed fluorescence image.

[0113] In this embodiment, the convolution layer cluster 810 includes a first convolution layer 811, a second convolution layer 812, and a third convolution layer 813, each of which has sequentially connected inputs and outputs. The convolution kernel size of each convolution layer is 3*3, with a stride of 2. The transformation layer cluster 820 includes eight transformation layers 821, each of which has sequentially connected inputs and outputs. The upsampling layer cluster 840 includes a first upsampling layer 841, a second upsampling layer 842, a third upsampling layer 843, and a fourth upsampling layer 844, each of which has sequentially connected inputs and outputs. The second upsampling layer 842 also obtains the local perception data and local feature extraction data of the target fluorescence image output by the third convolution layer 813. The third upsampling layer 843 also obtains the local perception data and local feature extraction data of the target fluorescence image output by the second convolution layer 812. The fourth upsampling layer 844 also obtains the local perception data and local feature extraction data of the target fluorescence image output by the first convolution layer 811.

[0114] Among them, the conversion layer cluster 820 focuses on extracting global information and tends to ignore image details at low resolution. This will cause great damage to the subsequent upsampling layer cluster 840 to restore the image size, resulting in serious loss of detail information and failing to achieve the expected effect.

[0115] In this embodiment, convolutional layer cluster 810 is placed before conversion layer cluster 820, and local and detailed feature extraction is performed on the target fluorescence image before processing by conversion layer cluster 820. This prevents loss of detail information in the target fluorescence image after processing by conversion layer cluster 820. Furthermore, upsampling layer cluster 840 is combined with convolutional layer cluster 810 to restore the processed fluorescence image. This integrates local detail feature data from the target fluorescence image, compensating for data loss in conversion layer cluster 820 and improving the restoration of the processed fluorescence image.

[0116] For example, if the input target fluorescence image size is 1200*1200*3, the data size obtained after the first convolution layer 811 is 600*600*6, and the data size generated after the second convolution layer 812 is 300*300*12. Then, the data is convolved for the third time by the third convolution layer 813, and the data size is 150*150*24. After that, the data is nonlinearly transformed, and the size remains unchanged. After that, the data is input into the conversion layer cluster 820, and the output data size is 75*75*192. The data is input into the hidden layer 830, and the data size remains unchanged. Then, the data is passed through the first upsampling layer 841. The first upsampling is performed, and the data size after sampling is 150*150*24. The data of the first upsampling is added to the data of the third convolutional layer 813, and then the second upsampling is performed through the second upsampling layer 842 to obtain a data size of 300*300*12. The data of the second upsampling is then added to the data of the second convolutional layer 812, and then the third upsampling is performed through the third upsampling layer 843 to obtain a data size of 600*600*6. The data is then added to the first convolutional layer 811, and then upsampled through the fourth upsampling layer 844. After obtaining the data, an upsampling with a convolution kernel size of 1*1 and a step size of 1 is performed to obtain the output image.

[0117] Step S7: performing edge smoothing on the processed fluorescence image to form an edge texture image Zfluorescence {z 荧1 ,z 荧2 ,z 荧3 …z 荧n}; where Z 荧n , is the grayscale value of any fluorescent pixel in the edge texture image of the fluorescence image, and n is greater than or equal to 1.

[0118] The fluorescent pixels in the edge texture image of the fluorescent image correspond one-to-one to the wet film pixels in the edge texture image of the wet film image described in step S5.

[0119] Figure 9 Shown Figure 7 The processed fluorescence image in step S7 is processed to form an edge texture image of the fluorescence image.

[0120] Combine Figure 7 and Figure 9 It can be seen that step S7 extracts Figure 7 The boundary features and texture features of targets such as white blood cells and naked nuclei in the processed fluorescence images are formed Figure 9 The edge texture image of the fluorescence image shown.

[0121] Exemplarily, the above step S7 may be implemented through the following steps S71 to S75 .

[0122] Step S71: obtaining the original fluorescence grayscale value of each fluorescence pixel in a target fluorescence image;

[0123] Step S72: traverse each fluorescent pixel in the target fluorescent image to determine the related fluorescent pixel and the related fluorescent pixel grayscale value matrix of each fluorescent pixel.

[0124] The related fluorescent pixels of a fluorescent pixel have factors that affect the grayscale value of the fluorescent pixel. Optionally, the related fluorescent pixels are all other fluorescent pixels centered on the fluorescent pixel and surrounding the fluorescent pixel; and a matrix formed by the grayscale values ​​of all other fluorescent pixels centered on the fluorescent pixel and surrounding the fluorescent pixel is the related fluorescent pixel grayscale value matrix.

[0125] In this embodiment, all other fluorescent pixel points within a 5*5 range around the fluorescent pixel point and centered on the fluorescent pixel point may be selected as relevant fluorescent pixel points.

[0126] Step S73: performing matrix multiplication of the first Gaussian kernel weight matrix and the relevant fluorescence pixel gray value matrix to form the first updated fluorescence gray value of the fluorescence pixel; the first updated fluorescence gray values ​​of all fluorescence pixels in the target fluorescence image form a first updated fluorescence image.

[0127] Step S74: performing matrix multiplication of the second Gaussian kernel weight matrix and the relevant fluorescence pixel gray value matrix to form a second updated fluorescence gray value of the fluorescence pixel; the second updated fluorescence gray values ​​of all fluorescence pixels in the target fluorescence image form a second updated fluorescence image.

[0128] The second Gaussian kernel weight matrix is ​​smaller than the first Gaussian kernel weight matrix.

[0129] Step S75: subtracting the first updated fluorescence image from the second updated fluorescence image and taking an absolute value to form an edge texture image of the fluorescence image.

[0130] The subtraction of the first updated fluorescence image and the second updated fluorescence image to obtain the absolute value in step S75 is the subtraction of the first updated fluorescence grayscale value and the second updated fluorescence grayscale value corresponding to all fluorescence pixels in the first updated fluorescence image and the second updated fluorescence image to obtain the absolute value.

[0131] Through the above steps S71 to S75 , an edge texture image of a fluorescence image with clear edge features and texture features can be obtained.

[0132] Exemplarily, step S73 may be implemented by following steps S731 to S732:

[0133] Step S731: establishing a coordinate system with a fluorescent pixel point as the center, and determining the coordinates of the relevant fluorescent pixel point in the coordinate system.

[0134] Step S732: Based on The formula calculates the Gaussian kernel corresponding to the fluorescent pixel and the Gaussian kernel corresponding to the related fluorescent pixel to form a first Gaussian kernel weight matrix; where (x new ,y new ) represents the coordinates of the fluorescent pixel or the related fluorescent pixel, σ1 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

[0135] In this embodiment, σ1 is set to 1. and All are set to 0.

[0136] Based on the above steps S731 to S732, step S74 can be implemented by the following steps:

[0137] based on The formula calculates the Gaussian kernel corresponding to the fluorescent pixel and the Gaussian kernel corresponding to the related fluorescent pixel to form a second Gaussian kernel weight matrix; where (x new ,y new ) represents the coordinates of the fluorescent pixel or the related fluorescent pixel, σ2 is a variable constant with an arbitrary value in the range [0,10], and is a variable constant with any value in the range [0,1].

[0138] In this embodiment, σ2 is set to 1.6. and All are set to 0.

[0139] Step S8: Based on the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image, the neural network model is trained to form a fluorescence detection image processing neural network model.

[0140] From step S7 and step S5, we can know that the set of grayscale values ​​of all wet pixel points in the edge texture image of the wet image is {z 湿1 ,z 湿2 ,z 湿3 …z 湿n}; The set of grayscale values ​​of all fluorescent pixels in the edge texture image of the fluorescence image is {z 荧1 ,z 荧2 ,z荧3 …z 荧n}.

[0141] For example, it can be achieved by The difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image is calculated. When the Loss value is less than 3, it is considered that the neural network model is trained to form a fluorescence detection image processing neural network model. It is considered that the fluorescence detection image processing neural network model can generate fluorescence images with clear textures and edges, and training continues.

[0142] Where n is the length of the grayscale value set of the fluorescent pixel point and the grayscale value set of the wet film pixel point, z 荧i is the i-th element in the grayscale value set of the fluorescent pixel, z 湿i is the i-th element of the grayscale value set of the wet film pixel, δ is a variable constant, and the range is any number in [0, 10]. In this embodiment, δ is 2.56.

[0143] The present invention also provides a fluorescence detection image processing device, the fluorescence detection image processing device includes a processor and a memory, the memory stores executable instructions; the processor is configured to read the instructions in the memory to execute the present invention Figures 1 to 9 The fluorescence detection image processing method is shown.

[0144] The present invention utilizes the characteristics of a wet film that does not show bright fluorescence and has clear texture and edges. First, an edge texture image of the wet film image is obtained to obtain clear edges and texture features of a target in an original sample (fluorescent sample). Subsequently, the target fluorescence image is processed by a neural network model to obtain a processed fluorescence image with unknown edge and texture clarity. Then, an edge texture image of the fluorescence image is obtained to obtain edge and texture features of the processed fluorescence image. The difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image is used to train the neural network model to form a fluorescence detection image processing neural network model. The difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image can be used to train a neural network model for generating a fluorescence image with clear texture and edges, thereby solving the problem of blurred edges in the generated fluorescence image.

[0145] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A fluorescence detection image processing method, characterized in that: The fluorescence detection image processing method comprises the following steps: Acquire multiple original samples; each of the original samples includes a smear area, and each of the smear areas includes multiple target units of the same size; staining the original sample to form a fluorescent sample; In a fluorescence environment, target units of the fluorescence sample smear area are photographed one by one within a first photographing window range to form a target fluorescence image group, and all the target fluorescence image groups form a fluorescence training set; In a non-fluorescent environment, photographing target units of the fluorescent sample smear area one by one within the first photographing window range to form a target wet mount image group, wherein all the target wet mount image groups form a wet mount training set; Traversing each target wet film image in the wet film training set, performing edge smoothing processing on the target wet film image, and forming an edge texture image of the wet film image corresponding to the target wet film image; Traversing each target fluorescence image in the fluorescence training set, processing the target fluorescence image through a neural network model, and obtaining a processed fluorescence image corresponding to the target fluorescence image; performing edge smoothing processing on the processed fluorescence image to form an edge texture image of the fluorescence image; training the neural network model to form a fluorescence detection image processing neural network model based on a difference between edge texture image parameters of the fluorescence image and edge texture image parameters of the corresponding wet film image; the edge texture image parameters of the fluorescence image are a set of grayscale values ​​of all fluorescent pixels in the edge texture image of the fluorescence image, and the edge texture image parameters of the wet film image are a set of grayscale values ​​of all wet film pixels in the edge texture image of the wet film image; The fluorescence detection image processing neural network model is used to generate fluorescence images with clear textures and edges.

2. The fluorescence detection image processing method according to claim 1, wherein: The step of traversing each target wet film image in the wet film training set, performing edge smoothing on the target wet film image, and forming an edge texture image of a wet film image corresponding to the target wet film image includes: Obtaining the original wet film grayscale value of each wet film pixel in a target wet film image; Traversing each wet film pixel in the target wet film image, and determining a related wet film pixel and a related wet film pixel grayscale value matrix for each wet film pixel; forming first updated wet film grayscale values ​​of the wet film pixels by performing matrix multiplication of a first Gaussian kernel weight matrix and a grayscale value matrix of related wet film pixels; and forming a first updated wet film image by the first updated wet film grayscale values ​​of all wet film pixels in the target wet film image; performing matrix multiplication of a second Gaussian kernel weight matrix and a grayscale value matrix of relevant wet film pixels to form second updated wet film grayscale values ​​of the wet film pixels; and forming a second updated wet film image with the second updated wet film grayscale values ​​of all wet film pixels in the target wet film image. The first updated wet film image and the second updated wet film image are subtracted and their absolute values ​​are taken to form an edge texture image of the wet film image.

3. The fluorescence detection image processing method according to claim 2, wherein: The second Gaussian kernel weight matrix is ​​smaller than the first Gaussian kernel weight matrix.

4. The fluorescence detection image processing method according to claim 2, wherein: The step of performing matrix multiplication of the first Gaussian kernel weight matrix and the relevant wet film pixel grayscale value matrix to form the first updated wet film grayscale value of the wet film pixel; and forming the first updated wet film image by the first updated wet film grayscale values ​​of all wet film pixels in the target wet film image comprises: Establishing a coordinate system with a wet film pixel point as the center, and determining the coordinate of the relevant wet film pixel point in the coordinate system; based on The formula calculates the Gaussian kernel corresponding to the wet film pixel and the Gaussian kernel corresponding to the related wet film pixel to form a first Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ1 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

5. The fluorescence detection image processing method according to claim 4, wherein: σ1 is 1, and All are set to 0, and the first Gaussian kernel weight matrix is:

6. The fluorescence detection image processing method according to claim 4, wherein: The step of performing matrix multiplication of the second Gaussian kernel weight matrix and the relevant wet film pixel grayscale value matrix to form the second updated wet film grayscale value of the wet film pixel; and forming the second updated wet film image from the second updated wet film grayscale values ​​of all wet film pixels in the target wet film image comprises: based on The formula calculates the Gaussian kernel corresponding to the wet film pixel and the Gaussian kernel corresponding to the related wet film pixel to form a second Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ2 is a variable constant with any value in the range [0,10], and σ2 is greater than σ1 in step S532, and is a variable constant with any value in the range [0,1].

7. The fluorescence detection image processing method according to claim 6, wherein: σ2 is taken as 1.6, and All are set to 0, and the second Gaussian kernel weight matrix is:

8. The fluorescence detection image processing method according to claim 2, wherein: The step of subtracting the first updated wet film image from the second updated wet film image and taking the absolute value to form the edge texture image of the wet film image includes: subtracting the first updated wet film grayscale value and the second updated wet film grayscale value corresponding to all wet film pixels in the first updated wet film image and the second updated wet film image and taking the absolute value.

9. The fluorescence detection image processing method according to claim 1, wherein: The step of training the neural network model to form a fluorescence detection image processing neural network model based on the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image includes: The set of grayscale values ​​of all wet pixel points in the edge texture image of the wet image is {z 湿1 ,z 湿2 ,z 湿3 …z 湿n }; The set of grayscale values ​​of all fluorescent pixels in the edge texture image of the fluorescence image is {z 荧1 ,z 荧2 ,z 荧3 …z 荧n }; pass Calculating the difference between edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image, and when the loss value is less than 3, training the neural network model to form a fluorescence detection image processing neural network model, wherein the fluorescence detection image processing neural network model can generate a fluorescence image with clear texture and edges; Where n is the length of the grayscale value set of the fluorescent pixel point and the grayscale value set of the wet film pixel point, z 荧 i is the i-th element in the grayscale value set of the fluorescent pixel, z 湿 i is the i-th element of the grayscale value set of the wet film pixel, and δ is a variable constant whose range is any number in [0,10].

10. The fluorescence detection image processing method according to claim 1, wherein: In the step of traversing each target fluorescence image in the fluorescence training set, processing the target fluorescence image through a neural network model, and obtaining a processed fluorescence image corresponding to the target fluorescence image, the neural network model includes: The convolution layer cluster, transformation layer cluster, hidden layer cluster and upsampling layer cluster are arranged in sequence from the input end to the output end; The convolution layer cluster performs local perception and local feature extraction on the input target fluorescence image, and inputs it into the conversion layer cluster after nonlinear change. The conversion layer cluster performs global perception and global feature extraction and then inputs it into the hidden layer. The hidden layer performs mapping and then inputs it into the upsampling layer cluster. The upsampling layer cluster obtains the local perception data and local feature extraction data of the target fluorescence image in the convolution layer cluster and restores it to form a processed fluorescence image.

11. A fluorescence detection image processing device, characterized in that: The fluorescence detection image processing device includes a processor and a memory, wherein the memory stores executable instructions; The processor is configured to read instructions in the memory to execute the fluorescence detection image processing method according to any one of claims 1 to 10.

Images