Fluorescence detection image processing method and device

Abstract

The invention relates to the technical field of data processing, in particular to a fluorescence detection image processing method and device. Obtaining a plurality of original samples; dyeing the original sample to form a fluorescent sample; in a fluorescent environment, shooting the target units in the fluorescent sample smear area one by one to form a target fluorescent picture; in a non-fluorescent environment, shooting the target units in the fluorescent sample smear area one by one to form a target wet film picture; traversing each target wet film picture, and carrying out edge smoothing processing on the target wet film pictures to form edge texture pictures of the wet film pictures; traversing each target fluorescence picture, and processing the target fluorescence picture through a neural network model to obtain a processed fluorescence picture; performing edge smoothing processing on the processed fluorescent picture to form an edge texture picture of the fluorescent picture; and training a neural network model to form a fluorescence detection image processing neural network model based on the difference between the edge texture image parameter of the fluorescence image and the edge texture image parameter of the corresponding wet image.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a fluorescence detection image processing method and device. Background Art

[0002] The share of routine leucorrhea testing in the medical testing market has been increasing, because routine leucorrhea testing is an important indicator for evaluating the health of the female urogenital tract. There are many testing methods for routine leucorrhea testing, among which the most mainstream morphological testing methods include wet mount testing, Gram staining testing, fluorescence testing, etc.

[0003] At present, the more common morphological detection methods are wet mount detection and Gram staining detection, both of which have major defects. First of all, wet mounts are easily disturbed in preparation, and the requirements for the interpretation personnel are very high, so it is impossible to achieve large-scale and accurate detection. The biggest problem with Gram staining is that it cannot effectively distinguish some flora in clue cells, and it will destroy Trichomonas. As the only target for diagnosing Trichomonas vaginitis, Trichomonas is indispensable in the inspection process. Because of the defects of the above two inspection methods, fluorescent staining has gradually occupied the market. However, the principle of fluorescent light emission is excitation light. For this reason, there is always a halo around the target. This halo will cause the texture and edges of some core targets, such as white blood cells and naked nuclei, to be unable to be effectively seen, which will cause subsequent doctors or artificial intelligence to make mistakes in the judgment of samples. Therefore, how to take pictures with sharp edges and clear textures is very important for the fluorescence detection of reproductive tract microorganisms. If pictures with clear edges and textures can be taken, it is of great significance for gynecological morphological detection using fluorescence, and can greatly improve the accuracy. Summary of the invention

[0004] The present invention provides a fluorescence detection image processing method and device, which can solve the problem of blurred edges of fluorescence images in related technologies and generate fluorescence images with clear textures and edges.

[0005] In order to solve the technical problem in the background technology, the first aspect of the present invention provides a fluorescence detection image processing method, the fluorescence detection image processing method comprising the following steps:

[0006] Acquire a plurality of original samples; each of the original samples comprises a smear region, and each of the smear regions comprises a plurality of target units of the same size;

[0007] staining the original sample to form a fluorescent sample;

[0008] In a fluorescent environment, target units of the fluorescent sample smear area are photographed one by one within a first photographing window range to form a target fluorescent image group, and all the target fluorescent image groups form a fluorescent training set;

[0009] In a non-fluorescent environment, photographing target units of the fluorescent sample smear area one by one within the first photographing window range to form a target wet mount image group, and all the target wet mount image groups form a wet mount training set;

[0010] Traversing each target wet film image in the wet film training set, performing edge smoothing processing on the target wet film image, and forming an edge texture image of a wet film image corresponding to the target wet film image;

[0011] Traversing each target fluorescence image in the fluorescence training set, processing the target fluorescence image through a neural network model, and obtaining a processed fluorescence image corresponding to the target fluorescence image;

[0012] Performing edge smoothing on the processed fluorescence image to form an edge texture image of the fluorescence image;

[0013] Based on the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image, training the neural network model to form a fluorescence detection image processing neural network model;

[0014] The fluorescence detection image processing neural network model is used to generate fluorescence images with clear textures and edges.

[0015] Among them, the wet film that does not show fluorescent light has the characteristics of clear texture and edges. First, an edge texture image of the wet film image is obtained to obtain the clear edge and texture characteristics of the target in the original sample (fluorescent sample). Then, the target fluorescence image is processed by a neural network model to obtain a processed fluorescence image with unknown edge and texture clarity. Then, an edge texture image of the fluorescence image is obtained to obtain the edge and texture characteristics of the processed fluorescence image. The difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image is used to train the neural network model to form a fluorescence detection image processing neural network model. Through the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image, a neural network model for generating a fluorescence image with clear texture and edges can be trained to solve the problem of blurred edges of the generated fluorescence image.

[0016] Optionally, the step of traversing each target wet film image in the wet film training set, performing edge smoothing processing on the target wet film image, and forming an edge texture image of a wet film image corresponding to the target wet film image comprises:

[0017] Obtaining the original wet film grayscale value of each wet film pixel in a target wet film image;

[0018] Traversing each wet film pixel in the target wet film image, and determining the related wet film pixel and the related wet film pixel gray value matrix of each wet film pixel;

[0019] A first updated wet film grayscale value of the wet film pixel is formed by performing matrix multiplication of a first Gaussian kernel weight matrix and a grayscale value matrix of related wet film pixels; the first updated wet film grayscale values ​​of all wet film pixels in the target wet film image form a first updated wet film image;

[0020] The second updated wet film grayscale values ​​of the wet film pixels are formed by performing matrix multiplication of the second Gaussian kernel weight matrix and the grayscale value matrix of the relevant wet film pixels; the second updated wet film grayscale values ​​of all the wet film pixels in the target wet film image form a second updated wet film image;

[0021] The first updated wet film image is subtracted from the second updated wet film image and the absolute value is taken to form an edge texture image of the wet film image.

[0022] Among them, unlike the target fluorescence image, the target wet film image has clear boundaries and textures because the target therein does not undergo fluorescence reaction, and no halo with blurred boundaries is produced due to fluorescence. The edge features and texture features of the target in the edge texture image of the wet film image formed by the above steps have high reliability.

[0023] Optionally, the second Gaussian kernel weight matrix is ​​smaller than the first Gaussian kernel weight matrix.

[0024] By making the second Gaussian kernel weight matrix smaller than the first Gaussian kernel weight matrix, an edge texture image of a wet film with higher definition is obtained.

[0025] Optionally, the step of performing matrix multiplication of the first Gaussian kernel weight matrix and the relevant wet film pixel gray value matrix to form the first updated wet film gray value of the wet film pixel; and the step of forming the first updated wet film image from the first updated wet film gray values ​​of all wet film pixels in the target wet film image comprises:

[0026] Establishing a coordinate system with a wet film pixel point as the center, and determining the coordinate of the relevant wet film pixel point in the coordinate system;

[0027] based on The formula calculates the Gaussian kernel corresponding to the wet film pixel point and the Gaussian kernel corresponding to the related wet film pixel to form a first Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ 1 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

[0028] Optionally, σ 1 Take 1, and All are taken as 0, and the first Gaussian kernel weight matrix is:

[0029]

[0030] Optionally, the step of performing matrix multiplication of the second Gaussian kernel weight matrix with the relevant wet film pixel gray value matrix to form the second updated wet film gray value of the wet film pixel; and the step of forming the second updated wet film image from the second updated wet film gray values ​​of all wet film pixels in the target wet film image comprises:

[0031] based on The formula calculates the Gaussian kernel corresponding to the wet film pixel point and the Gaussian kernel corresponding to the related wet film pixel to form a second Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ 2 is a variable constant with any value in the range [0,10], and σ 2 Greater than σ in step S532 1 , and is a variable constant with any value in the range [0,1].

[0032] Optionally, σ 2 Take 1.6, and All are taken as 0, and the second Gaussian kernel weight matrix is:

[0033]

[0034] Optionally, the step of subtracting the first updated wet film image from the second updated wet film image and taking the absolute value to form an edge texture image of the wet film image includes: subtracting the first updated wet film grayscale value from the second updated wet film grayscale value corresponding to all wet film pixels in the first updated wet film image and the second updated wet film image and taking the absolute value.

[0035] Optionally, the step of training the neural network model to form a fluorescence detection image processing neural network model based on the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image comprises:

[0036] The set of grayscale values ​​of all wet pixel points in the edge texture image of the wet image is {z 湿1 ,z 湿2 ,z 湿3 …z湿n}; The set of gray values ​​of all fluorescent pixels in the edge texture image of the fluorescence image is {z 荧1 ,z 荧2 ,z 荧3 …z 荧n};

[0037] pass Calculating the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image, and when the Loss value is less than 3, training the neural network model to form a fluorescence detection image processing neural network model, wherein the fluorescence detection image processing neural network model can generate a fluorescence image with clear texture and edges;

[0038] Where n is the length of the grayscale value set of the fluorescent pixel point and the grayscale value set of the wet film pixel point, z 荧i is the i-th element in the grayscale value set of the fluorescent pixel, z 湿i is the i-th element of the grayscale value set of the wet film pixel, δ is a variable constant, and the range is any number in [0,10].

[0039] Optionally, in the step of traversing each target fluorescence image in the fluorescence training set, processing the target fluorescence image through a neural network model, and obtaining a processed fluorescence image corresponding to the target fluorescence image, the neural network model includes:

[0040] A convolution layer cluster, a transformation layer cluster, a hidden layer cluster, and an upsampling layer cluster are sequentially arranged from the input end to the output end;

[0041] The convolution layer cluster performs local perception and local feature extraction on the input target fluorescence image, and inputs it into the conversion layer cluster after nonlinear change. The conversion layer cluster performs global perception and global feature extraction and inputs it into the hidden layer. The hidden layer performs mapping and inputs it into the upsampling layer cluster. The upsampling layer cluster obtains the local perception data and local feature extraction data of the target fluorescence image in the convolution layer cluster and restores it to form a processed fluorescence image.

[0042] Among them, the conversion layer cluster focuses on the extraction of global information, but tends to ignore the image details at low resolution, which will cause great damage to the subsequent upsampling layer cluster to restore the image size, and will cause the problem of serious loss of detail information and fail to achieve the expected effect. In this embodiment, by setting a convolution layer cluster before the conversion layer cluster, the local and detailed features of the target fluorescence image are extracted before the conversion layer cluster processing, so as to avoid the loss of detail information of the target fluorescence image after the conversion layer cluster processing. In addition, the upsampling layer cluster is combined with the convolution layer cluster to restore the processed fluorescence image, which can fuse the local detail feature data of the target fluorescence image, make up for the problem of data loss in the conversion layer cluster, and improve the restoration effect of the processed fluorescence image.

[0043] In order to solve the technical problem in the background technology, the second aspect of the present invention provides a fluorescence detection image processing device, the fluorescence detection image processing device includes a processor and a memory, and the memory stores executable instructions;

[0044] The processor is configured to read instructions in the memory to execute the fluorescence detection image processing method described in the first aspect of the present invention.

[0045] The fluorescence detection image processing method and device provided by the present invention can train a neural network model for generating fluorescence images with clear textures and edges, thereby solving the problem of blurred edges of the generated fluorescence images. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For non-fluorescence technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A schematic diagram of a fluorescence detection image processing method according to an embodiment of the present invention is shown;

[0048] Figure 2 A schematic diagram of a smear area of ​​an original sample in this embodiment is shown;

[0049] Figure 3 A schematic diagram of a target wet film image in one embodiment is shown;

[0050] Figure 4 Shows Figure 3 The edge texture image of the wet film image formed after the target wet film image in step S5 is processed;

[0051] Figure 5A schematic diagram showing a wet film pixel and related wet film pixels;

[0052] Figure 6 A schematic diagram of a target fluorescence image in one embodiment is shown;

[0053] Figure 7 Shows Figure 6 The target fluorescence image in step S6 is processed to form a processed fluorescence image;

[0054] Figure 8 A schematic diagram of a neural network model in step S6 is shown in one embodiment of the present invention;

[0055] Fig. 9 Shows Figure 7 The processed fluorescence image in step S7 is an edge texture image of the fluorescence image formed after the processed fluorescence image in step S7. DETAILED DESCRIPTION

[0056] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] In the description of the present invention, it should be noted that the description of the orientation or positional relationship indicated by "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like in the present embodiment is only used to explain the relative positional relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly. Therefore, it cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0058] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, fixed installation, or a detachable connection, detachable installation, or an integral connection, an integral installation; it can be a mechanical connection or an electrical connection, wherein the electrical connection can include any one or a combination of two of a power drive connection and a communication connection; it can be a direct connection, a direct installation, or an indirect connection or indirect installation through an intermediate medium, or it can be the internal connection of two elements, which can be a wireless connection or a wired connection. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0059] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0060] Figure 1 The following is a schematic diagram of a fluorescence detection image processing method according to an embodiment of the present invention. Figure 1 It can be seen that the fluorescence detection image processing method includes the following steps S1 to S8:

[0061] Step S1: Acquire multiple original samples; each of the original samples includes a smear area, and each of the smear areas includes multiple target units of the same size.

[0062] The original sample in this embodiment is a glass slide with a female leucorrhea sample in the smear area, and the smear area of ​​all original samples is the same size. The number of the multiple original samples is 800 to 2000, preferably 1000, and a larger number of original samples is conducive to improving the accuracy of image processing. There are 80 to 200 target units in each original sample smear area, preferably 100.

[0063] The target unit in the smear area is a range window for each shooting in steps S3 and S4, not an objectively existing physical unit, and there is no actual boundary between the target units.

[0064] Figure 2 A schematic diagram of the smear area of ​​an original sample in this embodiment is shown.

[0065] from Figure 2 It can be seen that the size of the smear area 100 of the original sample in this embodiment is 0.4mm*0.3mm, and the size of the target unit 110 is 0.004mm*0.003mm, so there are 100 target units 110 in the smear area 100 in this embodiment, which are {M1, M2, M3..., M100}.

[0066] It should be noted that, for the convenience of description Figure 2 The boundary of the target unit 110 is drawn in FIG. 1 , but in reality, the target unit 110 does not have an objective physical boundary, and it is only for the convenience of describing a fictitious unit module.

[0067] Step S2: dyeing the original sample to form a fluorescent sample.

[0068] Among them, the original sample can be stained using existing fluorescent staining reagents and fluorescent staining methods.

[0069] Step S3: In a fluorescent environment, target units in the fluorescent sample smear area are photographed one by one within the first photographing window range to form a target fluorescent image group, and all target fluorescent image groups form a fluorescent training set.

[0070] In a fluorescent environment, photographing target units of the fluorescent sample smear area one by one within a first photographing window range includes: irradiating the first photographing window with incident light of a specific wavelength, and photographing target units of the fluorescent sample smear area one by one within the first photographing window range.

[0071] The wavelength of the incident light corresponds to the fluorescent dyeing reagent used in step S2, that is, the target dyed by the fluorescent reagent will undergo a fluorescent reaction when irradiated with incident light of this wavelength.

[0072] By photographing the fluorescent sample smear area in the fluorescent environment, a fluorescent reaction can occur in the smear area of ​​the fluorescent sample, thereby obtaining a target fluorescent picture in which the target emits fluorescence.

[0073] The fluorescence training set includes a plurality of fluorescence samples formed by the original samples; each of the fluorescence samples has a target fluorescence picture group, each of the target fluorescence picture groups includes a plurality of target fluorescence pictures, and the target fluorescence pictures correspond to the target units one by one. Figure 2 Taking the smear area 100 shown in FIG. 1 as an example, a target fluorescence image group of a fluorescence sample corresponding to the smear area 100 is: 荧光1 ,A 荧光2 ,A 荧光3 …,A 荧光100}, the target fluorescence image group includes Figure 2 The target units {M1, M2, M3…, M100} shown in FIG. 1 correspond to multiple target fluorescence images {A 荧光1 ,A 荧光2 ,A 荧光3 …,A 荧光100}.

[0074] Step S4: In a non-fluorescent environment, target units of the fluorescent sample smear area are photographed one by one within the first photographing window range to form a target wet mount image group, and all target wet mount image groups form a wet mount training set.

[0075] The non-fluorescent environment is an environment without any lighting conditions that can cause the smear area of ​​the fluorescent sample to produce a fluorescent reaction.

[0076] In this non-fluorescent environment, the smear area of ​​the fluorescent sample does not undergo a fluorescent reaction, resulting in a target wet mount image in which the target in the smear area does not emit fluorescence.

[0077] The wet film training set includes a plurality of wet film samples formed by the original samples; each of the wet film samples has a target wet film picture group, each of the target wet film picture groups includes a plurality of target wet film pictures, and the target wet film pictures correspond to the target units one by one. Figure 2 Taking the smear area 100 shown in FIG. 1 as an example, a target wet film image group of a wet film sample corresponding to the smear area 100 is: 湿片1 ,A 湿片2 ,A 湿片3 …,A 湿片100}, the target wet film image group includes Figure 2 The target units {M1, M2, M3…, M100} shown in FIG. 1 correspond to multiple target wet film images {A 湿片1 ,A 湿片2 ,A 湿片3 …,A 湿片100}.

[0078] The size of the first shooting window range in step S3 and step S4 is equal to the size of the target unit. Figure 2 Taking the smear area 100 as an example, the range of the first shooting window is equal to the size of the target unit 110, which is 0.004mm*0.003mm.

[0079] Step S5: traverse each target wet film image in the wet film training set, perform edge smoothing on the target wet film image, and form an edge texture image Zwet{zwet 湿1 ,z 湿2 ,z 湿3 …z 湿n}; where Z 湿n , is the grayscale value of any wet pixel in the edge texture image of the wet image, and n is greater than or equal to 1.

[0080] Figure 3 A schematic diagram of a target wet film image in an embodiment is shown. Figure 3 As can be seen in the figure, the target wet film image has a target image, and the target image does not have fluorescent light.

[0081] Figure 4 Shows Figure 3 The edge texture image of the wet film image is formed after the target wet film image in step S5 is processed.

[0082] Combination Figure 3 and Figure 4 It can be seen that step S5 extracts Figure 3 The boundary features and texture features of targets such as white blood cells and naked nuclei in the wet mount image shown in the figure are formed Figure 4 The edge texture image of the wet image is shown.

[0083] Unlike the target fluorescence image, the target wet film image has clear boundaries and textures because the target therein does not undergo fluorescence reaction, and no halo with blurred boundaries is produced due to fluorescence. The edge features and texture features of the target in the edge texture image of the wet film formed in step S5 have high reliability.

[0084] Exemplarily, the above step S5 may be implemented through the following steps S51 to S55.

[0085] Step S51: obtaining the original wet film grayscale value of each wet film pixel in a target wet film image;

[0086] Step S52: traverse each wet film pixel in the target wet film image, and determine the related wet film pixel and the related wet film pixel gray value matrix of each wet film pixel.

[0087] The related wet film pixels of a wet film pixel have factors that affect the grayscale value of the wet film pixel. Optionally, the related wet film pixels are all other wet film pixels centered on the wet film pixel and surrounded by the wet film pixel; the matrix formed by the grayscale values ​​of all other wet film pixels centered on the wet film pixel and surrounded by the wet film pixel is the related wet film pixel grayscale value matrix.

[0088] In this embodiment, all other wet film pixels within a 5*5 range around the wet film pixel point and centered on the wet film pixel point may be selected as relevant wet film pixels.

[0089] Step S53: performing matrix multiplication of the first Gaussian kernel weight matrix and the relevant wet film pixel gray value matrix to form the first updated wet film gray value of the wet film pixel; the first updated wet film gray value of all wet film pixels in the target wet film image forms the first updated wet film image.

[0090] Step S54: performing matrix multiplication of the second Gaussian kernel weight matrix and the relevant wet film pixel gray value matrix to form second updated wet film gray values ​​of the wet film pixels; the second updated wet film gray values ​​of all wet film pixels in the target wet film image form a second updated wet film image.

[0091] The second Gaussian kernel weight matrix is ​​smaller than the first Gaussian kernel weight matrix.

[0092] Step S55: Subtract the first updated wet film image from the second updated wet film image and take the absolute value to form an edge texture image of the wet film image.

[0093] Among them, the subtraction of the absolute value of the first updated wet film image and the second updated wet film image in step S55 is to subtract the absolute value of the first updated wet film grayscale value and the second updated wet film grayscale value corresponding to all wet film pixels in the first updated wet film image and the second updated wet film image.

[0094] Through the above steps S51 to S55, an edge texture image of a wet film with clear edge features and texture features can be obtained.

[0095] Exemplarily, step S53 may be implemented by following steps S531 to S532:

[0096] Step S531: establishing a coordinate system with a wet film pixel point as the center, and determining the coordinates of the relevant wet film pixel point in the coordinate system.

[0097] Reference Figure 5 , which shows a schematic diagram of a wet film pixel and related wet film pixels.

[0098] from Figure 5 It can be seen that a coordinate system is established with the wet film pixel point as the center (0,0), and all other wet film pixels within a 5*5 range around the wet film pixel point (0,0) are related wet film pixels, and each related wet film pixel point has corresponding coordinates.

[0099] Step S532: Based on The formula calculates the Gaussian kernel corresponding to the wet film pixel point and the Gaussian kernel corresponding to the related wet film pixel to form a first Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ 1 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

[0100] In this embodiment, the σ 1 Take 1, and All are taken as 0.

[0101] by Figure 5 As an example of the embodiment shown in FIG. 1 , when σ in the above formula 1 Take 1, and When all are 0, the first Gaussian kernel weight matrix calculated through the above steps S531 to S532 is:

[0102]

[0103] On the basis of the above steps S531 to S532, step S54 can be implemented by the following steps:

[0104] based on The formula calculates the Gaussian kernel corresponding to the wet film pixel point and the Gaussian kernel corresponding to the related wet film pixel to form a second Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ 2 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

[0105] In this embodiment, the σ 2 Take 1.6, and All are taken as 0.

[0106] by Figure 5 As an example of the embodiment shown in FIG. 1 , when σ in the above formula 2 Take 1.6, and All are taken as 0. The second Gaussian kernel weight matrix calculated by the above steps is:

[0107]

[0108] Step S6: traverse each target fluorescence image in the fluorescence training set, process the target fluorescence image through a neural network model, and obtain a processed fluorescence image corresponding to the target fluorescence image.

[0109] Figure 6 A schematic diagram of a target fluorescence image in an embodiment is shown. Figure 6 As can be seen from the figure, the target fluorescence picture has a target image, and the target image has fluorescent brightness. Due to the fluorescent brightness, the edge and texture of the target image are blurred.

[0110] Figure 7 Shows Figure 6 The target fluorescence image in is processed in step S6 to form a processed fluorescence image.

[0111] Combination Figure 6 and Figure 7 It can be seen that Figure 6 The target fluorescence image shown is a processed fluorescence image formed after step S6

[0112] Figure 8 FIG. 1 shows a schematic diagram of a neural network model in step S6 provided in an embodiment of the present invention, from Figure 8It can be seen that the neural network model includes a convolution layer cluster 810, a conversion layer cluster 820, a hidden layer 830 and an upsampling layer cluster 840 which are arranged in sequence from the input end to the output end. The convolution layer cluster 810 performs local perception and local feature extraction on the input target fluorescence image, and inputs it into the conversion layer cluster 820 after nonlinear change. The conversion layer cluster 820 performs global perception and global feature extraction and then inputs it into the hidden layer 830. The hidden layer 830 performs mapping and then inputs it into the upsampling layer cluster 840. The upsampling layer cluster 840 obtains the local perception data and local feature extraction data of the target fluorescence image in the convolution layer cluster 810 to restore and form a processed fluorescence image.

[0113] In this embodiment, the convolution layer cluster 810 includes a first convolution layer 811, a second convolution layer 812, and a third convolution layer 813 whose inputs and outputs are connected in sequence, and the size of the convolution kernel of each convolution layer is 3*3, and the step length is 2. The conversion layer cluster 820 includes eight conversion layers 821 whose inputs and outputs are connected in sequence. The upsampling layer cluster 840 includes a first upsampling layer 841, a second upsampling layer 842, a third upsampling layer 843, and a fourth upsampling layer 844 whose inputs and outputs are connected in sequence. The second upsampling layer 842 also obtains the local perception data and local feature extraction data of the target fluorescence image output by the third convolution layer 813, the third upsampling layer 843 also obtains the local perception data and local feature extraction data of the target fluorescence image output by the second convolution layer 812, and the fourth upsampling layer 844 also obtains the local perception data and local feature extraction data of the target fluorescence image output by the first convolution layer 811.

[0114] Among them, the conversion layer cluster 820 focuses on the extraction of global information, but tends to ignore the image details at low resolution, which will cause great harm to the subsequent upsampling layer cluster 840 to restore the image size, resulting in serious loss of detail information and failing to achieve the expected effect.

[0115] In this embodiment, by setting the convolution layer cluster 810 before the conversion layer cluster 820, the local and detailed features of the target fluorescence image are extracted before the conversion layer cluster 820 processes, so as to avoid the loss of the detail information of the target fluorescence image after the conversion layer cluster 820 processes. In addition, the upsampling layer cluster 840 is combined with the convolution layer cluster 810 to restore the processed fluorescence image, which can integrate the local detail feature data of the target fluorescence image, make up for the problem of data loss in the conversion layer cluster 820, and improve the restoration effect of the processed fluorescence image.

[0116] For example, the input target fluorescence image size is 1200*1200*3, and the data size is 600*600*6 after the first convolution layer 811 is convolved for the first time. The data size generated after the second convolution layer 812 is 300*300*12, and then the third convolution layer 813 is convolved for the third time to obtain a data size of 150*150*24. After that, the data is nonlinearly transformed, and the size remains unchanged. After that, the data is input into the conversion layer cluster 820, and the output data size is 75*75*192. The data is input into the hidden layer 830, and the data size remains unchanged. Then, the data is passed through the first upsampling layer 841. The first upsampling is performed, and the data size after sampling is 150*150*24. The data of the first upsampling is added to the data of the third convolutional layer 813, and then the second upsampling is performed through the second upsampling layer 842 to obtain a data size of 300*300*12. Subsequently, the data of the second upsampling is added to the data of the second convolutional layer 812, and then the third upsampling is performed through the third upsampling layer 843 to obtain a data size of 600*600*6. Subsequently, the data is added to the first convolutional layer 811, and then upsampled through the fourth upsampling layer 844. After obtaining the data, an upsampling with a convolution kernel size of 1*1 and a step size of 1 is performed to obtain the output image.

[0117] Step S7: performing edge smoothing on the processed fluorescence image to form an edge texture image Zfluorescence {z 荧1 ,z 荧2 ,z 荧3 …z 荧n}; where Z 荧n , is the grayscale value of any fluorescent pixel in the edge texture image of the fluorescence image, and n is greater than or equal to 1.

[0118] The fluorescent pixel points in the edge texture image of the fluorescent image correspond one-to-one to the wet film pixel points in the edge texture image of the wet film image described in step S5.

[0119] Fig. 9 Shows Figure 7 The processed fluorescence image in step S7 is an edge texture image of the fluorescence image formed after the processed fluorescence image in step S7.

[0120] Combination Figure 7 and Fig. 9 It can be seen that step S7 extracts Figure 7 The boundary features and texture features of targets such as white blood cells and naked nuclei in the processed fluorescence images are formed Fig. 9 The edge texture image of the fluorescence image shown.

[0121] Exemplarily, the above step S7 may be implemented through the following steps S71 to S75.

[0122] Step S71: obtaining the original fluorescence grayscale value of each fluorescence pixel in a target fluorescence image;

[0123] Step S72: traverse each fluorescent pixel point in the target fluorescent image, and determine the related fluorescent pixel points and the related fluorescent pixel gray value matrix of each fluorescent pixel point.

[0124] The related fluorescent pixel points of a fluorescent pixel point have factors that affect the grayscale value of the fluorescent pixel point. Optionally, the related fluorescent pixel points are all other fluorescent pixel points surrounding the fluorescent pixel point with the fluorescent pixel point as the center; the matrix formed by the grayscale values ​​of all other fluorescent pixel points surrounding the fluorescent pixel point with the fluorescent pixel point as the center is the related fluorescent pixel grayscale value matrix.

[0125] In this embodiment, all other fluorescent pixel points within a 5*5 range around the fluorescent pixel point and centered on the fluorescent pixel point may be selected as relevant fluorescent pixel points.

[0126] Step S73: performing matrix multiplication of the first Gaussian kernel weight matrix and the relevant fluorescence pixel gray value matrix to form the first updated fluorescence gray value of the fluorescence pixel; the first updated fluorescence gray values ​​of all the fluorescence pixels in the target fluorescence image form a first updated fluorescence image.

[0127] Step S74: performing matrix multiplication of the second Gaussian kernel weight matrix and the relevant fluorescence pixel gray value matrix to form the second updated fluorescence gray value of the fluorescence pixel; the second updated fluorescence gray values ​​of all the fluorescence pixels in the target fluorescence image form a second updated fluorescence image.

[0128] The second Gaussian kernel weight matrix is ​​smaller than the first Gaussian kernel weight matrix.

[0129] Step S75: subtract the first updated fluorescence image from the second updated fluorescence image and take the absolute value to form an edge texture image of the fluorescence image.

[0130] The subtraction of the first updated fluorescent image from the second updated fluorescent image to obtain the absolute value in step S75 is to subtract the first updated fluorescent grayscale value from the second updated fluorescent grayscale value corresponding to all fluorescent pixels in the first updated fluorescent image and the second updated fluorescent image to obtain the absolute value.

[0131] Through the above steps S71 to S75, an edge texture picture of the fluorescence image with clear edge features and texture features can be obtained.

[0132] Exemplarily, step S73 may be implemented by following steps S731 to S732:

[0133] Step S731: establishing a coordinate system with a fluorescent pixel point as the center, and determining the coordinates of the relevant fluorescent pixel point in the coordinate system.

[0134] Step S732: Based on The formula calculates the Gaussian kernel corresponding to the fluorescent pixel and the Gaussian kernel corresponding to the related fluorescent pixel to form a first Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the fluorescent pixel or the related fluorescent pixel, σ 1 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

[0135] In this embodiment, the σ 1 Take 1, and All are taken as 0.

[0136] On the basis of the above steps S731 to S732, step S74 can be implemented by the following steps:

[0137] based on The formula calculates the Gaussian kernel corresponding to the fluorescent pixel and the Gaussian kernel corresponding to the related fluorescent pixel to form a second Gaussian kernel weight matrix; where (x new ,y new ) represents the coordinates of the fluorescent pixel or the related fluorescent pixel, σ 2 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

[0138] In this embodiment, the σ 2 Take 1.6, and All are taken as 0.

[0139] Step S8: Based on the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image, the neural network model is trained to form a fluorescence detection image processing neural network model.

[0140] From step S7 and step S5, it can be known that the set of grayscale values ​​of all wet pixel points in the edge texture image of the wet image is {z 湿1 ,z 湿2 ,z 湿3 …z 湿n}; The set of gray values ​​of all fluorescent pixels in the edge texture image of the fluorescence image is {z 荧1 ,z 荧2 ,z 荧3 …z 荧n}.

[0141] For example, it can be achieved by The difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image is calculated. When the Loss value is less than 3, it is considered that the neural network model is trained to form a fluorescence detection image processing neural network model. It is considered that the fluorescence detection image processing neural network model can generate fluorescence images with clear textures and edges, and the training continues.

[0142] Where n is the length of the grayscale value set of the fluorescent pixel point and the grayscale value set of the wet film pixel point, z 荧i is the i-th element in the grayscale value set of the fluorescent pixel, z 湿i is the i-th element of the grayscale value set of the wet film pixel, δ is a variable constant, and the range is any number in [0,10]. In this embodiment, δ is 2.56.

[0143] The present invention also provides a fluorescence detection image processing device, the fluorescence detection image processing device comprises a processor and a memory, the memory stores executable instructions; the processor is configured to read the instructions in the memory to execute the present invention Figures 1 to 9 The fluorescence detection image processing method is shown.

[0144] The present invention utilizes the characteristics of a wet film that does not show fluorescent light and has clear texture and edges. An edge texture image of the wet film image is first obtained to obtain the clear edge and texture characteristics of the target in the original sample (fluorescent sample). Subsequently, the target fluorescent image is processed through a neural network model to obtain a processed fluorescent image with unknown edge and texture clarity. Then, an edge texture image of the fluorescent image is obtained to obtain the edge and texture characteristics of the processed fluorescent image. The difference between the edge texture image parameters of the fluorescent image and the edge texture image parameters of the corresponding wet film image is used to train the neural network model to form a fluorescent detection image processing neural network model. The difference between the edge texture image parameters of the fluorescent image and the edge texture image parameters of the corresponding wet film image can be used to train a neural network model for generating a fluorescent image with clear texture and edges, thereby solving the problem of blurred edges of the generated fluorescent image.

[0145] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A fluorescence detection image processing method, characterized in that: The fluorescence detection image processing method comprises the following steps: Acquire a plurality of original samples; each of the original samples comprises a smear region, and each of the smear regions comprises a plurality of target units of the same size; staining the original sample to form a fluorescent sample; In a fluorescent environment, target units of the fluorescent sample smear area are photographed one by one within a first photographing window range to form a target fluorescent image group, and all the target fluorescent image groups form a fluorescent training set; In a non-fluorescent environment, photographing target units of the fluorescent sample smear area one by one within the first photographing window range to form a target wet mount image group, and all the target wet mount image groups form a wet mount training set; Traversing each target wet film image in the wet film training set, performing edge smoothing processing on the target wet film image, and forming an edge texture image of a wet film image corresponding to the target wet film image; Traversing each target fluorescence image in the fluorescence training set, processing the target fluorescence image through a neural network model, and obtaining a processed fluorescence image corresponding to the target fluorescence image; Performing edge smoothing on the processed fluorescence image to form an edge texture image of the fluorescence image; Based on the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image, training the neural network model to form a fluorescence detection image processing neural network model; The fluorescence detection image processing neural network model is used to generate fluorescence images with clear textures and edges.

2. The fluorescence detection image processing method according to claim 1, characterized in that: The step of traversing each target wet film image in the wet film training set, performing edge smoothing processing on the target wet film image, and forming an edge texture image of a wet film image corresponding to the target wet film image comprises: Obtaining the original wet film grayscale value of each wet film pixel in a target wet film image; Traversing each wet film pixel in the target wet film image, and determining the related wet film pixel and the related wet film pixel gray value matrix of each wet film pixel; A first updated wet film grayscale value of the wet film pixel is formed by performing matrix multiplication of a first Gaussian kernel weight matrix and a grayscale value matrix of related wet film pixels; the first updated wet film grayscale values ​​of all wet film pixels in the target wet film image form a first updated wet film image; The second updated wet film grayscale values ​​of the wet film pixels are formed by performing matrix multiplication of the second Gaussian kernel weight matrix and the grayscale value matrix of the relevant wet film pixels; the second updated wet film grayscale values ​​of all the wet film pixels in the target wet film image form a second updated wet film image; The first updated wet film image is subtracted from the second updated wet film image and the absolute value is taken to form an edge texture image of the wet film image.

3. The fluorescence detection image processing method according to claim 2, characterized in that: The second Gaussian kernel weight matrix is ​​smaller than the first Gaussian kernel weight matrix.

4. The fluorescence detection image processing method according to claim 2, characterized in that: The step of performing matrix multiplication of the first Gaussian kernel weight matrix and the grayscale value matrix of the relevant wet film pixels to form the first updated wet film grayscale values ​​of the wet film pixels; and the step of forming the first updated wet film image by the first updated wet film grayscale values ​​of all the wet film pixels in the target wet film image comprises: Establishing a coordinate system with a wet film pixel point as the center, and determining the coordinate of the relevant wet film pixel point in the coordinate system; based on The formula calculates the Gaussian kernel corresponding to the wet film pixel point and the Gaussian kernel corresponding to the related wet film pixel to form a first Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ1 is a variable constant with any value in the range [0,10], and is a variable constant with any value in the range [0,1].

5. The fluorescence detection image processing method according to claim 4, characterized in that: σ1 is 1, and All are taken as 0, and the first Gaussian kernel weight matrix is:

6. The fluorescence detection image processing method according to claim 4, characterized in that: The step of performing matrix multiplication of the second Gaussian kernel weight matrix with the relevant wet film pixel gray value matrix to form the second updated wet film gray value of the wet film pixel; and the step of forming the second updated wet film image with the second updated wet film gray values ​​of all wet film pixels in the target wet film image comprises: based on The formula calculates the Gaussian kernel corresponding to the wet film pixel point and the Gaussian kernel corresponding to the related wet film pixel to form a second Gaussian kernel weight matrix; wherein (x new ,y new ) represents the coordinates of the wet film pixel or the related wet film pixel, σ2 is a variable constant with any value in the range of [0,10], and σ2 is greater than σ1 in step S532, and is a variable constant with any value in the range [0,1].

7. The fluorescence detection image processing method according to claim 6, characterized in that: σ2 is 1.6, and All are taken as 0, and the second Gaussian kernel weight matrix is:

8. The fluorescence detection image processing method according to claim 2, characterized in that: The step of subtracting the first updated wet film image from the second updated wet film image and taking the absolute value to form an edge texture image of the wet film image includes: subtracting the first updated wet film grayscale value and the second updated wet film grayscale value corresponding to all wet film pixels in the first updated wet film image and the second updated wet film image and taking the absolute value.

9. The fluorescence detection image processing method according to claim 1, characterized in that: The step of training the neural network model to form a fluorescence detection image processing neural network model based on the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image comprises: The set of grayscale values ​​of all wet pixel points in the edge texture image of the wet image is {z 湿1 ,z 湿2 ,z 湿3 …z 湿n }; The set of gray values ​​of all fluorescent pixels in the edge texture image of the fluorescence image is {z 荧1 ,z 荧2 ,z 荧3 …z 荧n }; pass Calculating the difference between the edge texture image parameters of the fluorescence image and the edge texture image parameters of the corresponding wet film image, and when the Loss value is less than 3, training the neural network model to form a fluorescence detection image processing neural network model, wherein the fluorescence detection image processing neural network model can generate a fluorescence image with clear texture and edges; Where n is the length of the grayscale value set of the fluorescent pixel point and the grayscale value set of the wet film pixel point, z 荧i is the i-th element in the grayscale value set of the fluorescent pixel, z 湿i is the i-th element of the grayscale value set of the wet film pixel, δ is a variable constant, and the range is any number in [0,10].

10. The fluorescence detection image processing method according to claim 1, characterized in that: In the step of traversing each target fluorescence image in the fluorescence training set, processing the target fluorescence image through a neural network model, and obtaining a processed fluorescence image corresponding to the target fluorescence image, the neural network model includes: A convolution layer cluster, a transformation layer cluster, a hidden layer cluster, and an upsampling layer cluster are sequentially arranged from the input end to the output end; The convolution layer cluster performs local perception and local feature extraction on the input target fluorescence image, and inputs it into the conversion layer cluster after nonlinear change. The conversion layer cluster performs global perception and global feature extraction and inputs it into the hidden layer. The hidden layer performs mapping and inputs it into the upsampling layer cluster. The upsampling layer cluster obtains the local perception data and local feature extraction data of the target fluorescence image in the convolution layer cluster and restores it to form a processed fluorescence image.

11. A fluorescence detection image processing device, characterized in that: The fluorescence detection image processing device comprises a processor and a memory, wherein executable instructions are stored in the memory; The processor is configured to read instructions in the memory to execute the fluorescence detection image processing method according to any one of claims 1 to 10.

Images