Gene amplification fluorescence in situ hybridization image intelligent identification system based on deep learning

Abstract

The invention discloses a gene amplification fluorescence in situ hybridization image intelligent identification system based on deep learning. According to the technical scheme provided by the invention, in the system, an image acquisition module is used for performing image acquisition on a sample marked by a FISH technology by using a microscope to obtain an initial image; the SAM segmentation module is used for carrying out image segmentation on the initial image based on an SAM model to determine a segmented image; the ResNet comparison module is used for comparing each quasi-cell image by using the standard cell image, and topping a real cell image; the fluorescent dot counting module is used for carrying out fluorescent dot identification and counting based on a color filtering technology, and carrying out gene amplification detection based on a target gene signal and a chromosome centromere signal. According to the method, more accurate intelligent segmentation of the FISH image is realized based on the SAM model, the accuracy of cell judgment is further improved by using a deep learning technology, the identification and counting of target gene signals are quickly realized, and the gene amplification level is better judged.

Description

Technical Field

[0001] The present invention relates to the field of electric energy storage, and in particular to a deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system, method, computing device and storage medium. Background Art

[0002] Since the 1980s, Fluorescence in situ hybridization (FISH) technology has been able to visually display the location and quantity of specific DNA or RNA in cells or tissue sections by using specific fluorescent-labeled probes to hybridize with target nucleic acid sequences in cells. It has become a powerful means for studying gene abnormalities, chromosomal abnormalities, etc. in cytogenetics, molecular biology, and pathology research.

[0003] Especially in the field of gene amplification detection, the detection of specific gene amplification status through FISH technology has become an important parameter and standard in the diagnosis and treatment of many types of tumors such as well-differentiated liposarcoma, dedifferentiated liposarcoma and breast cancer.

[0004] However, in practical applications, traditional FISH technology requires manual image analysis and result judgment. The judgment process is time-consuming and labor-intensive, with the judgment time for a single case exceeding 30 minutes. In addition, the judgment process is easily affected by the subjective judgment of medical staff, which adversely affects the accuracy and is not repeatable for the same case. In current automated FISH image analysis methods, image processing techniques such as threshold segmentation and edge detection are usually used to identify fluorescent spots. However, such methods have high requirements for image quality, poor adaptability to noise and uneven illumination, and difficulty in accurately distinguishing partially overlapping or densely arranged fluorescent spots. However, due to the attenuation of the fluorescent signal and the presence of cell autofluorescence, the image background noise is high, which greatly affects the accuracy of such methods. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning and a corresponding gene amplification fluorescence in situ hybridization image intelligent recognition method, computing device and computer storage medium based on deep learning.

[0006] According to one aspect of the present invention, a gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning is provided, comprising: an image acquisition module, a SAM segmentation module, a ResNet comparison module and a fluorescence point counting module; wherein,

[0007] The image acquisition module is used to acquire images of the cell detection samples labeled by FISH technology using a microscope to obtain the corresponding initial images;

[0008] The SAM segmentation module is used to perform image segmentation on the initial image based on the SAM model, segment all cells displayed in the initial image, and determine a segmented image, wherein the segmented image includes at least one quasi-cell image and a segmentation prompt for each quasi-cell image;

[0009] The ResNet comparison module is used to compare each quasi-cell image with a standard cell image to screen out a real cell image;

[0010] The fluorescent spot counting module is used to identify and count fluorescent spots in each real cell image based on color filtering technology, and determine the number and ratio of target gene signals and their corresponding chromosome centromere signals under a preset number of tumor cells.

[0011] In the above solution, the SAM segmentation module is further used to:

[0012] Convert the initial image into an image embedding using an image encoder;

[0013] Combining the image embedding with the convolution-processed input mask and the prompt point and / or prompt box and / or prompt text to determine at least one mask region, and treating each mask region as a quasi-cell image;

[0014] Generate a segmented image after segmentation.

[0015] In the above solution, the SAM segmentation module is further used to:

[0016] Using the prompt encoder, determine the position encoding of the prompt point and / or prompt box, and combine the position encoding with the learning embedding of the corresponding prompt type, and finally obtain the prompt point and / or prompt box; based on the text encoder, represent the preset free-format text to obtain the prompt text; wherein,

[0017] The density of the prompt dot matrix is ​​adjusted by setting hyperparameters; the entire image is segmented based on the prompt dot matrix in the image embedding, the cell structure in the initial image is determined, and a segmented image is obtained.

[0018] In the above solution, the ResNet comparison module is further used for:

[0019] ResNet is used to process each quasi-cell image captured after the segmentation image;

[0020] A norm is calculated based on the processed quasi-cell image and the standard cell image processed by ResNet to determine whether the quasi-cell image is a real cell image; wherein,

[0021] If the calculation result of the one-norm is less than the preset threshold, the corresponding quasi-cell image is determined to be a real cell image; if the calculation result of the one-norm is not less than the preset threshold, the corresponding quasi-cell image is determined to be a non-cell image.

[0022] In the above scheme, the fluorescent spot counting module is further used for:

[0023] Take the intersection of each identified real cell image and the initial image to determine the target area for fluorescent spot counting;

[0024] Create a color filter for identifying fluorescent spots, and search for corresponding fluorescent spots in the target area;

[0025] Since the target gene signal is displayed in red and the chromosome centromere signal is displayed in green in the cell detection sample labeled by FISH technology, the color filter is used to target the red fluorescent spot and the green fluorescent spot.

[0026] In the above scheme, the fluorescent spot counting module is further used for:

[0027] Set up a Hue-color comparison table based on the HSV color model, and divide Hue into m intervals containing at least red and green, each interval corresponding to one color;

[0028] Convert each real cell image in the segmented image into a corresponding HSV image, and use a color filter to identify each color region from the HSV image corresponding to the real cell image;

[0029] Red pixel regions and green pixel regions are obtained by screening out each color region, and the number of red pixel regions and green pixel regions is counted respectively.

[0030] In the above scheme, the fluorescent spot counting module is further used for:

[0031] For real cell images, a HS histogram is established based on hue H and saturation S. The number of columns in the HS histogram is the number of intervals m divided by Hue.

[0032] Calculate the value HS(i) of the i-th column in the HS histogram corresponding to the real cell image

[0033] HS(i)=Σ(i-1)·step≤H(x,y)<i·step·S(x,y)

[0034] step=360° / m

[0035] Where i∈[1,m]; step is the span of each interval divided by Hue; H(x,y) is the chromaticity value of the non-edge point (x,y) in the HSV image; S(x,y) is the saturation value of the non-edge point (x,y) in the HSV image;

[0036] Based on HS(i), the interval corresponding to the non-edge point is determined, and then the color of the non-edge point is determined.

[0037] According to another aspect of the present invention, a method for intelligent recognition of gene amplification fluorescence in situ hybridization images based on deep learning is provided, comprising:

[0038] Use a microscope to collect images of cell detection samples labeled by FISH technology to obtain corresponding initial images;

[0039] Performing image segmentation on the initial image based on the SAM model to segment all cells displayed in the initial image and determine a segmented image, wherein the segmented image includes at least one quasi-cell image and a segmentation hint for each quasi-cell image;

[0040] Each quasi-cell image is compared with a standard cell image, and the real cell image is screened out;

[0041] Based on color filtering technology, fluorescent spots are identified and counted in each real cell image to determine the number and ratio of target gene signals and their corresponding chromosome centromere signals under a preset number of tumor cells.

[0042] According to another aspect of the present invention, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0043] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations implemented by the above-mentioned deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system.

[0044] According to another aspect of the present invention, a computer storage medium is provided, wherein the storage medium stores at least one executable instruction, and the executable instruction enables a processor to perform operations implemented by the above-mentioned deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system.

[0045] According to the technical solution provided by the present invention, a deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system includes: an image acquisition module, a SAM segmentation module, a ResNet comparison module and a fluorescence point counting module; wherein the image acquisition module is used to use a microscope to perform image acquisition on a cell detection sample labeled by FISH technology to obtain a corresponding initial image; the SAM segmentation module is used to perform image segmentation on the initial image based on the SAM model, segment all cells displayed in the initial image, and determine the segmented image, wherein the segmented image includes at least one quasi-cell image and a segmentation prompt for each quasi-cell image; the ResNet comparison module is used to compare each quasi-cell image with a standard cell image, and screen out a real cell image therefrom; the fluorescence point counting module is used to perform fluorescence point recognition and counting on each real cell image based on color filtering technology, and determine the number and ratio of target gene signals and their corresponding chromosome centromere signals under a preset number of tumor cells. The collected FISH images are automatically segmented through the SAM model. Based on its efficient, accurate and universal segmentation capabilities, the model training process is greatly reduced, the waste of manpower and time costs caused by manual recognition, and the reduction of recognition accuracy are avoided, which greatly improves the processing efficiency of FISH images. In the segmentation process, the segmentation efficiency and accuracy are further improved based on the prompt dot matrix, and the flexibility of the segmentation process is improved; based on the ResNet deep residual convolutional network, the obtained segmented images are re-judged using standard cell images, and the misjudged cell images are eliminated, further improving the accuracy of the system; through the organic combination of SAM segmentation, ResNet alignment and color filtering, the overall performance of the recognition system is effectively improved, making it more scalable, adaptable and robust; at the same time, the SAM model is used to annotate the relevant data of the segmented images, further avoiding the extra workload caused by manual annotation, and is more adaptable to situations with large data volumes or high annotation difficulty, so that medical staff can better focus on diagnosis and treatment. In addition, by performing HSV conversion on the segmented real cell image and determining the color based on the numerical value of the HS histogram corresponding to the pixel point, the saturation judgment dimension is added, which further improves the accuracy of determining the fluorescent point in gene amplification detection, thereby further improving the accuracy of gene amplification detection.

[0046] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0047] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0049] Figure 1 A structural block diagram of a gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning according to an embodiment of the present invention is shown;

[0050] Figure 2 A schematic flow chart of a FISH image segmentation method of a SAM model according to an embodiment of the present invention is shown;

[0051] Figure 3 A schematic diagram of a process of determining an overproduction violation based on big data analysis according to an embodiment of the present invention is shown;

[0052] Figure 4 A schematic diagram of a feedforward neural network of a ResNet deep residual network according to an embodiment of the present invention is shown;

[0053] Figure 5 A schematic diagram of a process of a fluorescent point determination and statistics method based on color filtering according to an embodiment of the present invention is shown;

[0054] Figure 6 A schematic diagram of the process of a method for intelligent recognition of gene amplification fluorescence in situ hybridization images based on deep learning according to an embodiment of the present invention is shown;

[0055] Figure 7 A schematic structural diagram of a computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0056] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0057] Figure 1 The structure block diagram of the gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning according to an embodiment of the present invention is shown, and the system includes: an image acquisition module 101, a SAM segmentation module 102, a ResNet comparison module 103 and a fluorescence point counting module 104; wherein,

[0058] The image acquisition module 101 is used to acquire images of cell detection samples labeled by FISH technology using a microscope to obtain corresponding initial images.

[0059] Preferably, a high-resolution fluorescence microscope is used to collect images of cell detection samples labeled by FISH (Fluorescence In Situ Hybridization) technology. By collecting images with a high-resolution fluorescence microscope, the labeled fluorescent spots can be displayed more clearly for subsequent identification and counting.

[0060] The SAM segmentation module 102 is used to perform image segmentation on the initial image based on the SAM model (Segment Anything Model), segment all cells displayed in the initial image, and determine a segmented image.

[0061] Specifically, the segmented image includes at least one quasi-cell image and a segmentation hint for each quasi-cell image.

[0062] Preferably, the segmentation prompt may be a prompt box.

[0063] The ResNet comparison module 103 is used to compare each quasi-cell image with a standard cell image to screen out a real cell image.

[0064] The fluorescent spot counting module 104 is used to identify and count fluorescent spots in each real cell image based on color filtering technology, and determine the number and ratio of target gene signals and their corresponding chromosome centromere signals under a preset number of tumor cells.

[0065] Preferably, the preset number of tumor cells may be 100.

[0066] Preferably, the number and ratio of the target gene signal and its corresponding chromosome centromere signal are used to determine whether the tumor cell has gene amplification or is polyploid. The specific ratio is determined according to the specific type of tumor and is not limited here.

[0067] The deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system provided in this embodiment includes: an image acquisition module, a SAM segmentation module, a ResNet comparison module and a fluorescence point counting module; wherein the image acquisition module is used to use a microscope to perform image acquisition on a cell detection sample labeled by FISH technology to obtain a corresponding initial image; the SAM segmentation module is used to perform image segmentation on the initial image based on the SAM model, segment all cells displayed in the initial image, and determine the segmented image, wherein the segmented image includes at least one quasi-cell image and a segmentation prompt for each quasi-cell image; the ResNet comparison module is used to compare each quasi-cell image with a standard cell image, and filter out a real cell image therefrom; the fluorescence point counting module is used to perform fluorescence point recognition and counting on each real cell image based on color filtering technology, and determine the number and ratio of target gene signals and their corresponding chromosome centromere signals under a preset number of tumor cells. The gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning provided by this embodiment automatically segments the collected FISH images through the SAM model. Based on its efficient, accurate and universal segmentation capabilities, the model training process is greatly reduced, the waste of manpower and time costs caused by manual recognition, and the reduction of recognition accuracy are avoided, and the processing efficiency of FISH images is greatly improved; based on the ResNet deep residual convolutional network, the obtained segmented image is re-judged using the standard cell image, and the misjudged cell image is eliminated, further improving the accuracy of the system; through the organic combination of SAM segmentation, ResNet comparison and color filtering, the overall performance of the recognition system is effectively improved; at the same time, the relevant data of the image after segmentation is annotated by the SAM model, further avoiding the extra workload caused by manual annotation, and being more adaptable to situations with large data volume or high annotation difficulty, so that medical personnel can better focus on diagnosis and treatment.

[0068] Figure 2 FIG. 4 is a flow chart showing a FISH image segmentation method of a SAM model according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0069] Step S201, using an image encoder to convert an initial image into an image embedding.

[0070] Preferably, the image encoder may select MAE (Masked Autoencoders) as a pre-trained visual converter to convert the initial image.

[0071] Step S202: combining the image embedding with the input mask after convolution processing and the prompt point and / or prompt box and / or prompt text to determine at least one mask area, and taking each mask area as a quasi-cell image.

[0072] Specifically, a prompt encoder is used to determine the position encoding of the prompt point and / or prompt box, and the position encoding is combined with the learned embedding of the corresponding prompt type to finally obtain the prompt point and / or prompt box; based on the text encoder, a preset free-format text is represented to obtain the prompt text.

[0073] This is done by aligning cell images.

[0074] Step S203, generating a segmented image after segmentation.

[0075] Specifically, the automatic segmentation mode of the SAM algorithm is used to adjust the density of the prompting dot matrix by setting hyperparameters;

[0076] The entire image is segmented based on the hint dot matrix in the image embedding, the cell structure in the initial image is determined, and a segmented image is obtained.

[0077] According to the above method, the FISH image is segmented using the SAM algorithm, making the image segmentation process very efficient, and by using prompt points, prompt boxes and prompt texts, the accuracy of the initial image segmentation is improved, and the cell image can be accurately segmented for subsequent analysis and counting. This greatly reduces the workload of relevant personnel and improves the overall processing efficiency.

[0078] Figure 3 FIG. 1 is a flow chart showing a method for determining overproduction violation based on big data analysis according to an embodiment of the present invention. Figure 3 As shown, the method comprises the following steps:

[0079] Step S301, using ResNet to process each quasi-cell image captured from the segmented image.

[0080] Preferably, when using ResNet (Residual Network) for image processing, it is divided into two branches after data input, wherein one branch passes through the convolutional network in ResNet to obtain the convolutional layer output and is connected to the output end, and the other branch is directly connected from the input to the output end, and the outputs of the two branches are added at the output end to form a feedforward neural network with "quick connection" to obtain the output after ResNet processing. Figure 4 As shown, Figure 4 A schematic diagram of a feedforward neural network of a ResNet deep residual network according to an embodiment of the present invention is shown.

[0081] Preferably, ResNet-50 is selected for the above image processing task.

[0082] Step S302 , determining whether a norm determined according to the quasi-cell image and the standard cell image is less than a preset threshold.

[0083] Specifically, a norm is calculated based on the processed quasi-cell image and the standard cell image processed by ResNet to determine whether the quasi-cell image is a real cell image;

[0084] If the calculation result of the norm is less than the preset threshold, step S303 is executed;

[0085] If the calculation result of the norm is not less than the preset threshold, step S304 is executed.

[0086] Step S303: determining that the corresponding quasi-cell image is a real cell image.

[0087] Step S304: determining that the corresponding quasi-cellular image is a non-cellular image.

[0088] According to the above method, based on the ResNet deep residual convolutional network, the obtained segmented image is re-judged using the standard cell image, which eliminates the misjudged cell images and further improves the accuracy of the system. In addition, the degradation problem of the deep convolutional network when the number of layers is deepened is solved by introducing the residual learning framework.

[0089] Figure 5 A schematic diagram of a process of a fluorescent point determination and statistics method based on color filtering according to an embodiment of the present invention is shown;

[0090] like Figure 5 As shown, the method comprises the following steps:

[0091] Step S501 : Take the intersection of each identified real cell image and the initial image to determine the target area for fluorescence point counting.

[0092] Specifically,

[0093] Step S502: Create a color filter for identifying fluorescent spots, and search for corresponding fluorescent spots in the target area.

[0094] Specifically, since in the cell detection sample labeled by FISH technology, the target gene signal is displayed in red and the chromosome centromere signal is displayed in green, the color filter is used to target the red fluorescent spot and the green fluorescent spot.

[0095] Preferably, a Hue-color comparison table is set based on the HSV color model, and Hue is divided into m intervals including at least red and green, and each interval corresponds to a color.

[0096] Each real cell image in the segmented image is converted into a corresponding HSV image, and each color region is identified from the HSV image corresponding to the real cell image using a color filter; wherein,

[0097] For real cell images, a HS histogram is established based on hue H and saturation S. The number of columns in the HS histogram is the number of intervals m divided by Hue.

[0098] Calculate the value HS(i) of the i-th column in the HS histogram corresponding to the real cell image

[0099] HS(i)=∑(i-1)·step≤H(x,y)<i·step·S(x,y)

[0100] step=360° / m

[0101] Where i∈[1,m]; step is the span of each interval divided by Hue; H(x,y) is the chromaticity value of the non-edge point (x,y) in the HSV image; S(x,y) is the saturation value of the non-edge point (x,y) in the HSV image;

[0102] Based on HS(i), the interval corresponding to the non-edge point is determined, and then the color of the non-edge point is determined; the original color of the non-edge point is used as the color of the area.

[0103] Red pixel regions and green pixel regions are obtained by screening out each color region, and the number of red pixel regions and green pixel regions is counted respectively.

[0104] Preferably, in order to make the project have a good visualization effect, after the model completes the segmentation and obtains the real cell image, each real cell image can be framed with a rectangular prompt box, and the number of red pixel areas and green pixel areas determined in each real cell image can be marked in a corner of the prompt box of the real cell image.

[0105] According to the above method, the red and green fluorescent points in the real cell image are accurately determined through color filtering, and the relevant data of the image after segmentation is annotated through the SAM model, which further avoids the extra workload caused by manual annotation and is more adaptable to situations with large amounts of data or high annotation difficulty, so that medical staff can better focus on diagnosis and treatment. In addition, by performing HSV conversion on the segmented real cell image and determining the color based on the value of the HS histogram corresponding to the pixel point, the saturation judgment dimension is added, which further improves the accuracy of determining the fluorescent points in gene amplification detection, thereby further improving the accuracy of gene amplification detection.

[0106] Figure 6 A schematic diagram of the process of a method for intelligent recognition of gene amplification fluorescence in situ hybridization images based on deep learning according to an embodiment of the present invention is shown;

[0107] like Figure 6 As shown, the method comprises the following steps:

[0108] Step S601: Use a microscope to collect images of cell detection samples labeled by FISH technology to obtain corresponding initial images.

[0109] Step S602: performing image segmentation on the initial image based on the SAM model to segment all cells displayed in the initial image and determine a segmented image.

[0110] Specifically, the segmented image includes at least one quasi-cell image and a segmentation hint for each quasi-cell image.

[0111] Preferably, the segmentation prompt may be a prompt box.

[0112] Specifically, the initial image is segmented based on the SAM model to segment all cells displayed in the initial image, and the segmented image is determined, further comprising:

[0113] Convert the initial image into an image embedding using an image encoder;

[0114] Combining the image embedding with the convolution-processed input mask and the prompt point and / or prompt box and / or prompt text to determine at least one mask region, and treating each mask region as a quasi-cell image;

[0115] Generate a segmented image after segmentation.

[0116] Preferably, a prompt encoder is used to determine the position encoding of the prompt point and / or the prompt box, and the position encoding is combined with the learning embedding of the corresponding prompt type to finally obtain the prompt point and / or the prompt box; a text encoder is used to represent the preset free-format text to obtain the prompt text; wherein,

[0117] The density of the prompt dot matrix is ​​adjusted by setting hyperparameters; the entire image is segmented based on the prompt dot matrix in the image embedding, the cell structure in the initial image is determined, and a segmented image is obtained.

[0118] Step S603: compare each quasi-cell image with a standard cell image to select a real cell image.

[0119] Specifically, each quasi-cell image is compared with a standard cell image to screen out a real cell image, further comprising:

[0120] ResNet is used to process each quasi-cell image captured after the segmentation image;

[0121] A norm is calculated based on the processed quasi-cell image and the standard cell image processed by ResNet to determine whether the quasi-cell image is a real cell image; wherein,

[0122] If the calculation result of the one-norm is less than the preset threshold, the corresponding quasi-cell image is determined to be a real cell image; if the calculation result of the one-norm is not less than the preset threshold, the corresponding quasi-cell image is determined to be a non-cell image.

[0123] Step S604, based on the color filtering technology, the fluorescent spots of each real cell image are identified and counted to determine the number and ratio of the target gene signal and its corresponding chromosome centromere signal under the preset number of tumor cells.

[0124] Specifically, the fluorescent spot recognition and counting of each real cell image is performed based on the color filtering technology, further including:

[0125] Take the intersection of each identified real cell image and the initial image to determine the target area for fluorescent spot counting;

[0126] Create a color filter for identifying fluorescent spots, and search for corresponding fluorescent spots in the target area;

[0127] Since the target gene signal is displayed in red and the chromosome centromere signal is displayed in green in the cell detection sample labeled by FISH technology, the color filter is used to target the red fluorescent spot and the green fluorescent spot.

[0128] Preferably, a Hue-color comparison table is set based on the HSV color model, and Hue is divided into m intervals including at least red and green, each interval corresponding to one color;

[0129] Convert each real cell image in the segmented image into a corresponding HSV image, and use a color filter to identify each color area from the HSV image corresponding to the real cell image;

[0130] Red pixel regions and green pixel regions are obtained by screening out each color region, and the number of red pixel regions and green pixel regions is counted respectively.

[0131] Among them, for real cell images, based on hue H and saturation S, a HS histogram is established; the number of columns in the HS histogram is the number of intervals m divided by Hue;

[0132] Calculate the value HS(i) of the i-th column in the HS histogram corresponding to the real cell image

[0133] HS(i)=∑(i-1)·step≤H(x,y)<i·step·S(x,y)

[0134] step=360° / m

[0135] Where i∈[1,m]; step is the span of each interval divided by Hue; H(x,y) is the chromaticity value of the non-edge point (x,y) in the HSV image; S(x,y) is the saturation value of the non-edge point (x,y) in the HSV image;

[0136] Based on HS(i), the interval corresponding to the non-edge point is determined, and then the color of the non-edge point is determined.

[0137] According to the deep learning-based intelligent recognition method for gene amplification fluorescence in situ hybridization images provided in this embodiment, a microscope is used to collect images of cell detection samples labeled with FISH technology to obtain a corresponding initial image; image segmentation is performed on the initial image based on the SAM model to segment all cells displayed in the initial image to determine a segmented image, wherein the segmented image includes at least one quasi-cell image and segmentation prompts for each quasi-cell image; each quasi-cell image is compared with a standard cell image to screen out a real cell image; fluorescent spots are identified and counted for each real cell image based on color filtering technology to determine the number and ratio of target gene signals and their corresponding chromosome centromere signals under a preset number of tumor cells. Through the deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition method provided in this embodiment, the collected FISH images are automatically segmented through the SAM model. Based on its efficient, accurate and universal segmentation capabilities, the model training process is greatly reduced, the waste of manpower and time costs caused by manual recognition, and the reduction of recognition accuracy are avoided, and the processing efficiency of FISH images is greatly improved. In the segmentation process, the segmentation efficiency and accuracy are further improved based on the prompt dot matrix, and the flexibility of the segmentation process is improved; based on the ResNet deep residual convolutional network, the obtained segmented image is re-judged using the standard cell image, and the misjudged cell image is excluded, further improving the accuracy of the system; through the organic combination of SAM segmentation, ResNet comparison and color filtering, the overall performance of the recognition system is effectively improved, so that it has better scalability, adaptability and robustness; at the same time, the SAM model is used to annotate the relevant data of the image after the segmentation is completed, further avoiding the extra workload caused by manual annotation, and it can better adapt to the situation of large data volume or high annotation difficulty, so that medical staff can better focus on diagnosis and treatment. In addition, by performing HSV conversion on the segmented real cell image and determining the color based on the numerical value of the HS histogram corresponding to the pixel point, the saturation judgment dimension is added, which further improves the accuracy of determining the fluorescent point in gene amplification detection, thereby further improving the accuracy of gene amplification detection.

[0138] The present invention also provides a non-volatile computer storage medium, which stores at least one executable instruction, and the executable instruction can execute the operations that can be achieved by the deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system in any of the above-mentioned system embodiments.

[0139] Figure 7 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0140] like Figure 7 As shown, the computing device may include: a processor (processor) 702 , a communications interface (Communications Interface) 704 , a memory (memory) 706 , and a communication bus 708 .

[0141] in:

[0142] The processor 702 , the communication interface 704 , and the memory 706 communicate with each other via a communication bus 708 .

[0143] The communication interface 704 is used to communicate with other devices such as clients or other servers.

[0144] The processor 702 is used to execute the program 710, and specifically can execute the operations that can be achieved by the gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning in the above-mentioned gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning embodiment.

[0145] Specifically, the program 710 may include program code, and the program code includes computer operation instructions.

[0146] The processor 702 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0147] The memory 706 is used to store the program 710. The memory 706 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0148] Program 710 can be specifically used to enable processor 702 to perform operations that can be implemented by the deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system in any of the above-mentioned system embodiments. The specific implementation of each step in program 710 can refer to the corresponding descriptions in the corresponding steps and units in the above-mentioned deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described equipment and modules can refer to the corresponding process description in the aforementioned method embodiment, which will not be repeated here.

[0149] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0150] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0151] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0152] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0153] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0154] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0155] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. Gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning, including: Image acquisition module, SAM segmentation module, ResNet comparison module and fluorescence point counting module; among them, The image acquisition module is used to acquire images of the cell detection samples labeled by FISH technology using a microscope to obtain the corresponding initial images; The SAM segmentation module is used to perform image segmentation on the initial image based on the SAM model, segment all cells displayed in the initial image, and determine a segmented image, wherein the segmented image includes at least one quasi-cell image and a segmentation prompt for each quasi-cell image; The ResNet comparison module is used to compare each quasi-cell image with a standard cell image to screen out a real cell image; The fluorescent spot counting module is used to identify and count fluorescent spots in each real cell image based on color filtering technology, and determine the number and ratio of target gene signals and their corresponding chromosome centromere signals under a preset number of tumor cells.

2. The deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system according to claim 1 is characterized in that: The SAM segmentation module is further used for: Convert the initial image into an image embedding using an image encoder; Combining the image embedding with the convolution-processed input mask and the prompt point and / or prompt box and / or prompt text to determine at least one mask region, and treating each mask region as a quasi-cell image; Generate a segmented image after segmentation.

3. The gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning according to claim 2 is characterized in that: The SAM segmentation module is further used for: Using the prompt encoder, determine the position encoding of the prompt point and / or prompt box, and combine the position encoding with the learning embedding of the corresponding prompt type, and finally obtain the prompt point and / or prompt box; based on the text encoder, represent the preset free-format text to obtain the prompt text; wherein, The density of the prompt dot matrix is ​​adjusted by setting hyperparameters; the entire image is segmented based on the prompt dot matrix in the image embedding, the cell structure in the initial image is determined, and a segmented image is obtained.

4. The gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning according to claim 1 is characterized in that: The ResNet comparison module is further used for: ResNet is used to process each quasi-cell image captured after the segmentation image; A norm is calculated based on the processed quasi-cell image and the standard cell image processed by ResNet to determine whether the quasi-cell image is a real cell image; wherein, If the calculation result of the one-norm is less than the preset threshold, the corresponding quasi-cell image is determined to be a real cell image; if the calculation result of the one-norm is not less than the preset threshold, the corresponding quasi-cell image is determined to be a non-cell image.

5. The gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning according to claim 1 is characterized in that: The fluorescent point counting module is further used for: Take the intersection of each identified real cell image and the initial image to determine the target area for fluorescent spot counting; Create a color filter for identifying fluorescent spots, and search for corresponding fluorescent spots in the target area; Since the target gene signal is displayed in red and the chromosome centromere signal is displayed in green in the cell detection sample labeled by FISH technology, the color filter is used to target the red fluorescent spot and the green fluorescent spot.

6. The gene amplification fluorescence in situ hybridization image intelligent recognition system based on deep learning according to claim 1 is characterized in that: The fluorescent point counting module is further used for: Set up a Hue-color comparison table based on the HSV color model, and divide Hue into m intervals containing at least red and green, each interval corresponding to one color; Convert each real cell image in the segmented image into a corresponding HSV image, and use a color filter to identify each color region from the HSV image corresponding to the real cell image; Red pixel regions and green pixel regions are obtained by screening out each color region, and the number of red pixel regions and green pixel regions is counted respectively.

7. The deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system according to claim 6, characterized in that: The fluorescent point counting module is further used for: For real cell images, a HS histogram is established based on hue H and saturation S. The number of columns in the HS histogram is the number of intervals m divided by Hue. Calculate the value HS(i) of the i-th column in the HS histogram corresponding to the real cell image HS(i)=∑(i-1)·step≤H(x,y) <i·step·S(x,y) step=360° / m Where, i∈[1,m]; step is the span of each interval divided by Hue; H(x,y) is the chromaticity value of the non-edge point (x,y) in the HSV image; S(x,y) is the saturation value of the non-edge point (x,y) in the HSV image; Based on HS(i), the interval corresponding to the non-edge point is determined, and then the color of the non-edge point is determined.

8. Intelligent recognition method of gene amplification fluorescence in situ hybridization images based on deep learning, including: Use a microscope to collect images of cell detection samples labeled by FISH technology to obtain corresponding initial images; Performing image segmentation on the initial image based on the SAM model to segment all cells displayed in the initial image and determine a segmented image, wherein the segmented image includes at least one quasi-cell image and a segmentation hint for each quasi-cell image; Each quasi-cell image is compared with a standard cell image, and the real cell image is screened out; Based on color filtering technology, fluorescent spots are identified and counted in each real cell image to determine the number and ratio of target gene signals and their corresponding chromosome centromere signals under a preset number of tumor cells.

9. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operations implemented by the deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system as described in any one of claims 1 to 7.

10. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute the operations implemented by the deep learning-based gene amplification fluorescence in situ hybridization image intelligent recognition system as described in any one of claims 1 to 7.

Images