A method and system for fusion of fluorescence features in fluorescence in situ hybridization images

Abstract

The present invention relates to a method and system for fusing fluorescence features in a fluorescence in situ hybridization image. The method comprises acquiring fluorescence in situ hybridization images under different filters; storing the fluorescence in situ hybridization images under different filters and reading them into a Data_list linked list of the Type type; locating the cell position of the fluorescence in situ hybridization image under the image type identification of a white light filter; and using the located image as a cell template; matching the cell template with the fluorescence in situ hybridization image under the image type identification of a green light filter, a blue light filter, and a red light filter to complete the adaptive shape marking of cells in the target image; extracting and fusing the fluorescence features of the labeled fluorescence in situ hybridization images under the green light filter, the blue light filter, and the red light filter, respectively. The present invention can improve the accuracy of detection and enhance the accuracy of judging complex chromosome abnormalities.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for fusing fluorescence features in fluorescence in situ hybridization images. Background Art

[0002] Fluorescence in situ hybridization is a non-radioactive cytogenetic technique that uses fluorescent labeling instead of isotope labeling. On the basis of in situ hybridization, the RNA or DNA probe labeled with hapten is complementary paired with nucleic acid bases, and the labeled RNA or DNA is recognized by fluorescent labeled antibodies, or the probe is directly labeled with fluorescent substances and combined with the target sequence. Without changing the original morphology of biological cells (cytoplasm, organelles, nucleus) or chromosomes, the labeled RNA or DNA probe is applied to the target to locate and detect specific target nucleic acid sequences. Because of its high specificity and ability to achieve precise positioning, it is widely used in biology, medicine, genetics and other fields.

[0003] Fluorescence in situ hybridization technology is used in the detection of tumor tissue sections or peripheral blood circulating tumor cells. Due to the large number of cells contained in the fluorescence in situ hybridization image, the cells are adhered and difficult to distinguish, and the cell nucleus is unevenly distributed. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for fusing fluorescence features in fluorescence in situ hybridization images, which can improve the accuracy of detection and enhance the accuracy of judging complex chromosome abnormalities.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for fusing fluorescence features in a fluorescence in situ hybridization image, comprising:

[0007] Obtain fluorescence in situ hybridization images under different filters; the different filters include: white light filter, green light filter, blue light filter and red light filter;

[0008] The fluorescence in situ hybridization images under different filters are stored and read into the Data_list linked list of Type type; the Data_list linked list of Type type includes: MatWhite, MatBlue, MatGreen, MatRed and Temp; the Temp is the image type identifier;

[0009] The cell position is located in the fluorescence in situ hybridization image under the white light filter, and the located image is used as a cell template;

[0010] The cell template is used to match the fluorescence in situ hybridization images with the image type identifications of green light filter, blue light filter and red light filter to complete the adaptive shape marking of the cells in the target image;

[0011] Fluorescence feature extraction is performed on the labeled fluorescence in situ hybridization images under the green light filter, blue light filter and red light filter respectively;

[0012] The extracted fluorescence features are fused.

[0013] Optionally, the storing of fluorescence in situ hybridization images under different filters and reading them into a Data_list linked list of the Type type specifically includes:

[0014] Grayscale processing of fluorescence in situ hybridization images under different filters;

[0015] The Data_file file is used to store the fluorescence in situ hybridization images under different filters after grayscale processing.

[0016] Optionally, the step of extracting fluorescence features from the labeled fluorescence in situ hybridization images under the green filter, the blue filter, and the red filter respectively specifically includes:

[0017] The color channel separation method was used to extract fluorescence features of the labeled fluorescence in situ hybridization images under the green filter, blue filter and red filter.

[0018] Optionally, the step of fusing the extracted fluorescence features specifically includes:

[0019] The extracted fluorescence features are fused using multi-channel image fusion method.

[0020] A fluorescence feature fusion system in a fluorescence in situ hybridization image, comprising:

[0021] A fluorescence in situ hybridization image acquisition module is used to acquire fluorescence in situ hybridization images under different filters; the different filters include: white light filter, green light filter, blue light filter and red light filter;

[0022] The image storage and reading module is used to store the fluorescence in situ hybridization images under different filters and read them into the Data_list linked list of Type type; the Data_list linked list of Type type includes: MatWhite, MatBlue, MatGreen, MatRed and Temp; the Temp is the image type identifier;

[0023] A cell template determination module is used to locate the cell position of the fluorescence in situ hybridization image under the white light filter as the image type; and use the located image as the cell template;

[0024] An adaptive shape marking module is used to match the cell template with the fluorescence in situ hybridization image under the image type identification of the green filter, the blue filter and the red filter to complete the adaptive shape marking of the cells in the target image;

[0025] A fluorescence feature extraction module is used to extract fluorescence features of the fluorescence in situ hybridization images under the marked green light filter, blue light filter and red light filter respectively;

[0026] The feature fusion module is used to fuse the extracted fluorescence features.

[0027] Optionally, the image storage and reading module specifically includes:

[0028] An image preprocessing unit, used for grayscale processing of fluorescence in situ hybridization images under different filters;

[0029] The image storage unit is used to store the fluorescence in situ hybridization images under different filters after grayscale processing using the Data_file file.

[0030] Optionally, the fluorescence feature extraction module specifically includes:

[0031] The fluorescence feature extraction unit is used to extract fluorescence features of the fluorescence in situ hybridization images under the marked green light filter, blue light filter and red light filter respectively by using the method of separating color channels.

[0032] Optionally, the feature fusion module specifically includes:

[0033] The feature fusion unit is used to fuse the extracted fluorescence features using a multi-channel image fusion method.

[0034] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0035] The present invention provides a method and system for fusion of fluorescence features in fluorescence in situ hybridization images. The method uses a Data_list linked list of the Type type to complete the storage and classification of four filter images, matches images with different image type identifiers with cell templates, completes the adaptive shape marking of cells in the target image, and then extracts and fuses fluorescence features of the labeled green filter, blue filter, and red filter fluorescence in situ hybridization images respectively; performs in situ hybridization on probes labeled with multiple colors of fluorescent markers alone or in combination, quickly processes and analyzes cell images, and detects whether there are abnormalities in the cell shape and chromosome sites in the cell. The accuracy of detection is improved, and the accuracy of judgment of complex chromosome abnormalities is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 A schematic diagram of the process flow of a method for fusing fluorescence features in fluorescence in situ hybridization images provided by the present invention;

[0038] Figure 2 It is a schematic diagram of the steps of the present invention;

[0039] Figure 3 Schematic diagram of the Fish image cell position completion positioning of the present invention; (a) is a white light template image, (b) is a cell positioning image;

[0040] Figure 4 Schematic diagram of the Fish image template matching adaptive shape labeling of the present invention; (a) is a blue light image, (b) is a blue light matching adaptive shape labeling diagram, (c) is a red light image, (d) is a red light matching adaptive shape labeling diagram, (e) is a green light image, and (f) is a green light matching adaptive shape labeling;

[0041] Figure 5 Schematic diagram of fluorescence characteristics of the present invention; (a) is a blue light fluorescence characteristic image, (b) is a red light fluorescence characteristic image, (c) is a green fluorescence characteristic image; (d) is a white fluorescence characteristic image, wherein the white bright spot portion represents the detected fluorescence marking point, and the black portion is the background image;

[0042] Figure 6 It is a schematic diagram of fluorescence feature fusion of the present invention;

[0043] Figure 7This is a schematic diagram of the structure of a fluorescence feature fusion system in a fluorescence in situ hybridization image provided by the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The purpose of the present invention is to provide a method and system for fusing fluorescence features in fluorescence in situ hybridization images, which can improve the accuracy of detection and enhance the accuracy of judging complex chromosome abnormalities.

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] Figure 1 A schematic diagram of the process flow of a method for fusing fluorescence features in fluorescence in situ hybridization images provided by the present invention; Figure 2 It is a schematic diagram of the steps of the present invention, such as Figure 1 and Figure 2 As shown, the present invention provides a method for fusing fluorescence features in a fluorescence in situ hybridization image, comprising:

[0048] S101, acquiring fluorescence in situ hybridization images under different filters; the different filters include: white light filter, green light filter, blue light filter and red light filter; using an Olympus microscope to acquire images;

[0049] S102, storing the fluorescence in situ hybridization images under different filters and reading them into a Data_list linked list of Type type; the Data_list linked list of Type type includes: MatWhite, MatBlue, MatGreen, MatRed and Temp; the Temp is an image type identifier;

[0050] S102 specifically includes:

[0051] Grayscale processing of fluorescence in situ hybridization images under different filters;

[0052] The Data_file file is used to store the fluorescence in situ hybridization images under different filters after grayscale processing.

[0053] S103, locating the cell position of the fluorescence in situ hybridization image under the white light filter with the image type marked; and using the located image as a cell template, and Figure 3 As shown;

[0054] S103 specifically includes:

[0055] Step 1: Define an object of the DataRow class in the System.Data namespace, represented by fData; define a List class object of the Type type in the System.Collections.Generic namespace, represented by Data_list; define a DataTable class object in the System.Data namespace, represented by Data_file.

[0056] Step 2: The SData object calls the NewRow function to create a new data row object, represented by dr, and calls the Substring function in a loop, assigning the return value of the Substring function to dr. Determine whether the expression value of the loop process is True. If so, execute step 3, otherwise continue to execute step 2;

[0057] Step 3: The Data_file object calls the Rows.Add function to loop the data in the SData object to the Data_file object and determine whether the expression value of the loop process is True. If so, execute step 4, otherwise continue to execute step 3;

[0058] Step 4: The Data_file object calls the Foreach function to retrieve the data in the data container, and assigns the return value of the Foreach function to Data_list and determines whether the expression value of the loop process is False. If so, execute step 5, otherwise continue to execute step 4;

[0059] Step 5: Assign the data in Data_list to MatWhite, MatBlue, MatGreen, MatRed, and Temp data members respectively, and assign the values ​​of Temp data member in Type to 1, 2, 3, and 4. Determine whether the expression value of the loop process is False. If so, execute step 6, otherwise continue to execute step 5.

[0060] Step 6: Define five objects of the Mat class in the Emgu.CV namespace, represented by Redmat, Greenmat, Bluemat, Imagefile, and Whitemat. Whitemat calls the Mat instantiated object with the parameter MatWhite and the ImreadModes.AnyColor flag, which is assigned to Whitemat. Define the Image file in the Emgu.CV namespace.<Bgr,byte> An object of the class is defined, represented by _image. Whitemat calls the ToImage class to mirror the image. Define an object of the Mat class in the Emgu.CV namespace, represented by src, and assign _image to src. Call the Threshold function under CvInvoke to perform binarization. The parameters are Whitemat: original image, Whitemat: result image, 50: threshold, 255: maximum value, and ThresholdType.Binary: binary thresholding.

[0061] Step 7: Call the CvtColor function under CvInvoke to convert the color space, the parameters are Whitemat: original image, Whitemat: result image, 50: threshold, 180: maximum value, ColorConversion.BayerBg2Gray: conversion to grayscale. Call the FindContours function under CvInvoke to find contours, the parameters are Whitemat: gray image, contours: result image, null: output phase, Emgu.CV.CvEnum.RetrType.Tree: contour type, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxNone: point connection mode, assign Whitemat to MatWhite, and get the image as a cell template.

[0062] S104, using the cell template to match the fluorescence in situ hybridization images with image type identification as green light filter, blue light filter and red light filter, to complete the adaptive shape marking of the cells in the target image; the adaptive shape marking of the cells in the target image is as follows: Figure 4 shown.

[0063] S104 specifically includes:

[0064] Step 1: Take the green light as an example: define an object of the Mat class in the Emgu.CV namespace, represented by Imagefile, Imagefile calls the Mat instantiated object, the parameter is Greenmat, the ImreadModes.AnyColor identifier, define the Image in the Emgu.CV namespace<Bgr,byte> An object of the class is defined, represented by _image. Imagefile calls the ToImage class to mirror the image. Two objects of the Mat class in the Emgu.CV namespace are defined, represented by src and mask. _image is assigned to src. src calls the Copy method under the ToImage class to create an image with the same size as the original image and assigns it to mask. An object of the VectorOfVectorOfPoint class in the Emgu.CV.Util namespace is defined, represented by contours. The DrawContours function under CvInvoke is called with the following parameters: mask: target image, Contours: contour group, -1: all contours, newMCvScalar(0,255,0): color, 1: contour line.

[0065] Step 2: Define the Image in the Emgu.CV namespace<Bgr,byte> 3 objects of the class are represented by _image1 and _image2. MatWhite calls the ToImage class to mirror the image and assigns it to _image1. Mask calls the ToImage class to mirror the image and assigns it to _image2. 2 objects of the Bitmap class in the System.Drawing namespace are defined, represented by _bitmap and _bitmap1. _image1 is assigned to _bitmap and _image1 is assigned to _image2. 4 objects of the Int class in the System namespace are defined, represented by width, height, stride, and offset. The width and height of the _bitmap object are obtained and assigned to width and height respectively.

[0066] Step 3: Define an object of the BitmapData class in the System.Drawing.Imaging namespace, represented by data. _bitmap calls the LockBits function with parameters such as image width, height, access method (ImageLockMode.ReadWrite), and data format (PixelFormat.Format24bppRgb). Define an object of the IntPtr class in the System namespace, represented by Scan0. _bitmap calls the Scan0 function to obtain the memory starting position of bmpData, that is, the address of the first pixel data. _bitmap calls the Stride function to obtain the scan width and assign it to stride.

[0067] Step 4: Define a pointer p of byte type, get the pixel address of Scan0 data, and assign it to p. Get the gap between the display width and the scan line width, assign it to offset, call the For function in a loop, and traverse the length and width of the image. Call the if function to determine whether parameters 1, 2, and 3 are 0, where parameter 1 is R in the RGB component of the image pixel, parameter 2 is G in the RGB component of the image pixel, and parameter 2 is B in the RGB component of the image pixel. Call the SetPixel function to set the pixel color of the current color component, remove other noise points, and assign the result to _bitmap2, and shift the pointer p left by three bits. Complete the traversal and complete the cell image matching.

[0068] Step 5: _bitmap calls the UnlockBits function to unlock the data object in this memory, and _bitmap2 calls the Save function to save the current matching image. Instantiate the global MatGreen variable, convert _bitmap2 to a grayscale image, and assign it to mat. The template shape completes the adaptive shape marking of the cells in the target image. The operations of red light and blue light are the same, so I will not repeat them one by one here.

[0069] S105, extracting fluorescence features of the labeled fluorescence in situ hybridization images under the green filter, the blue filter, and the red filter respectively;

[0070] S105 specifically includes:

[0071] The color channel separation method is used to extract fluorescence features from the labeled green filter, blue filter, and red filter fluorescence in situ hybridization images. Figure 5 shown.

[0072] S106, performing feature fusion on the extracted fluorescence features.

[0073] S106 specifically includes:

[0074] The extracted fluorescence features are fused using the multi-channel image fusion method. Figure 6 shown.

[0075] The specific process of feature extraction and fusion is as follows:

[0076] Step 1: Taking the green light as an example: define 5 objects of the Mat class in the Emgu.CV namespace, represented by currentImage, result, resultscr, mattemp, and matfinal, define 2 objects of the VectorOfVectorOfPoint class in the Emgu.CV.Util namespace, represented by contours and contours1, and define 1 object of the VectorOfPoint class in the Emgu.CV.Util namespace, represented by vp.

[0077] Step 2: Call the Threshold function under CvInvoke to perform binarization processing, the parameters are MatGreen: original image, MatGreen: result image, 120: threshold, 255: maximum value, ThresholdType.Binary: binary thresholding. Call the FindContours function under CvInvoke to find contours, the parameters are MatGreen: gray image,

[0078] contours: saves the index numbers of the next contour, the previous contour, the parent contour, and the embedded contour. RetrType.Ccomp: monitors all contours. ChainApproxMethod.ChainApproxNone: none saves all continuous contours on the boundary into the contours vector.

[0079] Step 3: Call the For function in a loop to traverse contours, assign each group of contours to vp, define a double type object in the System.Collections.Generic namespace, represented by area, and call the ContourArea function under CvInvoke to calculate the area. Define a List class object of double type in the System.Collections.Generic namespace, represented by sarea, sarea calls the Add function to store the area, calls the if function to determine the current area, and when the current area is less than 60, contours1 calls the push function to save the current contour.

[0080] Step 4: Call Mat to instantiate the object and convert the image format. The parameters are MatGreen, Emgu.CV.CvEnum.DepthType.Cv8U: 8-bit unsigned integer, 3: depth. And assign it to result. Call the DrawContours function under CvInvoke. The parameters are result: target image, contours1: contour group, -1: all contours, newMCvScalar(0,255,0): color, 1: contour line.

[0081] Step 5: Call Mat to instantiate the object, the parameters are result, Emgu.CV.CvEnum.ImreadModes.Grayscale: grayscale, assign it to resultscr. Call CvtColor function under CvInvoke to convert the color space, the parameters are resultscr: original image, resultscr: result image, ColorConversion.BayerBg2Gray: convert to grayscale. Call Mat to instantiate the object, the parameters are result size, Emgu.CV.CvEnum.DepthType.Cv8U: 8-bit unsigned integer, 1: depth, assign it to mattemp. Create other blank channels. Call push function under Channelss, the first parameter is mattemp. Call push function under Channelss, the second parameter is resultscr, call push function under Channelss, the third parameter is mattemp, fill 3 channels. Call the Merge function under CvInvoke to perform channel fusion processing on the image. The parameters are Channelss: original image, matfinal: result image, to complete the extraction of fluorescence features under green light cells. The operations for red and blue light are the same, so they will not be repeated here.

[0082] Step 6: Define an object of the Mat class in the Emgu.CV namespace, expressed in RGB, and call the AddWeighted function under CvInvoke to fuse the images. The parameters are MatWhite: Image 1, 0.5: weight value, MatGreen: Image 2, 0.5: weight value, 0: gamma correction, RGB: output image. Call the AddWeighted function under CvInvoke to fuse the images. The parameters are RGB: Image 1, 0.5: weight value, MatBlue: Image 2, 0.5: weight value, 0: gamma correction, RGB: output image. Call the AddWeighted function under CvInvoke to fuse the images. The parameters are RGB: Image 1, 0.5: weight value, MatRed: Image 2, 0.5: weight value, 0: gamma correction, RGB: output image to complete the fusion of the fluorescence features under the green light cells.

[0083] Figure 7 This is a schematic diagram of the structure of a fluorescence feature fusion system in a fluorescence in situ hybridization image provided by the present invention. Figure 7 As shown, the present invention provides a fluorescence feature fusion system in a fluorescence in situ hybridization image, comprising:

[0084] The fluorescence in situ hybridization image acquisition module 701 is used to acquire fluorescence in situ hybridization images under different filters; the different filters include: white light filter, green light filter, blue light filter and red light filter;

[0085] The image storage and reading module 702 is used to store the fluorescence in situ hybridization images under different filters and read them into the Data_list linked list of the Type type; the Data_list linked list of the Type type includes: MatWhite, MatBlue, MatGreen, MatRed and Temp; the Temp is the image type identifier;

[0086] The cell template determination module 703 is used to locate the cell position of the fluorescence in situ hybridization image under the white light filter as the image type; and use the located image as the cell template;

[0087] The adaptive shape marking module 704 is used to match the cell template with the fluorescence in situ hybridization images with the image type identification of green filter, blue filter and red filter to complete the adaptive shape marking of cells in the target image;

[0088] The fluorescence feature extraction module 705 is used to extract fluorescence features of the labeled fluorescence in situ hybridization images under the green filter, the blue filter and the red filter respectively;

[0089] The feature fusion module 706 is used to perform feature fusion on the extracted fluorescence features.

[0090] The image storage and reading module 702 specifically includes:

[0091] An image preprocessing unit, used for grayscale processing of fluorescence in situ hybridization images under different filters;

[0092] The image storage unit is used to store the fluorescence in situ hybridization images under different filters after grayscale processing using the Data_file file.

[0093] The fluorescence feature extraction module 705 specifically includes:

[0094] The fluorescence feature extraction unit is used to extract fluorescence features of the fluorescence in situ hybridization images under the marked green light filter, blue light filter and red light filter respectively by using the method of separating color channels.

[0095] The feature fusion module 706 specifically includes:

[0096] The feature fusion unit is used to fuse the extracted fluorescence features using a multi-channel image fusion method.

[0097] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0098] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for fusion of fluorescence features in fluorescence in situ hybridization images. It is characterized in that include: Acquire fluorescence in situ hybridization images under different filters; Different filters include: white light filter, green light filter, blue light filter and red light filter; The fluorescence in situ hybridization images under different filters are stored and read into the Data_list linked list of Type type; the Data_list linked list of Type type includes: MatWhite, MatBlue, MatGreen, MatRed and Temp; the Temp is the image type identifier; The cell position is located in the fluorescence in situ hybridization image under the white light filter, and the located image is used as a cell template; The cell template is used to match the fluorescence in situ hybridization images with the image type identifications of green light filter, blue light filter and red light filter to complete the adaptive shape marking of the cells in the target image; Fluorescence feature extraction is performed on the labeled fluorescence in situ hybridization images under the green light filter, blue light filter and red light filter respectively; Perform feature fusion on the extracted fluorescence features; The specific process of feature extraction and fusion is as follows: Step 1: Taking green light as an example: define five objects of the Mat class in the Emgu. CV namespace, represented by currentImage, result, resultscr, mattemp, and matfinal; define two objects of the VectorOfVectorOfPoint class in the Emgu.CV.Util namespace, represented by contours and contours1; define one object of the VectorOfPoint class in the Emgu.CV.Util namespace, represented by vp; Step 2: Call the Threshold function under CvInvoke for binarization, the parameters are MatGreen: original image, MatGreen: result image, 120: threshold, 255: maximum value, ThresholdType.Binary: binary thresholding; call the FindContours function under CvInvoke for contour search, the parameters are MatGreen: gray image, contours: saves the index numbers of the next contour, the previous contour, the parent contour, and the embedded contour. RetrType.Ccomp: monitors all contours. ChainApproxMethod.ChainApproxNone: none saves all continuous contours on the boundary into the contours vector. Step 3: Call the For function in a loop to traverse contours, assign each group of contours to vp, define a double type object in the System.Collections.Generic namespace, represented by area, and call the ContourArea function under CvInvoke to calculate the area; define a double type List class object in the System.Collections.Generic namespace, represented by sarea, sarea calls the Add function to store the area, and calls the if function to determine the current area. When the current area is less than 60, contours1 calls the push function to save the current contour; Step 4: Call Mat to instantiate the object and convert the image format. The parameters are MatGreen, Emgu.CV.CvEnum.DepthType.Cv8U: 8-bit unsigned integer, 3: depth; and assign it to result; call the DrawContours function under CvInvoke, the parameters are result: target image, contours1: contour group, -1: all contours, newMCvScalar(0, 255, 0): color, 1: contour line; Step 5: Call Mat to instantiate the object, the parameter is result, Emgu.CV.CvEnum.ImreadModes.Grayscale: grayscale, assign it to resultscr; call CvtColor function under CvInvoke to convert the color space, the parameters are resultscr: original image, resultscr: result image, ColorConversion.BayerBg2Gray: convert to grayscale; call Mat to instantiate the object, the parameter is result size, Emgu.CV.CvEnum.DepthType.Cv8U: 8-bit unsigned integer, 1: Depth, assign it to mattemp; create other blank channels; call the push function under Channelss, the first parameter is mattemp; call the push function under Channelss, the second parameter is resultscr, call the push function under Channelss, the third parameter is mattemp, fill the 3 channels; call the Merge function under CvInvoke to perform channel fusion processing on the image, the parameters are Channelss: original image, matfinal: result image, complete the extraction of fluorescence features under green light cells, the same operation for red light and blue light; Step 6: Define an object of the Mat class in the Emgu. CV namespace, expressed in RGB, and call the AddWeighted function under CvInvoke for image fusion, with the parameters MatWhite: image 1, 0.5: weight value, MatGreen: image 2, 0.5: weight value, 0: gamma correction, RGB: output image; call the AddWeighted function under CvInvoke for image fusion, with the parameters RGB: image 1, 0.5: weight value, MatBlue: image 2, 0.5: weight value, 0: gamma correction, RGB: output image; call the AddWeighted function under CvInvoke for image fusion, with the parameters RGB: image 1, 0.5: weight value, MatRed: image 2, 0.5: weight value, 0: gamma correction, RGB: output image, to complete the fusion of the fluorescence features under the green light cells; The storage of fluorescence in situ hybridization images under different filters and reading into the Data_list linked list of the Type type specifically includes: Grayscale processing of fluorescence in situ hybridization images under different filters; Data_file is used to store the fluorescence in situ hybridization images under different filters after grayscale processing; The cell position is located in the fluorescence in situ hybridization image under the white light filter with the image type marked; and the located image is used as a cell template, specifically including: Step 11: Define an object of the DataRow class in the System.Data namespace, represented by fData; define a List class object of the Type type in the System.Collections.Generic namespace, represented by Data_list; define a DataTable class object in the System.Data namespace, represented by Data_file; Step 12: The SData object calls the NewRow function to create a new data row object, represented by dr, and calls the Substring function in a loop, assigning the return value of the Substring function to dr. Determine whether the expression value of the loop process is True. If so, execute step 13, otherwise continue to execute step 12; Step 13: The Data_file object calls the Rows.Add function to loop the data in the SData object to the Data_file object and determine whether the expression value of the loop process is True. If so, execute step 14, otherwise continue to execute step 13; Step 14: The Data_file object calls the Foreach function to retrieve the data in the data container, and assigns the return value of the Foreach function to Data_list and determines whether the expression value of the loop process is False. If so, execute step 15, otherwise continue to execute step 14; Step 15: Assign the data in Data_list to MatWhite, MatBlue, MatGreen, MatRed, and Temp data members respectively, and assign the values ​​of Temp data member in Type to 1, 2, 3, and 4. Determine whether the expression value of the loop process is False. If so, execute step 16, otherwise continue to execute step 15; Step 16: Define five objects of the Mat class in the Emgu. CV namespace, represented by Redmat, Greenmat, Bluemat, Imagefile, and Whitemat. Whitemat calls the Mat instantiated object with the parameter MatWhite and the ImreadModes.AnyColor flag, which is assigned to Whitemat. Define the Image file in the Emgu. CV namespace.<Bgr, byte> An object of the class is defined, represented by _image. Whitemat calls the ToImage class to mirror the image. An object of the Mat class in the Emgu. CV namespace is defined, represented by src. _image is assigned to src. The Threshold function under CvInvoke is called for binarization. The parameters are Whitemat: original image, Whitemat: result image, 50: threshold, 255: maximum value, ThresholdType.Binary: binary thresholding. Step 17: Call the CvtColor function under CvInvoke to convert the color space, the parameters are Whitemat: original image, Whitemat: result image, 50: threshold, 180: maximum value, ColorConversion.BayerBg2Gray: conversion to grayscale; call the FindContours function under CvInvoke to find contours, the parameters are Whitemat: gray image, contours: result image, null: output phase, Emgu.CV.CvEnum.RetrType.Tree: contour type, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxNone: point connection mode, assign Whitemat to MatWhite, and get the image as a cell template; The cell template is used to match the fluorescence in situ hybridization image with the image type marked as a green filter, a blue filter, and a red filter to complete the adaptive shape marking of the cells in the target image, specifically including: Step 21: Take green light as an example: define an object of the Mat class in the Emgu. CV namespace, represented by Imagefile, Imagefile calls the Mat instantiated object, the parameter is Greenmat, ImreadModes.AnyColor identifier, define Image in the Emgu.CV namespace<Bgr, byte> An object of the class is defined, represented by _image. Imagefile calls the ToImage class to mirror the image. Two objects of the Mat class in the Emgu. CV namespace are defined, represented by src and mask. _image is assigned to src. src calls the Copy method under the ToImage class to create an image with the same size as the original image as the background, and assigns it to mask. An object of the VectorOfVectorOfPoint class in the Emgu.CV.Util namespace is defined, represented by contours. The DrawContours function under CvInvoke is called with the following parameters: mask: target image, Contours: contour group, -1: all contours, newMCvScalar(0, 255, 0): color, 1: contour line. Step 22: Define Image in Emgu.CV namespace<Bgr, byte> 3 objects of the class are represented by _image1 and _image2. MatWhite calls the ToImage class to mirror the image and assigns it to _image1. Mask calls the ToImage class to mirror the image and assigns it to _image2. 2 objects of the Bitmap class in the System.Drawing namespace are defined, represented by _bitmap and _bitmap1. _image1 is assigned to _bitmap and _image1 is assigned to _image2. 4 objects of the Int class in the System namespace are defined, represented by width, height, stride and offset. The width and height of the _bitmap object are obtained and assigned to width and height respectively. Step 23: Define an object of the BitmapData class in the System.Drawing.Imaging namespace, represented by data. _bitmap calls the LockBits function with the image width and height as parameters, the access mode as ImageLockMode.ReadWrite, and the data format as PixelFormat.Format24bppRgb. Define an object of the IntPtr class in the System namespace, represented by Scan0. _bitmap calls the Scan0 function to obtain the memory starting position of bmpData, that is, the address of the first pixel data. _bitmap calls the Stride function to obtain the scan width and assign it to stride. Step 24: Define a pointer p of byte type, obtain the pixel address of Scan0 data, and assign it to p; obtain the gap between the display width and the scan line width, assign it to offset, call the For function in a loop, traverse the length and width of the image; call the if function to determine whether parameter 1, parameter 2, and parameter 3 are 0, where parameter 1 is R in the RGB component of the image pixel, parameter 2 is G in the RGB component of the image pixel, and parameter 3 is B in the RGB component of the image pixel; call the SetPixel function to set the pixel color of the current color component, remove other noise points, and assign the result to _bitmap2, and shift the pointer p left by three bits; complete the traversal and complete the cell image matching; Step 25: _bitmap calls the UnlockBits function to unlock the data object in this memory, and _bitmap2 calls the Save function to save the current matching image; instantiate the global MatGreen variable, convert _bitmap2 into a grayscale image, and assign it to mat; the template shape completes the adaptive shape marking of the cells in the target image, and the operations for red light and blue light are the same.

2. A fluorescence feature fusion system in fluorescence in situ hybridization images, It is characterized in that include: A fluorescence in situ hybridization image acquisition module is used to acquire fluorescence in situ hybridization images under different filters; Different filters include: white light filter, green light filter, blue light filter and red light filter; The image storage and reading module is used to store the fluorescence in situ hybridization images under different filters and read them into the Data_list linked list of Type type; the Data_list linked list of Type type includes: MatWhite, MatBlue, MatGreen, MatRed and Temp; the Temp is the image type identifier; A cell template determination module is used to locate the cell position of the fluorescence in situ hybridization image under the white light filter as the image type; and use the located image as the cell template; An adaptive shape marking module is used to match the cell template with the fluorescence in situ hybridization image under the image type identification of the green filter, the blue filter and the red filter to complete the adaptive shape marking of the cells in the target image; A fluorescence feature extraction module is used to extract fluorescence features of the fluorescence in situ hybridization images under the marked green light filter, blue light filter and red light filter respectively; A feature fusion module is used to fuse the extracted fluorescence features; The specific process of feature extraction and fusion is as follows: Step 1: Taking green light as an example: define five objects of the Mat class in the Emgu. CV namespace, represented by currentImage, result, resultscr, mattemp, and matfinal; define two objects of the VectorOfVectorOfPoint class in the Emgu.CV.Util namespace, represented by contours and contours1; define one object of the VectorOfPoint class in the Emgu.CV.Util namespace, represented by vp; Step 2: Call the Threshold function under CvInvoke for binarization, the parameters are MatGreen: original image, MatGreen: result image, 120: threshold, 255: maximum value, ThresholdType.Binary: binary thresholding; call the FindContours function under CvInvoke for contour search, the parameters are MatGreen: gray image, contours: saves the index numbers of the next contour, the previous contour, the parent contour, and the embedded contour. RetrType.Ccomp: monitors all contours. ChainApproxMethod.ChainApproxNone: none saves all continuous contours on the boundary into the contours vector. Step 3: Call the For function in a loop to traverse contours, assign each group of contours to vp, define a double type object in the System.Collections.Generic namespace, represented by area, and call the ContourArea function under CvInvoke to calculate the area; define a double type List class object in the System.Collections.Generic namespace, represented by sarea, sarea calls the Add function to store the area, and calls the if function to determine the current area. When the current area is less than 60, contours1 calls the push function to save the current contour; Step 4: Call Mat to instantiate the object and convert the image format. The parameters are MatGreen, Emgu.CV.CvEnum.DepthType.Cv8U: 8-bit unsigned integer, 3: depth; and assign it to result; call the DrawContours function under CvInvoke, the parameters are result: target image, contours1: contour group, -1: all contours, newMCvScalar(0, 255, 0): color, 1: contour line; Step 5: Call Mat to instantiate the object, the parameter is result, Emgu.CV.CvEnum.ImreadModes.Grayscale: grayscale, assign it to resultscr; call CvtColor function under CvInvoke to convert the color space, the parameters are resultscr: original image, resultscr: result image, ColorConversion.BayerBg2Gray: convert to grayscale; call Mat to instantiate the object, the parameter is result size, Emgu.CV.CvEnum.DepthType.Cv8U: 8-bit unsigned integer, 1: Depth, assign it to mattemp; create other blank channels; call the push function under Channelss, the first parameter is mattemp; call the push function under Channelss, the second parameter is resultscr, call the push function under Channelss, the third parameter is mattemp, fill the 3 channels; call the Merge function under CvInvoke to perform channel fusion processing on the image, the parameters are Channelss: original image, matfinal: result image, complete the extraction of fluorescence features under green light cells, the same operation for red light and blue light; Step 6: Define an object of the Mat class in the Emgu. CV namespace, expressed in RGB, and call the AddWeighted function under CvInvoke for image fusion, with the parameters MatWhite: image 1, 0.5: weight value, MatGreen: image 2, 0.5: weight value, 0: gamma correction, RGB: output image; call the AddWeighted function under CvInvoke for image fusion, with the parameters RGB: image 1, 0.5: weight value, MatBlue: image 2, 0.5: weight value, 0: gamma correction, RGB: output image; call the AddWeighted function under CvInvoke for image fusion, with the parameters RGB: image 1, 0.5: weight value, MatRed: image 2, 0.5: weight value, 0: gamma correction, RGB: output image, to complete the fusion of the fluorescence features under the green light cells; The image storage and reading module specifically includes: An image preprocessing unit, used for grayscale processing of fluorescence in situ hybridization images under different filters; An image storage unit, used to store the fluorescence in situ hybridization images under different filters after grayscale processing using a Data_file file; The cell template determination module is used to locate the cell position of the fluorescence in situ hybridization image under the image type identification of the white light filter; and use the located image as the cell template, specifically including: Step 11: Define an object of the DataRow class in the System.Data namespace, represented by fData; define a List class object of the Type type in the System.Collections.Generic namespace, represented by Data_list; define a DataTable class object in the System.Data namespace, represented by Data_file; Step 12: The SData object calls the NewRow function to create a new data row object, represented by dr, and calls the Substring function in a loop, assigning the return value of the Substring function to dr. Determine whether the expression value of the loop process is True. If so, execute step 13, otherwise continue to execute step 12; Step 13: The Data_file object calls the Rows.Add function to loop the data in the SData object to the Data_file object and determine whether the expression value of the loop process is True. If so, execute step 14, otherwise continue to execute step 13; Step 14: The Data_file object calls the Foreach function to retrieve the data in the data container, and assigns the return value of the Foreach function to Data_list and determines whether the expression value of the loop process is False. If so, execute step 15, otherwise continue to execute step 14; Step 15: Assign the data in Data_list to MatWhite, MatBlue, MatGreen, MatRed, and Temp data members respectively, and assign the values ​​of Temp data member in Type to 1, 2, 3, and 4. Determine whether the expression value of the loop process is False. If so, execute step 16, otherwise continue to execute step 15; Step 16: Define five objects of the Mat class in the Emgu. CV namespace, represented by Redmat, Greenmat, Bluemat, Imagefile, and Whitemat. Whitemat calls the Mat instantiated object with the parameter MatWhite and the ImreadModes.AnyColor flag, which is assigned to Whitemat. Define the Image file in the Emgu. CV namespace.<Bgr, byte> An object of the class is defined, represented by _image. Whitemat calls the ToImage class to mirror the image. An object of the Mat class in the Emgu. CV namespace is defined, represented by src. _image is assigned to src. The Threshold function under CvInvoke is called for binarization. The parameters are Whitemat: original image, Whitemat: result image, 50: threshold, 255: maximum value, ThresholdType.Binary: binary thresholding. Step 17: Call the CvtColor function under CvInvoke to convert the color space, the parameters are Whitemat: original image, Whitemat: result image, 50: threshold, 180: maximum value, ColorConversion.BayerBg2Gray: conversion to grayscale; call the FindContours function under CvInvoke to find contours, the parameters are Whitemat: gray image, contours: result image, null: output phase, Emgu.CV.CvEnum.RetrType.Tree: contour type, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxNone: point connection mode, assign Whitemat to MatWhite, and get the image as a cell template; The adaptive shape marking module is used to match the cell template with the fluorescence in situ hybridization image under the image type identification of the green filter, the blue filter and the red filter to complete the adaptive shape marking of the cells in the target image, specifically including: Step 21: Take green light as an example: define an object of the Mat class in the Emgu.CV namespace, represented by Imagefile, Imagefile calls the Mat instantiated object, the parameter is Greenmat, the ImreadModes.AnyColor identifier, define the Image in the Emgu.CV namespace<Bgr, byte> An object of the class is defined, represented by _image. Imagefile calls the ToImage class to mirror the image. Two objects of the Mat class in the Emgu. CV namespace are defined, represented by src and mask. _image is assigned to src. src calls the Copy method under the ToImage class to create an image with the same size as the original image as the background, and assigns it to mask. An object of the VectorOfVectorOfPoint class in the Emgu.CV.Util namespace is defined, represented by contours. The DrawContours function under CvInvoke is called with the following parameters: mask: target image, Contours: contour group, -1: all contours, newMCvScalar(0, 255, 0): color, 1: contour line. Step 22: Define Image in Emgu.CV namespace<Bgr, byte> 3 objects of the class are represented by _image1 and _image2. MatWhite calls the ToImage class to mirror the image and assigns it to _image1. Mask calls the ToImage class to mirror the image and assigns it to _image2. 2 objects of the Bitmap class in the System.Drawing namespace are defined, represented by _bitmap and _bitmap1. _image1 is assigned to _bitmap and _image1 is assigned to _image2. 4 objects of the Int class in the System namespace are defined, represented by width, height, stride and offset. The width and height of the _bitmap object are obtained and assigned to width and height respectively. Step 23: Define an object of the BitmapData class in the System.Drawing.Imaging namespace, represented by data. _bitmap calls the LockBits function with the image width and height as parameters, the access mode as ImageLockMode.ReadWrite, and the data format as PixelFormat.Format24bppRgb. Define an object of the IntPtr class in the System namespace, represented by Scan0. _bitmap calls the Scan0 function to obtain the memory starting position of bmpData, that is, the address of the first pixel data. _bitmap calls the Stride function to obtain the scan width and assign it to stride. Step 24: Define a pointer p of byte type, obtain the pixel address of Scan0 data, and assign it to p; obtain the gap between the display width and the scan line width, assign it to offset, call the For function in a loop, traverse the length and width of the image; call the if function to determine whether parameter 1, parameter 2, and parameter 3 are 0, where parameter 1 is R in the RGB component of the image pixel, parameter 2 is G in the RGB component of the image pixel, and parameter 3 is B in the RGB component of the image pixel; call the SetPixel function to set the pixel color of the current color component, remove other noise points, and assign the result to _bitmap2, and shift the pointer p left by three bits; complete the traversal and complete the cell image matching; Step 25: _bitmap calls the UnlockBits function to unlock the data object in this memory, and _bitmap2 calls the Save function to save the current matching image; instantiate the global MatGreen variable, convert _bitmap2 into a grayscale image, and assign it to mat; the template shape completes the adaptive shape marking of the cells in the target image, and the operations for red light and blue light are the same.

Images