A breast cancer cell image classification method, system, device and storage medium

Abstract

The present invention discloses a breast cancer cell image classification method, system, device and storage medium, the method comprising: obtaining a cell suspension, which is obtained by preparing and diluting a breast cancer cell sample; controlling a dynamic light scattering imaging device to perform imaging processing on the cell suspension to obtain a plurality of frames of breast cancer cell images; obtaining an image classification model, which comprises a denoising module, a cell dynamic detection module and a classification module; grouping a plurality of frames of breast cancer cell images and then inputting them into the image classification model, using the denoising module to denoise a frame of breast cancer cell images with the most central time contained in each group of breast cancer cell images, using the cell dynamic detection module to generate a dynamic correlation map between every two frames of breast cancer cell images adjacent in time after denoising, and using the classification module to fuse and classify all dynamic correlation maps to obtain the category of the breast cancer cell sample. The present invention can accurately and quickly realize the classification of breast cancer cell samples.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a breast cancer cell image classification method, system, device and storage medium. Background Art

[0002] In clinical medical diagnosis, immunohistochemistry (IHC) is generally used to detect the expression of estrogen receptors, progesterone receptors and HER-2 in breast cancer cell samples, and then identify the type of breast cancer cell samples. However, this detection process is relatively complicated and costly, and requires the protein in breast cancer cell samples to be labeled in advance. The analysis is difficult and the accuracy of the test results depends largely on the operator's technical level. As image detection technology based on deep learning is widely used in the medical field, how to expand it to achieve classification detection of breast cancer cell images is an urgent problem to be solved. Summary of the invention

[0003] The present invention provides a breast cancer cell image classification method, system, device and storage medium to solve one or more technical problems existing in the prior art and at least provide a beneficial choice or create conditions.

[0004] In a first aspect, a method for classifying breast cancer cell images is provided, the method comprising:

[0005] obtaining a cell suspension obtained by preparing and diluting a breast cancer cell sample;

[0006] Controlling a dynamic light scattering imaging device to perform imaging processing on the cell suspension to obtain a plurality of time-continuous frames of breast cancer cell images;

[0007] Obtaining an image classification model, which includes a denoising module, a cell dynamics detection module, and a classification module;

[0008] The plurality of frames of breast cancer cell images are grouped and then input into the image classification model, the denoising module is used to denoise the frame of breast cancer cell image that is most central in time contained in each group of breast cancer cell images, the cell dynamic detection module is used to generate a dynamic correlation map between every two denoised frames of breast cancer cell images that are adjacent in time, and the classification module is used to fuse and classify all dynamic correlation maps to obtain a category of the breast cancer cell sample.

[0009] Furthermore, grouping the plurality of frames of breast cancer cell images includes:

[0010] According to the number of images processed by the denoising module at a time, the plurality of frames of breast cancer cell images are divided into a plurality of groups of breast cancer cell images in sequence, and there is only one different frame of breast cancer cell image between each two adjacent groups of breast cancer cell images.

[0011] Furthermore, the denoising module is built by the FastDVDNet network framework.

[0012] Further, the cell dynamics detection module includes a convolution unit and a cell dynamics detection unit, and the convolution unit includes a single 7×7 convolution layer and two 5×5 convolution layers connected in sequence;

[0013] The convolution unit is used to perform multiple convolution sampling on each frame of breast cancer cell image after denoising to obtain a corresponding feature image; and the cell dynamic detection unit is used to generate a dynamic correlation map between every two frames of feature images that are adjacent in time.

[0014] Furthermore, the step of using the cell dynamic detection unit to generate a dynamic correlation map between every two frames of feature images that are adjacent in time includes:

[0015] Setting a first scale window, and then reducing the first scale window by using a given scale adjustment coefficient to obtain a second scale window;

[0016] For every two frames of feature images that are adjacent in time, use the first scale window to traverse one of the frames of feature images to obtain a plurality of first window images, and synchronously traverse the other frame of feature images to obtain a plurality of second window images;

[0017] For each first window image, extract a second window image corresponding to the position of the first window image from the plurality of second window images, then use the second scale window to traverse the first window image to obtain a plurality of third window images, and synchronously traverse the second window image to obtain a plurality of fourth window images;

[0018] For each third window image, extract a fourth window image corresponding to the position of the third window image from the plurality of fourth window images, and then determine a first dynamic correlation map between the third window image and the fourth window image using a normalized cross-correlation matching algorithm;

[0019] The weighted average method is used to fuse multiple first dynamic association maps corresponding to each first window image to obtain a second dynamic association map, and then the multiple second dynamic association maps corresponding to the multiple first window images are combined according to position to obtain a final dynamic association map.

[0020] Further, the classification module includes two feature extraction units and a single feature fusion classification unit, each feature extraction unit includes two 3×3×3 convolutional layers connected in sequence and a single 2×2×2 pooling layer, and the single feature fusion classification unit includes two fully connected layers connected in sequence and a single classification layer;

[0021] The two feature extraction units are used to extract features from each dynamic association map to obtain corresponding feature information; the single feature fusion classification unit is used to fuse and classify all feature information to obtain the category of the breast cancer cell sample.

[0022] Furthermore, the dynamic light scattering imaging device comprises a stage, a laser, a focusing lens, a filter, a single-mode optical fiber, an imaging lens and an imaging sensor, the stage is provided with a microfluidic chip, and the microfluidic chip is provided with a sample chamber; the dynamic light scattering imaging device is controlled to perform imaging processing on the cell suspension to obtain a plurality of time-continuous frames of breast cancer cell images, including:

[0023] When the sample chamber is filled with the cell suspension, the laser is controlled to emit a laser beam, which is coupled to one end of the single-mode optical fiber after being processed by the focusing objective lens and the filter in sequence, and the other end of the single-mode optical fiber emits bright light to illuminate the sample chamber, thereby stimulating each cell in the cell suspension to continuously move and generate dynamic scattered light, and the dynamic scattered light is projected onto the imaging sensor via the imaging objective lens, thereby controlling the imaging sensor to output a plurality of frames of breast cancer cell images that are continuous in time at a given frame rate.

[0024] In a second aspect, a breast cancer cell image classification system is provided, the system comprising:

[0025] The first module is used to obtain a cell suspension, which is obtained by preparing and diluting a breast cancer cell sample;

[0026] The second module is used to control the dynamic light scattering imaging device to perform imaging processing on the cell suspension to obtain a plurality of time-continuous frames of breast cancer cell images;

[0027] The third module is used to obtain an image classification model, which includes a denoising module, a cell dynamics detection module and a classification module;

[0028] The fourth module is used to group the several frames of breast cancer cell images and input them into the image classification model, use the denoising module to denoise the frame of breast cancer cell image that is most central in time contained in each group of breast cancer cell images, use the cell dynamic detection module to generate a dynamic correlation map between every two denoised frames of breast cancer cell images that are adjacent in time, and use the classification module to fuse and classify all dynamic correlation maps to obtain the category of the breast cancer cell sample.

[0029] In a third aspect, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the breast cancer cell image classification method as described in the first aspect.

[0030] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for classifying breast cancer cell images as described in the first aspect is implemented.

[0031] The present invention has at least the following beneficial effects: by adopting a dynamic light scattering imaging device, the cell properties of breast cancer cell samples can be directly detected without the aid of fluorescent labels or chemical labels, and compared with the existing immunohistochemistry method, it has the advantages of simple operation and fast detection speed; in the image classification model, by adopting a denoising module, the noise interference in the dynamic light scattering imaging process can be effectively reduced, and by adopting a cell dynamic detection module, the dynamic motion characteristics of cells between adjacent frames can be captured at multiple scales, and it has a certain robustness in dealing with changes in illumination and contrast between adjacent frames, so that the subsequent classification module can more accurately predict the category information commonly characterized by all breast cancer cell images. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0033] Figure 1 is a schematic flow chart of a breast cancer cell image classification method in an embodiment of the present invention;

[0034] Figure 2 is a simplified schematic diagram of the composition of a dynamic light scattering imaging device in an embodiment of the present invention;

[0035] Figure 3 is a schematic diagram of the composition of a denoising module in an embodiment of the present invention;

[0036] Figure 4 Schematic diagram of the composition of the cell dynamic detection module in an embodiment of the present invention;

[0037] Figure 5 is a schematic diagram of the composition of the classification module in an embodiment of the present invention;

[0038] Figure 6 is a schematic diagram of the composition of a breast cancer cell image classification system in an embodiment of the present invention;

[0039] Figure 7 It is a schematic diagram of the hardware structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order from the module division in the system or the order in the flow chart. The terms "first", "second", "third", "fourth", etc. in the specification of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, such as processes, methods, systems, products or devices that include a series of steps or units, which are not necessarily limited to those steps or units clearly listed, but may include other steps or units inherent to these processes, methods, products or devices that are not clearly listed.

[0042] Please refer to Figure 1 , Figure 1 : is a schematic diagram of a process of a breast cancer cell image classification method provided by an embodiment of the present invention, the method comprising the following steps:

[0043] Step S110, obtaining a cell suspension, which is obtained by preparing and diluting a breast cancer cell sample;

[0044] Step S120, controlling the dynamic light scattering imaging device to perform imaging processing on the cell suspension to obtain a plurality of time-continuous frames of breast cancer cell images;

[0045] Step S130, obtaining an image classification model, which includes a denoising module, a cell dynamics detection module and a classification module;

[0046] Step S140, grouping the plurality of frames of breast cancer cell images and inputting them into the image classification model, using the denoising module to denoise the frame of breast cancer cell image that is most central in time contained in each group of breast cancer cell images, using the cell dynamic detection module to generate a dynamic correlation map between every two denoised frames of breast cancer cell images that are adjacent in time, and using the classification module to fuse and classify all dynamic correlation maps to obtain a category of the breast cancer cell sample.

[0047] In an embodiment of the present invention, the method for obtaining the cell suspension mentioned in the above step S110 includes but is not limited to the following: the operator prepares the breast cancer cell sample into a 100uL (microliter) cell suspension, then dilutes the 100uL cell suspension until its concentration is 100,000 cells / mL (milliliter), and finally uses an empty pipette to take out 20uL from the diluted 100uL cell suspension as the cell suspension required for the final imaging; wherein, the breast cancer cell sample is preferably a laboratory-cultured sample.

[0048] In the embodiments of the present invention, see Figure 2 As shown, the dynamic light scattering imaging device mentioned in the above step S120 includes a stage 210, a laser 220, a focusing lens 230, a filter 240, a single-mode optical fiber 250, an imaging lens 260 and an imaging sensor 270. A microfluidic chip 280 is placed on the stage 210, and a sample chamber 281 is opened on the microfluidic chip 280.

[0049] As a preferred embodiment, the stage 210 and the microfluidic chip 280 are both made of light-transmitting materials, the magnification of the focusing lens 230 is 4 times, the magnification of the imaging lens 260 is 10 times, and the core diameter of the single-mode optical fiber 250 is 50 um (micrometer).

[0050] On this basis, when the operator introduces the cell suspension loaded in the pipette into the sample chamber, the specific implementation process of the above step S120 includes the following: controlling the laser 220 to emit a laser beam, the wavelength of the laser beam is preferably 532nm (nanometers), the laser beam is coupled to one end of the single-mode optical fiber 250 after corresponding light processing through the focusing lens 230 and the filter 240 in sequence, the other end of the single-mode optical fiber 250 emits bright light and irradiates the inside of the sample chamber 281, stimulating each cell in the cell suspension to be in a continuous motion state to generate dynamic scattered light, the dynamic scattered light is projected to the imaging sensor 270 through the imaging lens 260, and at this time, the imaging sensor 270 is controlled to generate a plurality of frames of breast cancer cell images that are continuous in time according to a preset frame rate, the frame rate is preferably 103.09fps, and the imaging time of the imaging sensor 270 is preferably 10 seconds.

[0051] In the embodiments of the present invention, see Figure 3 As shown, the denoising module mentioned in the above step S130 is constructed using the existing FastDVDNet network framework, which includes a first denoising unit, a second denoising unit, a third denoising unit and a fourth denoising unit. The first denoising unit, the second denoising unit and the third denoising unit share weight parameters to improve the generalization ability of the model, and the output end of the first denoising unit, the output end of the second denoising unit and the output end of the third denoising unit are all connected to the input end of the fourth denoising unit.

[0052] More specifically, the network frameworks of the four denoising units are the same, and are all built using a U-net-like structure. Each denoising unit includes five convolution blocks connected in sequence, and an adder is added to the output ends of the three last convolution blocks, so that the input and output ends of the third convolution block are both connected to the input end of the first adder, the output end of the first adder is connected to the input end of the fourth convolution block, the input end of the second convolution block and the output end of the fourth convolution block are both connected to the input end of the second adder, the output end of the second adder is connected to the input end of the fifth convolution block, the input end of the first convolution block and the output end of the fifth convolution block are both connected to the input end of the third adder, and the output end of the third adder serves as the final output of the denoising unit.

[0053] The five convolution blocks contained in the denoising unit are described in detail as follows:

[0054] The first convolutional block includes two convolutional layers connected in sequence. Each convolutional layer performs convolution (Conv) operation, batch normalization (Bn) operation and nonlinear operation based on ReLU (Rectified Linear Unit) activation function in sequence. In addition, a 3×3 convolution kernel is used in the first convolutional layer and its output channel number is set to 30, and the output channel number of the second convolutional layer is set to 32.

[0055] The second convolutional block includes three convolutional layers connected in sequence. The first convolutional layer sequentially performs convolution operations with a stride (i.e., the sliding step size of the convolution kernel) of 2, batch normalization operations, and nonlinear operations based on the ReLU activation function to achieve downsampling. The following two convolutional layers sequentially perform convolution operations, batch normalization operations, and nonlinear operations based on the ReLU activation function, and the number of output channels of these three convolutional layers is set to 64.

[0056] The third convolution block includes six convolutional layers connected in sequence and a single upsampling layer. The first convolutional layer sequentially performs convolution operations with a stride of 2, batch normalization operations, and nonlinear operations based on the ReLU activation function to achieve downsampling functions. The middle four convolutional layers sequentially perform convolution operations, batch normalization operations, and nonlinear operations based on the ReLU activation function. The last convolutional layer performs a convolution operation. The upsampling layer performs a PixelShuffle operation, and sets the number of output channels of the first five convolutional layers to 128, and the number of output channels of the last convolutional layer to 256.

[0057] The fourth convolutional block includes three convolutional layers connected in sequence and a single upsampling layer. The first two convolutional layers perform convolution operations, batch normalization operations, and nonlinear operations based on the ReLU activation function in sequence. The last convolutional layer performs a convolution operation. The upsampling layer performs a PixelShuffle operation, and sets the number of output channels of the first two convolutional layers to 64, and the number of output channels of the last convolutional layer to 128.

[0058] The fifth convolution block includes two convolutional layers and a single upsampling layer connected in sequence. The first convolutional layer performs convolution operation, batch normalization operation and nonlinear operation based on ReLU activation function in sequence. The second convolutional layer performs convolution operation. The upsampling layer performs PixelShuffle operation and sets the output channel number of the first convolutional layer to 32 and the output channel number of the second convolutional layer to 3.

[0059] In actual application, when the denoising module receives five consecutive frames of breast cancer cell images (respectively recorded as the t-2 frame breast cancer cell image, the t-1 frame breast cancer cell image, the t frame breast cancer cell image, the t+1 frame breast cancer cell image and the t+2 frame breast cancer cell image), the three consecutive breast cancer cell images with the earliest time (i.e., the t-2 frame breast cancer cell image, the t-1 frame breast cancer cell image and the t frame breast cancer cell image) and a preset noise spectrum are input into the first denoising unit for processing to obtain a first intermediate image, and the three consecutive breast cancer cell images with the most central time (i.e., the t-1 frame breast cancer cell image, the t frame breast cancer cell image and the t+1 frame The first intermediate image, the second intermediate image, the third intermediate image and the noise spectrum are input into the fourth denoising unit for processing to obtain a denoising effect diagram of the breast cancer cell image frame that is most centered in time (that is, the t-th breast cancer cell image frame); wherein the noise spectrum contains annular noise and Gaussian noise that are easily generated in dynamic light scattering experiments.

[0060] In the embodiments of the present invention, see Figure 4 As shown, the cell dynamic detection module mentioned in the above step S130 includes a convolution unit and a cell dynamic detection unit connected in sequence, wherein the convolution unit includes a single 7×7 convolution layer (denoted as 7×7Conv1) and two 5×5 convolution layers (denoted as 5×5Conv2 and 5×5Conv3, respectively) connected in sequence, 64 convolution kernels are used in 7×7Conv1, 128 convolution kernels are used in 5×5Conv2, and 256 convolution kernels are used in 5×5Conv3, and weight parameters are shared between the three convolution layers, and each convolution layer is set to perform convolution operations with a stride of 2 (denoted as str2) and nonlinear operations based on the ReLU activation function in sequence.

[0061] In actual application, when the cell dynamic detection module receives all denoised breast cancer cell images output by the denoising module, the convolution unit performs multiple convolution sampling on each frame of the denoised breast cancer cell images to obtain corresponding feature images, and then the cell dynamic detection unit processes every two frames of feature images that have a temporally adjacent relationship among all feature images corresponding to all denoised breast cancer cell images to generate a corresponding dynamic association map.

[0062] Furthermore, taking any two frames of feature images (respectively recorded as the first frame feature image and the second frame feature image) that are temporally adjacent in all the feature images as an example, the internal processing process of the cell dynamic detection unit is described as follows:

[0063] Step 1: Set a first scale window, the size of which is preferably set to 4×4;

[0064] Step 2: Reduce the first scale window according to a preset scale adjustment coefficient to obtain a corresponding second scale window, wherein the scale adjustment coefficient is preferably set to 0.5;

[0065] Step 3: traverse the first frame feature image through the first scale window to obtain a corresponding plurality of first window images, and simultaneously traverse the second frame feature image through the first scale window to obtain a corresponding plurality of second window images;

[0066] Step 4: for each first window image, a single second window image having a position corresponding to the first window image is screened out from the multiple second window images, that is, the position of the second window image in the second frame feature image is the same as the position of the first window image in the first frame feature image, and then the first window image is traversed through the second scale window to obtain the corresponding multiple third window images, and the second window image is simultaneously traversed through the second scale window to obtain the corresponding multiple fourth window images;

[0067] Step 5: for each third window image, a single fourth window image having a position corresponding to the third window image is screened out from the plurality of fourth window images, that is, the position of the fourth window image in the second frame feature image is the same as the position of the third window image in the first frame feature image, and then a first dynamic correlation map between the third window image and the fourth window image is generated by a normalized cross-correlation matching algorithm, preferably solved by the following mathematical expression:

[0068]

[0069] Where NCC s (W1, W2) is the feature correlation value, s is the scale adjustment coefficient, W1 is the first frame feature image, W 1,s is the third window image, W2 is the second frame feature image, W 2,s is the fourth window image, N s is the number of pixels contained in the second scale window, and it can be understood that the number of pixels contained in the third window image and the number of pixels contained in the fourth window image are both Ns , sW is the pixel area that the second scale window is allowed to cover. It can be understood that the area occupied by the third window image in the first frame feature image and the area occupied by the fourth window image in the second frame feature image are both sW. F1(x,y) is the normalized pixel intensity value at the position (x,y) in the third window image. is the average value of all normalized pixel intensity values ​​contained in the third window image, is the standard deviation of all normalized pixel intensity values ​​contained in the third window image, F2(x,y) is the normalized pixel intensity value at the same position (x,y) in the fourth window image, is the average value of all normalized pixel intensity values ​​contained in the fourth window image, is the standard deviation of all normalized pixel intensity values ​​contained in the fourth window image; and the corresponding first dynamic correlation map is obtained by assigning the feature correlation value to each pixel contained in the second scale window;

[0070] Step 6: for each first window image, multiple first dynamic association maps corresponding to the first window image are fused by weighted average method, and the weight parameter used in the fusion process of each first dynamic association map is pre-defined according to the confidence level, so as to obtain the corresponding second dynamic association map;

[0071] Step 7: According to the position of each first window image in the first frame feature image, the multiple second dynamic association maps corresponding to the multiple first window images are combined to obtain the final required dynamic association map.

[0072] Of course, in the above steps 4 to 7, the second window image can be used as the main image and the first window image can be used as the auxiliary image to complete the subsequent window traversal tasks, dynamic association map generation tasks, dynamic association map fusion tasks and dynamic association map combination tasks in sequence. The present invention does not make any limitations on this.

[0073] In the embodiments of the present invention, see Figure 5As shown, the classification module mentioned in the above step S130 includes two feature extraction units connected in sequence (respectively denoted as the first feature extraction unit and the second feature extraction unit) and a single feature fusion classification unit, wherein the first feature extraction unit includes two 3×3×3 convolutional layers connected in sequence (respectively denoted as 3×3×3Conv4 and 3×3×3Conv5) and a single 2×2×2 pooling layer (denoted as 2×2×2Pool1), the second feature extraction unit also includes two 3×3×3 convolutional layers connected in sequence (respectively denoted as 3×3×3Conv6 and 3×3×3Conv7) and a single 2×2×2 pooling layer (denoted as 2×2×2Pool2), and the feature fusion classification unit includes two fully connected layers connected in sequence (respectively denoted as Fc1 and Fc2) and a single classification layer (denoted as Softmax).

[0074] 256 convolution kernels are used in 3×3×3Conv4 and 3×3×3Conv5, 512 convolution kernels are used in 3×3×3Conv6 and 3×3×3Conv7, 1024 neurons are set in Fc1, and m neurons are set in Fc2, where m is the number of breast cancer cell types that the image classification model allows to detect, and the breast cancer cell types are preferably Luminal A, Luminal Type B, HER-2 positive and triple-negative breast cancer, etc.; 3×3×3Conv4 and 3×3×3Conv6 are set to perform convolution operations with a stride of 2 (denoted as str2) and nonlinear operations based on the ReLU activation function in sequence, 3×3×3Conv5 and 3×3×3Conv7 are set to perform convolution operations with a stride of 1 (denoted as str1) and nonlinear operations based on the ReLU activation function in sequence, and 2×2×2Pool1 and 2×2×2Pool2 are set to perform pooling operations with a stride of 1; It should be noted that the purpose of setting 2×2×2Pool1 and 2×2×2Pool2 is to compress features and retain necessary information, and the purpose of setting Softmax is to calculate the probability distribution to output the category with the highest confidence.

[0075] In actual application, when the classification module receives all the dynamic association maps output by the cell dynamic detection module, the two feature extraction units perform feature extraction on each of the dynamic association maps to obtain corresponding feature information, and then the feature fusion classification unit fuses all the feature information corresponding to all the dynamic association maps and classifies them to obtain the category corresponding to the breast cancer cell sample.

[0076] In an embodiment of the present invention, the image grouping method of the above step S140 is explained as follows: the plurality of frames of breast cancer cell images are sequentially divided into a plurality of groups of breast cancer cell images according to the number of images that the denoising module allows to process at a single time, so that each two groups of breast cancer cell images that are adjacent to each other only contain one different frame of breast cancer cell image.

[0077] For example, assuming that the number of the plurality of frames of breast cancer cell images is N, and N is a positive integer greater than or equal to 5, the specific implementation process of the above step S140 includes the following:

[0078] Step S141: Since the number of images that the denoising module allows to process at a single time is 5 frames, N frames of breast cancer cell images are divided into N-4 groups of breast cancer cell images in sequence;

[0079] The first group of breast cancer cell images includes the first breast cancer cell image, the second breast cancer cell image, the third breast cancer cell image, the fourth breast cancer cell image and the fifth breast cancer cell image; the second group of breast cancer cell images includes the second breast cancer cell image, the third breast cancer cell image, the fourth breast cancer cell image, the fifth breast cancer cell image and the sixth breast cancer cell image; the third group of breast cancer cell images includes the third breast cancer cell image, the fourth breast cancer cell image, the fifth breast cancer cell image, the sixth breast cancer cell image and the seventh breast cancer cell image, and so on;

[0080] Step S142, inputting the N-4 groups of breast cancer cell images into the denoising module for processing respectively, so as to obtain corresponding N-4 frames of breast cancer cell images to be tested;

[0081] That is to say, for each group of breast cancer cell images, the denoising module performs joint denoising on the middle frame breast cancer cell image (i.e., the breast cancer cell image with the most central time) contained in the group of breast cancer cell images according to the time-related information reflected by the group of breast cancer cell images, so as to obtain the denoised middle frame breast cancer cell image and record it as a frame of breast cancer cell image to be tested;

[0082] Step S143, combining the N-4 frames of breast cancer cell images to be tested in pairs according to a temporal adjacent relationship to obtain corresponding N-3 groups of breast cancer cell images to be tested;

[0083] Step S144, inputting the N-3 groups of breast cancer cell images to be tested into the cell dynamic detection module for processing, so as to obtain corresponding N-3 dynamic association maps;

[0084] That is to say, for each group of breast cancer cell images to be tested, the cell dynamic detection module generates a final dynamic correlation map according to the corresponding distribution of pixel intensity values ​​of the group of breast cancer cell images to be tested at different scale positions;

[0085] Step S145: input the N-3 dynamic association maps into the classification module for fusion and classification to obtain the category corresponding to the breast cancer cell sample.

[0086] In the embodiment of the present invention, the cell properties of breast cancer cell samples can be directly detected by using a dynamic light scattering imaging device without the need for fluorescent or chemical labels. Compared with the existing immunohistochemistry method, it has the advantages of simple operation and fast detection speed. In the image classification model, the noise interference in the dynamic light scattering imaging process can be effectively reduced by using a denoising module. The dynamic motion characteristics of cells between adjacent frames can be captured at multiple scales by using a cell dynamic detection module. It has a certain robustness in dealing with illumination changes and contrast changes between adjacent frames, so that the subsequent classification module can more accurately predict the category information commonly represented by all breast cancer cell images.

[0087] Please refer to Figure 6 , Figure 6 : is a schematic diagram of the composition of a breast cancer cell image classification system provided by an embodiment of the present invention, the system comprising:

[0088] The first module 310 is used to obtain a cell suspension, which is obtained by preparing and diluting a breast cancer cell sample;

[0089] The second module 320 is used to control the dynamic light scattering imaging device to perform imaging processing on the cell suspension to obtain a plurality of time-continuous frames of breast cancer cell images;

[0090] The third module 330 is used to obtain an image classification model, which includes a denoising module, a cell dynamics detection module and a classification module connected in sequence;

[0091] The fourth module 340 is used to group the plurality of frames of breast cancer cell images and then input them into the image classification model, perform denoising on the frame of breast cancer cell image that is most central in time contained in each group of breast cancer cell images through the denoising module, generate a dynamic correlation map between every two denoised frames of breast cancer cell images that are adjacent in time through the cell dynamic detection module, and fuse and classify all dynamic correlation maps through the classification module to obtain the category of the breast cancer cell sample.

[0092] The contents of the above method embodiments are all applicable to the present system embodiments. The functions implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are the same as those of the above method embodiments, which will not be repeated here.

[0093] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a method for classifying breast cancer cell images in the above embodiment is implemented. Wherein, the computer-readable storage medium includes but is not limited to any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. In other words, the storage device includes any medium that stores or transmits information in a readable form by a device (such as a computer, a mobile phone, etc.), which can be a read-only memory, a disk or an optical disk, etc.

[0094] also, Figure 7 4 is a schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention, wherein the computer device includes a processor 420, a memory 430, an input unit 440, a display unit 450 and other devices. Those skilled in the art can understand that Figure 7 The device structure device shown does not constitute a limitation on all devices, and may include more or fewer components than shown, or combine certain components. The memory 430 can be used to store the computer program 410 and various functional modules, and the processor 420 runs the computer program 410 stored in the memory 430, thereby executing various functional applications and data processing of the device. The memory can be an internal memory or an external memory, or include an internal memory and an external memory. The internal memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, or a random access memory. The external memory may include a hard disk, a floppy disk, a U disk, a magnetic tape, etc. The memory 430 disclosed in the embodiment of the present invention includes but is not limited to the above-mentioned types of memory. The memory 430 disclosed in the embodiment of the present invention is only an example and not a limitation.

[0095] The input unit 440 is used to receive the input of the signal and the keyword input by the user. The input unit 440 may include a touch panel and other input devices. The touch panel can collect the user's touch operation on or near it (such as the user's operation on or near the touch panel using any suitable object or accessory such as a finger, a stylus, etc.), and drive the corresponding connection device according to a pre-set program; other input devices may include but are not limited to one or more of a physical keyboard, a function key (such as a playback control key, a switch key, etc.), a trackball, a mouse, a joystick, etc. The display unit 450 can be used to display the information input by the user or the information provided to the user and various menus of the terminal device. The display unit 450 can be in the form of a liquid crystal display, an organic light emitting diode, etc. The processor 420 is the control center of the terminal device, which uses various interfaces and lines to connect the various parts of the entire device, and executes various functions and processes data by running or executing software programs and / or modules stored in the memory 430, and calling the data stored in the memory 430.

[0096] As an embodiment, the computer device includes a processor 420, a memory 430 and a computer program 410, wherein the computer program 410 is stored in the memory 430 and is configured to be executed by the processor 420, and the computer program 410 is configured to execute a breast cancer cell image classification method in the above embodiment.

[0097] Although the description of the present application has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims by reference to the attached claims, taking into account the prior art, so as to effectively cover the intended scope of the present application. In addition, the above description of the present application is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present application that have not yet been foreseen may still represent equivalent changes to the present application.

Claims

1. A breast cancer cell image classification method, characterized in that: The method comprises: obtaining a cell suspension obtained by preparing and diluting a breast cancer cell sample; Controlling a dynamic light scattering imaging device to perform imaging processing on the cell suspension to obtain a plurality of time-continuous frames of breast cancer cell images; Obtaining an image classification model, which includes a denoising module, a cell dynamics detection module, and a classification module; The plurality of frames of breast cancer cell images are grouped and then input into the image classification model, the denoising module is used to denoise the frame of breast cancer cell image that is most central in time contained in each group of breast cancer cell images, the cell dynamic detection module is used to generate a dynamic correlation map between every two denoised frames of breast cancer cell images that are adjacent in time, and the classification module is used to fuse and classify all dynamic correlation maps to obtain the category of the breast cancer cell sample; The cell dynamics detection module includes a convolution unit and a cell dynamics detection unit, and the convolution unit includes a single 7×7 convolution layer and two 5×5 convolution layers connected in sequence; The convolution unit is used to perform multiple convolution sampling on each frame of breast cancer cell image after denoising to obtain a corresponding feature image; the cell dynamic detection unit is used to generate a dynamic correlation map between each two frames of feature images that are adjacent in time; Wherein, the step of using the cell dynamic detection unit to generate a dynamic correlation map between every two frames of feature images that are adjacent in time includes: Setting a first scale window, and then reducing the first scale window by using a given scale adjustment coefficient to obtain a second scale window; For every two frames of feature images that are adjacent in time, use the first scale window to traverse one of the frames of feature images to obtain a plurality of first window images, and synchronously traverse the other frame of feature images to obtain a plurality of second window images; For each first window image, extract a second window image corresponding to the position of the first window image from the plurality of second window images, then use the second scale window to traverse the first window image to obtain a plurality of third window images, and synchronously traverse the second window image to obtain a plurality of fourth window images; For each third window image, extract a fourth window image corresponding to the position of the third window image from the plurality of fourth window images, and then determine a first dynamic correlation map between the third window image and the fourth window image using a normalized cross-correlation matching algorithm; The weighted average method is used to fuse multiple first dynamic association maps corresponding to each first window image to obtain a second dynamic association map, and then the multiple second dynamic association maps corresponding to the multiple first window images are combined according to position to obtain a final dynamic association map.

2. The breast cancer cell image classification method according to claim 1, characterized in that: The grouping of the plurality of frames of breast cancer cell images comprises: According to the number of images processed by the denoising module at a time, the plurality of frames of breast cancer cell images are divided into a plurality of groups of breast cancer cell images in sequence, and there is only one different frame of breast cancer cell image between each two adjacent groups of breast cancer cell images.

3. The breast cancer cell image classification method according to claim 1, characterized in that: The denoising module is built by the FastDVDNet network framework.

4. The breast cancer cell image classification method according to claim 1, characterized in that: The classification module includes two feature extraction units and a single feature fusion classification unit, each feature extraction unit includes two 3×3×3 convolutional layers connected in sequence and a single 2×2×2 pooling layer, and the single feature fusion classification unit includes two fully connected layers connected in sequence and a single classification layer; Using the two feature extraction units to extract features from each dynamic association graph to obtain corresponding feature information; The single feature fusion classification unit is used to fuse and classify all feature information to obtain a category of the breast cancer cell sample.

5. The breast cancer cell image classification method according to claim 1, characterized in that: The dynamic light scattering imaging device comprises a stage, a laser, a focusing lens, a filter, a single-mode optical fiber, an imaging lens and an imaging sensor, wherein the stage is provided with a microfluidic chip, and the microfluidic chip is provided with a sample chamber; the dynamic light scattering imaging device is controlled to perform imaging processing on the cell suspension to obtain a plurality of time-continuous frames of breast cancer cell images, including: When the sample chamber is filled with the cell suspension, the laser is controlled to emit a laser beam, which is coupled to one end of the single-mode optical fiber after being processed by the focusing objective lens and the filter in sequence, and the other end of the single-mode optical fiber emits bright light to illuminate the sample chamber, thereby stimulating each cell in the cell suspension to continuously move and generate dynamic scattered light, and the dynamic scattered light is projected onto the imaging sensor via the imaging objective lens, thereby controlling the imaging sensor to output a plurality of frames of breast cancer cell images that are continuous in time at a given frame rate.

6. A breast cancer cell image classification system, characterized in that: The system comprises: The first module is used to obtain a cell suspension, which is obtained by preparing and diluting a breast cancer cell sample; The second module is used to control the dynamic light scattering imaging device to perform imaging processing on the cell suspension to obtain a plurality of time-continuous frames of breast cancer cell images; The third module is used to obtain an image classification model, which includes a denoising module, a cell dynamics detection module and a classification module; The fourth module is used to group the plurality of breast cancer cell images and input them into the image classification model, use the denoising module to denoise the most central frame of breast cancer cell images in each group of breast cancer cell images, use the cell dynamic detection module to generate a dynamic correlation map between every two denoised frames of breast cancer cell images that are adjacent in time, and use the classification module to fuse and classify all dynamic correlation maps to obtain the category of the breast cancer cell sample; The cell dynamics detection module includes a convolution unit and a cell dynamics detection unit, and the convolution unit includes a single 7×7 convolution layer and two 5×5 convolution layers connected in sequence; The convolution unit is used to perform multiple convolution sampling on each frame of breast cancer cell image after denoising to obtain a corresponding feature image; the cell dynamic detection unit is used to generate a dynamic correlation map between each two frames of feature images that are adjacent in time; Wherein, the step of using the cell dynamic detection unit to generate a dynamic correlation map between every two frames of feature images that are adjacent in time includes: Setting a first scale window, and then reducing the first scale window by using a given scale adjustment coefficient to obtain a second scale window; For every two frames of feature images that are adjacent in time, use the first scale window to traverse one of the frames of feature images to obtain a plurality of first window images, and synchronously traverse the other frame of feature images to obtain a plurality of second window images; For each first window image, extract a second window image corresponding to the position of the first window image from the plurality of second window images, then use the second scale window to traverse the first window image to obtain a plurality of third window images, and synchronously traverse the second window image to obtain a plurality of fourth window images; For each third window image, extract a fourth window image corresponding to the position of the third window image from the plurality of fourth window images, and then determine a first dynamic correlation map between the third window image and the fourth window image using a normalized cross-correlation matching algorithm; The weighted average method is used to fuse multiple first dynamic association maps corresponding to each first window image to obtain a second dynamic association map, and then the multiple second dynamic association maps corresponding to the multiple first window images are combined according to position to obtain a final dynamic association map.

7. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: The processor executes the computer program to implement the breast cancer cell image classification method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for classifying breast cancer cell images according to any one of claims 1 to 5 is implemented.

Images