Cell slice identification method and system based on digital microscope image

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a cell slice identification method and system based on a digital microscope image, and the method comprises the steps: receiving a first microscope image through a wireless channel; when it is detected that a cell image exists in the first microscope image, extracting cell feature data in the first microscope image by using a first feature extraction model; determining a plurality of predicted positions based on the plurality of pieces of coordinate information; searching a plurality of pieces of cell information based on the plurality of cell types in a preset database, and determining each piece of cell information as a plurality of pieces of prediction content; and based on the plurality of prediction contents, constructing a prediction frame at the corresponding prediction position, and generating a cell display image. The intelligent model is used for performing cell analysis on the microscope image, so that the intelligence and the accuracy of cell recognition are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a cell slice recognition method and system based on digital microscope images. Background Art

[0002] Microscopes are important research tools in biology, medicine and other fields, used to observe and study microscopic structures, and cell image analysis is a key topic in biomedical research.

[0003] At present, microscope images are usually collected through a microscope, and the collected data is transferred to a computer or mobile device through a USB interface for subsequent analysis. However, the system needs to connect the electronic eyepiece to a computer or mobile phone through a complex connection line, which is not only cumbersome to operate, but also prone to connection problems, affecting the overall user experience and stability of the system. In addition, the existing system is compatible with fewer terminals and has a single function, making it difficult to adapt to different application requirements and diverse application scenarios. This limitation limits the universal applicability and flexibility of the system, making it difficult to meet the needs of different fields and uses. In addition, in traditional technologies, cell analysis often relies on manual observation and simple image processing methods, which is not only time-consuming and labor-intensive, but also prone to human errors. Summary of the invention

[0004] The present invention provides a cell slice recognition method and system based on digital microscope images, which utilizes an intelligent model to perform cell analysis on microscope images, thereby improving the intelligence and accuracy of cell recognition.

[0005] In order to solve the above technical problems, the present invention provides a cell slice recognition method based on digital microscope images, comprising:

[0006] receiving a first microscope image;

[0007] When it is detected that there is a cell image in the first microscope image, extracting cell feature data from the first microscope image using a first feature extraction model; wherein the cell feature data includes a plurality of cell labels; each of the cell labels includes coordinate information and a cell type;

[0008] Determining a plurality of predicted positions based on the plurality of coordinate information;

[0009] In a preset database, a plurality of cell information is retrieved based on the plurality of cell types, and each of the cell information is determined as a plurality of prediction contents;

[0010] Based on the plurality of predicted contents, a prediction frame is constructed at a corresponding predicted position to generate a cell display image.

[0011] The present invention receives a first microscope image, determines whether there are cells in the first microscope image, and only extracts features from the first microscope image when cells are detected, thereby reducing image analysis costs; uses a first feature extraction model to extract cell feature data from the first microscope image, thereby improving the intelligence of cell analysis, and uses artificial intelligence instead of manual recognition, thereby effectively improving the accuracy of cell analysis; based on the extracted cell feature data, determines the coordinate information of the cell and the corresponding cell information, and then determines the predicted position and predicted content, constructs multiple prediction frames on the first microscope image, generates a cell display image, and can clearly know the cell information in the cell slice image, thereby improving the readability of the cell analysis result.

[0012] Further, the receiving the first microscope image specifically comprises:

[0013] The first microscope image is received by using a wireless channel; the first microscope image is obtained by performing data preprocessing on an original microscope image acquired in real time.

[0014] The present invention utilizes a wireless channel to receive the first microscope image, thereby realizing wireless transmission of image data and improving the convenience and efficiency of image data transmission;

[0015] Furthermore, the extracting of cell feature data from the first microscope image using the first feature extraction model is specifically as follows:

[0016] Using a first feature extraction model at a plurality of convolutional layers at different depths to respectively extract a plurality of local features of the first microscope image;

[0017] Splicing a plurality of the local features into a pyramid feature map;

[0018] Using an attention mechanism to extract global features on the pyramid feature map;

[0019] The dual detection module is used to perform fusion analysis on several local features and the global features to form cell feature data.

[0020] The present invention utilizes a first feature extraction model to extract cell feature data in a first microscope image, thereby improving the intelligence of image recognition; wherein, a plurality of convolutional layers are set at different depths of the model, so that tiny structures and complex forms in the image can be identified, thereby improving the recognition accuracy of features; an attention mechanism is introduced into the model, so that important features can be better focused on, unnecessary information interference can be reduced, and the accuracy of feature recognition can be improved; a dual detection module is set in the model, so that cells of different forms and sizes can be effectively processed, thereby enhancing the model's detection capability for complex cell forms.

[0021] Furthermore, the dual detection module is used to perform fusion analysis on several local features and the global features to form cell feature data, specifically:

[0022] Using a dual detection module to perform feature fusion on a plurality of the local features and the global features to form a fused feature;

[0023] Performing feature classification on the fused features to obtain several cell types;

[0024] Performing feature regression on the fused features to obtain a number of coordinate information;

[0025] Matching a plurality of the cell types with a plurality of the coordinate information to form a plurality of cell labels;

[0026] A plurality of the cell labels are determined as cell characteristic data.

[0027] The present invention utilizes the dual detection module in the first feature extraction model to fuse feature maps from different scales in the image, classify the fused features, identify the cell type of each cell, and perform boundary regression on the fused features to locate the coordinate position of each cell, thereby forming cell feature data, which can enhance the detection capability of complex cell morphology, thereby significantly improving the accuracy of cell slice recognition.

[0028] Furthermore, the model training process of the first feature extraction model is specifically as follows:

[0029] Using automated data acquisition technology to collect a number of cell slice image data; wherein the cell slice image data includes cell type, cell number and cell position;

[0030] Using a distributed training technology, a preset recognition network is used to train a plurality of cell slice image data to form a feature extraction model;

[0031] When each evaluation index of the feature extraction model reaches the corresponding preset evaluation index threshold, the model parameters of the feature extraction model are determined to form a first feature extraction model.

[0032] The present invention adopts automated data acquisition technology to efficiently collect a large amount of cell slice image data and improve work efficiency; the distributed training technology can decompose the model training task into multiple subtasks, and train in parallel on multiple computing devices, which significantly reduces the training time and can process more training data, thereby improving the accuracy of the model; the preset recognition network is used as the target detection algorithm, which can not only maintain high efficiency when the model is applied, but also further improve the accuracy of target detection and maintain the stability of the model.

[0033] Furthermore, after generating the cell display image, the method further comprises:

[0034] Get the data processing format of the display device;

[0035] Based on the data processing format, converting the cell display image into a format to form device display data;

[0036] The device display data is displayed using the display device.

[0037] After generating a cell display image, the present invention obtains the data processing format of the display device and then performs format conversion on the cell display image, and transmits the device display data after format conversion to the corresponding display device for display. Through format conversion, the present invention can ensure the accuracy and consistency of data, improve the compatibility and interoperability between devices, and ensure smooth exchange of data between different devices.

[0038] Accordingly, the present invention provides a cell slice recognition system based on digital microscope images, which is used to execute the above-mentioned cell slice recognition method based on digital microscope images, comprising: a digital microscope and a processing device;

[0039] The digital microscope is used to collect a first microscope image and transmit the first microscope image to a processing device using a wireless channel;

[0040] The processing device is used to receive a first microscope image; when a cell image is detected in the first microscope image, cell feature data is extracted from the first microscope image using a first feature extraction model; wherein the cell feature data includes a plurality of cell labels; each of the cell labels includes coordinate information and a cell type; a plurality of predicted positions are determined based on the plurality of coordinate information; in a preset database, a plurality of cell information is retrieved based on the plurality of cell types, and each of the cell information is determined as a plurality of predicted contents; based on the plurality of predicted contents, a prediction frame is constructed at a corresponding predicted position to generate a cell display image.

[0041] The cell slice identification system of the present invention comprises a digital microscope and a processing device, and data is transmitted between the two via a wireless channel, which can effectively improve the convenience and efficiency of data transmission, thereby ensuring the stability of the system.

[0042] Furthermore, the digital microscope comprises: an imaging module and a wireless transmission module;

[0043] The imaging module is used to collect a first microscope image;

[0044] The wireless transmission module is used to transmit the first microscope image to a processing device using a wireless channel.

[0045] The digital microscope of the present invention comprises an imaging module and a wireless transmission module, wherein the imaging module is used to collect a first microscope image, and the wireless transmission module is used to transmit the first microscope image to a processing device using a wireless channel. By arranging the wireless transmission module on the digital microscope, the present invention can realize wireless communication between the digital microscope and the processing device, thereby effectively improving the efficiency of data transmission.

[0046] Furthermore, the imaging module comprises: an image sensor and an image processor;

[0047] The image sensor is used to collect original microscope images in real time;

[0048] The image processor is used to perform data preprocessing on the original microscope image to generate a first microscope image.

[0049] The imaging module of the present invention comprises an image sensor and an image processor, wherein the image sensor is used to collect original microscope images in real time, and the image processor is used to perform data preprocessing on the collected original microscope images to form a first microscope image. After collecting the original microscope images, the image processor on the digital microscope can process the original microscope images in real time to improve the image quality.

[0050] Further, the processing device includes a first feature extraction model, and the first feature extraction model is used to extract cell feature data from the first microscope image, specifically:

[0051] Using a first feature extraction model at a plurality of convolutional layers at different depths to respectively extract a plurality of local features of the first microscope image;

[0052] Splicing a plurality of the local features into a pyramid feature map;

[0053] Using an attention mechanism to extract global features on the pyramid feature map;

[0054] The dual detection module is used to perform fusion analysis on several local features and the global features to form cell feature data. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of a flow chart of an embodiment of a cell slice identification method based on digital microscope images provided by the present invention;

[0056] Figure 2 A schematic diagram of the structure of an embodiment of a preset identification network provided by the present invention;

[0057] Figure 3 A schematic structural diagram of an embodiment of a cell slice recognition system based on digital microscope images provided by the present invention;

[0058] Figure 4 A schematic structural diagram of an embodiment of a digital microscope provided by the present invention;

[0059] Figure 5 A schematic structural diagram of another embodiment of a cell slice recognition system based on digital microscope images provided by the present invention;

[0060] Figure 6 The present invention is a schematic structural diagram of another embodiment of the cell slice recognition system based on digital microscope images provided by the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0063] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0064] like Figure 1 FIG. 1 is a flow chart of an embodiment of a cell slice identification method based on a digital microscope image provided by the present invention. The method includes steps 101 to 105, and each step is specifically as follows:

[0065] Step 101: Receive a first microscope image.

[0066] Further, in the embodiment of the present invention, receiving the first microscope image specifically includes:

[0067] The first microscope image is received by using a wireless channel; the first microscope image is obtained by performing data preprocessing on an original microscope image acquired in real time.

[0068] In actual application, the image data collected by the microscope is often transmitted to a computer or mobile device for subsequent analysis through a USB interface. This method requires the connection between the electronic eyepiece of the microscope and the computer or mobile phone to be connected through a complex connection line, which affects the accuracy of data transmission. Therefore, the present invention improves the transmission channel between the microscope and the processing device, and uses a wireless channel to receive the first microscope image, which can effectively improve the stability of image acquisition and achieve a smoother data acquisition and analysis process.

[0069] In an embodiment of the present invention, a high-resolution image sensor (such as a CCD or CMOS sensor) is integrated in the microscope eyepiece to capture the original microscope image under the microscope in real time. The collected original microscope image is subjected to preliminary data preprocessing such as denoising, correction and compression to generate a first microscope image, which can improve the quality of the original microscope image and reduce the amount of transmitted data. After the first microscope image is generated, a processing device (such as a computer or a mobile phone) receives the first microscope image using a wireless channel. Among them, the wireless channel supports a high data transmission rate (such as 802.11n / ac / ax), which can ensure the real-time nature of data transmission.

[0070] The present invention uses an image sensor integrated on a microscope to collect original microscope images, solves the complex connection problem of an electronic eyepiece, and improves the stability of image collection; and uses a wireless channel to receive a first microscope image transmitted by the microscope, which can improve the convenience of data transmission.

[0071] Step 102: When a cell image is detected in the first microscope image, cell feature data is extracted from the first microscope image using a first feature extraction model; wherein the cell feature data includes a plurality of cell labels; each of the cell labels includes coordinate information and a cell type.

[0072] In an embodiment of the present invention, after receiving the first microscope image, the trained cell recognition model is used to perform cell recognition on the first microscope image to determine whether there are cells in the first microscope image. Only when cells are detected, feature extraction is performed on the first microscope image, thereby effectively reducing the image analysis cost.

[0073] Further, in an embodiment of the present invention, the first feature extraction model is used to extract cell feature data from the first microscope image, specifically:

[0074] Using a first feature extraction model at a plurality of convolutional layers at different depths to respectively extract a plurality of local features of the first microscope image;

[0075] Splicing a plurality of the local features into a pyramid feature map;

[0076] Using an attention mechanism to extract global features on the pyramid feature map;

[0077] The dual detection module is used to perform fusion analysis on several local features and the global features to form cell feature data.

[0078] In an embodiment of the present invention, for a first microscope image in which a cell image exists, a first feature extraction model is used to extract features. Specifically, the first feature extraction model is used to capture features at different levels using multiple convolution kernels of different sizes and channel numbers at different depths, so that the first feature extraction model can extract various features in the first microscope image, such as cell boundaries, shapes, textures, etc. The different local features extracted using convolution kernels of different sizes are spliced ​​to form a pyramid-shaped feature map. The Pyramid Split Attention (PSA) attention mechanism is then introduced, and the attention mechanism is applied to the above-mentioned pyramid feature map to extract the global features of the first microscope image to obtain richer feature information. Finally, the dual detection module in the first feature extraction model is used to perform a fusion analysis of local features and global features, which solves the problem of insufficient feature extraction, effectively captures the tiny features of cells, and ensures the recognition accuracy of different types of cells.

[0079] The present invention utilizes a first feature extraction model to extract cell feature data in a first microscope image, thereby improving the intelligence of image recognition; wherein, a plurality of convolutional layers are set at different depths of the model, so that tiny structures and complex forms in the image can be identified, thereby improving the recognition accuracy of features; an attention mechanism is introduced into the model, so that important features can be better focused on, unnecessary information interference can be reduced, and the accuracy of feature recognition can be improved; a dual detection module is set in the model, so that cells of different forms and sizes can be effectively processed, thereby enhancing the model's detection capability for complex cell forms.

[0080] Furthermore, in the embodiment of the present invention, a dual detection module is used to perform fusion analysis on several local features and the global features to form cell feature data, specifically:

[0081] Using a dual detection module to perform feature fusion on a plurality of the local features and the global features to form a fused feature;

[0082] Performing feature classification on the fused features to obtain several cell types;

[0083] Performing feature regression on the fused features to obtain a number of coordinate information;

[0084] Matching a plurality of the cell types with a plurality of the coordinate information to form a plurality of cell labels;

[0085] A plurality of the cell labels are determined as cell characteristic data.

[0086] In an embodiment of the present invention, by using the dual detection module in the first feature extraction model, features from different scales can be fused to integrate multi-scale feature information, i.e., fused features; by classifying the fused features, the cell type of each cell on the first microscope image can be identified; by performing boundary regression on the fused features, each cell can be located to form the coordinate information of each cell. The present invention uses a dual detection module to effectively detect small targets in the first microscope image, thereby enhancing the detection capability of complex cell morphology.

[0087] The present invention utilizes the dual detection module in the first feature extraction model to fuse feature maps from different scales in the image, classify the fused features, identify the cell type of each cell, and perform boundary regression on the fused features to locate the coordinate position of each cell, thereby forming cell feature data, which can enhance the detection capability of complex cell morphology, thereby significantly improving the accuracy of cell slice recognition.

[0088] Furthermore, in the embodiment of the present invention, the first feature extraction model, its model training process is specifically as follows:

[0089] Using automated data acquisition technology to collect a number of cell slice image data; wherein the cell slice image data includes cell type, cell number and cell position;

[0090] Using a distributed training technology, a preset recognition network is used to train a plurality of cell slice image data to form a feature extraction model;

[0091] When each evaluation index of the feature extraction model reaches the corresponding preset evaluation index threshold, the model parameters of the feature extraction model are determined to form a first feature extraction model.

[0092] In practical applications, common microscope image cell slice recognition systems mainly rely on deep learning models, such as convolutional neural networks (CNN) and recurrent neural networks (RNN). Among these models, networks such as YOLO (You Only Look Once) and Faster R-CNN are often used for training. However, current deep learning models have the defect of low feature extraction efficiency. Many existing models use traditional backbone networks, such as ResNet and VGG, to extract image features. These networks may lose important spatial information when extracting features at different levels, resulting in a decrease in recognition accuracy. Existing detection technologies often use a single-scale feature extraction method, which is easily limited when faced with cell images of different sizes and morphologies, and cannot cope with the diversity of cells, thereby reducing the detection performance of the model.

[0093] In an embodiment of the present invention, a preset recognition network is used to construct a first feature extraction model. First, an automated data acquisition technology is used to collect multiple cell slice image data as model training data. The automated data acquisition technology can collect a large amount of cell slice image data in a short time, effectively improving work efficiency; then the distributed training technology is used to train the model, which can decompose the model training task into multiple subtasks and perform training in parallel on multiple computing devices, significantly reducing the training time, and can process more training data, thereby improving the accuracy of the model. By using a preset recognition network as a target detection algorithm, high efficiency can be maintained and the accuracy of target detection can be further improved, and the stability of the model can be maintained.

[0094] As an example of an embodiment of the present invention, see Figure 2, is a structural diagram of an embodiment of the preset recognition network provided by the present invention. YOLOv9 is an open source deep learning algorithm, and the improved YOLOv9 network can be used as the preset recognition network. The improved YOLOv9 network consists of three parts: Backbone, Neck and Head. Among them, Backbone is the backbone structure of the network, which is used to extract features. Backbone uses convolution kernels (Conv) of different sizes and channel numbers at different depths to capture features at different levels, so that the network can learn to extract various features in microscope images, such as cell boundaries, shapes, textures, etc. A deep feature map is established through multi-layer convolution and cell feature extraction modules, and average convolution (AConv) is used to improve the expression ability of the feature map by reducing the spatial resolution, and adapt to multi-scale feature transformation. Introducing the ELAN1 module and the RepNCSPELAN4 module in Backbone can solve the problem of insufficient feature extraction through local perception and deep feature fusion, effectively capture the tiny features of cells, and improve the real-time performance of the network in microscope image recognition. Neck is the neck structure of the network, which is used to further process the features extracted by the backbone network to achieve specific tasks. Multi-scale feature fusion is achieved through upsampling and concatenation operations, which helps to improve the accuracy of cell positioning and recognition; and spatial pyramid pooling combined with ELAN blocks (SPPELAN) is used to extract multi-scale features, ensuring the recognition accuracy of different types of cells. The PSA attention mechanism is introduced in Neck to introduce position information, enhance the ability of long-range dependence and feature fusion, and thus improve the accuracy of cell slice recognition and detection. Head is the head structure of the network, which is equipped with a dual detection module. It can fuse feature maps from different scales, integrate multi-scale feature information, and classify and regress the fused features to identify cell types and locate cell positions, enhancing the detection ability of complex cell morphology and having good performance for small target detection in microscope images.

[0095] In an embodiment of the present invention, when training a feature extraction model, the performance of the model can be comprehensively evaluated by testing evaluation indicators such as the model's accuracy, recall rate, and mAP50. When each evaluation indicator reaches a preset evaluation indicator threshold, it can be determined that the feature extraction model training is complete, and the current model parameters are determined to form a first feature extraction model.

[0096] As an example of an embodiment of the present invention, the precision, recall and mAP50 can be calculated by the following formula:

[0097]

[0098] Where TP is the number of pixels predicted as cells; FN is the number of pixels predicted as background pixels; FP is the number of pixels misdetected as cells; n is the accuracy at the nth threshold; R n is the recall rate at the nth threshold; R n and R n-1 are two adjacent but not equal intervals.

[0099] Step 103: Determine a plurality of predicted positions based on the plurality of coordinate information.

[0100] Step 104: In a preset database, a plurality of cell information is retrieved based on the plurality of cell types, and each of the cell information is determined as a plurality of prediction contents.

[0101] Step 105: Based on the plurality of predicted contents, a prediction frame is constructed at a corresponding predicted position to generate a cell display image.

[0102] In an embodiment of the present invention, the cell feature data obtained using the first feature extraction model includes multiple cell labels, each cell label includes the coordinate information and cell type of each corresponding cell. Based on the coordinate information of each cell, multiple predicted positions on the first microscope image can be obtained; based on the cell type of each cell, the cell information corresponding to the cell type can be retrieved in a preset database, including the cell name, cell picture and cell introduction. After obtaining the predicted position and corresponding predicted content of each cell in the first microscope image, a prediction frame is constructed at each predicted position, and the corresponding predicted content is written into the prediction frame to generate a cell display image.

[0103] Furthermore, in the embodiment of the present invention, after generating the cell display image, the method further includes:

[0104] Get the data processing format of the display device;

[0105] Based on the data processing format, converting the cell display image into a format to form device display data;

[0106] The device display data is displayed using the display device.

[0107] In an embodiment of the present invention, after the cell display image is generated, the cell display image is converted according to the data processing format (such as JSON format) of the display device so that the display device can read the data. After acquiring the cell display image, the display device can output the cell display image in a graphical manner, display the image of the cell slice and related information on the terminal screen, and generate a detailed report based on the cell display image, including analysis and detection information of the cell image.

[0108] After generating a cell display image, the present invention obtains the data processing format of the display device and then performs format conversion on the cell display image, and transmits the device display data after format conversion to the corresponding display device for display. Through format conversion, the present invention can ensure the accuracy and consistency of data, improve the compatibility and interoperability between devices, and ensure smooth exchange of data between different devices.

[0109] See also Figure 3 , is a schematic diagram of the structure of an embodiment of a cell slice recognition system based on a digital microscope image provided by the present invention, the system is used to execute the above-mentioned cell slice recognition method based on a digital microscope image, including a digital microscope and a processing device;

[0110] The digital microscope is used to collect a first microscope image and transmit the first microscope image to a processing device using a wireless channel;

[0111] The processing device is used to receive a first microscope image; when a cell image is detected in the first microscope image, cell feature data is extracted from the first microscope image using a first feature extraction model; wherein the cell feature data includes a plurality of cell labels; each of the cell labels includes coordinate information and a cell type; a plurality of predicted positions are determined based on the plurality of coordinate information; in a preset database, a plurality of cell information is retrieved based on the plurality of cell types, and each of the cell information is determined as a plurality of predicted contents; based on the plurality of predicted contents, a prediction frame is constructed at a corresponding predicted position to generate a cell display image.

[0112] In the embodiment of the present invention, the digital microscope is used to collect the first microscope image, and the processing device is used to analyze and process the first microscope image to generate a cell display image. The digital microscope and the processing device transmit data via a wireless channel.

[0113] Further, in an embodiment of the present invention, the digital microscope includes: an imaging module and a wireless transmission module;

[0114] The imaging module is used to collect a first microscope image;

[0115] The wireless transmission module is used to transmit the first microscope image to the processing device using a wireless channel.

[0116] In the embodiment of the present invention, the imaging module is responsible for collecting the first microscope image; the wireless transmission module is responsible for converting the first microscope image into digital signal data, and transmitting the digital signal data to the processing device through the wireless channel, ensuring the integrity and rapid transmission of the image data for subsequent processing. Further, when the processing device receives the digital signal data sent by the wireless transmission module, it converts the digital signal data into image data, and performs subsequent analysis and processing on the image data.

[0117] As an example of an embodiment of the present invention, see Figure 4 , is a schematic diagram of the structure of an embodiment of the digital microscope provided by the present invention. The digital microscope includes a lower light source dimming knob 1, an upper light source dimming knob 2, a power socket 3, a focusing hand wheel 4, a mirror arm 5, an upper light source lamp 6, an imaging module 7, a wireless transmission module 8, an LCD display screen 9, a converter 10, an objective lens 11, a biological slice 12, an object stage 13, a condenser 14, a lower light source lamp 15 and a base 16. Among them, the imaging module 7 is integrated on the eyepiece of the digital microscope and is used to collect the first microscope image in real time. The wireless transmission module 8 is used to transmit the first microscope image to the processing device. See Figure 5 , is a schematic diagram of the structure of another embodiment of a cell slice identification system based on a digital microscope image provided by the present invention, the system includes a digital microscope and a processing device 17, and data is transmitted between the two via a wireless channel (such as a WIFI channel). The present invention can realize wireless communication between the digital microscope and the processing device by setting a wireless transmission module on the digital microscope, effectively improving the efficiency of data transmission.

[0118] Furthermore, in an embodiment of the present invention, the imaging module includes: an image sensor and an image processor;

[0119] The image sensor is used to collect raw microscope images in real time;

[0120] The image processor is used to perform data preprocessing on the original microscope image to generate a first microscope image.

[0121] In an embodiment of the present invention, the imaging module includes an image sensor (such as a CCD or CMOS sensor) and an image processor. The image sensor is used to capture the microscope image under the digital microscope in real time. The image processor is connected to the optical system of the digital microscope, performs preliminary denoising, correction and compression on the captured microscope image to improve the image quality and reduce the amount of transmitted data, and converts the microscope image into a digital signal to ensure the data integrity in data transmission. After the original microscope image is collected, the image processor on the digital microscope can process the original microscope image in real time to improve the image quality.

[0122] Further, in an embodiment of the present invention, the processing device includes a first feature extraction model, and the first feature extraction model is used to extract cell feature data from the first microscope image, specifically:

[0123] Using a first feature extraction model at a plurality of convolutional layers at different depths to respectively extract a plurality of local features of the first microscope image;

[0124] Splicing a plurality of the local features into a pyramid feature map;

[0125] Using an attention mechanism to extract global features on the pyramid feature map;

[0126] The dual detection module is used to perform fusion analysis on several local features and the global features to form cell feature data.

[0127] The processing device of the present invention includes a computer device or an edge device. When a computer device is used as a processing device, the computer device is wirelessly connected to a digital microscope using the Internet. The computer device has high processing power, centralized storage and management, scalability, and strong learning ability, and can process complex and large amounts of data. The computer device is used to identify cell slices in microscope images, which can further improve the accuracy of data analysis. When an edge device is used as a processing device, a lightweight algorithm model is deployed on the edge device in view of the low computing power characteristics of the edge device. The connection between the edge device and the digital microscope does not rely on the Internet. The edge device has the characteristics of low latency, bandwidth saving, low cost, and offline capabilities, and can perform real-time, localized data processing.

[0128] In the embodiment of the present invention, the edge device receives the microscope image from the digital microscope, loads the trained cell slice recognition model, infers the pre-processed microscope image, recognizes the cell slice information (such as cell name, picture and related introduction) in real time, and returns the recognition result to the application interface in real time. In addition, the mobile terminal of the edge device can also realize the voice broadcast function, and supports Chinese, English, German and other languages, providing an excellent tool for scientific research.

[0129] The cell slice recognition system of the present invention supports a variety of application scenarios. It can develop various application forms such as computer APP and mobile phone APP, combined with the advantages of Wi-Fi with high-speed data transmission, wide coverage, no wiring, secure WPA2 or WPA3 and easy expansion, to meet different application needs and achieve efficient compatibility of deep learning models on computer and mobile phones for more flexible use.

[0130] As an example of an embodiment of the present invention, see Figure 6, is a structural diagram of another embodiment of the cell slice recognition system based on digital microscope images provided by the present invention. C-end users can call the wireless transmission module of the digital microscope through the APP to identify the image data obtained by the APP, and display the results on the page, so that users can view, analyze and study the results. The business widgets in the APP include stateless (StatelessWidget) and stateful (StatefulWidget). The stateless (StatelessWidget) belongs to a static page, which is only responsible for UI display and does not contain logic; the stateful (StatefulWidget) includes UI, data and logic. The UI components in the APP include buttons Button, navigation Navbar, pictures Image and layout Layout, etc. The basic components in the APP are mainly used for function implementation, such as using Pytorch to load deep learning models for recognition, using Video Player to play videos, using FFmpeg to load camera video streams, using Device Info to obtain APP environment information, and using Image Picker to select pictures, etc. The CLI in the APP is a command line tool. Among them, GetCLI is used to create project structure, components, internationalization, and manage dependencies, and FlutterCLI is used to create, test, package, and deploy Flutter applications. The APP uses the Pytorch plug-in to load the cell detection model, calls the microscope image transmitted by the wireless transmission module for detection, extracts the cell feature data in the microscope image, uses the cell feature data to determine the location information and cell type of each cell, and retrieves cell-related information (name, picture, introduction, etc.) from the preset database based on the cell type, and then builds a prediction box on the microscope image. The technical support in the APP mainly includes instructions for use and update precautions for the plug-in package. In the APP, the cell display image is packaged in the format of apk or ipa and distributed to the Android or iOS platform. After obtaining the cell display image, the platform outputs it in a graphical manner, that is, displays the image of the cell slice and related information on the terminal screen, and generates a detailed report based on the results, including analysis and detection information of the cell image.

[0131] In summary, the embodiments of the present invention provide a cell slice recognition method and system based on digital microscope images, which utilizes a wireless channel to receive a first microscope image, realizes wireless transmission of image data, and improves the convenience and efficiency of image data transmission; determines whether there are cells in the first microscope image, and only extracts features of the first microscope image when cells are detected, thereby reducing the image analysis cost; utilizes a first feature extraction model to extract cell feature data in the first microscope image, thereby improving the intelligence of cell analysis, and uses artificial intelligence instead of manual recognition, thereby effectively improving the accuracy of cell analysis; based on the extracted cell feature data, determines the coordinate information of the cell and the corresponding cell information, and then determines the predicted position and predicted content, constructs multiple prediction boxes on the first microscope image, generates a cell display image, and can clearly know the cell information in the cell slice image, thereby improving the readability of the cell analysis results.

[0132] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A cell slice recognition method based on digital microscope images, characterized in that: include: receiving a first microscope image; When it is detected that there is a cell image in the first microscope image, extracting cell feature data from the first microscope image using a first feature extraction model; wherein the cell feature data includes a plurality of cell labels; each of the cell labels includes coordinate information and a cell type; Determining a plurality of predicted positions based on the plurality of coordinate information; In a preset database, a plurality of cell information is retrieved based on the plurality of cell types, and each of the cell information is determined as a plurality of prediction contents; Based on the plurality of predicted contents, a prediction frame is constructed at a corresponding predicted position to generate a cell display image.

2. The cell slice recognition method based on digital microscope images according to claim 1, characterized in that: The receiving the first microscope image specifically comprises: The first microscope image is received by using a wireless channel; the first microscope image is obtained by performing data preprocessing on an original microscope image acquired in real time.

3. The cell slice identification method based on digital microscope images according to claim 1, characterized in that: The extracting of cell feature data from the first microscope image using the first feature extraction model is specifically: Using a first feature extraction model at a plurality of convolutional layers at different depths to respectively extract a plurality of local features of the first microscope image; Splicing a plurality of the local features into a pyramid feature map; Using an attention mechanism to extract global features on the pyramid feature map; The dual detection module is used to perform fusion analysis on several local features and the global features to form cell feature data.

4. The cell slice identification method based on digital microscope images according to claim 3, characterized in that: The dual detection module is used to perform fusion analysis on a plurality of local features and the global features to form cell feature data, specifically: Using a dual detection module to perform feature fusion on a plurality of the local features and the global features to form a fused feature; Performing feature classification on the fused features to obtain several cell types; Performing feature regression on the fused features to obtain a number of coordinate information; Matching a plurality of the cell types with a plurality of the coordinate information to form a plurality of cell labels; A plurality of the cell labels are determined as cell characteristic data.

5. The cell slice identification method based on digital microscope images according to claim 1, characterized in that: The first feature extraction model, its model training process is specifically as follows: Using automated data acquisition technology to collect a number of cell slice image data; wherein the cell slice image data includes cell type, cell number and cell position; Using a distributed training technology, a preset recognition network is used to train a plurality of cell slice image data to form a feature extraction model; When each evaluation index of the feature extraction model reaches the corresponding preset evaluation index threshold, the model parameters of the feature extraction model are determined to form a first feature extraction model.

6. The cell slice identification method based on digital microscope images according to claim 1, characterized in that: After generating the cell display image, the method further comprises: Get the data processing format of the display device; Based on the data processing format, converting the cell display image into a format to form device display data; The device display data is displayed using the display device.

7. A cell slice recognition system based on digital microscope images, characterized in that: Used to perform the cell slice identification method based on digital microscope images as described in any one of claims 1 to 6, comprising a digital microscope and a processing device; The digital microscope is used to collect a first microscope image and transmit the first microscope image to a processing device; The processing device is used to receive a first microscope image; when a cell image is detected in the first microscope image, cell feature data is extracted from the first microscope image using a first feature extraction model; wherein the cell feature data includes a plurality of cell labels; each of the cell labels includes coordinate information and a cell type; a plurality of predicted positions are determined based on the plurality of coordinate information; in a preset database, a plurality of cell information is retrieved based on the plurality of cell types, and each of the cell information is determined as a plurality of predicted contents; based on the plurality of predicted contents, a prediction frame is constructed at a corresponding predicted position to generate a cell display image.

8. The cell slice recognition system based on digital microscope images according to claim 7, characterized in that: The digital microscope comprises: an imaging module and a wireless transmission module; The imaging module is used to collect a first microscope image; The wireless transmission module is used to transmit the first microscope image to a processing device using a wireless channel.

9. The cell slice recognition system based on digital microscope images according to claim 8, characterized in that: The imaging module comprises: an image sensor and an image processor; The image sensor is used to collect original microscope images in real time; The image processor is used to perform data preprocessing on the original microscope image to generate a first microscope image.

10. The cell slice recognition system based on digital microscope images according to claim 7, characterized in that: The processing device includes a first feature extraction model, and the first feature extraction model is used to extract cell feature data from the first microscope image, specifically: Using a first feature extraction model at a plurality of convolutional layers at different depths to respectively extract a plurality of local features of the first microscope image; Splicing a plurality of the local features into a pyramid feature map; Using an attention mechanism to extract global features on the pyramid feature map; The dual detection module is used to perform fusion analysis on several local features and the global features to form cell feature data.

Images