Methods, devices, and equipment for sperm identification based on microscope images

Abstract

Embodiments of this application provide a method, apparatus, device, and computer-readable storage device for sperm identification based on microscope images. The method includes acquiring a microscope image; segmenting the microscope image to obtain all cell instances; classifying the cell instances to obtain the shape and position features of each cell; and inputting the shape and position features of each cell into a trained decision tree classification model to identify normal sperm cells. In this way, normal sperm can be accurately and quickly located from a microscope image.

Description

Technical Field

[0001] Embodiments of this application relate to the field of sperm identification, and more particularly to sperm identification methods, apparatus, devices, and computer-readable storage devices based on microscopic images. Background Technology

[0002] In current technologies, sperm morphology analysis is performed manually under a high-powered microscope. However, manual microscopy has many drawbacks, such as high subjectivity, inconsistent conclusions from different operators, and inconsistencies in results obtained by the same operator at different times.

[0003] Therefore, how to quickly identify normal sperm from microscopic images and distinguish them from other cells such as spermatogonia, stertoly cells, and red and white blood cells that may be present during surgery is a problem that urgently needs to be solved. Summary of the Invention

[0004] According to embodiments of this application, a sperm identification scheme based on microscope images is provided.

[0005] In a first aspect of this application, a method for sperm identification based on microscopic images is provided. The method includes:

[0006] Acquire microscope images;

[0007] The microscope image was segmented to obtain all cell instances;

[0008] The cell instances are classified to obtain the shape and location characteristics of each cell;

[0009] The shape and location features of each cell are input into a trained decision tree classification model to obtain a sperm image.

[0010] Furthermore, the step of segmenting the microscope image to obtain all cell instances includes:

[0011] Extract multi-level features from the microscope image;

[0012] The instance segmentation module, based on a multilayer perceptron, maps the multi-level features in the microscope image to the semantic segmentation space to obtain the instance segmentation result.

[0013] Furthermore, the extraction of multi-level features from the microscope image includes:

[0014] The microscope image is upsampled layer by layer, and the features of the intermediate layers are fused to obtain multi-level features including spatial and semantic information.

[0015] Furthermore, the shape and position characteristics of each cell include area, equivalent diameter, eccentricity, and centroid offset.

[0016] Furthermore, the decision tree classification model is trained in the following manner:

[0017] Construct a sample set; the sample set includes non-sperm, inactive sperm, or motile sperm;

[0018] Predict the class of each cell in the sample set;

[0019] Based on the difference between the predicted category and the true category, the decision threshold of each node is adjusted. When the decision threshold meets the preset threshold, the training of the decision tree classification model is completed.

[0020] In a second aspect of this application, a sperm identification device based on microscope images is provided. The device includes:

[0021] The acquisition module is used to acquire microscope images;

[0022] The segmentation module is used to segment the microscope image into instances to obtain all cell instances;

[0023] The classification module is used to classify the cell instances and obtain the shape and position features of each cell;

[0024] The determination module is used to input the shape and position features of each cell into a trained decision tree classification model to obtain a sperm image.

[0025] In a third aspect of this application, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0026] In a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method according to the first aspect of this application.

[0027] The sperm identification method based on microscope images provided in this application involves acquiring a microscope image; segmenting the microscope image to obtain all cell instances; classifying the cell instances to obtain the shape and position features of each cell; and inputting the shape and position features of each cell into a trained decision tree classification model to obtain a sperm image. This method can accurately and quickly identify real sperm from a microscope image.

[0028] It should be understood that the description in the Summary Section is not intended to limit the key or essential features of the embodiments of this application, nor is it intended to restrict the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0029] The above and other features, advantages, and aspects of the embodiments of this application will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0030] Figure 1 A system architecture diagram involving the method provided in the embodiments of this application is shown.

[0031] Figure 2 A flowchart of a sperm identification method based on microscope images according to an embodiment of this application is shown;

[0032] Figure 3 A schematic diagram of the algorithm architecture according to an embodiment of this application is shown;

[0033] Figure 4 A block diagram of a sperm identification device based on microscope images according to an embodiment of this application is shown;

[0034] Figure 5 A schematic diagram of a terminal device or server suitable for implementing embodiments of this application is shown. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0036] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0037] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the microscopic image-based sperm identification method or microscopic image-based sperm identification device of this application can be applied.

[0038] like Figure 1As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0039] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as model training applications, video recognition applications, web browser applications, social platform software, etc.

[0040] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services) or as a single software program or software module. No specific limitations are imposed here.

[0041] When terminals 101, 102, and 103 are hardware devices, video capture devices can also be installed on them. These video capture devices can be various devices capable of capturing video, such as cameras, sensors, etc. Users can use the video capture devices on terminals 101, 102, and 103 to capture video.

[0042] Server 105 can be a server that provides various services, such as a backend server for processing data displayed on terminal devices 101, 102, and 103. The backend server can analyze and process the received data and can feed back the processing results (such as recognition results) to the terminal devices.

[0043] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0044] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. In particular, if the target data does not need to be obtained remotely, the above system architecture may exclude the network and include only terminal devices or servers.

[0045] like Figure 2 The diagram shown is a flowchart of a sperm identification method based on microscope images, according to an embodiment of this application. Figure 2 As can be seen from the image, the sperm identification method based on microscope images in this embodiment includes the following steps:

[0046] S210, acquire microscope images.

[0047] In this embodiment, the execution subject for the sperm identification method based on microscope images (e.g.) Figure 1 The server shown can acquire microscope images via wired or wireless connection.

[0048] Furthermore, the aforementioned executing entity can acquire electronic devices (e.g., those connected to it in communication) Figure 1 The microscope images sent by the terminal device shown can also be microscope images pre-stored locally.

[0049] Typically, the microscope images include sperm cells (active and inactive), spermatogonia, stertoly cells, and red blood cells and white blood cells that may be present during the procedure.

[0050] S220, Perform instance segmentation on the microscope image to obtain all cell instances.

[0051] In some embodiments, a coding module based on a convolutional neural network is used to extract multi-level features from microscope images, wherein deeper features have more semantic information and less spatial information.

[0052] Furthermore, by using a decoding module built on a convolutional neural network, starting from the deepest features, the features of the intermediate layers are upsampled and fused layer by layer to obtain a feature that combines spatial and semantic information.

[0053] In some embodiments, an instance segmentation module is constructed using a multilayer perceptron. The instance segmentation module maps the features generated by the decoding module to the semantic segmentation space, and then distinguishes different cells based on connected components to obtain the instance segmentation result. At the same time, the region of each cell instance is cropped out for use in the subsequent step of distinguishing sperm cells.

[0054] S230, classify the cell instances to obtain the shape and location characteristics of each cell.

[0055] In some embodiments, the cropped cell regions are individually input into the sperm cell classification module to distinguish whether they are sperm.

[0056] Among them, the sperm cell classification module uses the feature calculation module to calculate the shape features of each cell based on the segmented cell regions, including area, equivalent diameter, eccentricity and centroid offset;

[0057] The area is used to describe the number of pixels occupied by cells in the image; it is determined by counting the number of pixels in the cell region.

[0058] The equivalent diameter and eccentricity are used to describe the eccentricity and diameter of an ellipse that has the same second moment as the region. The eccentricity is the ratio of the focal distance (distance between the focal points) to the length of the principal axis, and the value range is [0,1). When the value is 0, it is a circle.

[0059] The centroid offset is used to describe the amount of offset of the cell centroid between two temporally adjacent images It and It+τ (τ is a selected time interval).

[0060] Furthermore, the area, equivalent diameter, eccentricity, and centroid offset are compared with the values ​​of a standard sperm cell to determine the sperm cells within the trimmed cell region. For example, if the eccentricity is less than 0.8, the cells are considered non-sperm cells.

[0061] In some embodiments, the shape and location of sperm cells are determined based on the shape characteristics of each cell.

[0062] S240, the shape and position features of each cell are input into the trained decision tree classification model to obtain a sperm image.

[0063] In some embodiments, a decision tree model is used to identify normal sperm cells among the sperm cells.

[0064] The decision tree classification model is trained in the following way:

[0065] Construct a sample set; the sample set includes non-sperm, inactive sperm, or motile sperm;

[0066] Predict the class of each cell in the sample set;

[0067] Based on the difference between the predicted category and the true category, the decision threshold of each node is adjusted. When the decision threshold meets the preset threshold, the training of the decision tree classification model is completed.

[0068] In some embodiments, the shape and position features of each sperm cell are input into a trained decision tree classification model, and candidate sperm cells are selected based on the eccentricity (e.g., eccentricity ≥ 0.8).

[0069] Sperm candidates were further screened based on area and equivalent diameter characteristics to obtain high-quality sperm cells (e.g., area ≤ 20 μm). 2 (Equivalent diameter ≥ 4μm);

[0070] The high-quality sperm cells are further distinguished based on the centroid offset, identifying motile and inactive sperm cells (the centroid offset of motile sperm is ≥1μm).

[0071] In summary, we obtain active (normal) sperm cells.

[0072] According to the embodiments of this disclosure, the following technical effects are achieved:

[0073] refer to Figure 3 The method disclosed herein allows for the rapid and effective identification of normal sperm cells from microscopic images.

[0074] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0075] The above is an introduction to the method embodiments. The following describes the solution described in this application through device embodiments.

[0076] Figure 4 A block diagram of a sperm identification device 400 based on a microscope image according to an embodiment of this application is shown as follows. Figure 4 As shown, the device 400 includes:

[0077] Acquisition module 410 is used to acquire microscope images;

[0078] Segmentation module 420 is used to segment the microscope image into instances to obtain all cell instances;

[0079] The classification module 430 is used to classify the cell instances and obtain the shape and position features of each cell;

[0080] The determination module 440 is used to input the shape and position features of each cell into a trained decision tree classification model to obtain a sperm image.

[0081] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0082] Figure 5 A schematic diagram of a terminal device or server suitable for implementing embodiments of this application is shown.

[0083] like Figure 5 As shown, the terminal device or server 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0084] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0085] Specifically, according to embodiments of this application, the above method flow steps can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined in the system of this application.

[0086] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0088] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be located in a processor. The names of these units or modules do not, in certain circumstances, constitute a limitation on the unit or module itself.

[0089] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the methods described in this application.

[0090] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing application concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions claimed in this application.

Claims

1. A sperm identification method based on microscope images, characterized in that, include: Acquire microscope images; The microscope image was segmented to obtain all cell instances; The cell instances are classified to obtain the shape and location characteristics of each cell; The shape and location features of each cell are input into a trained decision tree classification model to identify normal sperm cells; The step of segmenting the microscope image to obtain all cell instances includes: Extract multi-level features from the microscope image; The instance segmentation module, based on a multilayer perceptron, maps the multi-level features in the microscope image to the semantic segmentation space to obtain the instance segmentation result. The extraction of multi-level features from the microscope image includes: The microscope image is upsampled layer by layer, and the features of the intermediate layers are fused to obtain multi-level features including spatial and semantic information; The shape and position features of each cell include area, equivalent diameter, eccentricity, and centroid offset; the shape and position features of each sperm cell are input into the classification model, and candidate sperm cells are selected by eccentricity. The candidate sperm cells were further screened based on area and equivalent diameter characteristics to obtain high-quality sperm cells. The high-quality sperm cells are further differentiated based on the centroid offset, identifying active and inactive sperm cells.

2. The method according to claim 1, characterized in that, The decision tree classification model is trained in the following way: Construct a sample set; the sample set includes non-sperm, inactive sperm, or motile sperm; Predict the class of each cell in the sample set; Based on the difference between the predicted category and the true category, the decision threshold of each node is adjusted. When the decision threshold meets the preset threshold, the training of the decision tree classification model is completed.

3. A sperm identification device based on microscope images, used to implement the method of claim 1, characterized in that, include: The acquisition module is used to acquire microscope images; The segmentation module is used to segment the microscope image into instances to obtain all cell instances; The classification module is used to classify the cell instances and obtain the shape and position features of each cell; The determination module is used to input the shape and position features of each cell into a trained decision tree classification model to obtain a sperm image.

4. The apparatus according to claim 3, characterized in that, The segmentation module is specifically used for: Extract multi-level features from the microscope image; The instance segmentation module, based on a multilayer perceptron, maps the multi-level features in the microscope image to the semantic segmentation space to obtain the instance segmentation result.

5. The apparatus according to claim 4, characterized in that, The extraction of multi-level features from the microscope image includes: The microscope image is upsampled layer by layer, and the features of the intermediate layers are fused to obtain multi-level features including spatial and semantic information.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 2.

7. A computer-readable storage device having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1 to 2.

Images