Non-invasive sperm morphology evaluation method and system based on multi-target instance analysis and knowledge driving

Abstract

The invention relates to the technical field of medical image processing, in particular to a non-invasive sperm morphology evaluation method and system based on multi-target instance analysis and knowledge driving, and the method comprises the steps: obtaining a sperm microscopic image data set; constructing a multi-target instance analysis model, and performing iterative training on the multi-target instance analysis model through the sperm microscopic image data set to obtain a trained multi-target instance analysis model; obtaining a target analysis result of the sperm microscopic image to be evaluated based on the trained multi-target instance analysis model; and based on the target analysis result, obtaining morphological characteristics of the corresponding part, and taking the morphological characteristics as input of a sperm knowledge graph constructed in advance to obtain a sperm morphological evaluation result. According to the method, by combining technologies such as multi-target instance analysis and the knowledge graph, accurate and efficient evaluation of sperm morphology is realized, and key data support is provided for applications such as euploid prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a non-invasive sperm morphology evaluation method and system based on multi-target instance analysis and knowledge-driven. Background Art

[0002] The core position of sperm morphology analysis in the evaluation of male fertility cannot be ignored. It is directly related to the diagnosis and treatment strategies of infertility. The morphological structure of sperm not only affects the fertilization ability, but also determines the quality of embryo development. Therefore, accurate analysis of it is crucial. However, although the importance of this field is widely recognized, the following challenges are still faced in actual operation: First of all, as the main method for sperm morphology analysis, traditional staining technology can effectively improve the visibility of sperm structure, but its operation process requires extremely high precision of chemical substances and time control. Any instability in staining quality may lead to deviations in analysis results, which undoubtedly increases the risk of failure for high-precision operations such as intracytoplasmic sperm injection (ICSI).

[0003] Secondly, although high-power microscopes (such as 100X oil objective) provide detailed information on cell structure, their limited field of view means that each observation can only cover a small number of sperm. This not only prolongs the analysis time of a single sample, but also increases the complexity of the experiment. In order to obtain statistically significant results, a large number of sperm need to be analyzed, which undoubtedly further exacerbates the time-consuming and cumbersome analysis process.

[0004] Furthermore, the assessment of sperm morphology is highly dependent on the operator's experience and judgment. This subjectivity not only leads to significant differences in analysis results between different technicians and even different laboratories, but also affects the accuracy of diagnosis and treatment. Although the introduction of artificial intelligence technology has made it possible to reduce this subjectivity, the performance of AI models is still limited by the quality and representativeness of the training data set. In the process of data set production, the accuracy and consistency of manual annotation are key, which is often difficult to guarantee. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a non-invasive sperm morphology assessment method based on multi-objective instance analysis and knowledge-driven, comprising the following steps: S1: Acquire a microscopic image dataset of multiple live sperm targets without staining; S2: constructing a multi-target instance parsing model, and iteratively training the multi-target instance parsing model using the microscopic image dataset to obtain a trained multi-target instance parsing model; S3: Based on the trained multi-target instance parsing model, a target parsing result of the sperm microscopic image to be evaluated is obtained; S4: Based on the target analysis results, the morphological features of the corresponding parts are obtained, which are used as input to the sperm knowledge graph constructed in advance to obtain the sperm morphology evaluation results.

[0006] In one embodiment of the present invention, the multi-target instance parsing model includes an instance segmentation module, a parsing module and a fusion module; Among them, the instance segmentation module obtains the segmentation mask of the target sperm in the sperm microscopic image; the analysis module extracts the characteristic data of different parts of the target sperm; the fusion module aligns and fuses the segmentation mask and the characteristic data of different parts of the corresponding sperm in spatial position to obtain the analysis results of multiple parts of the sperm.

[0007] In one embodiment of the present invention, the method for constructing the sperm knowledge graph is as follows: Acquiring morphological classification features of different parts of multiple sperms, wherein the morphological classification features include head classification features, mid-section classification features, and tail classification features; Based on the morphological classification features, constructing a triple data structure, the triple data structure including a sperm part name, a classification feature and an attribute value of the classification feature; Using the top-down construction method of the graph database Neo4j, all triple data structures are connected one by one, and finally a knowledge graph of sperm part attribute characteristics is formed.

[0008] In one embodiment of the present invention, the head classification features include size features, Hu moment invariants and symmetry as position features, the middle section classification features include size features and position features, and the tail classification features include size features and curl features.

[0009] In one embodiment of the present invention, the method for obtaining the sperm morphology evaluation result is: Extracting the segmentation masks of each sperm instance and position from the target parsing result, and calculating the corresponding classification features; Input the classification features into the sperm knowledge graph, and match and compare the classification features with the knowledge in the knowledge graph through the conditional mapping mechanism of the sperm knowledge graph to obtain a matching result; According to the matching results, the morphological evaluation results of the sperm of each instance are output.

[0010] Based on the same inventive concept, the present invention also provides a non-invasive sperm morphology evaluation system based on multi-objective instance analysis and knowledge-driven, which is used to implement the steps of the non-invasive sperm morphology evaluation method based on multi-objective instance analysis and knowledge-driven described above. The non-invasive sperm morphology evaluation system based on multi-objective instance analysis and knowledge-driven includes the following modules: A data set acquisition module, used for acquiring a microscopic image data set of multiple live sperm targets without staining; A model building and training module, used for building a multi-target instance parsing model, iteratively training the multi-target instance parsing model through the microscopic image data set, and obtaining a trained multi-target instance parsing model; A target parsing result calculation module, used to obtain a target parsing result of the sperm microscopic image to be evaluated based on the trained multi-target instance parsing model; The morphological evaluation result acquisition module is used to obtain the morphological features of the corresponding parts based on the target analysis results, and use them as input to the sperm knowledge graph constructed in advance to obtain the sperm morphological evaluation results.

[0011] In one embodiment of the present invention, the morphological evaluation result acquisition module includes a knowledge graph construction module, and the knowledge graph construction module is configured as follows: Acquiring morphological classification features of different parts of multiple sperms, wherein the morphological classification features include head classification features, mid-section classification features, and tail classification features; Based on the morphological classification features, constructing a triple data structure, the triple data structure including a sperm part name, a classification feature and an attribute value of the classification feature; Using the top-down construction method of the graph database Neo4j, all triple data structures are connected one by one, and finally a knowledge graph of sperm part attribute characteristics is formed.

[0012] The present invention also provides a non-invasive sperm morphology evaluation device based on multi-objective instance analysis and knowledge-driven, and the device includes the above-mentioned non-invasive sperm morphology evaluation system based on multi-objective instance analysis and knowledge-driven.

[0013] The present invention also provides an electronic device, which includes a processor, a memory and a bus system, wherein the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the above-mentioned non-invasive sperm morphology assessment method based on multi-target instance parsing and knowledge-driven.

[0014] The present invention also provides a computer storage medium storing a computer software product, wherein the computer software product includes a number of instructions for enabling a computer device to execute the above-mentioned non-invasive sperm morphology assessment method based on multi-target instance parsing and knowledge-driven.

[0015] The above technical solution of the present invention has the following advantages compared with the prior art: The present invention combines multi-target instance analysis with knowledge-driven technology, and performs instance-level segmentation and pixel-level part analysis on sperm through a multi-target instance analysis model; the instance target and the corresponding part analysis results are input into a knowledge graph that integrates extensive knowledge of sperm morphology, which is used to guide the classification and judgment of various parts of sperm. The technical solution proposed by the present invention is relatively simple in operation, does not require the use of complex steps or equipment, can quickly complete the analysis of sperm morphology, and improve detection efficiency. By combining technologies such as knowledge graphs, this method has a high accuracy in sperm morphology analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein: Figure 1 It is a flow chart of a non-invasive sperm morphology assessment method based on multi-objective instance analysis and knowledge-driven provided in the first embodiment of the present invention; Figure 2 It is a specific flow chart of the non-invasive sperm morphology assessment method based on multi-objective instance analysis and knowledge-driven provided in the first embodiment of the present invention; Figure 3 It is a schematic diagram of the structure of a non-invasive sperm morphology assessment system based on multi-objective instance analysis and knowledge-driven provided in the second embodiment of the present invention; Description of the Figures in the Specification: 100. Data set acquisition module; 200. Model building and training module; 300. Target parsing result calculation module; 400. Morphological evaluation result acquisition module. DETAILED DESCRIPTION

[0017] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0018] Embodiment 1

[0019] See also Figure 1 and Figure 2 As shown, the present invention provides a non-invasive sperm morphology assessment method based on multi-objective instance parsing and knowledge-driven, the method comprising the following steps: S1: Acquire a microscopic image dataset of multiple live sperm targets without staining under low magnification (20X flat field objective); S2: constructing a multi-target instance parsing model, and iteratively training the multi-target instance parsing model using the microscopic image dataset to obtain a trained multi-target instance parsing model; S3: Based on the trained multi-target instance parsing model, a target parsing result of the sperm microscopic image to be evaluated is obtained; S4: Based on the target analysis results, the morphological features of the corresponding parts are obtained, which are used as input to the sperm knowledge graph constructed in advance to obtain the sperm morphology evaluation results.

[0020] It can be seen from the above technical solutions that the present invention does not need to stain or otherwise process the sperm, and directly obtains microscopic images under a low-power microscope, thereby realizing non-invasive sperm morphological evaluation, avoiding potential damage to sperm caused by chemical stains, and ensuring the accuracy and reliability of the evaluation; by constructing and training a multi-target instance parsing model, the method can automatically parse sperm microscopic images, quickly and accurately identify multiple live sperm targets and the morphological characteristics of their corresponding parts, which greatly improves the efficiency of sperm morphological evaluation and reduces the errors and subjectivity of manual operations. Combined with the sperm knowledge graph constructed in advance, the method can use a wide range of knowledge base information to accurately classify and judge the morphological characteristics of various parts of the sperm. This not only improves the accuracy of sperm morphological evaluation, but also provides strong data support for subsequent sperm quality analysis, disease diagnosis, etc.

[0021] Further, see Figure 2 As shown, the multi-target instance parsing model includes an instance segmentation module, a parsing module and a fusion module; The instance segmentation module obtains the segmentation mask of the target sperm in the sperm microscopic image to ensure that each sperm can be accurately distinguished and located in a complex background; preferably, the instance segmentation module is a Mask R-CNN instance segmentation model; The analysis module analyzes and extracts morphological feature data of different regions such as the head and midsection of the target sperm in detail; The fusion module performs an alignment and fusion operation on the segmentation mask and the feature data of different parts of the corresponding sperm in spatial position to obtain multiple part analysis results of the sperm, thereby realizing instance-level part analysis of multiple sperm.

[0022] Furthermore, the method for constructing the sperm knowledge graph is as follows: Acquire morphological classification features of different parts of multiple sperms, wherein the morphological classification features include head classification features, middle section classification features and tail classification features; the head classification features include size features, Hu moment invariants and symmetry as position features, the middle section classification features include size features and position features, and the tail classification features include size features and curling features; Based on the morphological classification features obtained by the above knowledge extraction, a triple data structure is constructed, each of which includes the sperm part name, classification features and attribute values ​​of the classification features; for example, a triple instance may be "sperm head-size feature-length = XX microns"; The advanced graph database Neo4j was selected as the carrier of knowledge storage. The top-down construction method of the graph database Neo4j was used to connect all triple data structures one by one, and finally a knowledge graph of sperm part attribute characteristics was formed to guide sperm morphology assessment. The nodes in the knowledge graph represent the names of sperm parts or classification features, and the edges connecting the nodes represent the attribute relationships.

[0023] Furthermore, in step S4, based on the target analysis result, the morphological characteristics of the corresponding parts are obtained, and the characteristics are used as the input of the sperm knowledge graph constructed in advance to obtain the sperm morphological evaluation result as follows: Extracting the segmentation masks of each sperm instance and position from the target parsing result, and calculating the corresponding classification features; Input the classification features into the sperm knowledge graph, and match and compare the classification features with the knowledge in the knowledge graph through the conditional mapping mechanism of the sperm knowledge graph to obtain a matching result; According to the matching results, the morphological evaluation results of the sperm of each instance are output, such as conical head, pear-shaped head, curved midsection, etc.

[0024] Embodiment 2

[0025] Based on the same inventive concept as that of the first embodiment, the present invention also provides a non-invasive sperm morphology assessment system based on multi-objective instance analysis and knowledge-driven, which is used to implement the steps of the non-invasive sperm morphology assessment method based on multi-objective instance analysis and knowledge-driven described in the first embodiment. Figure 3 As shown, the non-invasive sperm morphology evaluation system based on multi-objective instance parsing and knowledge-driven includes the following modules: The data set acquisition module 100 is used to acquire a microscopic image data set of multiple live sperm targets without staining under a low-power microscope (20X flat-field objective); A model building and training module 200 is used to build a multi-target instance parsing model, and iteratively train the multi-target instance parsing model through the microscopic image data set to obtain a trained multi-target instance parsing model; A target parsing result calculation module 300 is used to obtain a target parsing result of the sperm microscopic image to be evaluated based on the trained multi-target instance parsing model; The morphological evaluation result acquisition module 400 is used to obtain the morphological features of the corresponding parts based on the target analysis results, and use them as input to the sperm knowledge graph constructed in advance to obtain the sperm morphological evaluation results.

[0026] Furthermore, the morphological evaluation result acquisition module includes a knowledge graph construction module, and the knowledge graph construction module is configured as follows: Acquiring morphological classification features of different parts of multiple sperms, wherein the morphological classification features include head classification features, mid-section classification features, and tail classification features; Based on the morphological classification features, constructing a triple data structure, the triple data structure including a sperm part name, a classification feature and an attribute value of the classification feature; Using the top-down construction method of the graph database Neo4j, all triple data structures are connected one by one, and finally a knowledge graph of sperm part attribute characteristics is formed.

[0027] The present embodiment proposes a non-invasive sperm morphology assessment system based on multi-target instance analysis and knowledge-driven, which is used to implement the aforementioned non-invasive sperm morphology assessment method based on multi-target instance analysis and knowledge-driven. Therefore, the specific implementation method in the synchronous response single-point displacement monitoring system can be seen from the aforementioned embodiment part of the non-invasive sperm morphology assessment method based on multi-target instance analysis and knowledge-driven. For example, the data set acquisition module 100, the model construction and training module 200, the target analysis result calculation module 300 and the morphology assessment result acquisition module 400 are respectively used to implement the steps S1, S2, S3 and S4 in the non-invasive sperm morphology assessment method based on multi-target instance analysis and knowledge-driven in embodiment one. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part. In order to avoid redundancy, it will not be repeated here.

[0028] Embodiment 3

[0029] The present invention also provides a non-invasive sperm morphology assessment device based on multi-objective instance analysis and knowledge-driven, which includes the non-invasive sperm morphology assessment system based on multi-objective instance analysis and knowledge-driven described in the above-mentioned embodiment 1.

[0030] Embodiment 4

[0031] The present invention also provides an electronic device, comprising a processor, a memory and a bus system, wherein the processor and the memory are connected via the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, so as to implement the non-invasive sperm morphology assessment method based on multi-target instance parsing and knowledge-driven as described in the above-mentioned embodiment 1.

[0032] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0033] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0034] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0035] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0036] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A non-invasive sperm morphology assessment method based on multi-objective instance analysis and knowledge-driven, characterized in that: The following steps are involved: S1: Acquire a microscopic image dataset of multiple live sperm targets without staining; S2: constructing a multi-target instance parsing model, and iteratively training the multi-target instance parsing model using the microscopic image dataset to obtain a trained multi-target instance parsing model; S3: Based on the trained multi-target instance parsing model, a target parsing result of the sperm microscopic image to be evaluated is obtained; S4: Based on the target analysis results, the morphological features of the corresponding parts are obtained, which are used as input to the sperm knowledge graph constructed in advance to obtain the sperm morphology evaluation results.

2. The non-invasive sperm morphology assessment method based on multi-objective instance analysis and knowledge-driven according to claim 1 is characterized in that: The multi-target instance parsing model includes an instance segmentation module, a parsing module and a fusion module; Among them, the instance segmentation module obtains the segmentation mask of the target sperm in the sperm microscopic image; the analysis module extracts the characteristic data of different parts of the target sperm; the fusion module aligns and fuses the segmentation mask and the characteristic data of different parts of the corresponding sperm in spatial position to obtain the analysis results of multiple parts of the sperm.

3. The non-invasive sperm morphology assessment method based on multi-objective instance analysis and knowledge-driven according to claim 1 is characterized in that: The method for constructing the sperm knowledge graph is as follows: Acquiring morphological classification features of different parts of multiple sperms, wherein the morphological classification features include head classification features, mid-section classification features, and tail classification features; Based on the morphological classification features, constructing a triple data structure, the triple data structure including a sperm part name, a classification feature and an attribute value of the classification feature; Using the top-down construction method of the graph database Neo4j, all triple data structures are connected one by one, and finally a knowledge graph of sperm part attribute characteristics is formed.

4. The non-invasive sperm morphology assessment method based on multi-objective instance analysis and knowledge-driven according to claim 3 is characterized in that: The head classification features include size features, Hu moment invariants and symmetry as position features, the middle section classification features include size features and position features, and the tail classification features include size features and curl features.

5. The non-invasive sperm morphology assessment method based on multi-objective instance analysis and knowledge-driven according to claim 1 is characterized in that: The method for obtaining sperm morphology assessment results is: Extracting the segmentation masks of each sperm instance and position from the target parsing result, and calculating the corresponding classification features; Input the classification features into the sperm knowledge graph, and match and compare the classification features with the knowledge in the knowledge graph through the conditional mapping mechanism of the sperm knowledge graph to obtain a matching result; According to the matching results, the morphological evaluation results of the sperm of each instance are output.

6. A non-invasive sperm morphology assessment system based on multi-objective instance analysis and knowledge-driven, characterized in that: The steps for implementing the non-invasive sperm morphology evaluation method based on multi-objective instance analysis and knowledge-driven according to any one of claims 1 to 5, the non-invasive sperm morphology evaluation system based on multi-objective instance analysis and knowledge-driven comprises the following modules: A data set acquisition module, used for acquiring a microscopic image data set of multiple live sperm targets without staining; A model building and training module, used for building a multi-target instance parsing model, iteratively training the multi-target instance parsing model through the microscopic image data set, and obtaining a trained multi-target instance parsing model; A target parsing result calculation module, used to obtain a target parsing result of the sperm microscopic image to be evaluated based on the trained multi-target instance parsing model; The morphological evaluation result acquisition module is used to obtain the morphological features of the corresponding parts based on the target analysis results, and use them as input to the sperm knowledge graph constructed in advance to obtain the sperm morphological evaluation results.

7. The non-invasive sperm morphology assessment system based on multi-objective instance analysis and knowledge-driven according to claim 6 is characterized in that: The morphological evaluation result acquisition module includes a knowledge graph construction module, and the knowledge graph construction module is configured as follows: Acquiring morphological classification features of different parts of multiple sperms, wherein the morphological classification features include head classification features, mid-section classification features, and tail classification features; Based on the morphological classification features, constructing a triple data structure, the triple data structure including a sperm part name, a classification feature and an attribute value of the classification feature; Using the top-down construction method of the graph database Neo4j, all triple data structures are connected one by one, and finally a knowledge graph of sperm part attribute characteristics is formed.

8. A non-invasive sperm morphology assessment device based on multi-objective instance analysis and knowledge-driven, characterized in that: It includes a non-invasive sperm morphology assessment system based on multi-objective instance analysis and knowledge-driven as described in claim 6 or claim 7.

9. An electronic device, characterized in that: The electronic device includes a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the non-invasive sperm morphology assessment method based on multi-target instance parsing and knowledge-driven as described in any one of claims 1 to 5.

10. A computer storage medium, characterized in that: The computer storage medium stores a computer software product, which includes several instructions for enabling a computer device to execute the non-invasive sperm morphology assessment method based on multi-target instance parsing and knowledge-driven as described in any one of claims 1 to 5.

Images