Embryonic cell tracking method, device, equipment, medium and product

Abstract

The embodiment of the invention discloses an embryonic cell tracking method and device, equipment, a medium and a product, and the method comprises the steps: inputting image frames in an embryo image sequence to be analyzed into a preset cell recognition model, obtaining a cell recognition result in each image frame, and obtaining a cell recognition result in each image frame through image segmentation; a cell recognition image is obtained and input into an image feature analysis model to obtain image features; respectively calculating the cell feature similarity and the position similarity of the cell recognition images in any two continuous image frames based on the image features; and determining a cell counting marking result according to the cell similarity and the position similarity to realize embryonic cell tracking. According to the technical scheme, the problem of low cell counting accuracy at present is solved, analysis and cell counting can be carried out in combination with cell characteristics in the adjacent embryonic cell image sequence, the integrity and accuracy of cell counting are improved, and the method is particularly suitable for complex scenes such as cell stacking and boundary blur in the embryonic development process.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular, to an embryonic cell tracking method, device, equipment, medium and product. Background Art

[0002] Embryo delayed detection technology can take images of embryos at regular intervals and capture subtle changes in each stage of the in vitro culture process, such as cell division, embryo morphology and division timing, providing doctors with a reliable basis for assessing the health of the embryo.

[0003] However, when using image processing algorithms for cell counting, the fuzziness of cell boundaries, noise interference, and insufficient algorithm accuracy may lead to miscounting or omissions, thus affecting the accuracy of the evaluation results. Summary of the invention

[0004] The embodiments of the present invention provide an embryonic cell tracking method, device, equipment, medium and product, which can combine the cell features in the adjacent embryonic cell image sequence for analysis and cell counting, thereby improving the integrity and accuracy of cell counting, and are particularly suitable for complex scenes such as cell stacking and blurred boundaries during embryonic development.

[0005] In a first aspect, an embodiment of the present invention provides an embryonic cell tracking method, the method comprising:

[0006] Acquire an embryo image sequence to be analyzed, and input image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence to obtain a cell recognition result in each image frame;

[0007] Perform image segmentation based on the cell recognition results to obtain corresponding cell recognition images, and input the cell recognition images into an image feature analysis model to obtain image features corresponding to each cell recognition image;

[0008] Based on the image features, each cell identification image in the first image frame in any two consecutive image frames is calculated and compared with each cell identification image in the second image frame in the latter image frame to determine the similarity of cell features;

[0009] Determining the position similarity between each cell recognition image in the first image frame and each cell recognition image in the second image frame according to the position mark information of the cell recognition image;

[0010] According to the cell similarity and position similarity, on the basis of the cell counting and marking results in the first image frame, the cell counting and marking results in the second image frame are determined to achieve embryonic cell tracking.

[0011] In a second aspect, an embodiment of the present invention further provides an embryonic cell tracking device, the device comprising:

[0012] A cell recognition module is used to obtain an embryo image sequence to be analyzed, and input image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence to obtain a cell recognition result in each image frame;

[0013] An image feature analysis model is used to perform image segmentation based on the cell recognition results to obtain corresponding cell recognition images, and the cell recognition images are input into the image feature analysis model to obtain image features corresponding to each cell recognition image;

[0014] A cell feature similarity determination module is used to respectively calculate, based on image features, each cell identification image in a first image frame in any two consecutive image frames and each cell identification image in a second image frame in the latter image frame for comparative analysis to determine the cell feature similarity;

[0015] A position similarity determination module, used to determine the position similarity between each cell recognition image in the first image frame and each cell recognition image in the second image frame according to the position mark information of the cell recognition image;

[0016] The cell counting and marking result determination module is used to determine the cell counting and marking result in the second image frame based on the cell counting and marking result in the first image frame according to the cell similarity and the position similarity, so as to realize embryonic cell tracking.

[0017] In a third aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:

[0018] one or more processors;

[0019] A memory for storing one or more programs;

[0020] When one or more programs are executed by one or more processors, the one or more processors implement the embryonic cell tracking method provided by any embodiment of the present invention.

[0021] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the embryonic cell tracking method provided by any embodiment of the present invention.

[0022] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product, including a computer program, which, when executed by a processor, implements the embryonic cell tracking method provided by any embodiment of the present invention.

[0023] The embodiments of the above invention have the following advantages or beneficial effects:

[0024] The embodiment of the present invention obtains the embryo image sequence to be analyzed, and inputs the image frames in the embryo image sequence to be analyzed into the preset cell recognition model in sequence, so as to obtain the cell recognition result in each image frame; image segmentation is performed based on the cell recognition result to obtain the corresponding cell recognition image, and the cell recognition image is input into the image feature analysis model to obtain the image feature corresponding to each cell recognition image; based on the image feature, each cell recognition image in the first image frame in any two consecutive image frames is calculated and compared with each cell recognition image in the second image frame in the latter, so as to determine the cell feature similarity; according to the position marking information of the cell recognition image, the position similarity of each cell recognition image in the first image frame and each cell recognition image in the second image frame is determined; according to the cell similarity and the position similarity, on the basis of the cell counting marking result in the first image frame, the cell counting marking result in the second image frame is determined, so as to realize embryo cell tracking. The technical solution of the embodiment of the present invention solves the problem of low accuracy of current cell counting, and can analyze and count cells in combination with the cell features in the adjacent embryo cell image sequence, so as to improve the integrity and accuracy of cell counting, and is particularly suitable for complex scenes such as cell stacking and blurred boundaries during embryonic development. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of an embryonic cell tracking method provided by an embodiment of the present invention;

[0026] Figure 2 is a flow chart of an embryonic cell tracking method provided by an embodiment of the present invention;

[0027] Figure 3 is a flow chart of an embryonic cell tracking method provided by an embodiment of the present invention;

[0028] Figure 4 is an example diagram of an embryonic cell tracking method provided by an embodiment of the present invention;

[0029] Figure 5 is a schematic structural diagram of an embryonic cell tracking device provided by an embodiment of the present invention;

[0030] Figure 6 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0032] Figure 1 A flowchart of an embryonic cell tracking method provided by an embodiment of the present invention is provided. This embodiment is applicable to embryonic cell tracking scenarios, especially embryonic cell tracking in complex scenarios such as cell stacking and blurred boundaries during embryonic development. The method can be performed by an embryonic cell tracking device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0033] like Figure 1 As shown, the embryonic cell tracking method of this embodiment includes the following steps:

[0034] S110, obtaining an embryo image sequence to be analyzed, and inputting image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence to obtain a cell recognition result in each image frame.

[0035] During the embryonic cell tracking process, the embryonic cell is often imaged at regular intervals by an embryonic time-delay detection system to record developmental changes. The embryonic image sequence to be analyzed can be an image sequence obtained by regularly capturing the embryonic cell to be analyzed by the embryonic time-delay detection system at preset time intervals, which can record the developmental changes of the embryonic cells. The preset time interval can be determined according to actual needs, for example, 5 minutes.

[0036] Image frames in the embryo image sequence to be analyzed are input into the preset cell recognition model successively, and each embryo image to be analyzed is carried out cell recognition by the preset cell recognition model to obtain the cell recognition result in each image frame. The preset cell recognition model can be a model based on a target detection algorithm, and the target detection algorithm can identify the position and category of a specific target from an image or a video. The cell recognition result can be the position marking information of a cell. In the present embodiment, the preset cell recognition model can identify the position of a cell in the image, and generates the position marking information indicating the position of the cell, such as a position marking frame.

[0037] The target detection algorithm may be SSD (Single Shot MultiBox Detector), RetinaNet, CenterNet, and various versions of the YOLO (You Only Look Once) algorithm. In this embodiment, the YOLOv10 (You Only Look Once version 10) algorithm is preferably used. YOLOv10 has good detection effects and can improve the accuracy of cell recognition.

[0038] S120, performing image segmentation based on the cell recognition result to obtain corresponding cell recognition images, and inputting the cell recognition images into an image feature analysis model to obtain image features corresponding to each cell recognition image.

[0039] Based on the position mark information in the cell recognition result, the image within the mark information corresponding to each cell is segmented to obtain the corresponding cell recognition image. The number of cell recognition images corresponds to the number of cells recognized, that is, the number of position mark information. The cell recognition image is input into the image feature analysis model to obtain the image features corresponding to each cell recognition image. The image feature analysis model can be a deep convolutional neural network model, such as ResNet, Inception and VGGNet (Visual Geometry Group Network). This embodiment preferably uses VGGNet. VGGNet has a deep network structure and can perform multi-level feature extraction on the cell recognition image to obtain image features.

[0040] In this embodiment, the efficiency and accuracy of YOLOv10 are used to ensure accurate detection of cell targets in complex scenarios, while VGGNet further improves the reliability of cell tracking. YOLOv10 and VGGNet are combined to maximize tracking accuracy.

[0041] S130 , based on the image features, respectively calculate each cell recognition image in the first image frame in any two consecutive image frames and compare and analyze each cell recognition image in the second image frame in order to determine the similarity of the cell features.

[0042] In this embodiment, the trajectory information of the cells is determined by making full use of the image frame information arranged in time sequence. Specifically, for any two consecutive image frames, that is, two image frames separated by a preset time interval, the cell identification images in the first image frame acquired earlier and the second image frame acquired later than the first image frame are compared and analyzed, for example, by similarity calculation, to determine the similarity of cell features between every two cells in different images.

[0043] S140 . Determine, based on the position mark information of the cell recognition images, the position similarity between each cell recognition image in the first image frame and each cell recognition image in the second image frame.

[0044] For example, by calculating the frame overlap of the position mark frames, the position similarity between each cell recognition image in the first image frame and each cell recognition image in the second image frame is determined. The higher the frame overlap, the higher the position similarity.

[0045] S150, determining the cell counting and marking result in the second image frame based on the cell counting and marking result in the first image frame according to the cell similarity and the position similarity, so as to realize embryo cell tracking.

[0046] The cell similarity and position similarity are comprehensively considered. For example, the comprehensive similarity is determined according to the cell similarity and the position similarity, and the position of the same cell in the two image frames in the second image frame is determined according to the comprehensive similarity, that is, the position of the cell in the first image frame in the second image frame is determined, and the cell counting and marking result of the same cell in the first image frame is used as the cell counting and marking result of the same cell in the second image frame.

[0047] For example, the cells in the first image frame include cell 1 and cell 2, and the second image frame includes four cells. The positions of cell 1 and cell 2 in the first image frame in the second image frame can be determined by similarity comparison, and cell 1 and cell 2 can be marked accordingly in the second image frame. The other two cells in the second image frame can be marked as cell 3 and cell 4, respectively. Similarly, cell 1, cell 2, cell 3, cell 4, and other newly added cells can be marked in the third image frame after the second image frame, so as to realize the positioning and marking of cells and realize embryonic cell tracking.

[0048] The technical solution of this embodiment is to obtain the embryo image sequence to be analyzed, and input the image frames in the embryo image sequence to be analyzed into the preset cell recognition model in sequence, so as to obtain the cell recognition result in each image frame; perform image segmentation based on the cell recognition result, obtain the corresponding cell recognition image, and input the cell recognition image into the image feature analysis model, so as to obtain the image feature corresponding to each cell recognition image; based on the image feature, respectively calculate each cell recognition image in the first image frame in any two consecutive image frames and each cell recognition image in the second image frame in the latter for comparative analysis, and determine the cell feature similarity; according to the position marking information of the cell recognition image, determine the position similarity of each cell recognition image in the first image frame and each cell recognition image in the second image frame; according to the cell similarity and position similarity, on the basis of the cell counting marking result in the first image frame, determine the cell counting marking result in the second image frame, so as to realize embryo cell tracking. The technical solution of the embodiment of the present invention solves the problem of low accuracy of current cell counting, and can analyze and count cells in combination with the cell features in the adjacent embryo cell image sequence, so as to improve the integrity and accuracy of cell counting, and is particularly suitable for complex scenes such as cell stacking and blurred boundaries during embryonic development.

[0049] Figure 2 A flowchart of an embryonic cell tracking method provided by an embodiment of the present invention, this embodiment and the embryonic cell tracking method in the above embodiment belong to the same inventive concept, and further describes the process of determining the similarity of cells and positions. The method can be performed by an embryonic cell tracking device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0050] like Figure 2 As shown, the embryonic cell tracking method of this embodiment includes the following steps:

[0051] S210, obtaining an embryo image sequence to be analyzed, and inputting image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence to obtain a cell recognition result in each image frame.

[0052] S220, performing image segmentation based on the cell recognition result to obtain corresponding cell recognition images, and inputting the cell recognition images into an image feature analysis model to obtain image features corresponding to each cell recognition image.

[0053] S230 , calculating the cosine similarity and the Euclidean distance between the image features of each cell recognition image in the first image frame and the image features of each cell recognition image in the second image frame.

[0054] The image features of the cell recognition image can be the feature vector of the cell recognition image. Cosine similarity can measure the similarity between two feature vectors, and the value range is [-1, 1][-1, 1][-1, 1], where 1 means exactly the same, -1 means completely opposite, and 0 means no similarity. It is used to calculate the similarity of the cell feature values ​​of the previous and next frames, which can be expressed by the following formula:

[0055]

[0056] Wherein, A and B are feature vectors of any cell recognition image belonging to the first image frame and the second image frame respectively, and cosine_sim(A, B) is cosine similarity.

[0057] The Euclidean distance (L2 norm) is used to calculate the distance between two eigenvectors, which means the distance between the two eigenvectors, that is, the distance between the eigenvalues ​​of the cells in the previous and next frames, which can be expressed by the following formula:

[0058]

[0059] Wherein, A and B are feature vectors of any cell recognition image belonging to the first image frame and the second image frame respectively, distance is the Euclidean distance, and n is the spatial dimension.

[0060] S240, performing weighted sum calculation on the cosine similarity and the Euclidean distance to obtain the cell feature similarity of the corresponding cell recognition image pair.

[0061] According to the preset weights corresponding to the cosine similarity and the Euclidean distance respectively, the cosine similarity and the Euclidean distance are weighted and calculated to obtain the cell feature similarity of the corresponding cell recognition image pair.

[0062] S250 , calculating the intersection-and-union ratio of corresponding cell image pairs according to the position marking information of each cell recognition image in the first image frame and the position marking information of each cell recognition image in the second image frame.

[0063] The position marker information may be a position marker box. The intersection over union (IOU) of two cells is an indicator to measure the degree of overlap between two position marker boxes. It is the ratio of the area of ​​the intersection of the two position marker boxes to the area of ​​their union.

[0064]

[0065] Wherein, A and B represent the position mark information of any cell identification image belonging to the first image frame and the second image frame.

[0066] S260, using the intersection-over-union ratio as the position similarity of the corresponding cell image pair.

[0067] S270, determining the cell counting and marking result in the second image frame based on the cell counting and marking result in the first image frame according to the cell similarity and the position similarity, so as to realize embryo cell tracking.

[0068] According to the preset weights corresponding to the cell similarity and position similarity, a weighted sum calculation is performed to obtain a comprehensive similarity. According to the comprehensive similarity, it is determined whether the cells in the cell image pair are the same cell, and the cell counting and marking results of the same cells in the first image frame are used as the cell counting and marking results of the same cells in the second image frame.

[0069] In an optional embodiment, based on the cell counting and marking results in the first image frame, the cell counting and marking results in the second image frame are determined according to the cell similarity and the position similarity. The similarity value of the corresponding cell image pair can be determined according to the cell similarity and the position similarity; when the similarity value meets a preset similarity threshold, the corresponding cell image pair is determined to be the same cell; and the cell counting and marking results of the cells in the cell image pair in the first image frame are determined as the cell counting and marking results of the cells in the cell image pair in the second image frame.

[0070] The calculation formula of the similarity value of the cell image pair, that is, the comprehensive similarity, can be expressed as:

[0071] Cost value =α1*IOU+α2*cell_sim=β1*IOU+β2*cosine_sim+β3*distance;

[0072] α1 is the weight value of position similarity, α2 is the weight value of cell similarity, cell_sim is the cell similarity, β1 is the weight value of intersection ratio, β2 is the weight value of cosine similarity, β3 is the weight value of Euclidean distance, and β1 can be set to 1, β2 to 0.5, and β3 to 0.8.

[0073] At Cost value When a preset similarity threshold is met, the corresponding cell image pair is determined to be the same cell. The preset similarity threshold can be determined based on experience and experimental conditions, etc., and is not limited in this embodiment. The cell counting and marking result of the cell in the cell image pair in the first image frame is determined as the cell counting and marking result of the cell in the cell image pair in the second image frame, such as Figure 3 As shown, in images 625 to 697, by calculating the similarity of the cell recognition images of the previous and next frames, the cell position information corresponding to cell 6 in each image is determined and assigned the same cell counting mark "6", thereby realizing the trajectory tracking of cell "6".

[0074] The technical solution of this embodiment is as follows: by acquiring an embryo image sequence to be analyzed, and inputting the image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence, so as to obtain a cell recognition result in each image frame; performing image segmentation based on the cell recognition result to obtain a corresponding cell recognition image, and inputting the cell recognition image into an image feature analysis model to obtain an image feature corresponding to each cell recognition image; calculating the cosine similarity and Euclidean distance between the image feature of each cell recognition image in the first image frame and the image feature of each cell recognition image in the second image frame respectively; performing weighted sum calculation on the cosine similarity and the Euclidean distance to obtain the cell feature similarity of the corresponding cell recognition image pair; calculating the intersection-and-union ratio of the corresponding cell image pair according to the position marking information of each cell recognition image in the first image frame and the position marking information of each cell recognition image in the second image frame; using the intersection-and-union ratio as the position similarity of the corresponding cell image pair; determining the cell counting marking result in the second image frame according to the cell similarity and the position similarity, based on the cell counting marking result in the first image frame, to achieve embryo cell tracking. The technical solution of the embodiment of the present invention solves the problem of low accuracy in current cell counting. It can combine the cell features in adjacent embryonic cell image sequences for analysis and cell counting, determine the same cells in different image frames through multiple similarity calculation methods, dynamically track the position changes of cells at multiple time points, and improve the integrity and accuracy of cell counting and trajectory tracking. It is particularly suitable for complex scenes such as cell stacking and blurred boundaries during embryonic development, and provides strong support for the study of multi-dimensional behavior of embryonic cells.

[0075] Figure 3The present invention provides a flowchart of an embryonic cell tracking method. This embodiment and the embryonic cell tracking method in the above embodiment belong to the same inventive concept, and further illustrates the process of supplementing the missing cell marker frame. The method can be performed by an embryonic cell tracking device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0076] like Figure 3 As shown, the embryonic cell tracking method of this embodiment includes the following steps:

[0077] S310, obtaining an embryo image sequence to be analyzed, and inputting image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence to obtain a cell recognition result in each image frame.

[0078] S320, performing image segmentation based on the cell recognition result to obtain corresponding cell recognition images, and inputting the cell recognition images into an image feature analysis model to obtain image features corresponding to each cell recognition image.

[0079] S330 , based on the image features, respectively calculate each cell recognition image in the first image frame in any two consecutive image frames and compare and analyze each cell recognition image in the second image frame in the latter image frame to determine the similarity of cell features.

[0080] S340 , determining the position similarity between each cell recognition image in the first image frame and each cell recognition image in the second image frame according to the position mark information of the cell recognition image.

[0081] S350, determining the cell counting and marking result in the second image frame based on the cell counting and marking result in the first image frame according to the cell similarity and the position similarity, so as to realize embryo cell tracking.

[0082] S360, determining the missing-mark image frames having missing cell marking frames according to the cell counting marking results of each image frame in the embryo image sequence to be analyzed.

[0083] Understandably, due to there are complicated situations such as stacking, incomplete division and focal plane difference in the cell development process, therefore, can not correctly reflect actual cell development situation in the image frame, there is the possibility of cell loss.Therefore, in the present embodiment, by analyzing the cell counting labeling result of each image frame in the embryo image sequence to be analyzed, determine to lack the cell labeling frame.Example, if in the embryo image sequence to be analyzed, any cell labeling frame lacks in the 3rd image frame, and occurs in the 4th image frame after the 3rd image frame, then determine that the 3rd image frame, and all image frames between the 3rd image frame and the 4th image frame are the labeling lacking image frames with the missing cell labeling frame.

[0084] S370, determining the coordinates of the missing cell marker frame according to the coordinate information of the cell marker frame corresponding to the missing cell marker frame in the two non-marker missing image frames that are closest in time to the marked missing image frame.

[0085] The two non-marker missing image frames include an image frame before the mark missing image frame and an image frame after the mark missing image frame. For example, for the third image frame and all image frames between the third image frame and the fourth image frame, the two non-marker missing image frames closest in time are the previous image frame and the fourth image frame adjacent to the third image frame. Linear interpolation is performed based on the coordinate information of the cell marker frame corresponding to the missing cell marker frame in the previous image frame and the fourth image frame adjacent to the third image frame to determine the coordinates of the missing cell marker frame.

[0086] In an optional embodiment, the coordinates of the missing cell marking frame are determined based on the coordinate information of the cell marking frame corresponding to the missing cell marking frame in the two non-marked missing image frames that are closest in time to the marked missing image frame. The coordinate results can be obtained by performing linear interpolation calculation based on the coordinate information of the cell marking frame corresponding to the missing cell marking frame in the two non-marked missing image frames that are closest in time to the marked missing image frame; and the coordinates of the missing cell marking frame are determined based on the calculated coordinate results.

[0087] like Figure 4 As shown, 7 cells and 8 cells in the image frame 6698-cell overlap. Through steps S310-S350, only 8 cells are detected, and 7 cells are lost. Therefore, the lost 7 cells are supplemented. The closest previous non-labeled missing image frame of 6698-cell is 6658-cell. The coordinates of the upper left corner and the lower right corner of the cell marking box of 7 cells in 6658-cell are (x1_prev, y1_prev)(x2_prev, x2_prev). The closest subsequent non-labeled missing image frame of 6698-cell is 6978-cell. The coordinates of the upper left corner and the lower right corner of the cell marking box of 7 cells in 6978-cell are (x1_next, y1_next)(x2_next, x2_next). The coordinates (x1, y1)(x2, y2) of the missing cell box in 6698-cell are found by linear interpolation and can be expressed by the following formula:

[0088] x1=(x1_prev+x1_next) / 2;

[0089] y1=(y1_prev+y1_next) / 2;

[0090] x2=(x2_prev+x2_next) / 2;

[0091] y2=(y2_prev+y2_next) / 2.

[0092] S380 , supplementing the corresponding cell marker frame in the marker-missing image frame based on the coordinates of the missing cell marker frame.

[0093] like Figure 4 As shown, the missing cell box in 6698-cell has been marked by a black line in 6698-cell.

[0094] In this embodiment, according to the cell counting and marking results of each image frame in the embryo image sequence to be analyzed, a cell marking frame that appears only once can be determined as a misdetected cell marking frame, and the misdetected cell marking frame can be removed from the image frame, and the cell count in the image frame can be modified.

[0095] The technical solution of this embodiment is to obtain an embryo image sequence to be analyzed, and input the image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence to obtain a cell recognition result in each image frame; perform image segmentation based on the cell recognition result to obtain a corresponding cell recognition image, and input the cell recognition image into an image feature analysis model to obtain image features corresponding to each cell recognition image; based on the image features, respectively calculate each cell recognition image in the first image frame of any two consecutive image frames and each cell recognition image in the second image frame to compare and analyze to determine the similarity of cell features; determine the similarity of each cell recognition image in the first image frame and each cell recognition image in the second image frame according to the position mark information of the cell recognition image. position similarity; according to the cell similarity and position similarity, on the basis of the cell counting marking result in the first image frame, determine the cell counting marking result in the second image frame to realize embryonic cell tracking; according to the cell counting marking result of each image frame in the embryonic image sequence to be analyzed, determine the marked missing image frame with missing cell marking frame; according to the coordinate information of the cell marking frame corresponding to the missing cell marking frame in the two non-marked missing image frames closest to the time of the marked missing image frame, determine the coordinates of the missing cell marking frame, wherein the two non-marked missing image frames include the image frame before the marked missing image frame and the image frame after the marked missing image frame; based on the coordinates of the missing cell marking frame, supplement the corresponding cell marking frame in the marked missing image frame. The technical solution of the embodiment of the present invention solves the problem of low accuracy of cell counting at present, can be combined with the cell features in the adjacent embryonic cell image sequence for analysis and cell counting, and the missing cell marking frame is supplemented by the cell counting result, so as to improve the integrity and accuracy of cell counting, and is particularly suitable for complex scenes such as cell stacking and blurred boundaries during embryonic development.

[0096] Figure 5 A structural schematic diagram of an embryonic cell tracking device provided in an embodiment of the present invention. This embodiment can be applied to scenarios of embryonic cell tracking, especially situations where embryonic cell tracking is performed in complex scenarios such as cell stacking and blurred boundaries during embryonic development. The embryonic cell tracking device can be implemented by software and / or hardware and integrated into a computer terminal device with application development capabilities.

[0097] like Figure 5 The embryonic cell tracking device shown includes: a cell identification module 410 , an image feature analysis model 420 , a cell feature similarity determination module 430 , a position similarity determination module 440 and a cell counting mark result determination module 450 .

[0098] Among them, the cell recognition module 410 is used to obtain the embryo image sequence to be analyzed, and input the image frames in the embryo image sequence to be analyzed into the preset cell recognition model in sequence to obtain the cell recognition result in each image frame; the image feature analysis model 420 is used to perform image segmentation based on the cell recognition result to obtain the corresponding cell recognition image, and input the cell recognition image into the image feature analysis model to obtain the image feature corresponding to each cell recognition image; the cell feature similarity determination module 430 is used to calculate each cell recognition image in the first image frame of any two consecutive image frames based on the image features, and compare and analyze each cell recognition image in the second image frame to determine the cell feature similarity; the position similarity determination module 440 is used to determine the position similarity between each cell recognition image in the first image frame and each cell recognition image in the second image frame according to the position marking information of the cell recognition image; the cell counting marking result determination module 450 is used to determine the cell counting marking result in the second image frame based on the cell similarity and the position similarity, so as to realize embryo cell tracking.

[0099] The technical solution of this embodiment is to obtain the embryo image sequence to be analyzed, and input the image frames in the embryo image sequence to be analyzed into the preset cell recognition model in sequence, so as to obtain the cell recognition result in each image frame; perform image segmentation based on the cell recognition result, obtain the corresponding cell recognition image, and input the cell recognition image into the image feature analysis model, so as to obtain the image feature corresponding to each cell recognition image; based on the image feature, respectively calculate each cell recognition image in the first image frame in any two consecutive image frames and each cell recognition image in the second image frame in the latter for comparative analysis, and determine the cell feature similarity; according to the position marking information of the cell recognition image, determine the position similarity of each cell recognition image in the first image frame and each cell recognition image in the second image frame; according to the cell similarity and position similarity, on the basis of the cell counting marking result in the first image frame, determine the cell counting marking result in the second image frame, so as to realize embryo cell tracking. The technical solution of the embodiment of the present invention solves the problem of low accuracy of current cell counting, and can analyze and count cells in combination with the cell features in the adjacent embryo cell image sequence, so as to improve the integrity and accuracy of cell counting, and is particularly suitable for complex scenes such as cell stacking and blurred boundaries during embryonic development.

[0100] In an optional implementation, the cell feature similarity determination module 430 may be used to:

[0101] Calculate the cosine similarity and Euclidean distance between the image features of each cell recognition image in the first image frame and the image features of each cell recognition image in the second image frame; perform weighted sum calculation on the cosine similarity and Euclidean distance to obtain the cell feature similarity of the corresponding cell recognition image pair.

[0102] In an optional implementation, the location similarity determination module 440 may be used to:

[0103] According to the position marking information of each cell recognition image in the first image frame and the position marking information of each cell recognition image in the second image frame, the intersection-and-union ratio of the corresponding cell image pairs is calculated; and the intersection-and-union ratio is used as the position similarity of the corresponding cell image pairs.

[0104] In an optional embodiment, the cell counting labeling result determination module 450 may be used to:

[0105] According to the cell similarity and the position similarity, the similarity value of the corresponding cell image pair is determined; when the similarity value meets the preset similarity threshold, the corresponding cell image pair is determined to be the same cell; the cell counting and marking result of the cell in the cell image pair in the first image frame is determined as the cell counting and marking result of the cell in the cell image pair in the second image frame.

[0106] In an optional embodiment, the device further comprises:

[0107] A cell marking frame supplementing module is used to determine a marked missing image frame with a missing cell marking frame according to the cell counting and marking results of each image frame in the embryo image sequence to be analyzed; determine the coordinates of the missing cell marking frame according to the coordinate information of the cell marking frame corresponding to the missing cell marking frame in two non-marked missing image frames that are closest in time to the marked missing image frame, wherein the two non-marked missing image frames include an image frame before the marked missing image frame and an image frame after the marked missing image frame; and supplement the corresponding cell marking frame in the marked missing image frame based on the coordinates of the missing cell marking frame.

[0108] In an optional embodiment, the cell marker frame supplement module can be used to:

[0109] According to the coordinate information of the cell marker frame corresponding to the missing cell marker frame in the two non-marked missing image frames closest in time to the marked missing image frame, a linear interpolation calculation is performed to obtain a calculated coordinate result; and the coordinates of the missing cell marker frame are determined based on the calculated coordinate result.

[0110] The embryonic cell tracking device provided in the embodiment of the present invention can execute the embryonic cell tracking method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0111] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 6 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as an intelligent controller and server, a mobile phone and other terminal devices.

[0112] like Figure 6 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0113] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0114] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0115] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0116] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0117] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0118] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing the embryonic cell tracking method provided by the embodiment of the present invention, the method comprising:

[0119] Acquire an embryo image sequence to be analyzed, and input image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence to obtain a cell recognition result in each image frame;

[0120] Perform image segmentation based on the cell recognition results to obtain corresponding cell recognition images, and input the cell recognition images into an image feature analysis model to obtain image features corresponding to each cell recognition image;

[0121] Based on the image features, each cell identification image in the first image frame in any two consecutive image frames is calculated and compared with each cell identification image in the second image frame in the latter image frame to determine the similarity of cell features;

[0122] Determining the position similarity between each cell recognition image in the first image frame and each cell recognition image in the second image frame according to the position mark information of the cell recognition image;

[0123] According to the cell similarity and position similarity, on the basis of the cell counting and marking results in the first image frame, the cell counting and marking results in the second image frame are determined to achieve embryonic cell tracking.

[0124] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for tracking embryonic cells provided in any embodiment of the present invention is implemented. The method comprises:

[0125] Acquire an embryo image sequence to be analyzed, and input image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence to obtain a cell recognition result in each image frame;

[0126] Perform image segmentation based on the cell recognition results to obtain corresponding cell recognition images, and input the cell recognition images into an image feature analysis model to obtain image features corresponding to each cell recognition image;

[0127] Based on the image features, each cell identification image in the first image frame in any two consecutive image frames is calculated and compared with each cell identification image in the second image frame in the latter image frame to determine the similarity of cell features;

[0128] Determining the position similarity between each cell recognition image in the first image frame and each cell recognition image in the second image frame according to the position mark information of the cell recognition image;

[0129] According to the cell similarity and position similarity, on the basis of the cell counting and marking results in the first image frame, the cell counting and marking results in the second image frame are determined to achieve embryonic cell tracking.

[0130] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0131] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0132] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0133] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0135] The embodiments of the present disclosure also provide a computer program product, including a computer program, which, when executed by a processor, implements the embryonic cell tracking method provided in any one of the embodiments of the present disclosure.

[0136] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code for the disclosed operation, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0137] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for tracking embryonic cells, characterized in that: include: Acquire an embryo image sequence to be analyzed, and input image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence to obtain a cell recognition result in each image frame; Perform image segmentation based on the cell recognition results to obtain corresponding cell recognition images, and input the cell recognition images into an image feature analysis model to obtain image features corresponding to each of the cell recognition images; Based on the image features, each of the cell recognition images in the first image frame in any two consecutive image frames is calculated and compared with each of the cell recognition images in the second image frame in the latter, to determine the similarity of cell features; Determining, according to the position mark information of the cell recognition images, the position similarity between each of the cell recognition images in the first image frame and each of the cell recognition images in the second image frame; According to the cell similarity and the position similarity, on the basis of the cell counting and marking results in the first image frame, the cell counting and marking results in the second image frame are determined to achieve embryonic cell tracking.

2. The method according to claim 1, characterized in that The step of calculating, based on the image features, each of the cell identification images in the first image frame in any two consecutive image frames and each of the cell identification images in the second image frame in the latter image frame for comparative analysis to determine the similarity of cell features comprises: Calculating the cosine similarity and the Euclidean distance between the image features of each of the cell recognition images in the first image frame and the image features of each of the cell recognition images in the second image frame; A weighted sum is calculated for the cosine similarity and the Euclidean distance to obtain the cell feature similarity of the corresponding cell recognition image pair.

3. The method according to claim 1, characterized in that The step of determining the position similarity between each of the cell recognition images in the first image frame and each of the cell recognition images in the second image frame according to the position mark information of the cell recognition images comprises: Calculating the intersection-and-union ratio of corresponding cell image pairs according to the position marking information of each cell recognition image in the first image frame and the position marking information of each cell recognition image in the second image frame; The intersection-over-union ratio is used as the position similarity of the corresponding cell image pair.

4. The method according to claim 1, characterized in that: The step of determining the cell counting and marking result in the second image frame based on the cell counting and marking result in the first image frame according to the cell similarity and the position similarity includes: Determining similarity values ​​of corresponding cell image pairs according to the cell similarity and the position similarity; When the similarity value satisfies a preset similarity threshold, determining that the corresponding pair of cell images are the same cell; The cell counting and marking result of the cells in the cell image pair in the first image frame is determined as the cell counting and marking result of the cells in the cell image pair in the second image frame.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: According to the cell counting and marking results of each image frame in the embryo image sequence to be analyzed, determining the marked missing image frames having the missing cell marking frames; Determine the coordinates of the missing cell marker frame according to the coordinate information of the cell marker frame corresponding to the missing cell marker frame in the two non-marker missing image frames that are closest in time to the marked missing image frame, wherein the two non-marker missing image frames include an image frame before the marked missing image frame and an image frame after the marked missing image frame; Based on the coordinates of the missing cell marker frame, a corresponding cell marker frame is supplemented in the marker-missing image frame.

6. The method according to claim 5, characterized in that The step of determining the coordinates of the missing cell marking frame according to the coordinate information of the cell marking frame corresponding to the missing cell marking frame in the two non-marked missing image frames that are closest in time to the marked missing image frame comprises: According to the coordinate information of the cell marker frame corresponding to the missing cell marker frame in the two non-marker missing image frames closest in time to the mark missing image frame, a linear interpolation calculation is performed to obtain a calculated coordinate result; The coordinates of the missing cell marking frame are determined based on the calculated coordinate results.

7. An embryonic cell tracking device, characterized in that: include: A cell recognition module is used to obtain an embryo image sequence to be analyzed, and input the image frames in the embryo image sequence to be analyzed into a preset cell recognition model in sequence to obtain a cell recognition result in each image frame; An image feature analysis model is used to perform image segmentation based on the cell recognition results to obtain corresponding cell recognition images, and input the cell recognition images into the image feature analysis model to obtain image features corresponding to each of the cell recognition images; A cell feature similarity determination module is used to respectively calculate, based on the image features, each of the cell identification images in the first image frame in any two consecutive image frames and each of the cell identification images in the second image frame in the latter to compare and analyze, and determine the cell feature similarity; a position similarity determination module, used to determine the position similarity between each of the cell recognition images in the first image frame and each of the cell recognition images in the second image frame according to the position mark information of the cell recognition images; The cell counting mark result determination module is used to determine the cell counting mark result in the second image frame based on the cell counting mark result in the first image frame according to the cell similarity and the position similarity, so as to realize embryonic cell tracking.

8. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the embryonic cell tracking method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the embryonic cell tracking method as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the embryonic cell tracking method according to any one of claims 1 to 6.

Images