Ordered image extraction method, device, electronic equipment and medium

Abstract

The present invention provides an ordered image extraction method, device, electronic device and medium, and relates to the field of image processing technology. The method comprises: performing tail swing analysis on target sperm in a sperm swimming video image to obtain a contour image sequence; performing tail skeleton extraction on each frame of the contour image to obtain a tail skeleton sequence; selecting a reference tail skeleton from the tail skeleton sequence and moving the frame number corresponding to the reference tail skeleton to obtain multiple target frame numbers; calculating the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set; determining the start frame and end frame of the sperm tail swing cycle according to the maximum value in the similarity set; and intercepting the contour image between the start frame and the end frame from the contour image sequence to obtain an image sequence corresponding to the sperm tail swing cycle. The scheme of the present invention can identify a complete sperm tail swing cycle, and the obtained sperm tail swing cycle has a high accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an ordered image extraction method, device, electronic equipment and medium. Background Art

[0002] Sperm is the reproductive cell of male animals. Although it is tiny, it is a highly specialized mobile cell, which is mainly composed of a head and a tail. The head is the carrier of the sperm's genetic material, while the tail is the sperm's movement device. During the sperm's swimming process, the tail swings in a dynamic and regular manner. In theory, the sperm swimming video should be able to record the periodic changes in the tail swing.

[0003] At present, most of the existing sperm video images are recorded in specific scenes, but the image of the regular swinging of the tail cannot be segmented from the video. The main reasons why the regular swinging of the tail cannot be segmented are: on the one hand, the time accuracy of the video recording is completely not at the same order of magnitude as the time of the regular movement of the sperm, so it is impossible to directly obtain the sperm swimming image in units of cycles; on the other hand, if the video image is captured based on the theoretical sperm swinging time, the obtained image is not applicable to all sperm and has poor accuracy. Summary of the invention

[0004] The present invention provides an ordered image extraction method, device, electronic equipment and medium, which are used to solve the defect in the prior art that the tail swing cycle of sperm cannot be segmented during the swimming process, and realize automatic identification of the sperm tail swing cycle.

[0005] According to a first aspect of the present invention, the present invention provides an ordered image extraction method, the method comprising:

[0006] Perform tail swing analysis on target sperm in the sperm swimming video image to obtain a contour image sequence arranged according to frame numbers;

[0007] Extract the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number;

[0008] Selecting a reference tail skeleton from the tail skeleton sequence, and moving the frame number corresponding to the reference tail skeleton using a preset frame number moving rule to obtain a plurality of target frame numbers;

[0009] Calculate the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set;

[0010] Determine the start frame and the end frame of the sperm tail swing cycle according to the maximum value in the similarity set;

[0011] The contour image between the start frame and the end frame is intercepted from the contour image sequence to obtain an image sequence corresponding to the sperm tail swinging period.

[0012] In some possible implementations, before extracting the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number, the method further includes:

[0013] The contour image sequence is screened based on a preset head contour area threshold interval and a preset tail midline length threshold interval to obtain a screened contour image sequence;

[0014] Segmenting the filtered contour image sequence based on the continuity of frame numbers to obtain at least one segment, wherein each segment includes at least two frames of contour images and the frame numbers of contour images belonging to the same segment are continuous;

[0015] In response to the total number of frames of the contour image in any segment exceeding a preset number of frames, the any segment is taken as a stable segment.

[0016] In some possible implementations, the screening of the contour image sequence based on a preset head contour area threshold interval and a preset tail midline length threshold interval to obtain a screened contour image sequence includes:

[0017] Calculate the area of ​​the head contour and the midline length of the tail contour in each frame of the contour image;

[0018] The following operations are performed by traversing each frame of the contour image: determining whether the area of ​​the head contour belongs to the preset head contour area threshold interval; determining whether the midline length of the tail contour belongs to the preset tail midline length threshold interval; if the area of ​​the head contour does not belong to the preset head contour area threshold interval and / or the midline length of the tail contour does not belong to the preset tail midline length threshold interval, deleting the contour image from the contour image sequence, otherwise retaining the contour image;

[0019] The following operations are performed in a manner of traversing each frame of the retained contour image: the areas of the head contours in all the contour images retained in the contour image sequence are standardized by the standard fraction method to obtain a first standardized value; the midline lengths of the tail contours in all the contour images retained in the contour image sequence are standardized by the standard fraction method to obtain a second standardized value; and the retained contour image is retained again when both the first standardized value and the second standardized value are smaller than a preset value, otherwise the retained contour image is deleted.

[0020] In some possible implementations, extracting the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number includes:

[0021] The tail skeleton is extracted from each frame of contour image in each stable segment using the skeleton extraction algorithm;

[0022] For each stable segment, the tail skeletons extracted from each frame of the contour image are arranged in the order of frame numbers to obtain a tail skeleton sequence corresponding to each stable segment.

[0023] In some possible implementations, selecting a reference tail skeleton from the tail skeleton sequence, and moving the frame number corresponding to the reference tail skeleton using a preset frame number moving rule to obtain multiple target frame numbers include:

[0024] The first three tail skeletons in the tail skeleton sequence are used as the first reference tail skeleton, the second reference tail skeleton and the third reference tail skeleton in sequence;

[0025] Iterate over each offset value in the preset frame number offset set and do the following:

[0026] Calculate the sum of the frame number corresponding to the first reference tail skeleton and the offset value to obtain a first target frame number;

[0027] Calculate the sum of the frame number corresponding to the second reference tail skeleton and the offset value to obtain a second target frame number;

[0028] The third target frame number is obtained by calculating the sum of the frame number corresponding to the third reference tail skeleton and the offset value.

[0029] In some possible implementations, calculating the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set includes:

[0030] Calculating the similarity between the tail skeleton corresponding to the first target frame number and the first reference tail skeleton by a dynamic time warping algorithm to obtain a first similarity;

[0031] Calculating the similarity between the tail skeleton corresponding to the second target frame number and the second reference tail skeleton by a dynamic time warping algorithm to obtain a second similarity;

[0032] Calculating the similarity between the tail skeleton corresponding to the third target frame number and the third reference tail skeleton by a dynamic time warping algorithm to obtain a third similarity;

[0033] Calculating the average of the first similarity, the second similarity and the third similarity to obtain a similarity average;

[0034] A set consisting of all similarity means is taken as the similarity set.

[0035] In some possible implementations, determining the start frame and the end frame of the sperm tail swing cycle according to the maximum value in the similarity set includes:

[0036] Obtaining the maximum similarity mean in the similarity set;

[0037] Using the frame number corresponding to the first reference tail skeleton as the starting frame, and using the first target frame number corresponding to the maximum similarity mean as the ending frame; or

[0038] Using the frame number corresponding to the second reference tail skeleton as the starting frame, and using the second target frame number corresponding to the maximum similarity mean as the ending frame; or

[0039] The frame number corresponding to the third reference tail skeleton is used as the starting frame, and the third target frame number corresponding to the maximum similarity mean is used as the ending frame.

[0040] According to a second aspect of the present invention, the present invention also provides an ordered image extraction device, the device comprising:

[0041] An analysis module, the analysis module is used to perform tail swing analysis on target sperm in the sperm swimming video image to obtain a contour image sequence arranged according to frame numbers;

[0042] An extraction module, the extraction module is used to extract the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number;

[0043] A moving module, the moving module is used to select a reference tail skeleton from the tail skeleton sequence, and move the frame number corresponding to the reference tail skeleton using a preset frame number moving rule to obtain multiple target frame numbers;

[0044] A similarity calculation module, wherein the similarity calculation module is used to calculate the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set;

[0045] A determination module, the determination module is used to determine the start frame and the end frame of the sperm tail swing cycle according to the maximum value in the similarity set;

[0046] A capture module is used to capture the contour image between the start frame and the end frame from the contour image sequence to obtain the image sequence corresponding to the sperm tail swinging period.

[0047] According to the third aspect of the present invention, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an ordered image extraction method as described above is implemented.

[0048] According to a fourth aspect of the present invention, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the ordered image extraction method as described in any one of the above is implemented.

[0049] The invention provides an ordered image extraction method, which firstly performs tail swing analysis on target sperm in a sperm swimming video image to extract a contour image sequence, then performs tail skeleton extraction to obtain a tail skeleton sequence arranged according to frame numbers, then selects a reference tail skeleton from the tail skeleton sequence and uses a preset frame number movement rule to move the frame number corresponding to the reference tail skeleton to generate a target frame number, and then calculates the similarity between the tail skeleton of each target frame and the reference tail skeleton, then uses the maximum similarity to determine the start frame and the end frame, and finally uses the start frame and the end frame to intercept the contour image sequence, thereby obtaining an image sequence corresponding to the sperm tail swing cycle, and combining the tail swing morphology recognition to identify a small range of ordered image sequences that can characterize the complete sperm tail swing cycle from a large range of sequence images, the obtained sperm tail swing cycle has high accuracy, is suitable for video images of various frame rates, has good versatility, and the intercepted contour image can present the changing state of the target sperm tail within a cycle, providing rich data support for studying sperm motility.

[0050] In addition, an ordered image extraction method device, an electronic device and a non-transitory computer-readable storage medium provided by the present invention can also achieve the above-mentioned technical effects, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 It is a schematic diagram of the process of the ordered image extraction method provided by the present invention;

[0053] Figure 2 is a schematic diagram of a contour image sequence provided by the present invention;

[0054] Figure 3 It is a schematic diagram of the sperm tail swing cycle captured by the invention;

[0055] Figure 4 It is a schematic diagram of a process of searching for a stable section provided by the present invention;

[0056] Figure 5 is a schematic diagram of the movement of the reference tail skeleton provided by the present invention;

[0057] Figure 6 This is a schematic diagram of the target sperm identification process provided by the present invention;

[0058] Figure 7 It is a schematic diagram of the tail swing analysis process provided by the present invention;

[0059] Figure 8 It is a structural schematic diagram of the ordered image extraction device provided by the present invention;

[0060] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Combine the following Figures 1 to 9 The present invention describes an ordered image extraction method, a sperm tail swing cycle device, an electronic device and a non-transitory computer-readable storage medium.

[0063] Figure 1 This is a flow chart of the ordered image extraction method provided by the present invention. Please refer to Figure 1 As shown, this embodiment provides an ordered image extraction method, which can be implemented by steps S101 to S106. Each step will be described in detail below:

[0064] Step S101, performing tail swing analysis on target sperm in the sperm swimming video image to obtain a contour image sequence arranged according to frame numbers.

[0065] Specifically, the sperm swimming video image refers to an image of sperm swimming in semen taken under a microscope, which can be acquired through existing video acquisition equipment, such as a professional video recorder, a terminal with a video recording function, or a camera. The frame rate and clarity of the sperm swimming video image can meet the requirements.

[0066] Specifically, the target sperm refers to the swimming sperm identified from the sperm swimming video image. The target sperm can be determined by using the existing sperm recognition technology, for example, selecting a frame image and identifying the sperm in the image as the target sperm, or identifying multiple frames of the sperm swimming video and finding the sperm in motion as the target sperm. The target sperm can be one or more.

[0067] Specifically, please refer to Figure 2 As shown, tail swing analysis refers to aligning the head of the target sperm in each frame, wherein the head alignment is to align the head contours after the second frame with the head contours in the first frame in terms of direction and size; after tail swing analysis, a contour image sequence of multiple frames of contour images arranged in order according to the frame number can be obtained. It should be noted that Figure 2 Only the superposition state of a part of the sperm contour image sequence of a certain sperm is shown. Figure 2 The image frames and states shown are for ease of understanding only and should not be construed as limitations on the solutions of the present invention.

[0068] Step S102, extracting the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number.

[0069] Specifically, tail skeleton extraction can be achieved through any existing skeleton extraction algorithm. In this embodiment, skeleton extraction of the tail contour can simplify the structural shape representation of the planar area where the tail contour is located. Tail skeleton extraction includes but is not limited to the use of medial axis transformation, Zhang-Suen algorithm and its related improved algorithms.

[0070] Step S103, selecting a reference tail skeleton from the tail skeleton sequence, and moving the frame number corresponding to the reference tail skeleton using a preset frame number moving rule to obtain a plurality of target frame numbers.

[0071] In this embodiment, the reference tail skeleton is selected arbitrarily. The reference tail skeleton is preferably selected from the front position of the contour sequence. The reference tail skeleton can be one or more. In the specific implementation process, the reference tail skeleton is preferably three tail skeletons of consecutive frames. The preset frame number movement rule refers to the rule for transforming the frame number to generate a new frame number. In the specific implementation process, the preset frame number movement rule can be achieved by setting a frame number offset range and a frame number offset step, and then realizing the frame number movement by window sliding to obtain the target frame number multiple times. It should be noted that the target frame number obtained each time the frame number is moved corresponds to the frame number corresponding to the reference tail skeleton. Assuming that there are two consecutive reference tail skeletons, each time the two reference tail skeleton frame numbers are moved, two corresponding target frame numbers are generated.

[0072] Step S104, calculating the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set.

[0073] In this embodiment, the similarity between two tail skeletons can be calculated by dynamic time warping algorithm, Euclidean distance, tail curvature, etc.; the number of elements in the similarity set is equal to the number of frame numbers corresponding to the moving reference tail skeleton. When there are multiple reference tail skeletons, it can be converted into a unique similarity by calculating the mean similarity of multiple target frames, the median similarity, etc.

[0074] Step S105, determining the start frame and the end frame of the sperm tail swing cycle according to the maximum value in the similarity set.

[0075] In this embodiment, the similarity set is composed of elements, and the maximum value is the maximum similarity value in the similarity set. When there are multiple reference tail skeletons, the maximum value is the similarity mean or the maximum value of the similarity median.

[0076] Step S106, extracting the contour image between the start frame and the end frame from the contour image sequence to obtain an image sequence corresponding to the sperm tail swinging period.

[0077] In this embodiment, interception refers to extracting a valid image sequence from the contour image, wherein the valid image sequence is the portion starting from the start frame and ending at the end frame, and the obtained image is the sperm tail swing cycle. For example, Figure 3 This is a schematic diagram of the sperm tail swing cycle captured by the invention, please refer to Figure 3 As shown, the contour image obtained by interception can clearly segment the sperm tail swing cycle, and each frame of the contour image within the sperm tail swing cycle shows the morphological changes of the sperm tail.

[0078] In the specific implementation process, in order to better quantify the sperm tail swing cycle, the sperm tail swing cycle time can also be calculated using the number of contour images of the sperm tail swing cycle. The specific implementation method of the sperm tail swing cycle time is as follows: calculate the ratio of the number of contour images of the sperm tail swing cycle to the frame rate to obtain the sperm tail swing cycle time.

[0079] Among them, the frame rate is a parameter for capturing video images of sperm swimming. The frame rate indicates the number of images displayed per second. For example, a camera frame rate parameter of 60fps means that 60 image frames are displayed in 1 second.

[0080] The ordered image extraction method of this embodiment first performs tail swing analysis on the target sperm in the sperm swimming video image to extract the contour image sequence, then performs tail skeleton extraction to obtain a tail skeleton sequence arranged according to frame numbers, then selects a reference tail skeleton from the tail skeleton sequence and uses a preset frame number movement rule to move the frame number corresponding to the reference tail skeleton to generate a target frame number, and then calculates the similarity between the tail skeleton of each target frame and the reference tail skeleton, then uses the maximum similarity to determine the start frame and the end frame, and finally uses the start frame and the end frame to intercept the contour image sequence, thereby obtaining an image sequence corresponding to the sperm tail swing cycle, and combining the tail swing morphology to identify a small range of ordered image sequences that can characterize the complete sperm tail swing cycle from a large range of sequence images. The obtained sperm tail swing cycle has high accuracy, is suitable for video images of various frame rates, has good versatility, and the intercepted contour image can present the changing state of the target sperm tail within a cycle, providing rich data support for studying sperm motility.

[0081] In some embodiments, please refer to Figure 4 As shown, in step S102, before extracting the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number, the ordered image extraction method further includes searching for a stable segment, which can be specifically achieved by the following steps:

[0082] Step S401, filtering the contour image sequence based on a preset head contour area threshold interval and a preset tail midline length threshold interval to obtain a filtered contour image sequence;

[0083] Step S402, segmenting the filtered contour image sequence based on the continuity of frame numbers to obtain at least one segment, wherein each segment includes at least two frames of contour images and the frame numbers of contour images belonging to the same segment are continuous;

[0084] Step S403: in response to the total number of frames of the contour image in any segment exceeding a preset number of frames, any segment is regarded as a stable segment.

[0085] In the specific implementation process, the preset frame number is preferably thirty, that is, the segment is regarded as a stable segment only when and only when the continuous frames exceed thirty frames. It should be noted that the frame number of the stable segment obtained in this way is not fixed, and the number of stable segments is not fixed. A contour image sequence may be divided into multiple stable segments, or there may be no stable segments. In the case where there are no stable segments, it can be considered that the target sperm swimming is unstable.

[0086] The ordered image extraction method of this embodiment first uses the head contour and the tail contour to filter the contour image sequence, and implements segmentation based on the continuity of the frame number of the contour image sequence after screening. Finally, the preset number of frames is used to implement screening of the segments, so as to find the stable swimming section of the sperm, which can further improve the accuracy of tail swinging cycle recognition.

[0087] In some embodiments, the aforementioned step S401, filtering the contour image sequence based on the preset head contour area threshold interval and the preset tail midline length threshold interval to obtain the filtered contour image sequence, can be specifically implemented by the following steps:

[0088] Calculate the area of ​​the head contour and the midline length of the tail contour in each frame of the contour image;

[0089] The following operations are performed by traversing each frame of the contour image: determining whether the area of ​​the head contour belongs to a preset head contour area threshold interval; determining whether the midline length of the tail contour belongs to a preset tail midline length threshold interval; if the area of ​​the head contour does not belong to the preset head contour area threshold interval and / or the midline length of the tail contour does not belong to the preset tail midline length threshold interval, deleting the contour image from the contour image sequence, otherwise retaining the contour image; wherein the preset head contour area threshold interval is preferably [50, 1500], and the preset tail midline length threshold interval is preferably [100, 1000].

[0090] The following operations are performed by traversing each frame of the retained contour image: the area of ​​the head contour in all the contour images retained in the contour image sequence is standardized by the standard fraction method to obtain a first standardized value; the midline length of the tail contour in all the contour images retained in the contour image sequence is standardized by the standard fraction method to obtain a second standardized value; if both the first standardized value and the second standardized value are less than the preset value, the retained contour image is retained again, otherwise the retained contour image is deleted. The preset value used for comparison with the first standardized value and the second standardized value is preferably set to 3.

[0091] Exemplarily, the process of determining the stable segment is as follows:

[0092] (1) For the target sperm n, calculate its head contour The area of Represents the area of ​​the head outline of the nth target sperm in the tth frame of the video.

[0093] (2) For the target sperm n, calculate its tail profile The length of the center line, It represents the midline length of the tail outline of the nth target sperm in the tth frame of the video.

[0094] (3) For the outline of the target sperm n in the tth frame of the video, if and Then the contour of the target sperm n in the t-th frame image is retained; otherwise, the contour of the target sperm n in the t-th frame image is deleted.

[0095] (4) Using the standard score method (z-score) to calculate the head contour area vector in the video frame retained by the target sperm n After standardization, the head contour area vector after standardization is

[0096] (5) Use z-score to calculate the tail contour midline length vector in the video frame retained by the target sperm n After standardization, the tail contour midline length vector is

[0097] (6) If the maximum value of the absolute value of the standardized head contour area and the absolute value of the standardized tail contour midline length is less than 3, that is, Then the contour of the target sperm in the p-th frame image n is retained; otherwise, the contour of the target sperm in the p-th frame image n is deleted.

[0098] (7) For all video frames of the target sperm n that are finally retained after processing (3) and (6), extract the segments with continuous frame numbers based on the frame number sequence, and use Slot w Indicates the wth segment extracted. Use |Slot w | represents the length of segment w, that is, the number of video frames contained in segment w.

[0099] (8) If |Slot w |>30, then segment w is the stable segment where the target sperm n swims in the two-dimensional plane.

[0100] The ordered image extraction method of this embodiment, combined with a head contour area of ​​a specific size and a tail midline length of a specific length, realizes the screening of contour image sequences, eliminates sperm contours with poor imaging, and avoids the impact of poor imaging of some parts of the target sperm on subsequent stable segment analysis and cycle identification.

[0101] In some embodiments, the aforementioned step S102, extracting the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number, can be specifically implemented by the following steps:

[0102] The tail skeleton is extracted from each frame of contour image in each stable segment using the skeleton extraction algorithm;

[0103] For each stable segment, the tail skeletons extracted from each frame of the contour image are arranged in the order of frame numbers to obtain a tail skeleton sequence corresponding to each stable segment.

[0104] It should be noted that the tail skeleton sequence corresponds to the stable segment one by one. Let's assume that the original frame number of a sperm swimming video image is 120 frames. After screening the head contour, tail contour and segmentation screening, the frame number range of the first stable segment is from the 2nd frame to the 43rd frame, and the frame number range of the second stable segment is from the 64th frame to the 95th frame. At this time, the tail skeletons corresponding to the 2nd frame to the 43rd frame are arranged in sequence to obtain a tail skeleton sequence, and the tail skeletons from the 64th frame to the 95th frame are arranged in sequence to obtain another tail skeleton sequence.

[0105] The ordered image extraction method of this embodiment combines and sorts the tail estimates of each stable segment respectively to obtain an ordered tail skeleton sequence corresponding to each stable period, so that the tail swinging period can be analyzed in units of stable segments later, making the division of the sperm tail swinging period more accurate and improving the accuracy of the evaluation of the sperm swimming tail swing.

[0106] In some embodiments, the aforementioned step S103, selecting a reference tail skeleton from the tail skeleton sequence, and moving the frame number corresponding to the reference tail skeleton using a preset frame number moving rule to obtain multiple target frame numbers, can be specifically implemented by the following steps:

[0107] The first three tail skeletons in the tail skeleton sequence are used as the first reference tail skeleton, the second reference tail skeleton and the third reference tail skeleton in sequence;

[0108] Iterate over each offset value in the preset frame number offset set and do the following:

[0109] Calculate the sum of the frame number corresponding to the first reference tail skeleton and the offset value to obtain the first target frame number;

[0110] Calculate the sum of the frame number corresponding to the second reference tail skeleton and the offset value to obtain the second target frame number;

[0111] The third target frame number is obtained by calculating the sum of the frame number corresponding to the third reference tail skeleton and the offset value.

[0112] For example, Figure 5 This is a reference tail skeleton movement diagram provided by the present invention, please refer to Figure 5 As shown in the figure, assuming that the selected reference tail skeleton is three consecutive frames, the reference tail skeleton of these three frames is used as a window, and the window is moved back one frame at a time from the position of the window to achieve window sliding, so as to obtain each target frame number in turn. Assuming that a certain stable segment is represented as Slot w, the tail contour in the g-th frame is represented as use Indicates that the target sperm n is in the stable swimming segment Slot w The skeleton of the tail outline in the g-th frame image, then the relationship between the reference tail skeleton and the tail skeleton corresponding to the target frame is and correspond, and correspond, and Corresponding to, where e∈[5,|Slot w |-3].

[0113] In some embodiments, the aforementioned step S104, calculating the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton by a dynamic time warping algorithm to obtain a similarity set, can be specifically implemented by the following steps:

[0114] Calculating the similarity between the tail skeleton corresponding to the first target frame number and the first reference tail skeleton by a dynamic time warping algorithm to obtain a first similarity;

[0115] Calculating the similarity between the tail skeleton corresponding to the second target frame number and the second reference tail skeleton by a dynamic time warping algorithm to obtain a second similarity;

[0116] Calculating the similarity between the tail skeleton corresponding to the third target frame number and the third reference tail skeleton by a dynamic time warping algorithm to obtain a third similarity;

[0117] Calculate the average of the first similarity, the second similarity and the third similarity to obtain the similarity average;

[0118] The set consisting of all similarity means is taken as the similarity set.

[0119] For example, the dynamic time warping (DTW) algorithm is used to calculate and Similarity and Similarity and Similarity Use S e Represents a similarity set In the specific implementation process, in order to reduce the amount of calculation, the set {S e} can be calculated by letting e traverse all the values ​​of [thd_l, thd_r], the set {S eThe minimum value of} is the period of sperm tail swing. For human sperm, when the video frame rate is 60, the value range of thd_l is [5, 10], and the value range of thd_r is [20, 40].

[0120] In some embodiments, the aforementioned step S105, determining the start frame and the end frame of the sperm tail swing cycle according to the maximum value in the similarity set, can be specifically implemented by the following steps:

[0121] Get the maximum similarity mean in the similarity set;

[0122] The frame number corresponding to the first reference tail skeleton is used as the starting frame, and the first target frame number corresponding to the maximum similarity mean is used as the ending frame; or

[0123] The frame number corresponding to the second reference tail skeleton is used as the starting frame, and the second target frame number corresponding to the maximum similarity mean is used as the ending frame; or

[0124] The frame number corresponding to the third reference tail skeleton is used as the starting frame, and the third target frame number corresponding to the maximum similarity mean is used as the ending frame.

[0125] For some possible implementations, see Figure 6 As shown, the target sperm in the previous step S101 can be identified by the following steps:

[0126] Step S601, selecting a frame from the sperm swimming video image as a reference frame;

[0127] Step S602, performing sperm head recognition on the reference frame to obtain a plurality of candidate sperms and a candidate head contour corresponding to each candidate sperm;

[0128] Step S603, target tracking is performed on the remaining frames of the sperm swimming video image except the reference frame based on the candidate head contour corresponding to each candidate sperm, to obtain the tracking area of ​​each candidate sperm in each remaining frame;

[0129] Step S604, searching for the contour center of each tracking area to obtain the sperm contour center of each candidate sperm in the remaining frames except the reference frame;

[0130] Step S605, calculating the Euclidean distance of the center of the sperm contour in adjacent frames for each candidate sperm, and obtaining the maximum Euclidean distance corresponding to each candidate sperm;

[0131] Step S606: The candidate sperm whose maximum Euclidean distance corresponding to the candidate sperm exceeds the preset distance is taken as the target sperm.

[0132] For ease of understanding, the following uses the first frame of the sperm swimming video image as a reference frame to explain in detail the process of determining the target sperm and the process of obtaining the target sperm's outline in each frame:

[0133] First, a video image of sperm swimming is obtained; and a video of sperm swimming is obtained by recording a video under a microscope lens with a preset magnification through a camera, wherein the preset magnification refers to a pre-set magnification of the microscope lens. In the specific implementation process, the microscope lens is preferably a ten-fold lens, and the first frame of the sperm swimming video image is used as a reference frame.

[0134] Next, the sperm head is identified; the identification process is as follows: obtain the first frame of the sperm swimming video, convert the image into a grayscale image, calculate the cumulative probability function (CDF) of the image pixel value, and detect the inflection point in the CDF function through an inflection point detection algorithm such as knee. i represents the detected i-th inflection point, p i Indicates the pixel value corresponding to the detected i-th inflection point; if the image is a white background, find p i The value p in the interval [10, 100] th If there are multiple values ​​in the interval [10, 100], take the minimum value as p th ; If the image has a black background, then find p i The value p in the interval [200, 250] th If there are multiple values ​​in the interval [200, 250], take the maximum value as p th ; Use the fixed threshold algorithm to binarize the image, and the fixed threshold is set to p th ; Use the contour detection algorithm to analyze the above binarized image, and the detected contour is the candidate sperm head contour.

[0135] Next, sperm swimming tracking is performed; the swimming tracking process is as follows: the circumscribed ellipse of each candidate sperm head contour is calculated, and the center point of the circumscribed ellipse is used as the center point of the contour. Indicates the center point of the head outline of the mth candidate sperm in the first frame of the video. All candidate sperm are tracked by the multi-target tracking algorithm of computer vision. The multi-target tracking algorithm needs to set the tracking area of ​​each tracked object. For the mth candidate sperm, the tracking area of ​​the sperm is set to the center point of the candidate sperm head outline. A square area with a side length of 400 pixels and centered on Represents the tracking area of ​​the mth candidate sperm in the tth frame image of the video.

[0136] Then, determine the true sperm and get the tail outline; the specific process is as follows: for the mth candidate sperm, The area is converted into a grayscale image; the head contour of the mth candidate sperm is calculated by sperm head recognition, and the Represents the head outline of the mth candidate sperm in the tth frame of the video, using represents the center point of the head contour of the mth candidate sperm in the tth frame of the video; the image is binarized using an adaptive threshold algorithm, which uses a Gaussian weighted sum method; the binarized image is analyzed using a contour detection algorithm; for all detected contours, the center point of the head contour of the candidate sperm m is calculated Is it inside the contour? If it is inside a certain contour, then the contour is the contour of the candidate sperm m. Represents the outline of the mth candidate sperm in the tth frame of the video; calculate The center point of the contour, Represents the center point of the outline of the mth candidate sperm in the tth frame of the video; first calculate and The Euclidean distance between them; then find the maximum value of all distances Max dsp ; If Max dsp >2, then the candidate sperm m is the correct sperm, and n represents the true sperm; otherwise, delete the candidate sperm m; for the contour of the true sperm n Remove the sperm head outline The remaining part is the tail outline of sperm n, Represents the tail of the real sperm n in the tth frame of the video.

[0137] Finally, the determined true sperm is taken as the target sperm, thereby obtaining the complete outline, head outline and tail outline of each target sperm in each frame.

[0138] For some possible implementations, see Figure 7 As shown, the aforementioned step S101, performing tail swing analysis on the target sperm in the sperm swimming video image to obtain a contour image sequence arranged according to frame numbers, can be specifically implemented by the following steps:

[0139] Step S701, extracting the head contour and tail contour of the target sperm from the first frame of the sperm swimming video image to obtain a first contour image with a frame number;

[0140] Step S702, extracting the head contour and tail contour of the target sperm from the remaining frames of the sperm swimming video image except the first frame, and obtaining a second contour image with a frame number;

[0141] Step S703, registering the head contour in each second contour image with the head contour in the first contour image, to obtain an affine transformation matrix corresponding to each second contour image;

[0142] Step S704, transforming the corresponding second contour image based on the affine transformation matrix to obtain a transformed second contour image;

[0143] Step S705 , sorting the first contour image and all transformed second contour images based on the frame number to obtain a contour image sequence arranged according to the frame number.

[0144] In the aforementioned step S703, before registering the head contour in each second contour image with the head contour in the first contour image, the following step is further included:

[0145] respectively calculating the circumscribed triangle of the head contour in the first contour image and the head contour in each second contour image;

[0146] The three vertices of each circumscribed triangle are sorted according to the sum of the distances from each vertex of each circumscribed triangle to the remaining vertices to obtain three vertices with order.

[0147] Before the aforementioned step S703, respectively registering the head contour in each second contour image with the head contour in the first contour image, the method further includes:

[0148] respectively calculating the inscribed triangle of the head contour in the first contour image and the head contour in each second contour image;

[0149] The three vertices of each inscribed triangle are sorted according to the sum of the distances from each vertex of each inscribed triangle to the remaining vertices to obtain three vertices with order.

[0150] The previous step 703, registering the head contour in each second contour image with the head contour in the first contour image, to obtain an affine transformation matrix corresponding to each second contour image, can be specifically implemented by the following steps:

[0151] In a traversal manner, take the head contour in each second contour image as the target contour and perform the following operations:

[0152] Match the three vertices corresponding to each target contour with the three vertices corresponding to the first contour image in vertex order to obtain a vertex correspondence relationship;

[0153] An affine transformation matrix is ​​calculated using the three vertex coordinates corresponding to the target contour and the three vertex coordinates corresponding to the first contour image as well as the vertex correspondence.

[0154] The aforementioned step S704, transforming the corresponding second contour image based on the affine transformation matrix to obtain the transformed second contour image, includes:

[0155] Take each second contour image as the target contour image in a traversal manner and perform the following operations:

[0156] The coordinates of the target contour image are transformed using the affine transformation matrix corresponding to the second contour image to which the target contour belongs.

[0157] The ordered image extraction device provided by the present invention is described below. The ordered image extraction device described below and the ordered image extraction method described above can be referred to each other.

[0158] Please refer to Figure 8 As shown, this embodiment provides an ordered image extraction device, specifically, the device includes: an analysis module 810, an extraction module 820, a movement module 830, a similarity calculation module 840, a determination module 850 and a capture module 860, and the above modules are described in detail below:

[0159] An analysis module 810 is used to perform tail swing analysis on target sperm in the sperm swimming video image to obtain a contour image sequence arranged according to frame numbers;

[0160] An extraction module 820 is used to extract the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number;

[0161] A moving module 830 is used to select a reference tail skeleton from the tail skeleton sequence, and move the frame number corresponding to the reference tail skeleton using a preset frame number moving rule to obtain multiple target frame numbers;

[0162] A similarity calculation module 840 is used to calculate the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set;

[0163] A determination module 850 is used to determine the start frame and the end frame of the sperm tail swing cycle according to the maximum value in the similarity set;

[0164] The interception module 860 is used to intercept the contour image between the start frame and the end frame from the contour image sequence to obtain an image sequence corresponding to the sperm tail swinging period.

[0165] In some possible implementations, the ordered image extraction device further includes:

[0166] A contour screening module, which is used to screen the contour image sequence based on a preset head contour area threshold interval and a preset tail midline length threshold interval to obtain a screened contour image sequence;

[0167] A segmentation module, the segmentation module is used to segment the filtered contour image sequence based on the continuity of frame numbers to obtain at least one segment, wherein each segment includes at least two frames of contour images and the frame numbers of contour images belonging to the same segment are continuous;

[0168] The stable segment determination module is used for taking any segment as a stable segment in response to the total number of frames of the contour image in any segment exceeding a preset number of frames.

[0169] In some possible implementations, the profile screening module is further used to:

[0170] Calculate the area of ​​the head contour and the midline length of the tail contour in each frame of the contour image;

[0171] Perform the following operations by traversing each frame of the contour image: determine whether the area of ​​the head contour belongs to a preset head contour area threshold interval; determine whether the midline length of the tail contour belongs to a preset tail midline length threshold interval; if the area of ​​the head contour does not belong to the preset head contour area threshold interval and / or the midline length of the tail contour does not belong to the preset tail midline length threshold interval, delete the contour image from the contour image sequence, otherwise retain the contour image;

[0172] The following operations are performed in a manner of traversing each frame of the retained contour image: the areas of the head contours in all the contour images retained in the contour image sequence are standardized by the standard fraction method to obtain a first standardized value; the midline lengths of the tail contours in all the contour images retained in the contour image sequence are standardized by the standard fraction method to obtain a second standardized value; the retained contour image is retained again when both the first standardized value and the second standardized value are less than a preset value, otherwise the retained contour image is deleted.

[0173] In some possible implementations, the extraction module 820 is further configured to:

[0174] The tail skeleton is extracted from each frame of contour image in each stable segment using the skeleton extraction algorithm;

[0175] For each stable segment, the tail skeletons extracted from each frame of the contour image are arranged in the order of frame numbers to obtain a tail skeleton sequence corresponding to each stable segment.

[0176] In some possible implementations, the moving module 830 is further configured to:

[0177] The first three tail skeletons in the tail skeleton sequence are used as the first reference tail skeleton, the second reference tail skeleton and the third reference tail skeleton in sequence;

[0178] Iterate over each offset value in the preset frame number offset set and do the following:

[0179] Calculate the sum of the frame number corresponding to the first reference tail skeleton and the offset value to obtain the first target frame number;

[0180] Calculate the sum of the frame number corresponding to the second reference tail skeleton and the offset value to obtain the second target frame number;

[0181] The third target frame number is obtained by calculating the sum of the frame number corresponding to the third reference tail skeleton and the offset value.

[0182] In some possible implementations, the similarity calculation module 840 is further configured to:

[0183] Calculating the similarity between the tail skeleton corresponding to the first target frame number and the first reference tail skeleton by a dynamic time warping algorithm to obtain a first similarity;

[0184] Calculating the similarity between the tail skeleton corresponding to the second target frame number and the second reference tail skeleton by a dynamic time warping algorithm to obtain a second similarity;

[0185] Calculating the similarity between the tail skeleton corresponding to the third target frame number and the third reference tail skeleton by a dynamic time warping algorithm to obtain a third similarity;

[0186] Calculate the average of the first similarity, the second similarity and the third similarity to obtain the similarity average;

[0187] The set consisting of all similarity means is taken as the similarity set.

[0188] In some possible implementations, the determination module 850 is further configured to:

[0189] Get the maximum similarity mean in the similarity set;

[0190] Using the frame number corresponding to the first reference tail skeleton as the starting frame, and using the first target frame number corresponding to the maximum similarity mean as the ending frame; or

[0191] Using the frame number corresponding to the second reference tail skeleton as the starting frame, and using the second target frame number corresponding to the maximum similarity mean as the ending frame; or

[0192] The frame number corresponding to the third reference tail skeleton is used as the starting frame, and the third target frame number corresponding to the maximum similarity mean is used as the ending frame.

[0193] Each module in the above-mentioned ordered image extraction device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0194] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the ordered image extraction method, which includes: performing tail swing analysis on the target sperm in the sperm swimming video image to obtain a contour image sequence arranged according to the frame number; performing tail skeleton extraction on each frame contour image to obtain a tail skeleton sequence arranged according to the frame number; selecting a reference tail skeleton from the tail skeleton sequence, and moving the frame number corresponding to the reference tail skeleton using a preset frame number movement rule to obtain multiple target frame numbers; calculating the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set; determining the start frame and end frame of the sperm tail swing cycle according to the maximum value in the similarity set; intercepting the contour image between the start frame and the end frame from the contour image sequence to obtain an image sequence corresponding to the sperm tail swing cycle.

[0195] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0196] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the ordered image extraction method provided by the above-mentioned methods, which includes: performing tail swing analysis on the target sperm in the sperm swimming video image to obtain a contour image sequence arranged according to frame numbers; performing tail skeleton extraction on each frame of the contour image to obtain a tail skeleton sequence arranged according to frame numbers; selecting a reference tail skeleton from the tail skeleton sequence, and moving the frame number corresponding to the reference tail skeleton using a preset frame number movement rule to obtain multiple target frame numbers; calculating the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set; determining the start frame and end frame of the sperm tail swing cycle according to the maximum value in the similarity set; and intercepting the contour image between the start frame and the end frame from the contour image sequence to obtain an image sequence corresponding to the sperm tail swing cycle.

[0197] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the ordered image extraction method provided by the above-mentioned methods, the method comprising: performing tail swing analysis on the target sperm in the sperm swimming video image to obtain a contour image sequence arranged according to frame numbers; performing tail skeleton extraction on each frame of the contour image to obtain a tail skeleton sequence arranged according to frame numbers; selecting a reference tail skeleton from the tail skeleton sequence, and moving the frame number corresponding to the reference tail skeleton using a preset frame number movement rule to obtain multiple target frame numbers; calculating the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set; determining the start frame and end frame of the sperm tail swing cycle based on the maximum value in the similarity set; and capturing the contour image between the start frame and the end frame from the contour image sequence to obtain an image sequence corresponding to the sperm tail swing cycle.

[0198] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0199] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting an ordered image, characterized in that: The method comprises: Perform tail swing analysis on target sperm in the sperm swimming video image to obtain a contour image sequence arranged according to frame numbers; Extract the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number; Selecting a reference tail skeleton from the tail skeleton sequence, and moving the frame number corresponding to the reference tail skeleton using a preset frame number moving rule to obtain a plurality of target frame numbers; Calculate the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set; Determine the start frame and the end frame of the sperm tail swing cycle according to the maximum value in the similarity set; Intercepting the contour image between the start frame and the end frame from the contour image sequence to obtain an image sequence corresponding to the sperm tail swinging period; Before extracting the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number, the method further includes: The contour image sequence is screened based on a preset head contour area threshold interval and a preset tail midline length threshold interval to obtain a screened contour image sequence; Segmenting the filtered contour image sequence based on the continuity of frame numbers to obtain at least one segment, wherein each segment includes at least two frames of contour images and the frame numbers of contour images belonging to the same segment are continuous; In response to the total number of frames of the contour image in any segment exceeding a preset number of frames, the any segment is regarded as a stable segment; The step of screening the contour image sequence based on a preset head contour area threshold interval and a preset tail midline length threshold interval to obtain a screened contour image sequence includes: Calculate the area of ​​the head contour and the midline length of the tail contour in each frame of the contour image; The following operations are performed by traversing each frame of the contour image: determining whether the area of ​​the head contour belongs to the preset head contour area threshold interval; determining whether the midline length of the tail contour belongs to the preset tail midline length threshold interval; if the area of ​​the head contour does not belong to the preset head contour area threshold interval and / or the midline length of the tail contour does not belong to the preset tail midline length threshold interval, deleting the contour image from the contour image sequence, otherwise retaining the contour image; The following operations are performed in a manner of traversing each frame of the retained contour image: the areas of the head contours in all the contour images retained in the contour image sequence are normalized by a standard fraction method to obtain a first normalized value; the midline lengths of the tail contours in all the contour images retained in the contour image sequence are normalized by a standard fraction method to obtain a second normalized value; the retained contour image is retained again when both the first normalized value and the second normalized value are less than a preset value, otherwise the retained contour image is deleted; The step of extracting the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number includes: The tail skeleton is extracted from each frame of contour image in each stable segment using the skeleton extraction algorithm; For each stable segment, the tail skeletons extracted from each frame of the contour image are arranged in the order of frame numbers to obtain a tail skeleton sequence corresponding to each stable segment.

2. The ordered image extraction method according to claim 1, characterized in that: The step of selecting a reference tail skeleton from the tail skeleton sequence and moving the frame number corresponding to the reference tail skeleton using a preset frame number moving rule to obtain a plurality of target frame numbers includes: The first three tail skeletons in the tail skeleton sequence are used as the first reference tail skeleton, the second reference tail skeleton and the third reference tail skeleton in sequence; Iterate over each offset value in the preset frame number offset set and do the following: Calculate the sum of the frame number corresponding to the first reference tail skeleton and the offset value to obtain a first target frame number; Calculate the sum of the frame number corresponding to the second reference tail skeleton and the offset value to obtain a second target frame number; The third target frame number is obtained by calculating the sum of the frame number corresponding to the third reference tail skeleton and the offset value.

3. The ordered image extraction method according to claim 2, characterized in that: The calculating the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set includes: Calculating the similarity between the tail skeleton corresponding to the first target frame number and the first reference tail skeleton by a dynamic time warping algorithm to obtain a first similarity; Calculating the similarity between the tail skeleton corresponding to the second target frame number and the second reference tail skeleton by a dynamic time warping algorithm to obtain a second similarity; Calculating the similarity between the tail skeleton corresponding to the third target frame number and the third reference tail skeleton by a dynamic time warping algorithm to obtain a third similarity; Calculating the average of the first similarity, the second similarity and the third similarity to obtain a similarity average; A set consisting of all similarity means is taken as the similarity set.

4. The ordered image extraction method according to claim 3, characterized in that: The step of determining the start frame and the end frame of the sperm tail swing cycle according to the maximum value in the similarity set comprises: Obtaining the maximum similarity mean in the similarity set; Using the frame number corresponding to the first reference tail skeleton as the starting frame, and using the first target frame number corresponding to the maximum similarity mean as the ending frame; or Using the frame number corresponding to the second reference tail skeleton as the starting frame, and using the second target frame number corresponding to the maximum similarity mean as the ending frame; or The frame number corresponding to the third reference tail skeleton is used as the starting frame, and the third target frame number corresponding to the maximum similarity mean is used as the ending frame.

5. An ordered image extraction device, characterized in that: The device comprises: An analysis module, the analysis module is used to perform tail swing analysis on target sperm in the sperm swimming video image to obtain a contour image sequence arranged according to frame numbers; An extraction module, the extraction module is used to extract the tail skeleton from each frame of the contour image to obtain a tail skeleton sequence arranged according to the frame number; A moving module, the moving module is used to select a reference tail skeleton from the tail skeleton sequence, and move the frame number corresponding to the reference tail skeleton using a preset frame number moving rule to obtain multiple target frame numbers; A similarity calculation module, wherein the similarity calculation module is used to calculate the similarity between the tail skeleton corresponding to each target frame number and the reference tail skeleton to obtain a similarity set; A determination module, the determination module is used to determine the start frame and the end frame of the sperm tail swing cycle according to the maximum value in the similarity set; A capture module, the capture module is used to capture the contour image between the start frame and the end frame from the contour image sequence to obtain an image sequence corresponding to the sperm tail swinging period; A contour screening module, which is used to screen the contour image sequence based on a preset head contour area threshold interval and a preset tail midline length threshold interval to obtain a screened contour image sequence; A segmentation module, the segmentation module is used to segment the filtered contour image sequence based on the continuity of frame numbers to obtain at least one segment, wherein each segment includes at least two frames of contour images and the frame numbers of contour images belonging to the same segment are continuous; A stable segment determination module, the stable segment determination module is used to respond to the total number of frames of the contour image in any segment exceeding a preset number of frames, and then regard any segment as a stable segment; The contour screening module is further used to: Calculate the area of ​​the head contour and the midline length of the tail contour in each frame of the contour image; Perform the following operations by traversing each frame of the contour image: determine whether the area of ​​the head contour belongs to a preset head contour area threshold interval; determine whether the midline length of the tail contour belongs to a preset tail midline length threshold interval; if the area of ​​the head contour does not belong to the preset head contour area threshold interval and / or the midline length of the tail contour does not belong to the preset tail midline length threshold interval, delete the contour image from the contour image sequence, otherwise retain the contour image; The following operations are performed in a manner of traversing each frame of the retained contour image: the areas of the head contours in all the contour images retained in the contour image sequence are normalized by a standard fraction method to obtain a first normalized value; the midline lengths of the tail contours in all the contour images retained in the contour image sequence are normalized by a standard fraction method to obtain a second normalized value; if both the first normalized value and the second normalized value are less than a preset value, the retained contour image is retained again, otherwise the retained contour image is deleted; The extraction module is further used to: The tail skeleton is extracted from each frame of contour image in each stable segment using the skeleton extraction algorithm; For each stable segment, the tail skeletons extracted from each frame of the contour image are arranged in the order of frame numbers to obtain a tail skeleton sequence corresponding to each stable segment; The mobile module is further used to: The first three tail skeletons in the tail skeleton sequence are used as the first reference tail skeleton, the second reference tail skeleton and the third reference tail skeleton in sequence; Iterate over each offset value in the preset frame number offset set and do the following: Calculate the sum of the frame number corresponding to the first reference tail skeleton and the offset value to obtain the first target frame number; Calculate the sum of the frame number corresponding to the second reference tail skeleton and the offset value to obtain the second target frame number; Calculate the sum of the frame number corresponding to the third reference tail skeleton and the offset value to obtain the third target frame number; The similarity calculation module is further used to: Calculating the similarity between the tail skeleton corresponding to the first target frame number and the first reference tail skeleton by a dynamic time warping algorithm to obtain a first similarity; Calculating the similarity between the tail skeleton corresponding to the second target frame number and the second reference tail skeleton by a dynamic time warping algorithm to obtain a second similarity; Calculating the similarity between the tail skeleton corresponding to the third target frame number and the third reference tail skeleton by a dynamic time warping algorithm to obtain a third similarity; Calculate the average of the first similarity, the second similarity and the third similarity to obtain the similarity average; The set consisting of all similarity means is taken as the similarity set; The determination module is further used to: Get the maximum similarity mean in the similarity set; Using the frame number corresponding to the first reference tail skeleton as the starting frame, and using the first target frame number corresponding to the maximum similarity mean as the ending frame; or Using the frame number corresponding to the second reference tail skeleton as the starting frame, and using the second target frame number corresponding to the maximum similarity mean as the ending frame; or The frame number corresponding to the third reference tail skeleton is used as the starting frame, and the third target frame number corresponding to the maximum similarity mean is used as the ending frame.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the ordered image extraction method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the ordered image extraction method according to any one of claims 1 to 4 is implemented.

Images