A sperm quality assessment method and system based on multimodal fusion

Abstract

The present invention relates to the field of multimodal fusion technology, specifically a sperm quality assessment method and system based on multimodal fusion, comprising the following steps: acquiring image and parameter data to generate a spatiotemporal alignment set, extracting trajectory linearity, analyzing parameter fluctuations to mark abnormal time windows, extracting images to assess deformity rates to generate feature matrices, and dividing levels to generate multimodal sperm quality assessment results. In the present invention, by introducing linearity calculations in the trajectory path extraction process and combining them with spatial distribution thermodynamic mapping, the dynamic characteristics of the local sperm population are made explicit, an abnormal time window is constructed based on the parameter value change rate and a set threshold, and combined with a motion consistency index, the depth of interpretation of sperm head deformity data in quality judgment is enhanced, the dimensional space of sperm quality description is effectively expanded, the consistency ratio between vitality and morphology is introduced and combined with trajectory continuity features, a more hierarchical grading system is established to ensure the accuracy of sperm quality assessment results.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal fusion, and in particular to a sperm quality assessment method and system based on multimodal fusion. Background Art

[0002] The field of multimodal fusion technology encompasses comprehensive analytical methods that integrate clinical imaging data with semen parameter measurement data. The core of this approach lies in establishing a correlation model between standardized semen reports, ultrasound imaging features, and direct-connected data from laboratory equipment. This approach focuses on aligning heterogeneous data across multi-center databases. This includes cross-modal matching techniques for varicocele grading parameters, testicular volume measurements, and sperm trajectory parameters. Dynamic adaptation of sperm morphology thresholds is achieved through a WHO standard conversion module. Time series alignment algorithms are used to process the interval data from multiple semen analyses. Finally, a CNN-based automatic head vacuolar recognition system and an LSTM-driven sperm motility trend prediction model are developed.

[0003] Among them, the sperm quality assessment method and system based on multimodal fusion refers to the use of a parallel processing architecture of structured semen parameters and morphological staining images, covering the spatiotemporal correlation modeling of VCL motion trajectory data output by the computer-assisted semen analysis system and transrectal ultrasound imaging features, using ResNet to extract abnormal acrosome morphological features and synchronously integrate the ejaculatory duct structure ultrasound parameters, realizing quantitative analysis of the total active sperm count based on the derived indicator calculation framework, and using the standardized splicing technology in feature-level fusion to complete the dimensionality unification of semen volume parameters and motion velocity vectors, and using multiple interpolation algorithms to handle the clinical guideline filling logic when missing values ​​are detected in laboratory tests.

[0004] In the current sperm quality assessment process, during sperm trajectory analysis, there is a lack of quantitative extraction methods for local area trajectory morphology, resulting in insufficient coverage of sperm microscopic dynamic characteristics and affecting the understanding of distribution differences within the population. Parameter change judgments often rely on the comparison of a single time point, and fail to form a dynamic recognition logic for the continuous change process, limiting the ability to capture fluctuation-sensitive areas. In the morphological image processing link, data selection is disconnected from motion information, failing to reflect the synergistic impact of morphological characteristics on dynamic performance, reducing the depth of assessment and the strength of interpretation. Multimodal data fusion lacks unified temporal anchor support, and the logical connection between features is weak, resulting in limited decision-making value of the fused data. Scoring models are mostly constructed based on the stacking of static classification results. The weight distribution and sample performance cannot be effectively linked, making it difficult to support the precise judgment and hierarchical management of abnormal samples. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose a sperm quality assessment method based on multimodal fusion.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a sperm quality assessment method based on multimodal fusion, comprising the following steps:

[0007] S1: Acquire a real-time sperm motion trajectory image sequence, collect sperm density, total motility percentage, and forward motion ratio parameter records, combine the image sequence with the time tags in the parameter records, determine whether the adjacent time error is less than the time synchronization threshold, and generate a spatiotemporal alignment dataset;

[0008] S2: Based on the spatiotemporal alignment dataset, all sperm trajectory paths in the image are extracted frame by frame, the linearity index of the motion trajectory is calculated, the distribution area is marked, and a motion feature heat map is generated;

[0009] S3: Obtain density, vitality, and forward motion data from the spatiotemporal alignment dataset, calculate parameter differences and change rate gradients, determine whether the change amplitude is greater than the corresponding parameter fluctuation threshold, and generate a parameter anomaly time window;

[0010] S4: extracting sperm morphology images with the same numbers according to the motion feature heat map and the image frame numbers in the parameter abnormality time window, evaluating the head deformity rate, and generating a multimodal feature matrix by combining the motion trajectory linearity index and parameter changes;

[0011] S5: Based on the multimodal feature matrix, the vitality-morphology consistency index is calculated, and the trajectory continuity index is combined to allocate the scoring weights, divide the evaluation levels, and generate the multimodal sperm quality evaluation results.

[0012] As a further solution of the present invention, the spatiotemporal alignment dataset includes image frame sequence numbers, synchronization label mapping relationships, and a density and motion parameter correspondence table; the motion feature heat map includes a trajectory linearity distribution map, a regional density map, and a motion direction vector map; the parameter anomaly time window includes a density fluctuation interval, a vitality anomaly segment, and a motion rate offset interval; the multimodal feature matrix includes a morphological anomaly label set, a trajectory stability index group, and a parameter synchronization anomaly label set; and the multimodal sperm quality assessment results include a vitality-morphology consistency index score, a trajectory continuity assessment grade, and a sperm quality classification result.

[0013] As a further solution of the present invention, the steps of acquiring the spatiotemporal alignment dataset are specifically as follows:

[0014] S111: Acquire a sequence of real-time sperm motion trajectory images and corresponding time tags, collect three parameters recorded by a semen analyzer: sperm density, total motility percentage, and forward motion ratio, extract the corresponding time tags respectively, calculate the time difference between adjacent ones, and obtain a set of image and parameter time differences;

[0015] S112: Based on the image and parameter time difference set, performing a judgment operation on each time difference and a time synchronization threshold, screening all time tag pairs whose time difference is less than the time synchronization threshold, and obtaining a time consistency data pair set;

[0016] S113: Based on the temporally consistent data pair set, the corresponding image sequence is associated with three parameters: sperm density, total motility percentage, and forward motion ratio. Data pairs with the same time tags are matched. Each set of data is subjected to consistency verification and validity marking to establish a spatiotemporally aligned data set.

[0017] As a further solution of the present invention, the steps of obtaining the motion feature heat map are specifically as follows:

[0018] S211: Based on the spatiotemporal alignment dataset, extract the sperm trajectory path in the image frame by frame, identify the start and end positions of the trajectory, record the sperm head position coordinates in all consecutive frames along the path, record the Euclidean distance between the start and end points of each path and the entire trajectory path length, and obtain a path structure parameter set;

[0019] S212: Calculating the sperm path trajectory linearity index based on the path structure parameter set and the time interval of the trajectory frame segments, and mapping it to the corresponding image frame to obtain the sperm motion linearity parameter set;

[0020] S213: Based on the sperm motion linearity parameter set and the corresponding image frame coordinate system, the image area is divided into grid cells, the distribution density of trajectory points in each cell that meet the trajectory linearity index greater than the judgment threshold is counted, the color intensity in each grid is proportionally assigned to the image thermal color scale, and a motion feature heat map is generated.

[0021] As a further solution of the present invention, the step of obtaining the parameter abnormality time window is specifically as follows:

[0022] S311: Obtain sperm density, motility, and forward motion ratio data from the spatiotemporal aligned dataset, set a sliding time window, traverse all consecutive time points in the dataset in chronological order, record the density change, motility change, and forward motion ratio change in each window, and obtain a parameter change difference sequence;

[0023] S312: Calculate the gradient of the rate of change per unit time based on the parameter change difference sequence, perform normalization, and make judgments item by item based on the corresponding parameter fluctuation threshold reference value, complete screening and marking of each gradient exceeding the limit state in the continuous window, and obtain a change gradient judgment value sequence;

[0024] S313: Based on the change gradient judgment value sequence, screen the time periods that continuously exceed the parameter fluctuation threshold according to the continuity of the time axis, extract the start time and end time of the time interval that continuously meets the abnormal condition, and establish the parameter abnormal time window.

[0025] As a further solution of the present invention, the step of obtaining the multimodal feature matrix is ​​specifically as follows:

[0026] S411: Retrieving the image frame numbers within the time-overlapping interval based on the motion feature heat map and the time information in the parameter abnormality time window, extracting all sperm target regions in the corresponding image by frame number, marking the target sperm head region, and obtaining a sperm image group of the time-overlapping frames;

[0027] S412: Based on the sperm image group of overlapping time frames, identifying the edge of the sperm head structure in the image, marking each sperm for morphological determination, judging it according to the standard of normal sperm head, calculating the sperm head deformity rate, and obtaining a head deformity rate sequence;

[0028] S413: Based on the head deformity rate sequence, the motion trajectory linearity parameter set under the corresponding image frame number is combined with the data at the same numbered position in the parameter change difference sequence, the data structure is unified, data fusion and format standardization are performed, and a multimodal feature matrix is ​​established.

[0029] As a further embodiment of the present invention, the steps for obtaining the multimodal sperm quality assessment results are specifically as follows:

[0030] S511: Based on the multimodal feature matrix, the deformity rate and vitality value corresponding to each frame image are extracted row by row, and whether the synchronization is in a deviated state in the same frame is determined. The consistency frequency of the relative change direction is calculated and uniformly output in percentage form to obtain the vitality morphology consistency index;

[0031] S512: Reading the trajectory linearity index in the motion trajectory linearity parameter set corresponding to the frame image based on the vitality morphology consistency index, extracting the density, vitality, and forward motion parameter change trends of the aforementioned frames, and calculating to obtain a comprehensive score value sequence;

[0032] S513: Based on the comprehensive score value sequence and in accordance with the semen quality grade standard, all score value intervals are judged, and grade labels are marked. The scores are arranged in time series to obtain a multimodal sperm quality assessment grade result.

[0033] A sperm quality assessment system based on multimodal fusion, comprising:

[0034] The temporal alignment module obtains a sequence of real-time sperm motion trajectory images, collects parameters such as sperm density, total motility percentage, and forward motion ratio, retains data pairs with consistent time, and generates a spatiotemporal alignment dataset;

[0035] The trajectory extraction module extracts all sperm trajectory paths in the image frame by frame based on the spatiotemporal alignment dataset, calculates the linearity index of the motion trajectory and locates the active area, annotates the pixel heat of the area, and generates a motion feature heat map;

[0036] The parameter fluctuation module obtains the density, vitality and forward motion data in the spatiotemporal alignment data set, calculates the parameter difference and the gradient of the rate of change, determines whether it exceeds the WHO threshold and marks the interval, and generates a parameter abnormality time window;

[0037] The morphological analysis module extracts the corresponding sperm morphological image based on the motion feature heat map and the image frame number that coincides with the time in the parameter abnormality time window, evaluates the head deformity rate, and generates a multimodal feature matrix by combining the motion trajectory linearity index and parameter changes;

[0038] The grade scoring module calculates the consistency index based on the multimodal feature matrix, divides the evaluation grades by using the scoring weight distribution, and generates a multimodal sperm quality evaluation result.

[0039] Compared with the prior art, the advantages and positive effects of the present invention are:

[0040] In the present invention, by introducing linearity calculation in the trajectory path extraction process and combining it with spatial distribution thermodynamic mapping, the dynamic characteristics of the local sperm population are made explicit, and an abnormal time window is constructed based on the parameter value change rate and the set threshold. Combined with the motion consistency index, multi-layer information composite analysis is achieved, the interpretation depth of sperm head deformity data in quality judgment is enhanced, feature intersection is achieved, and the internal structure logic of morphological, trajectory and functional data is connected, effectively expanding the dimensional space of sperm quality description. The scoring mechanism introduces the consistency ratio between vitality and morphology and combines the trajectory continuity feature to establish a more hierarchical grading system to ensure the accuracy of sperm quality assessment results. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of the main steps of the present invention;

[0042] Figure 2 A flow chart for obtaining a spatiotemporal alignment dataset of the present invention;

[0043] Figure 3 A flow chart for obtaining a motion characteristic heat map according to the present invention;

[0044] Figure 4 This is a flow chart for obtaining the parameter abnormality time window of the present invention;

[0045] Figure 5 A flowchart for obtaining a multimodal feature matrix of the present invention;

[0046] Figure 6 The flowchart for obtaining the multimodal sperm quality assessment results of the present invention is shown. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0048] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0049] See also Figure 1 , a sperm quality assessment method based on multimodal fusion, comprising the following steps:

[0050] S1: Acquire real-time sperm motion trajectory image sequences and time tags, collect sperm density, total motility percentage, and forward motion ratio parameter records from the structured semen analyzer, combine the image sequence and the time tags in the parameter records, and determine whether the time error between the two is less than the time synchronization threshold (200ms, according to the equipment synchronization requirements set by the ISO23162:2021 semen analysis standard). Retain data pairs with consistent time and generate a spatiotemporally aligned dataset;

[0051] S2: Based on the spatiotemporal alignment dataset, all sperm trajectory paths in the image are extracted frame by frame, the motion trajectory linearity (VSL, a motion parameter defined in the World Health Organization's "Human Semen Examination and Processing Laboratory Manual") index is calculated, and the distribution area position is marked in the image to generate a motion feature heat map;

[0052] S3: Obtain density, motility, and forward motility data from the spatiotemporally aligned dataset, calculate parameter differences and change rate gradients according to a sliding time window (the analysis window is set based on the sampling frequency of the CASA (computer-assisted semen analysis) system, 500 ms), and determine whether the amplitude of the change is greater than the density fluctuation threshold (based on the concentration change warning value set by the 5th edition of the WHO standard (≥15×10^6 / mL is normal)), the motility fluctuation threshold (referring to the WHO standard for the proportion of progressively motile sperm (≥32% is normal)), and the motility fluctuation threshold (referring to the WHO standard, usually set at ±3%). Mark the time interval where the continuous change exceeds the threshold to generate a parameter abnormality time window;

[0053] S4: Based on the image frame numbers that coincide with the motion feature heat map and the parameter abnormality time window, sperm morphology images with the same numbers are extracted to evaluate the head deformity rate (the core indicator of morphological evaluation defined in ISO23162:2021). Combined with the motion trajectory linearity index and parameter changes of the corresponding frames, a multimodal feature matrix is ​​generated.

[0054] S5: Based on the multimodal feature matrix, the vitality-morphology consistency index is statistically calculated, and the trajectory continuity index is combined to allocate comprehensive score weights. The evaluation levels are divided according to the WHO semen quality grading standard (the four-level classification standard (normal / critical / abnormal / severely abnormal) released by the World Health Organization in 2021) to generate multimodal sperm quality assessment results.

[0055] The spatiotemporal alignment dataset includes image frame sequence numbers, synchronization label mapping relationships, and density and motion parameter correspondence tables. The motion feature heat map includes trajectory linearity distribution maps, regional density maps, and motion direction vector maps. The parameter anomaly time window includes density fluctuation intervals, vitality anomaly segments, and motion rate offset intervals. The multimodal feature matrix includes a morphological anomaly label set, a trajectory stability index group, and a parameter synchronization anomaly label set. The multimodal sperm quality assessment results include vitality-morphology consistency index scores, trajectory continuity assessment grades, and sperm quality classification results.

[0056] See also Figure 2 , S1 step is:

[0057] S111: Acquire a sequence of real-time sperm motion trajectory images and corresponding time tags, collect three parameters recorded by a semen analyzer: sperm density, total motility percentage, and forward motion ratio, extract the corresponding time tags respectively, calculate the time difference between adjacent ones, and obtain a set of image and parameter time differences;

[0058] To obtain the real-time sperm motion trajectory image sequence and the corresponding time label, a high-speed camera system is used to continuously shoot the sperm movement process in the semen sample under a microscope to obtain image frames, and a capture time label is added to each frame image. The time label is accurate to the millisecond level. For example, if the image sequence is 25 frames per second, the time interval between each frame is 40ms. Then, the parameters such as sperm density, total motility percentage, and forward motion ratio extracted from the structured semen analyzer are classified according to the acquisition time. The system records the acquisition time of this group of parameters. The time label matching each group of parameter records can be obtained through the timestamp extraction operation. For example, if a group of parameters is acquired at 10:01:03.420 seconds on June 15, 2024, its label is set to "20240615100103420". Then, the time label of each frame in the image sequence is compared with the label in the structured parameter record one by one, and the absolute difference operation is used to compare the time difference between the two. The specific process is: the image frame T1 label "20240615100103400" is compared with the parameter record. Record the P1 label "20240615100103420" for difference calculation, and get the difference value of 20ms. Repeat this operation to perform calculation on all image frames and parameter label pairs. If the number of image frames is 750 frames and the parameter records are 100 groups, the actual comparison will involve 750×100 difference judgments. The data volume is large, but the comparison speed can be accelerated by optimizing the retrieval through data pointers. After that, the paired differences of all image frames and parameter records are collected into an array, such as ΔT array = {20, 180, 300, 2 5, ...}, all numerical values ​​are in ms. It should be noted that some parameter records cannot accurately obtain time tags or are missing fields. In this case, the corresponding records are not included in the calculation and are deemed invalid and eliminated. To enhance data consistency, the initial parameter density threshold can be set to only filter sample data with a density greater than 50 million / mL to participate in the calculation to avoid low-density interference. For example, if the density of sample A is 63 million / mL, it is retained. The difference between the image frame and the structured parameter record of this sample constitutes the image and parameter time difference set.

[0059] S112: Based on the image and parameter time difference set, a judgment operation is performed on each time difference and a time synchronization threshold, and all time tag pairs whose time difference is less than the time synchronization threshold are screened to obtain a time consistency data pair set;

[0060] Based on the image and parameter time difference value set, each difference value item in the array is judged against the 200ms time synchronization threshold. The specific operation is as follows: Set the threshold T to 200ms, and perform a comparison operation on each item in the ΔT array. If ΔT_i < T, then consider the corresponding image frame and parameter record of this item to meet the synchronization condition, mark and save them; otherwise, eliminate them. For example, when the ΔT array is {20, 180, 300, 25, 190, 250}, the indexes that meet the condition of being less than 200ms are 0, 1, 3, 4, that is, the frames with the corresponding image frame numbers T0, T1, T3, T4 and the parameter records are paired effectively, forming data pairs that meet the time consistency condition. To verify the rationality of the threshold selection, different synchronization requirement conditions are introduced for testing. The thresholds are set to 150ms, 200ms, and 250ms respectively, calculate the number of remaining data pairs under each threshold, and present them in the form of Table 1 as follows:

[0061] Table 1 Data pair screening result table under different time synchronization thresholds

[0062]

[0063] As shown in Table 1, the retention ratio at the 200ms threshold is 81.81%, which is the benchmark set to balance the synchronization accuracy and the data volume. Subsequently, an effective data pair set composed of the image frames and parameter records that meet the conditions is constructed, and the time label pairing result will be used as the pre-input for the integration of the image and structured parameter data in the next stage.

[0064] S113: According to the time consistency data pair set, associate the corresponding image sequence with the three parameters of sperm density, total motility percentage, and forward motility ratio, match the data pairs with the same time labels, perform consistency verification and validity marking on each group of data, and establish a spatio-temporal alignment data set;

[0065] According to the time consistency data pair set, each group of image frames with a confirmed time difference of less than 200ms is paired with the corresponding structured parameter record, and a data group is established by synchronously combining the image frame sequence number and the parameter sequence number. The information in the image frame is obtained through image analysis to obtain dynamic information such as sperm trajectory, head position and velocity vector. The structured record contains the density, total activity percentage and forward motion ratio value at the corresponding moment of the frame. For example, the time label of image frame T15 is "20240615100201400", and the corresponding parameter record P15 label is "20240615100201410". The difference is 10ms, which is less than the threshold and belongs to a valid data pair. The system converts the sperm trajectory vector group V_T15 in image frame T15 into [(x1, y1, t1), (x2, y2, t2 )...] and the structured parameters {S = 71 million / mL, M = 72.4%, A = 61.1%} are integrated to form a complete set of data items. The combination operation is performed frame by frame, traversing all time-consistent data pairs, resulting in a total of 589 sets of image frame and parameter record combinations. Then, outliers of the three parameters within each data set are removed. If the forward motion ratio is less than 20% or greater than 90%, it is marked as an outlier and excluded from database construction. Missing frames in the image frame trajectory are repaired by interpolation of adjacent frames. Only cases with no more than two consecutive missing frames are repaired. The final data set constitutes a unified three-dimensional array structure with the fields unified as [trajectory vector set, density value, total activity percentage, forward motion ratio]. This data set will be further used as the original input for spatiotemporal analysis to complete the construction of the spatiotemporal alignment dataset.

[0066] See also Figure 3 , step S2 is:

[0067] S211: Based on the spatiotemporal alignment dataset, extract the sperm trajectory path in the image frame by frame, identify the start and end positions of the trajectory, record the position coordinates of the sperm head in all consecutive frames along the path, record the Euclidean distance between the start and end points of each path and the total trajectory path length, and obtain the path structure parameter set;

[0068] Based on the spatiotemporal alignment dataset, the sperm image sequence is traversed frame by frame to extract the trajectory path. First, each sperm in the image frame is identified, its head coordinate point is extracted, and its number in the image frame is recorded to form a correspondence between the sperm number and the coordinate. In the image frame F001, the trajectory numbered T01 is detected. The initial position coordinates of its head are (12, 30) and the ending position is (48, 70). The starting and ending points are connected, and the starting and ending distances of the path are calculated as follows:

[0069] ;

[0070] If the trajectory has 24 consecutive recorded points in the middle frame, and the distance between each adjacent coordinate point is accumulated as follows to obtain a total path length of 75.2 μm, then the corresponding path information is organized as follows to form a path structure parameter set as shown in Table 2:

[0071] Table 2 Path structure parameter table

[0072]

[0073] As shown in Table 2, by extracting the starting and ending coordinates of each trajectory in the image frame and calculating its path length and starting and ending distances, the standard path structure parameters can be obtained, and the path structure parameter set can be obtained.

[0074] S212: Based on the path structure parameter set and the trajectory frame interval, the formula is used:

[0075] ;

[0076] Calculate the linearity index of the sperm path trajectory and map it to the corresponding image frame to obtain the linearity parameter set of sperm movement, where: Indicates the Sperm in the Trajectory linearity index in the frame, Indicates the straight-line distance between the starting point and the end point of the path. 、 Respectively Sperm in frame The two-dimensional coordinates of Indicates the total time interval corresponding to the path, is the total number of frames in the path, is the position jitter compensation item in the current trajectory;

[0077] Based on the path structure parameter set, the linearity index VSL of the motion trajectory is calculated for each path. The trajectory duration frame number and the corresponding time interval are introduced for compensation processing. Taking trajectory T01 as an example, the path length is 75.2μm, the start-end distance is 62.4μm, the total number of trajectory frames is 22, and the time interval between each frame is 40ms. Therefore, Δt = 880ms, and the position jitter compensation term σ is set to 2 frames. The formula is:

[0078] ;

[0079] Continue to calculate the same method for tracks T02 to T05. Assuming the results are 4.98, 5.03, 5.68, and 5.00 respectively, map each VSL value to the corresponding image frame and track number to form the following array:

[0080] ;

[0081] The VSL value is assigned to the corresponding position area of ​​the image frame according to the trajectory number, which is used for subsequent thermal map data superposition to obtain the sperm movement linearity parameter set.

[0082] The trajectory linearity index is an important indicator to measure whether the sperm movement path in the image sequence is close to a straight line. Its essence is the ratio between the straight-line distance between the starting point and the end point and the actual trajectory path length, and is corrected by the time factor. The closer the value is to a larger value, the more stable the sperm movement trajectory in the continuous frame image, the smaller the offset, and the lower the path curvature, indicating that it has a stronger forward propulsion ability. Conversely, if the trajectory linearity value is low, it usually means that the sperm movement direction fluctuates greatly, the path frequently bends or turns, and the movement is not directional. Therefore, the trajectory linearity value is not only used to describe the spatial characteristics of a single trajectory, but also can be used as an important data parameter for screening sperm activity and judging its functional status. It has the statistical function of quantitatively characterizing the movement pattern characteristics of sperm groups.

[0083] The calculation logic of the formula comprehensively considers the geometric characteristics and time stability of the sperm movement path, aiming to measure the linearity strength of each trajectory. Its overall structure consists of three parts: First, the molecular part It represents the ratio of the starting and ending distances of the path to the total path length, and is used to quantify the linearity of the trajectory. The closer the ratio is to 1, the closer the trajectory is to a straight line. Its value is greatly affected by the path offset and curvature. Secondly, the ratio is multiplied by a square root term. , used to integrate the effects of time persistence and trajectory stability, where the numerator is the total time that the trajectory experiences, reflecting its continuous motion time, and the denominator The total number of frames and the trajectory stability correction term σ are introduced. σ reflects the degree of displacement jitter in the trajectory. The square root structure is introduced to avoid excessive pull of the time parameter in the product and maintain the smooth adjustment effect of the time factor on the linearity index. The final overall expression uses the absolute value to ensure that the output result is positive and avoid directional interference with the overall value. Therefore, the addition, subtraction, multiplication, and division of the entire formula and the square root structure respectively assume the dual functions of path shape evaluation and time compensation, and jointly reflect the comprehensive evaluation of the motion linearity of the trajectory.

[0084] S213: Based on the sperm motion linearity parameter set and the corresponding image frame coordinate system, the image area is divided into grid cells, the distribution density of trajectory points in each cell that meet the trajectory linearity index greater than the judgment threshold is counted, and the color intensity in each grid is proportionally assigned to the image thermal color scale to generate a motion feature heat map;

[0085] According to the sperm motility linearity parameter set, all VSL index values ​​are mapped to the coordinate grid area of ​​the image frame. The image is set to 800×800 pixels and divided into 100×100 pixels per grid, forming 8×8 grid units, a total of 64. All frame point positions of each trajectory are traversed and assigned to the corresponding grid. It is judged whether the corresponding VSL value exceeds the judgment threshold of 4.5. If it exceeds, the corresponding counter in the grid is increased by 1. For example, if the point T01 of trajectory falls on the (1, 1) grid and the VSL is 5.25, the count is recorded as +1. After completing the traversal of all trajectories, the number of points with VSL greater than 4.5 in each grid of 64 grids is counted and the proportion is calculated. The points are mapped to the color scale array proportionally, with the maximum density mapped to red and the minimum mapped to blue. The RGB value matrix is ​​generated and written back to the original image space. The image rendering process is completed to generate a motion feature heat map.

[0086] See also Figure 4 , S3 steps are:

[0087] S311: Obtain sperm density, motility, and forward motion ratio data from the spatiotemporal alignment dataset, set a sliding time window, traverse all consecutive time points in the dataset in chronological order, record the density change, motility change, and forward motion ratio change in each window, and obtain a parameter change difference sequence;

[0088] After obtaining the density, vitality, and forward motion ratio data in the spatiotemporal alignment dataset, we first divided the sliding time windows into 500ms in chronological order and extracted the sperm parameter data of two adjacent frames in the continuous time window. In each group, we obtained the density, vitality, and forward motion ratio values ​​of the current window and the next window, and calculated the difference between the parameters in turn. For example, the density in the time window W1 is 1820×10 6 / mL, the density in W2 is 1670×10 6 / mL, the density difference is -150×10 6 / mL, if the motility of W1 is 36% and that of W2 is 40%, then the motility difference is +4%. If the forward motion ratio decreases from 31% to 28%, then the difference is -3%. This difference indicates that within 500ms, the motility of sperm in semen increases slightly while the density and motility decrease slightly. Repeat the operation to obtain parameter differences for more time window groups and construct a complete parameter change sequence, as shown in Table 3:

[0089] Table 3 Example of parameter change difference

[0090]

[0091] As shown in Table 3, by sliding and comparing the parameter change trends window by window and recording the continuous change direction and difference state of each parameter item, the short-term dynamic change process of density, vitality and forward motion can be clarified, and the parameter change difference sequence can be obtained.

[0092] S312: Calculate the gradient of the rate of change per unit time based on the parameter change difference sequence, perform normalization, and make judgments item by item based on the corresponding parameter fluctuation threshold reference value. Complete the screening and marking of each gradient exceeding the limit state in the continuous window to obtain a change gradient judgment value sequence;

[0093] According to the parameter change difference sequence, the change rate per unit time, i.e., the change gradient value, is obtained by dividing the change amount of each parameter by the window interval of 0.5 seconds. For each set of data in Table 6, the density change rate, vitality change rate, and forward motion ratio change rate are calculated respectively. Taking W1-W2 as an example, the density change rate is -150÷0.5=-300×10 6 / mL / s, the activity change rate is 4÷0.5=+8% / s, the forward motion change rate is -3÷0.5=-6% / s, and the same operation is performed on all windows in this way. The absolute value of the change rate is extracted for abnormal judgment. The system presets the reference fluctuation threshold of the density change rate as 300×10 6 / mL / s, the vitality change rate is 6% / s, and the forward motion ratio is 3% / s. The calculated results are compared with the above benchmarks. If any one of them exceeds the limit, the window is recorded as abnormal. For example, if the density change rate and the forward motion ratio change rate in W1-W2 both exceed the set range, the window status is "multiple abnormalities". Compare all the results to construct a judgment sequence, as shown in Table 4:

[0094] Table 4 Parameter change gradient and judgment status table

[0095]

[0096] As shown in Table 4, the constructed judgment sequence marks whether the parameter gradient exceeds the threshold in each time period and indicates the type of abnormal source, obtaining a change gradient judgment value sequence.

[0097] S313: Based on the change gradient judgment value sequence, screen the time periods that continuously exceed the parameter fluctuation threshold according to the time axis continuity, extract the start time and end time of the time interval that continuously meets the abnormal condition, and establish the parameter abnormal time window;

[0098] Based on the change gradient judgment value sequence, the windows with abnormal states in multiple consecutive time periods are retrieved in chronological order, and the abnormal start and end time intervals are extracted. For example, if W1-W2, W2-W3, and W3-W4 are three consecutive abnormal segments, the segments are merged into a single abnormal time window starting at time point T1 and ending at T4, marked as abnormal segment A1, and the abnormal type is recorded as "multiple continuous abnormalities". In this way, all continuous abnormal window intervals that meet the conditions are extracted. Each interval item records the start and end time, the number of included windows, and the dominant abnormal source to construct a complete abnormal interval array, which is shown in Table 5 after sorting:

[0099] Table 5 Abnormal time window record table

[0100]

[0101] As shown in Table 5, the abnormal time window records the time range and dominant abnormal indicator type of the parameters in different stages of the detection period when continuous and violent fluctuations occur, and establishes the parameter abnormal time window.

[0102] See also Figure 5 , step S4 is:

[0103] S411: Based on the time information in the motion feature heat map and the parameter abnormality time window, the image frame numbers in the time overlapping interval are retrieved, all sperm target areas in the corresponding image are extracted according to the frame numbers, the target sperm head areas are marked, and a sperm image group of the time overlapping frames is obtained;

[0104] According to the time periods recorded in the motion feature heat map and the parameter anomaly time window, an intersection operation is performed on their time tags to screen out the image frame numbers that exist simultaneously in the two records. The image frames corresponding to these frame numbers in the image dataset are extracted in sequence, and target detection is performed on individual sperm in the image. Each sperm structure area is separated by segmentation, and the corresponding relationship between the number of individual sperm in the image frame and the image number is recorded one by one. For example, 20 sperm areas are detected in frame F001 and 18 targets are detected in frame F002. Both are used as input image sets for subsequent morphological judgment. The image frame numbers and the sperm image segments contained therein are combined into a dataset, uniformly numbered and structured for storage. The summary structure is shown in Table 6:

[0105] Table 6 Sperm image extraction record table

[0106]

[0107] As shown in Table 6, the image frame numbers are mapped to the number of detected sperm individuals. Subsequently, morphological determination will be performed frame by frame based on this data to obtain a sperm image group with time-overlapping frames.

[0108] S412: Based on the sperm image group with overlapping time frames, identify the edges of the sperm head structure in the image and mark each sperm for morphological determination. The sperm head is judged according to the normal sperm head standards specified in ISO23162:2021. If any of the head width-to-height ratio, symmetry, or curvature characteristics do not meet the standards, the sperm head is considered deformed using the formula:

[0109] ;

[0110] Calculate the sperm head deformity rate, obtain the head deformity rate sequence, and construct the deformity rates in all frames into a vector array, where Indicates the Head deformity rate in frame images (%), Indicates the The number of spermatozoa identified as having a deformed head in the frame, Indicates the The total number of sperm in the frame was multiplied by 100% to convert to percentage expression;

[0111] According to the sperm image group of time-overlapping frames, the head structure of each sperm area in each frame image is extracted to obtain its width, height, edge curvature and symmetry index. According to the structural definition of normal sperm head in ISO23162:2021 standard, if the aspect ratio of the sperm head is not between 1.5 and 1.8, or the symmetry score is lower than 80%, it is judged as a deformed individual. In image F001, 20 sperm were identified, of which 4 did not meet the standard definition and were therefore deformed sperm. When calculating the deformity rate, the formula was substituted, where the number of deformities in frame F001 is , the total is , then it is calculated as , similarly in F002 , and so on, the statistics are shown in Table 7:

[0112] Table 7 Head deformity rate calculation table

[0113]

[0114] As shown in Table 7, the above results are used as important indicators for morphological quality assessment in each frame of the image, and a head deformity rate sequence is obtained.

[0115] The head deformity rate is one of the core indicators used to evaluate the morphological quality of sperm. Its specific significance lies in the numerical expression of the proportion of abnormal head structures in the individual sperm detected in the image. This indicator determines head deformity based on the ISO23162:2021 standard, including inconsistent head size, unbalanced width-to-height ratio, structural asymmetry, morphological distortion, etc. The higher the deformity rate value, the greater the proportion of sperm morphological abnormalities in the sample, and the greater the possible impact on biological activity and fertilization ability. Therefore, this indicator is not only used for the morphological quality evaluation of single-frame images, but also as a key reference parameter in fertility assessment, assisted reproductive diagnosis and image screening tasks.

[0116] The formula operation logic is based on the calculation of the direct proportional relationship between the total number of sperm in a single frame image and the number of abnormal sperm. Indicates the number of sperm individuals identified as abnormal structures in the current image frame. The denominator It represents the total number of all identified sperm in the frame image. The two are divided to obtain the proportion of deformed individuals to the total number of individuals. This value is a decimal. In order to convert it into a commonly used percentage for visualization and subsequent model input processing, it is multiplied by 100%. The absolute value symbol is used to prevent negative values ​​caused by logical inversion during image label recognition or calculation, ensuring that the calculation result is a positive real number to meet the positive representation requirements of morphological indicators, thereby clearly reflecting the proportion of deformed sperm head structures in the total sample in each image frame.

[0117] S413: Based on the head deformity rate sequence, the motion trajectory linearity parameter set under the corresponding image frame number is combined with the data at the same number position in the parameter change difference sequence, the data structure is unified, data fusion and format standardization are performed, and a multimodal feature matrix is ​​established;

[0118] According to the head deformity rate sequence, the linearity parameter value of the motion trajectory and the parameter change difference group under the corresponding image number are matched frame by frame. The linearity value, density difference, vitality difference, forward motion difference and other contents corresponding to each frame are extracted and combined with the deformity rate into a five-dimensional vector to form a unified record structure. For example, in image frame F001, the deformity rate is 20.0%, the trajectory linearity is 5.25, and the density change is -150×10 6 / mL, the vitality change is +4%, and the forward motion change is -3%. Then the multimodal vector of this frame is [20.0, 5.25, -150, +4, -3]. Similarly, the matrix is ​​constructed as shown in Table 8:

[0119] Table 8 Multimodal feature matrix

[0120]

[0121] As shown in Table 8, each row of data corresponds to an image frame, which integrates the morphology, motion trajectory and parameter change information to form a unified vector structure and establish a multimodal feature matrix.

[0122] See also Figure 6 , step S5 is:

[0123] S511: Based on the multimodal feature matrix, the deformity rate and vitality value corresponding to each frame image are extracted row by row, and whether the synchronization is deviated in the same frame is determined. The consistency frequency of the relative change direction is calculated and uniformly output in percentage form to obtain the vitality morphology consistency index;

[0124] Based on the image parameters of each frame collected in the multimodal feature matrix, we first identify the fields where the deformity rate and vitality value are located in each frame. By traversing the image frame number sequence, we compare the vitality trend and deformity rate trend of adjacent frames. If the vitality value of the current frame decreases compared to the previous frame and the deformity rate increases, it is considered a morphological vitality deviation consistency event. This event is marked and accumulated. The number of such synchronization offsets in all image frames is counted and divided by the total number of frames to obtain the vitality-morphological consistency index. For example, if the vitality of frame F001 decreases from 0.85 to 0.82 in the previous frame and the deformity rate increases from 18% to 20%, it is a consistent event. If such an event occurs 3 times in 5 frames, the consistency index is 0.6, which is 60.0% in percentage. The following records are established, as shown in Table 9:

[0125] Table 9 Vitality form consistency event table

[0126]

[0127] As shown in Table 9, there are 2 frames that meet the conditions of decreased vitality and increased deformity rate. The consistency index is calculated for the total number of frames 5: , converted into a percentage of 40.0%, to obtain the vitality morphology consistency index.

[0128] S512: Based on the vitality morphology consistency index, read the trajectory linearity index in the motion trajectory linearity parameter set under the corresponding frame image, and extract the density, vitality and forward motion parameter change trends of the aforementioned frames using the formula:

[0129] ;

[0130] Operation to obtain a comprehensive score value sequence, where: Indicates the The comprehensive score of the frame image, 、 、 Respectively represent Normalized vibrancy, density, and forward motion metrics of frames, represents the trajectory linearity value, Indicates the The frame's vitality morphology consistency index, 、 、 are the scoring weights of the participation items, which are set to 0.4, 0.3, and 0.3 respectively. The constant term 0.5 in the denominator of the formula is the theoretical neutral value of the vitality-morphology consistency index, which corresponds to the ideal symmetrical baseline state where there is no obvious correlation between decreased vitality and increased deformity. The degree of fluctuation above and below this point represents the degree of deviation from consistency, and is therefore used as the denominator normalization base value;

[0131] According to the vitality morphology consistency index, the normalized vitality, density and forward motion values ​​in each frame are extracted, and the trajectory linearity index is read. The three parameters are weighted by 0.4, 0.3 and 0.3 respectively. Among them, the weight setting of 0.4, 0.3 and 0.3 is based on the classification of the influence of the three parameters on overall fertility in the WHO semen quality definition. Vitality is the core indicator of sperm motility and directly determines its possibility of combining with egg cells. Therefore, it is given the highest weight of 0.4; density reflects the number of sperm in a unit volume and is linked to the total available amount. Its influence is second only to vitality. The weight is 0.3. Although the forward motion rate also reflects the state of movement, it has a certain degree of overlap with the vitality index, and the fluctuation sensitivity in some frames is slightly lower than the first two. Therefore, it is given a weight of 0.3. The sum of the three is 1 to ensure the normalization of the score. This setting remains stable when comparing the data performance of individuals with different fertility. When the variation range of density and vitality increases, the scores are more likely to be widened, forming a difference judgment space. Taking frame F001 as an example, its vitality value is 0.82, density is 0.79, forward motion is 0.81, linearity is 5.25, and consistency index is 0.61. The calculation process is as follows:

[0132] ;

[0133] In the calculation, the weighted part is , the root value of the trajectory is , the consistency deviation is , the denominator is 1.11, and the final score is:

[0134] ;

[0135] The calculation results of the scores of each frame are as follows, as shown in Table 10:

[0136] Table 10 Multimodal score calculation table

[0137]

[0138] As shown in Table 10, the score values ​​are all in a relatively high range. The overall comprehensive performance of each frame is displayed in percentage to obtain a comprehensive score value sequence.

[0139] The comprehensive score is a unified quantitative result reflecting the multidimensional performance of sperm quality in a single-frame image. Its value integrates five key indicators: normalized motility, density, forward motion, trajectory linearity, and consistency of vitality and morphology. It reflects the weighted contribution and mutual checks and balances of each parameter in each frame in the quality evaluation system. The score not only takes into account static characteristics such as the steady-state level of morphology and density, but also integrates dynamic parameters such as trajectory stability and the changing trend of motility. At the same time, the consistency index is introduced as an adjustment item to evaluate the coordination between multiple parameters. Therefore, the comprehensive score can be regarded as a numerical reflection of the overall biological quality of sperm in the current spatiotemporal state. It is the core basic indicator for subsequent grading assessment, status judgment and diagnostic support.

[0140] The calculation logic design of the formula is aimed at realizing the integrated scoring of multi-dimensional sperm quality indicators, among which the normalized motility value, density value and forward motion value are positive evaluation indicators, which have the significance of improving in the same direction and being beneficial to quality. Therefore, the three parameters are weightedly added and assigned weights of 0.4, 0.3 and 0.3 respectively to reflect their relative importance in the comprehensive evaluation. This weighted sum structure ensures that each indicator can participate in the superposition of the score value according to the set proportion. At the same time, trajectory linearity is an independent dynamic indicator. The larger its value, the more stable the motion trajectory. In order to maintain its independent influence and suppress the pulling effect of its excessive value, the opening and closing of the indicator is introduced. The absolute value is then taken after the square to achieve scale compression and smoothing effects, and then deducted from the weighted value in the final score to express the negative correction effect of trajectory deviation on quality. In addition, the consistency index is a signal item reflecting the common changes of vitality and morphology. The farther it deviates from the theoretical neutral value of 0.5, the stronger its bias is. Therefore, the denominator structure is constructed as the absolute value of the difference between it and 0.5 plus 1, so that the closer the item is to 0.5, the smaller the overall impact. Conversely, the farther it is, the lower the final score will be. This normalization factor effectively reflects the fluctuation degree of vitality-morphology consistency, thereby jointly completing the association modeling and quality index scoring between multimodal parameters.

[0141] S513: Based on the comprehensive score value sequence, the boundaries of the four-level intervals are set according to the WHO 2021 version of the semen quality grade standard. The intervals to which all score values ​​belong are judged, and the grade labels are marked. The scores are arranged in time series to obtain the multimodal sperm quality assessment grade results.

[0142] According to the comprehensive score sequence and the WHO 2021 four-level quality standard, the grading rules are set as scores greater than 75 as "normal", 50 to 75 as "critical", 25 to 50 as "abnormal", and below 25 as "severely abnormal". The scores of each frame are mapped to their corresponding levels in turn. For example, F001 is 133.6, which is classified as "normal". The complete grading results are shown in Table 11:

[0143] Table 11 Multimodal quality assessment grade table

[0144]

[0145] As shown in Table 11, all image frames were evaluated and classified into the normal grade, and the grade mapping structure under the multimodal indicators was established to establish the grade results of multimodal sperm quality assessment.

[0146] A sperm quality assessment system based on multimodal fusion, comprising:

[0147] The temporal alignment module obtains a sequence of real-time sperm motion trajectory images, collects parameters such as sperm density, total motility percentage, and forward motion ratio, retains data pairs with consistent time, and generates a spatiotemporal alignment dataset;

[0148] The trajectory extraction module extracts all sperm trajectory paths in the image frame by frame based on the spatiotemporal alignment dataset, calculates the linearity index of the motion trajectory and locates the active area, annotates the pixel heat of the area, and generates a motion feature heat map;

[0149] The parameter fluctuation module obtains the density, vitality and forward motion data in the spatiotemporal alignment data set, calculates the parameter difference and the gradient of the rate of change, determines whether it exceeds the WHO threshold and marks the interval, and generates a parameter abnormality time window;

[0150] The morphological analysis module extracts the corresponding sperm morphological image based on the motion feature heat map and the image frame number that coincides with the time in the parameter abnormality time window, evaluates the head deformity rate, and generates a multimodal feature matrix by combining the motion trajectory linearity index and parameter changes;

[0151] The grade scoring module calculates the consistency index based on the multimodal feature matrix, divides the evaluation grades by using the scoring weight distribution, and generates a multimodal sperm quality evaluation result.

[0152] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A sperm quality assessment method based on multimodal fusion, characterized in that: The following steps are involved: S1: Acquire a real-time sperm motion trajectory image sequence, collect sperm density, total motility percentage, and forward motion ratio parameter records, combine the image sequence with the time tags in the parameter records, determine whether the adjacent time error is less than the time synchronization threshold, and generate a spatiotemporal alignment dataset; S2: Based on the spatiotemporal alignment dataset, all sperm trajectory paths in the image are extracted frame by frame, the linearity index of the motion trajectory is calculated, the distribution area is marked, and a motion feature heat map is generated; S3: Obtain density, vitality, and forward motion data from the spatiotemporal alignment dataset, calculate parameter differences and change rate gradients, determine whether the change amplitude is greater than the corresponding parameter fluctuation threshold, and generate a parameter anomaly time window; S4: extracting sperm morphology images with the same numbers according to the motion feature heat map and the image frame numbers in the parameter abnormality time window, evaluating the head deformity rate, and generating a multimodal feature matrix by combining the motion trajectory linearity index and parameter changes; S5: Based on the multimodal feature matrix, the vitality-morphology consistency index is calculated, and the scoring weight is allocated in combination with the trajectory continuity index, and the evaluation level is divided to generate a multimodal sperm quality assessment result; The steps for obtaining the motion feature heat map are specifically as follows: S211: Based on the spatiotemporal alignment dataset, extract the sperm trajectory path in the image frame by frame, identify the start and end positions of the trajectory, record the sperm head position coordinates in all consecutive frames along the path, record the Euclidean distance between the start and end points of each path and the entire trajectory path length, and obtain a path structure parameter set; S212: Based on the path structure parameter set and the trajectory frame interval, the formula is used: ； Calculate the linearity index of the sperm path trajectory and map it to the corresponding image frame to obtain the linearity parameter set of sperm movement, where: Indicates the Sperm in the Trajectory linearity index in the frame, Indicates the straight-line distance between the starting point and the end point of the path. 、 Respectively Sperm in frame The two-dimensional coordinates of Indicates the total time interval corresponding to the path, is the total number of frames in the path, is the position jitter compensation item in the current trajectory; S213: Based on the sperm motion linearity parameter set and the corresponding image frame coordinate system, the image area is divided into grid cells, the distribution density of trajectory points in each cell that meet the trajectory linearity index greater than the judgment threshold is counted, the color intensity in each grid is proportionally assigned to the image thermal color scale, and a motion feature heat map is generated.

2. The sperm quality assessment method based on multimodal fusion according to claim 1, characterized in that: The spatiotemporal alignment dataset includes image frame sequence numbers, synchronization label mapping relationships, and a density and motion parameter correspondence table. The motion feature heat map includes a trajectory linearity distribution map, a regional density map, and a motion direction vector map. The parameter anomaly time window includes a density fluctuation interval, a vitality anomaly segment, and a motion rate offset interval. The multimodal feature matrix includes a morphological anomaly label set, a trajectory stability index group, and a parameter synchronization anomaly label set. The multimodal sperm quality assessment results include a vitality-morphology consistency index score, a trajectory continuity assessment grade, and a sperm quality classification result.

3. The sperm quality assessment method based on multimodal fusion according to claim 1, characterized in that: The steps for obtaining the spatiotemporal alignment dataset are specifically as follows: S111: Acquire a sequence of real-time sperm motion trajectory images and corresponding time tags, collect three parameters recorded by a semen analyzer: sperm density, total motility percentage, and forward motion ratio, extract the corresponding time tags respectively, calculate the time difference between adjacent ones, and obtain a set of image and parameter time differences; S112: Based on the image and parameter time difference set, performing a judgment operation on each time difference and a time synchronization threshold, screening all time tag pairs whose time difference is less than the time synchronization threshold, and obtaining a time consistency data pair set; S113: Based on the temporally consistent data pair set, the corresponding image sequence is associated with three parameters: sperm density, total motility percentage, and forward motion ratio. Data pairs with the same time tags are matched. Each set of data is subjected to consistency verification and validity marking to establish a spatiotemporally aligned data set.

4. The sperm quality assessment method based on multimodal fusion according to claim 1, characterized in that: The steps for obtaining the parameter abnormal time window are specifically as follows: S311: Obtain sperm density, motility, and forward motion ratio data from the spatiotemporal aligned dataset, set a sliding time window, traverse all consecutive time points in the dataset in chronological order, record the density change, motility change, and forward motion ratio change in each window, and obtain a parameter change difference sequence; S312: Calculate the gradient of the rate of change per unit time based on the parameter change difference sequence, perform normalization, and make judgments item by item based on the corresponding parameter fluctuation threshold reference value, complete screening and marking of each gradient exceeding the limit state in the continuous window, and obtain a change gradient judgment value sequence; S313: Based on the change gradient judgment value sequence, screen the time periods that continuously exceed the parameter fluctuation threshold according to the continuity of the time axis, extract the start time and end time of the time interval that continuously meets the abnormal condition, and establish the parameter abnormal time window.

5. The sperm quality assessment method based on multimodal fusion according to claim 1, characterized in that: The steps for obtaining the multimodal feature matrix are specifically as follows: S411: According to the time information in the motion feature heat map and the parameter abnormality time window, the image frame number in the time overlapping interval is retrieved, all sperm target areas in the corresponding image are extracted according to the frame number, the target sperm head area is marked, and the sperm image group of the time overlapping frame is obtained. S412: Based on the sperm image group of overlapping time frames, identifying the edge of the sperm head structure in the image, marking each sperm for morphological determination, judging it according to the standard of normal sperm head, calculating the sperm head deformity rate, and obtaining a head deformity rate sequence; S413: Based on the head deformity rate sequence, the motion trajectory linearity parameter set under the corresponding image frame number is combined with the data at the same numbered position in the parameter change difference sequence, the data structure is unified, data fusion and format standardization are performed, and a multimodal feature matrix is ​​established.

6. The sperm quality assessment method based on multimodal fusion according to claim 1, characterized in that: The steps for obtaining the multimodal sperm quality assessment results are specifically as follows: S511: Based on the multimodal feature matrix, the deformity rate and vitality value corresponding to each frame image are extracted row by row, and whether the synchronization is in a deviated state in the same frame is determined. The consistency frequency of the relative change direction is calculated and uniformly output in percentage form to obtain the vitality morphology consistency index; S512: Reading the trajectory linearity index in the motion trajectory linearity parameter set corresponding to the frame image based on the vitality morphology consistency index, extracting the density, vitality, and forward motion parameter change trends of the aforementioned frames, and calculating to obtain a comprehensive score value sequence; S513: Based on the comprehensive score value sequence and in accordance with the semen quality grade standard, all score value intervals are judged, and grade labels are marked. The scores are arranged in time series to obtain a multimodal sperm quality assessment grade result.

7. A sperm quality assessment system based on multimodal fusion, characterized in that: The system is used to implement the sperm quality assessment method based on multimodal fusion according to any one of claims 1 to 6, comprising: The temporal alignment module obtains a sequence of real-time sperm motion trajectory images, collects parameters such as sperm density, total motility percentage, and forward motion ratio, retains data pairs with consistent time, and generates a spatiotemporal alignment dataset; The trajectory extraction module extracts all sperm trajectory paths in the image frame by frame based on the spatiotemporal alignment dataset, calculates the linearity index of the motion trajectory and locates the active area, annotates the pixel heat of the area, and generates a motion feature heat map; The parameter fluctuation module obtains the density, vitality and forward motion data in the spatiotemporal alignment data set, calculates the parameter difference and the gradient of the rate of change, determines whether it exceeds the WHO threshold and marks the interval, and generates a parameter abnormality time window; The morphological analysis module extracts the corresponding sperm morphological image based on the motion feature heat map and the image frame number that coincides with the time in the parameter abnormality time window, evaluates the head deformity rate, and generates a multimodal feature matrix by combining the motion trajectory linearity index and parameter changes; The grade scoring module calculates the consistency index based on the multimodal feature matrix, divides the evaluation grades by using the scoring weight distribution, and generates a multimodal sperm quality evaluation result.

Images