Sperm quality evaluation method and system based on multi-modal fusion

Abstract

The invention relates to the technical field of multi-modal fusion, in particular to a sperm quality assessment method and system based on multi-modal fusion, and the method comprises the following steps: obtaining image and parameter data to generate a space-time alignment set, extracting trajectory calculation linearity, analyzing parameter fluctuation to mark an abnormal time window, extracting an image assessment deformity rate to generate a feature matrix, and grading to generate a multi-modal sperm quality evaluation result. According to the method, linear calculation is introduced in the trajectory path extraction process, spatial distribution thermodynamic mapping is combined, the dynamic characteristics of local sperm populations are dominated, an abnormal time window is constructed based on the parameter value change rate and a set threshold value, and motion consistency indexes are combined, so that the dynamic characteristics of the local sperm populations are optimized. The explanation depth of sperm head malformation data in quality judgment is enhanced, the dimension space of sperm quality description is effectively expanded, the consistency ratio between vitality and morphology is introduced, track continuous characteristics are combined, a hierarchical grading system is established, and the accuracy of sperm quality evaluation results is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal fusion, and particularly to a sperm quality assessment method and system based on multimodal fusion. Background Art

[0002] The technical field of multimodal fusion includes comprehensive analysis methods that integrate clinical imaging data and semen parameter detection data. The core lies in establishing an association model for standardized semen reports, ultrasonic imaging features, and directly connected data from laboratory equipment, focusing on solving the problem of heterogeneous data alignment in multi-center databases, covering cross-modal matching technologies for varicocele grading parameters, testicular volume measurement values, and sperm movement trajectory parameters, dynamically adapting to changes in sperm morphology thresholds through the WHO standard conversion module, processing interval period data of multiple semen analyses using time series alignment algorithms, and constructing an automatic head vacuole recognition system based on CNN and an LSTM-driven sperm movement trend prediction model.

[0003] Among them, the sperm quality assessment method and system based on multimodal fusion refer to a parallel processing architecture that uses structured semen parameters and morphological staining images, covering spatio-temporal correlation modeling of VCL movement trajectory data output by a computer-aided semen analysis system and transrectal ultrasonic imaging features, using ResNet to extract abnormal acrosome morphological features and synchronously integrating ultrasonic parameters of the ejaculatory duct structure, realizing quantitative analysis of the total number of motile sperm based on a derived index calculation framework, using standardized splicing technology in feature-level fusion to unify the dimensions of semen volume parameters and movement speed vectors, and filling clinical guidelines when processing missing values in laboratory tests through multiple imputation algorithms.

[0004] In the existing sperm quality assessment process, during sperm trajectory analysis, there is a lack of quantitative extraction means for the trajectory morphology of local regions, resulting in insufficient coverage of sperm micro-dynamic characteristics and affecting the grasp of distributional differences within the population. The judgment of parameter changes mostly relies on the comparison of a single time point, and no dynamic recognition logic for the continuous change process is formed, restricting the capture ability of regions sensitive to fluctuations. In the morphological image processing link, data selection is disconnected from movement information, and the synergistic influence of morphological features on dynamic performance is not reflected, reducing the depth and interpretability of the assessment. Multimodal data fusion lacks unified temporal anchor support, and the logical connection between features is weak, resulting in limited decision-making value of the fused data. Most scoring models are constructed by stacking static classification results, and the weight distribution cannot effectively interact with the sample performance, making it difficult to support the fine determination and hierarchical management of abnormal samples. Summary of the Invention

[0005] The object of the present invention is to solve the deficiencies existing in the prior art, and to propose a sperm quality assessment method based on multimodal fusion.

[0006] To achieve the above object, the present invention adopts the following technical solutions: A sperm quality assessment method based on multimodal fusion, comprising the following steps:

[0007] S1: Obtain a sperm real-time movement trajectory image sequence, collect records of sperm density, total motility percentage, and forward motility ratio parameters, combine the image sequence with the time tags in the parameter records, determine whether the adjacent time error is less than the time synchronization threshold, and generate a spatio-temporal alignment data set;

[0008] S2: Based on the spatio-temporal alignment data set, extract all sperm trajectory paths in the image frame by frame, calculate the linearity index of the movement trajectory, mark the distribution area, and generate a movement feature heat map;

[0009] S3: Obtain the density, motility, and forward motility data in the spatio-temporal alignment data set, calculate the parameter difference and the change rate gradient, determine whether the change amplitude is greater than the corresponding parameter fluctuation threshold, and generate a parameter anomaly time window;

[0010] S4: According to the image frame numbers with the same time in the movement feature heat map and the parameter anomaly time window, extract the sperm morphology images with the same numbers, evaluate the head malformation rate, and combine the movement trajectory linearity index and parameter changes to generate a multimodal feature matrix;

[0011] S5: Based on the multimodal feature matrix, statistically calculate the motility-morphology consistency index, combine the trajectory continuity index for scoring weight allocation, divide the evaluation grades, and generate a multimodal sperm quality assessment result.

[0012] As a further solution of the present invention, the spatio-temporal alignment data set includes the image frame sequence number, the synchronous label mapping relationship, and the density and movement parameter correspondence table. The movement feature heat map includes the trajectory linearity distribution map, the regional density map, and the movement direction vector map. The parameter anomaly time window includes the density fluctuation interval, the motility anomaly section, and the movement rate offset interval. The multimodal feature matrix includes the morphological anomaly label set, the trajectory stability index group, and the parameter synchronization anomaly mark set. The multimodal sperm quality assessment result includes the motility-morphology consistency index score, the trajectory continuity evaluation grade, and the sperm quality classification result.

[0013] As a further solution of the present invention, the specific steps for obtaining the spatio-temporal alignment data set are as follows:

[0014] S111: Obtain a sperm real-time movement trajectory image sequence and the corresponding time tags, collect the three parameters of sperm density, total motility percentage, and forward motility ratio recorded by the semen analyzer, respectively extract the corresponding time tags, and calculate the time difference between adjacent ones to obtain the image and parameter time difference set;

[0015] S112: Based on the image and the parameter time difference set, perform a judgment operation on each time difference and the time synchronization threshold, screen all time tag pairs with time differences less than the time synchronization threshold, and obtain a time consistency data pair set;

[0016] S113: According to the time consistency data pair set, associate the corresponding image sequence with the three parameters of sperm density, total motility percentage, and progressive motility ratio, match the data pairs with the same time tags, perform consistency verification and validity marking on each group of data, and establish a spatio-temporal alignment data set.

[0017] As a further solution of the present invention, the obtaining step of the motion feature heat map is specifically as follows:

[0018] S211: Based on the spatio-temporal alignment data set, extract the sperm trajectory paths in the images frame by frame, identify the start and end positions of the trajectories, record the sperm head position coordinates in all consecutive frames on the paths, record the Euclidean distance and the total trajectory path length between the start and end points of each path, and obtain a path structure parameter set;

[0019] S212: According to the path structure parameter set, combine the time interval of the trajectory frame segments, calculate the linearity index of the sperm path trajectory, map it to the corresponding image frame, and obtain a sperm motion linearity parameter set;

[0020] S213: According to the sperm motion linearity parameter set, combine the coordinate system of the corresponding image frame, divide the grid units according to the image area, count the distribution density of the trajectory points in each cell that satisfy the trajectory linearity index greater than the determination threshold, assign the color intensity in each grid to the image heat level according to the ratio, and generate a motion feature heat map.

[0021] As a further solution of the present invention, the obtaining step of the parameter abnormal time window is specifically as follows:

[0022] S311: Obtain the sperm density, motility value, and progressive motility ratio data in the spatio-temporal alignment data set, set a sliding time window, traverse all consecutive time points in the data set in chronological order, record the density change amount, motility change amount, and progressive motility ratio change amount in each window, and obtain a parameter change difference sequence;

[0023] S312: According to the parameter change difference sequence, calculate the change rate gradient per unit time, perform normalization processing, and perform item-by-item judgment based on the corresponding parameter fluctuation threshold reference value to complete the screening and marking process of the over-limit state of each gradient in consecutive windows, and obtain a change gradient judgment value sequence;

[0024] S313: Based on the sequence of variation gradient judgment values, screen the time periods that continuously exceed the parameter fluctuation threshold according to the continuity of the time axis. Extract the start time and end time of the time intervals that continuously meet the abnormal conditions, and establish a parameter abnormal time window.

[0025] As a further solution of the present invention, the steps for obtaining the multi-modal feature matrix are specifically as follows:

[0026] S411: According to the motion feature heat map and the time information in the parameter abnormal time window, retrieve the image frame numbers within the time overlapping interval, extract all sperm target regions in the corresponding images frame by frame, mark the head regions of the target sperm, and obtain a group of sperm images of the overlapping frames.

[0027] S412: According to the group of sperm images of the overlapping frames, identify the edges of the sperm head structures in the images, make morphological judgment marks for each sperm, judge according to the normal sperm head standard, calculate the sperm head malformation rate, and obtain a sequence of head malformation rates.

[0028] S413: According to the sequence of head malformation rates, combine the data at the same numbered positions in the set of motion trajectory linearity parameters and the sequence of parameter change differences corresponding to the image frame numbers, unify the data structure, perform data fusion and format standardization processing, and establish a multi-modal feature matrix.

[0029] As a further solution of the present invention, the steps for obtaining the multi-modal sperm quality assessment result are specifically as follows:

[0030] S511: Based on the multi-modal feature matrix, extract the malformation rate and motility value corresponding to each frame of the image row by row, judge whether they are synchronously in a deviated state in the same frame, calculate the consistency frequency of the relative change direction, and uniformly output it in the form of a percentage to obtain a motility and morphology consistency index.

[0031] S512: According to the motility and morphology consistency index, read the trajectory linearity index in the set of motion trajectory linearity parameters corresponding to the image frame, and extract the density, motility, and forward movement parameter change trends of the foregoing frames, and calculate to obtain a sequence of comprehensive score values.

[0032] S513: Based on the sequence of comprehensive score values, judge the intervals to which all the score values belong according to the semen quality grade standard, mark the grade labels, and arrange them in time sequence to obtain the multi-modal sperm quality assessment grade result.

[0033] A sperm quality assessment system based on multi-modal fusion, comprising:

[0034] The timing alignment module obtains the real-time sperm movement trajectory image sequence, collects the records of sperm density, total motility percentage, and forward motility ratio parameters, retains the data pairs with consistent time, and generates a spatio-temporal alignment dataset;

[0035] Based on the spatio-temporal alignment dataset, the trajectory extraction module extracts all sperm trajectory paths in the image frame by frame, calculates the linearity index of the movement trajectory and locates the active area, annotates the pixel heat of the area, and generates a movement feature heat map;

[0036] The parameter fluctuation module obtains the density, motility, and forward motility data in the spatio-temporal alignment dataset, calculates the parameter difference and the change rate gradient, determines whether it exceeds the WHO threshold and marks the interval, and generates a parameter abnormal time window;

[0037] According to the image frame numbers with consistent time in the movement feature heat map and the parameter abnormal time window, the morphology analysis module extracts the corresponding sperm morphology images, evaluates the head abnormality rate, combines the linearity index of the movement trajectory and the parameter change, and generates a multi-modal feature matrix;

[0038] Based on the multi-modal feature matrix, the grade scoring module calculates the consistency index, uses the scoring weight distribution to divide the evaluation grades, and generates a multi-modal sperm quality evaluation result.

[0039] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0040] In the present invention, by introducing linearity calculation in the process of trajectory path extraction and combining spatial distribution heat mapping, the dynamic characteristics of local sperm populations are made explicit. An abnormal time window is constructed based on the parameter value change rate and the set threshold, and combined with the movement consistency index, multi-layer information composite analysis is realized, enhancing the explanatory depth of sperm head abnormality data in quality judgment, achieving feature intersection, connecting the internal structural logic of morphology, trajectory, and function data, effectively expanding the dimensional space of sperm quality description. The scoring mechanism introduces the consistency ratio between motility and morphology and combines the trajectory continuous feature to establish a more hierarchical grading system, ensuring the accuracy of the sperm quality evaluation result. Brief Description of the Drawings

[0041] Figure 1 is the main step flowchart of the present invention;

[0042] Figure 2 is the flowchart for obtaining the spatio-temporal alignment dataset of the present invention;

[0043] Figure 3 is the flowchart for obtaining the movement feature heat map of the present invention;

[0044] Figure 4 is the flowchart for obtaining the parameter abnormal time window of the present invention;

[0045] Figure 5 This is the flowchart for obtaining the multi-modal feature matrix of the present invention;

[0046] Figure 6 This is the flowchart for obtaining the multi-modal sperm quality assessment result of the present invention. Detailed implementation manners

[0047] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention. In addition, in the description of the present invention, "a plurality of" means two or more, unless otherwise specifically defined.

[0049] Please refer to Figure 1 , a sperm quality assessment method based on multi-modal fusion, comprising the following steps:

[0050] S1: Obtain the real-time sperm movement trajectory image sequence and time tags, collect the sperm density, total motility percentage, and forward motility ratio parameter records in the structured semen analyzer, and combine the time tags in the image sequence and parameter records to determine whether the time error between adjacent two is less than the time synchronization threshold (200 ms, the device synchronization requirement set according to the ISO23162:2021 semen analysis standard), and retain the data pairs with consistent time to generate a spatio-temporal alignment data set;

[0051] S2: Based on the spatio-temporal alignment data set, extract all sperm trajectory paths in the image frame by frame, calculate the linearity of the movement trajectory (VSL, a movement parameter defined in the World Health Organization's Laboratory Manual for the Examination and Processing of Human Semen), and mark the distribution area positions in the image to generate a movement feature heat map;

[0052] S3: Obtain the density, vitality, and forward motility data in the spatio-temporal alignment dataset, calculate the parameter difference and rate of change gradient according to a sliding time window (an analysis window set according to the sampling frequency of the CASA (Computer-Aided Sperm Analysis) system, 500 ms), and determine whether the change amplitude is greater than the density fluctuation threshold (a concentration change warning value set based on the WHO 5th Edition standard (≥15×10^6 / mL is normal)), the vitality fluctuation threshold (referring to the WHO progressive motility sperm proportion standard (≥32% is normal)), and the motility rate fluctuation threshold (usually set at ±3% according to the WHO standard). Mark the time intervals with continuous changes exceeding the threshold to generate parameter abnormal time windows;

[0053] S4: According to the image frame numbers that are consistent in time between the motility feature heatmap and the parameter abnormal time window, extract the sperm morphology images with the same numbers, evaluate the head abnormality rate (the core morphological evaluation index defined by ISO23162:2021), and combine the motility trajectory linearity index and parameter changes of the corresponding frames to generate a multi-modal feature matrix;

[0054] S5: Based on the multi-modal feature matrix, calculate the vitality-morphology consistency index, combine the trajectory continuity index for comprehensive scoring weight assignment, and classify the evaluation level according to the WHO semen quality classification standard (a four-level classification standard (normal / critical / abnormal / severely abnormal) released by the World Health Organization in 2021) to generate a multi-modal sperm quality evaluation result.

[0055] The spatio-temporal alignment dataset includes the image frame sequence number, the synchronous tag mapping relationship, and the density and motility parameter correspondence table. The motility feature heatmap includes the trajectory linearity distribution map, the regional density map, and the motility direction vector map. The parameter abnormal time window includes the density fluctuation interval, the vitality abnormal section, and the motility rate deviation interval. The multi-modal feature matrix includes the morphological abnormality tag set, the trajectory stability index group, and the parameter synchronization abnormal mark set. The multi-modal sperm quality evaluation result includes the vitality-morphology consistency index score, the trajectory continuity evaluation level, and the sperm quality classification result.

[0056] Please refer to Figure 2 For step S1, it is as follows:

[0057] S111: Obtain the sperm real-time motility trajectory image sequence and the corresponding time tags, collect the three parameters of sperm density, total vitality percentage, and forward motility proportion recorded by the semen analyzer, extract the corresponding time tags respectively, and calculate the time difference between adjacent ones to obtain the image and parameter time difference set;

[0058] Obtaining the real-time sperm motility trajectory image sequence and the corresponding time tags is achieved by continuously capturing image frames of the sperm movement process in a semen sample under a microscope using a high-speed imaging system, and attaching the capture time tag to each frame of the image. The time tag is accurate to the millisecond level. If the image sequence is 25 frames per second, the time interval for each frame is 40 ms. Subsequently, parameters such as sperm density, total motility percentage, and forward motility ratio extracted from the structured semen analyzer are classified according to the acquisition time. The system records the acquisition time of this set of parameters. Through the time stamp extraction operation, the time tag matching each set of parameter records can be obtained. For example, if a set of parameters is obtained at 10:01:03.420 seconds on June 15, 2024, its tag is set as "20240615100103420". Then, the time tags of each frame in the image sequence are compared one by one with the tags in the structured parameter records, and the absolute difference operation is used to compare the time difference between the two. The specific process is as follows: Calculate the difference between the tag "20240615100103400" of image frame T1 and the tag "20240615100103420" of parameter record P1, and the obtained difference is 20 ms. Repeat this operation to perform calculations for all pairs of image frames and parameter tags. If the number of image frames is 750 and the number of parameter records is 100, the actual comparison will involve 750×100 difference judgments. Although the amount of data is large, the comparison speed can be accelerated by optimizing the retrieval with data pointers. After that, the paired differences of all image frames and parameter records are collected into an array, such as the ΔT array = {20, 180, 300, 25,...}, and the unit of each value is ms. It should be noted that in some parameter records, the time tag cannot be accurately obtained or there are missing fields. In this case, the corresponding records are not included in the operation and are regarded as invalid records and excluded. To enhance data consistency, an initial parameter density threshold can be set to only select sample data with a density greater than 50 million / mL to participate in the operation and avoid low-density interference. For example, if the density of sample A is 63 million / mL, it is retained. The differences between the image frames and the structured parameter records under this sample constitute the image and parameter time difference set.

[0059] S112: Based on the image and parameter time difference set, perform a judgment operation on each time difference and the time synchronization threshold, and screen out all time tag pairs with time differences less than the time synchronization threshold to obtain the time consistency data pair set;

[0060] Based on the image and parameter time difference set, each difference value item in the array is judged against the 200ms time synchronization threshold one by one. The specific operation is as follows: Set the threshold T to 200ms, and perform a comparison operation on each item in the ΔT array. If ΔT_i < T, then consider the corresponding image frame and parameter record of this item to meet the synchronization condition, mark and save them; otherwise, eliminate them. For example, when the ΔT array is {20, 180, 300, 25, 190, 250}, the indices that meet the condition of being less than 200ms are 0, 1, 3, 4, that is, the frames with corresponding image frame numbers T0, T1, T3, T4 and the parameter records are paired effectively, forming data pairs that meet the time consistency condition. To verify the rationality of the threshold selection, different synchronization requirement conditions are introduced for testing. The thresholds are set to 150ms, 200ms, and 250ms respectively. Calculate the number of retained data pairs under each threshold and present it in the form of Table 1 as follows:

[0061] Table 1 Data pair screening result table under different time synchronization thresholds

[0062] As shown in Table 1, the retention ratio at the 200ms threshold is 81.81%, which is the benchmark set to balance the synchronization accuracy and data volume. Subsequently, an effective data pair set composed of image frames and parameter records that meet the conditions is constructed. The time label pairing result will be used as the pre-input for the integration of images and structured parameter data in the next stage.

[0063] S113: According to the time consistency data pair set, associate the corresponding image sequence with the three parameters of sperm density, total motility percentage, and progressive motility ratio, match the data pairs with the same time label, perform consistency verification and validity marking on each group of data, and establish a spatio-temporal alignment data set;

[0064] Based on the time-consistent data pair set, for each group of image frames with a confirmed time difference less than 200 ms, a pairing and matching operation is performed with the corresponding structured parameter records. A data group is established by synchronously combining the image frame serial numbers and parameter serial numbers. The information within the image frame is obtained through image parsing to obtain dynamic information such as sperm trajectories, head positions, and velocity vectors. The structured record contains the density, total motility percentage, and forward motility ratio value at the corresponding moment of this frame. For example, the time label of image frame T15 is "20240615100201400", and the corresponding parameter record P15 label is "20240615100201410", with a difference of 10 ms less than the threshold, belonging to a valid data pair. The system integrates the sperm trajectory vector group V_T15 = [(x1, y1, t1), (x2, y2, t2)...] in image frame T15 with the structured parameters {S = 71 million / mL, M = 72.4%, A = 61.1%} to form a complete data item. The combination operation is performed frame by frame, traversing all time-consistent data pairs, and a total of 589 groups of image frame and parameter record combinations are obtained. Then, outliers of the three parameters within each group of data are removed. If the forward motility ratio is less than 20% or greater than 90%, it is marked as abnormal and does not participate in database construction. At the same time, for the missing frame situation in the image frame trajectory, it is repaired by interpolating adjacent frames, and only the situation where the continuous missing frames do not exceed 2 frames is patched. The finally obtained data group constitutes a three-dimensional array structure in a unified format, with the fields unified as [trajectory vector set, density value, total motility percentage, forward motility ratio]. The data group will further serve as the original input for spatio-temporal analysis to complete the establishment of the spatio-temporal alignment data set.

[0065] Please refer to Figure 3 , step S2 is as follows:

[0066] S211: Based on the spatio-temporal alignment data set, extract the sperm trajectory paths in the image frame by frame, identify the start and end positions of the trajectories, record the sperm head position coordinates in all consecutive frames on the path, record the Euclidean distance and the total trajectory path length between the start and end points of each path, and obtain the path structure parameter set;

[0067] Based on the spatio-temporal alignment data set, traverse the sperm image sequence frame by frame to extract the trajectory paths. First, identify each sperm individual in the image frame, extract its head coordinate point, and record its number in the image frame to form the corresponding relationship between the sperm number and the coordinates. In image frame F001, a trajectory numbered T01 is detected, and its initial head position coordinates are (12, 30), and the end position is (48, 70). Connect the initial point and the end point, and calculate the start and end distance of this path as:

[0068] ;

[0069] If there are 24 consecutive recording points on the trajectory in the middle frame, and the cumulative total path length is 75.2 μm based on the distances between adjacent coordinate points as follows, then the corresponding path information is organized into a path structure parameter set as shown in Table 2:

[0070] Table 2 Path Structure Parameter Table

[0071] As shown in Table 2, by extracting the starting and ending coordinates of each trajectory in the image frame and calculating its path length and start - end distance, standard path structure parameters can be obtained, and a path structure parameter set can be acquired.

[0072] S212: According to the path structure parameter set, combined with the time interval of the trajectory frame segment, use the formula:

[0073] ;

[0074] Calculate the linearity index of the sperm path trajectory, map it to the corresponding image frame, and obtain the sperm motility linearity parameter set. Among them, represents the linearity index of the th sperm in the th frame, represents the straight - line distance between the starting point and the ending point of this path, and are the two - dimensional coordinates of the sperm in the th frame respectively, represents the total time interval corresponding to this path, is the total number of frames of the path, is the position jitter compensation term in the current trajectory;

[0075] Based on the path structure parameter set, calculate the linearity index VSL of the movement trajectory for each path, introduce the number of consecutive frames of the trajectory and the corresponding time interval for compensation processing. Taking trajectory T01 as an example, the path length is 75.2 μm, the start - end distance is 62.4 μm, the total number of frames of the trajectory is 22 frames, and the time interval for each frame is 40 ms, then Δt = 880 ms, and the position jitter compensation term σ is set to 2 frames. The formula is:

[0076] ;

[0077] Continue to calculate the trajectories T02 - T05 in the same way. Assuming the obtained results are 4.98, 5.03, 5.68, and 5.00 respectively, then map each VSL value to the corresponding image frame and trajectory number to form the following array:

[0078] ;

[0079] Assign the VSL values to the corresponding position areas of the image frames according to the track numbers for subsequent heat map data superposition, and obtain the sperm motility linearity parameter set.

[0080] The track linearity index is an important indicator to measure whether the movement path of sperm in the image sequence is close to a straight line. Its essence is the ratio of the straight-line distance between the starting point and the ending point to the length of the actual track path, and it is corrected by considering the time factor. The closer this value is to a larger number, the more stable the movement track of the sperm in the consecutive frame images, the smaller the deviation, and the lower the degree of path bending, indicating that it has stronger forward propulsion ability. On the contrary, if the track linearity value is low, it usually means that the movement direction of the sperm fluctuates greatly, the path bends or turns frequently, and the movement is non-directional. Therefore, the track linearity value is not only used to describe the spatial characteristics of a single track, but also can be used as an important data parameter for screening sperm activity and judging its functional state, and has the function of quantitatively describing the movement pattern characteristics of the sperm population statistically.

[0081] The operation logic of the formula comprehensively considers the geometric characteristics and time stability of the sperm movement path, aiming to measure the movement linearity intensity of each track. Its overall structure consists of three parts: First, the numerator part represents the ratio of the start-end distance of the path to the total path length, which is used to quantify the linearity degree of the track. The closer this ratio is to 1, the more the track tends to be a straight line, and its value is greatly affected by path deviation and curvature; Second, this ratio is then multiplied by a square root term , which is used to comprehensively consider the influence of time persistence and track stability. Among them, the numerator is the total time experienced by this track, reflecting its continuous movement time, while the denominator introduces the total number of frames and the track stability correction term σ. σ reflects the displacement jitter degree in the track. The introduction of the square root structure is to avoid excessive pulling of the time parameter in the product and maintain the smooth adjustment effect of the time factor on the linearity index. Finally, the overall expression uses the absolute value to ensure that the output result is positive and avoid directional interference with the overall quantity value. Therefore, the addition, subtraction, multiplication, division and square root structure of the whole formula respectively undertake the dual functions of path shape evaluation and time compensation, and jointly reflect the comprehensive evaluation of the movement linearity of the track.

[0082] S213: According to the sperm motility linearity parameter set, combined with the corresponding image frame coordinate system, divide the grid cells according to the image area, count the distribution density of the track points in each cell that meet the track linearity index greater than the determination threshold, and assign the color intensity in each grid to the image heat color scale according to the ratio to generate the movement feature heat map;

[0083] According to the sperm motility linearity parameter set, all VSL index values are mapped to the coordinate grid area of the image frame. The image is set to 800×800 pixels, divided into 100×100 pixel grids each, forming a total of 64 grid cells of 8×8. Traverse all frame point positions of each trajectory and assign them to the corresponding grids, and judge whether the corresponding VSL value exceeds the judgment threshold of 4.5. If it exceeds, increment the corresponding counter in that grid. For example, if the point of trajectory T01 falls in the (1, 1) grid and the VSL is 5.25, record the count +1. After traversing all trajectories, count the number of points with VSL greater than 4.5 in each of the 64 grids and calculate the proportion, map it to the color scale array according to the proportion, set the maximum density mapped to red and the minimum mapped to blue, generate the RGB value matrix and write it back to the original image space to complete the image rendering process and generate the motility feature heat map.

[0084] Please refer to Figure 4 , step S3 is as follows:

[0085] S311: Obtain the sperm density, motility value and progressive motility ratio data in the spatio-temporal alignment dataset, set a sliding time window, traverse all consecutive time points in the dataset in chronological order, record the density change amount, motility change amount and progressive motility ratio change amount of each window, and obtain the parameter change difference sequence;

[0086] After obtaining the density, motility and progressive motility ratio data in the spatio-temporal alignment dataset, first divide the sliding time window in chronological order, set the length of each window to 500 ms, extract the sperm parameter data of adjacent two frames in consecutive time windows, respectively obtain the density, motility and progressive motility ratio values of the current window and the next window in each group, and calculate the difference between the parameters in turn. For example, the density in time window W1 is 1820×10 6 / mL, and the density in W2 is 1670×10 6 / mL, then the density difference is -150×10 6 / mL. If the motility in W1 is 36% and in W2 is 40%, then the motility difference is +4%. If the progressive motility ratio drops from 31% to 28%, the difference is -3%. This difference indicates that within 500 ms, the sperm motility in the semen has increased slightly while the density and motility have decreased slightly. Repeat the operation to obtain the parameter differences of more time window groups and construct a complete parameter change sequence, as shown in Table 3:

[0087] Table 3 Example table of parameter change differences

[0088] As shown in Table 3, by sliding the window by window and comparing the parameter change trends, recording the continuous change directions and difference states of each parameter item, the short-term dynamic change process of density, motility and progressive motility can be clarified, and the parameter change difference sequence can be obtained.

[0089] S312: Calculate the rate-of-change gradient per unit time based on the parameter change difference sequence, perform normalization processing, and make item-by-item judgments according to the corresponding parameter fluctuation threshold reference value to complete the screening and marking process of the gradient overrun status for each item within the continuous window, obtaining the change gradient judgment value sequence;

[0090] Based on the parameter change difference sequence, divide the change amount of each parameter by the window interval of 0.5 seconds to obtain the rate of change per unit time, i.e., the change gradient value. Calculate the density change rate, vitality change rate, and forward movement ratio change rate for each set of data in Table 6 respectively. Taking W1 - W2 as an example, the density change rate is -150÷0.5 = -300×10 6 / mL / s, the vitality change rate is 4÷0.5 = +8% / s, and the forward movement change rate is -3÷0.5 = -6% / s. Perform the same operation for all windows in sequence and extract the absolute value of the change rate for anomaly judgment. The system presets the reference fluctuation threshold for the density change rate as 300×10 6 / mL / s, the vitality change rate is 6% / s, and the forward movement ratio is 3% / s. Compare the calculation results with the above benchmarks. If any item exceeds, record the window as abnormal. For example, if both the density change rate and the forward movement ratio change rate in W1 - W2 exceed the set range, the status of this window is "multiple anomalies". Compare all the results to construct a judgment sequence, as shown in Table 4:

[0091] Table 4 Parameter Change Gradient and Judgment Status Table

[0092] As shown in Table 4, the constructed judgment sequence marks whether the parameter gradient exceeds the threshold within each time period and indicates the type of anomaly source, obtaining the change gradient judgment value sequence.

[0093] S313: Based on the change gradient judgment value sequence, screen the time periods that continuously exceed the parameter fluctuation threshold according to the continuity of the time axis, extract the start time and end time of the time intervals that continuously meet the abnormal conditions, and establish parameter abnormal time windows;

[0094] Based on the change gradient judgment value sequence, retrieve the windows with abnormal status in multiple consecutive time periods in chronological order, and extract the abnormal start time and end time intervals. For example, if W1 - W2, W2 - W3, and W3 - W4 are three consecutive abnormal sections, then merge this segment into a single abnormal time window starting from time point T1 and ending at T4, mark it as abnormal segment A1, and record the anomaly type as "multiple continuous anomalies". Continue to extract all continuous abnormal window intervals that meet the conditions in this way. Each interval item records the start and end times, the number of windows included, and the dominant anomaly source, constructing a complete abnormal interval array. After sorting, it is shown in Table 5:

[0095] Table 5 Abnormal Time Window Record Table

[0096] As shown in Table 5, the abnormal time window records the time range and the dominant abnormal index type when parameters show continuous and drastic fluctuations in different stages during the detection period, and an abnormal time window for parameters is established.

[0097] Please refer to Figure 5 , and the steps of S4 are as follows:

[0098] S411: According to the time information in the motion feature heat map and the parameter abnormal time window, retrieve the image frame numbers within the time overlap interval, extract all sperm target regions in the corresponding images according to the frame numbers, mark the head regions of the target sperm, and obtain a group of sperm images for the time overlap frames;

[0099] According to the time periods recorded in the motion feature heat map and the parameter abnormal time window, perform an intersection operation on their time tags, filter out the image frame numbers that exist in both records, sequentially extract the corresponding image frames of these frame numbers in the image dataset, perform target detection on the sperm individuals in the images, separate each sperm structure region through segmentation means, and record one by one the correspondence between the number of sperm individuals in the image frame and the image number. For example, 20 sperm regions are detected in frame F001 and 18 targets are detected in frame F002, all of which are used as the input image set for subsequent morphological judgment. Combine the image frame numbers and the included sperm image segments into a dataset, uniformly number and structurally save it, and the summary structure is shown in Table 6:

[0100] Table 6 Sperm Image Extraction Record Table

[0101] As shown in Table 6, a mapping is formed between the image frame numbers and the detected number of sperm individuals. Subsequently, morphological judgment will be carried out frame by frame based on this data to obtain a group of sperm images for the time overlap frames.

[0102] S412: According to the group of sperm images for the time overlap frames, identify the edges of the sperm head structures in the images, perform morphological judgment and marking on each sperm, and make a judgment according to the normal sperm head standards specified in ISO23162:2021. If any one of the head aspect ratio, symmetry, or curvature characteristics does not meet the standard, it is counted as a deformed head, and the formula is used:

[0103] ;

[0104] Calculate the sperm head deformity rate, obtain a sequence of head deformity rates, and construct a vector array of the deformity rates for all frames. Among them, represents the head deformity rate (%) in the th frame image, Indicates the number of spermatozoa identified as having head deformities in the frame, Indicates the total number of spermatozoa in the frame, multiplied by 100% for conversion to percentage expression;

[0105] According to the sperm image group of time-overlapping frames, the head structure is extracted for each sperm region in each frame of image, and its width, height, edge curvature and symmetry index are obtained. According to the structural definition of normal sperm heads in ISO23162:2021 standard, if the aspect ratio of the sperm head is not between 1.5 and 1.8, or the symmetry score is less than 80%, it is judged as a deformed individual. In image F001, 20 spermatozoa are identified, and 4 of them do not meet the standard definition, so they are deformed spermatozoa. When calculating the deformity rate, substitute into the formula. Among them, the number of deformed spermatozoa in frame F001 is , and the total number is , then the calculation is , similarly in F002 it is , and so on, the statistics are shown in Table 7:

[0106] Table 7 Calculation Table of Head Deformity Rate

[0107] As shown in Table 7, the above results are input as important indicators for morphological quality assessment in each frame of image, and a head deformity rate sequence is obtained.

[0108] The head deformity rate is one of the core indicators for evaluating sperm morphological quality. Its specific significance lies in reflecting the numerical expression of the proportion of abnormal head structures among the detected sperm individuals in the image. This indicator determines head deformities according to the ISO23162:2021 standard, including inconsistent head sizes, unbalanced aspect ratios, structural asymmetry, morphological distortion, etc. The higher the deformity rate value, the greater the proportion of abnormal sperm morphology in the sample, and the greater the possible impact on biological activity and fertilization ability. Therefore, this indicator is not only used for morphological quality assessment of single-frame images, but also as a key reference parameter in fertility assessment, assisted reproductive diagnosis and image screening tasks.

[0109] The formula operation logic is based on the direct proportional relationship calculation of the total number of spermatozoa and the number of deformed spermatozoa in a single-frame image. The numerator part represents the number of spermatozoa individuals identified as having deformed structures in the current image frame, and the denominator part It represents the total number of all recognized sperm in the frame image. The two are divided to obtain the proportion value of abnormal individuals in the total number of individuals. This value is a decimal. To convert it into a common percentage form for visual display and subsequent model input processing, it is multiplied by 100%. The absolute value symbol is used to prevent negative values caused by logical inversion during image label recognition or calculation, ensuring that the calculation result is a positive real number, meeting the requirement of positive representation of morphological indicators, so as to clearly reflect the proportion of sperm head malformation structures in each image frame in the total sample.

[0110] S413: According to the head malformation rate sequence, combined with the data at the same numbered positions in the linearity parameter set of the movement trajectory and the parameter change difference sequence corresponding to the image frame number, unify the data structure, perform data fusion and format standardization processing, and establish a multi-modal feature matrix;

[0111] According to the head malformation rate sequence, frame by frame match the linearity parameter values and parameter change difference groups of the movement trajectory corresponding to the image numbers, extract the corresponding linearity values, density differences, vitality differences, forward movement differences, etc. for each frame, and combine them side by side with the malformation rate to form a five-dimensional vector, forming a unified record structure. For example, in image frame F001, the malformation rate is 20.0%, the trajectory linearity is 5.25, the density change is -150×10 6 / mL, the vitality change is +4%, and the forward movement change is -3%. Then the multi-modal vector of this frame is [20.0, 5.25, -150, +4, -3]. And so on, the matrix is constructed as shown in Table 8:

[0112] Table 8 Multi-modal Feature Matrix

[0113] As shown in Table 8, each row of data corresponds to an image frame, fusing morphological, movement trajectory, and parameter change information to form a unified vector structure and establish a multi-modal feature matrix.

[0114] Please refer to Figure 6 , the steps of S5 are as follows:

[0115] S511: Based on the multi-modal feature matrix, extract the malformation rate and vitality values corresponding to each frame of image row by row, judge whether they are synchronously in a deviated state in the same frame, calculate the consistency frequency of the relative change direction, and uniformly output it in percentage form to obtain the vitality morphology consistency index;

[0116] Based on the parameters of each frame of image collected in the multimodal feature matrix, first identify the fields where the malformation rate and vitality value are located in each frame. By traversing the order of the image frame numbers, compare the vitality trend and malformation rate trend of adjacent frames. If the vitality value of the current frame decreases compared to the previous frame while the malformation rate increases, it is regarded as an event of deviation from morphological vitality consistency. Mark and accumulate this event, count the number of such synchronous offsets in all image frames, and divide by the total number of frames to obtain the vitality-morphology consistency index. For example, if the vitality of frame F001 drops from 0.85 in the previous frame to 0.82, and the malformation rate rises from 18% to 20%, it is a consistency event. If such an event occurs 3 times in 5 frames, the consistency index is 0.6, which is 60.0% when expressed as a percentage. The following record is established, as shown in Table 9:

[0117] Table 9 Vitality-Morphology Consistency Event Table

[0118] As shown in Table 9, there are 2 frames that meet the conditions of decreasing vitality and increasing malformation rate. Calculate the consistency index for the total of 5 frames as , which is 40.0% when converted to a percentage, and obtain the vitality-morphology consistency index.

[0119] S512: According to the vitality-morphology consistency index, read the trajectory linearity index in the set of motion trajectory linearity parameters corresponding to the image of the corresponding frame, and extract the density, vitality, and forward motion parameter change trends of the aforementioned each frame. Use the formula:

[0120] ;

[0121] Perform operations to obtain a sequence of comprehensive score values, where represents the comprehensive score value of the th frame image, , , respectively represent the normalized vitality, density, and forward motion indexes of the th frame, represents the trajectory linearity value, represents the vitality-morphology consistency index of the th frame, , , are the scoring weights of the participating items, which are set to 0.4, 0.3, and 0.3 respectively. The constant term 0.5 in the denominator part of the formula is the theoretical neutral value of the vitality-morphology consistency index. This value corresponds to the ideal symmetric reference state where there is no obvious correlation between decreasing vitality and increasing malformation. The degree of fluctuation above and below this point represents the degree of consistency deviation, so it is used as the denominator normalization base value;

[0122] According to the vitality form consistency index, the normalized vitality, density, and forward movement values in each frame are extracted, and the trajectory linearity index is read. The three parameters are weighted with weights of 0.4, 0.3, and 0.3. Among them, the weight settings of 0.4, 0.3, and 0.3 are based on the influence level division of the three parameters on the overall fertility ability in the WHO semen quality definition. As the core index of sperm motility, vitality directly determines the possibility of its combination with the egg cell, so the highest weight of 0.4 is given; density reflects the number of sperm per unit volume and is related to the total available amount. Its influence degree is second to vitality, and the weight is set to 0.3; although the forward movement rate also reflects the motility state, there is a certain overlap with vitality, and its fluctuation sensitivity is slightly lower than the first two in some frames, so a weight of 0.3 is assigned. The sum of the three is 1 to ensure the normalization of the score, and this setting remains stable when comparing the data performance of individuals with different fertility abilities. When the variation ranges of density and vitality expand, the scores are more likely to be separated, forming a differential judgment space. Taking frame F001 as an example, its vitality value is 0.82, density is 0.79, forward movement is 0.81, linearity is 5.25, and the consistency index is 0.61. The calculation process is as follows:

[0123] ;

[0124] In the calculation, the weighted part is , the root value of the trajectory is , the consistency deviation is , the denominator is 1.11, and the final score is:

[0125] ;

[0126] The calculation results of the scores for each frame are as follows, as shown in Table 10:

[0127] Table 10 Multimodal Score Calculation Table

[0128] As shown in Table 10, the score values are all in a relatively high range. The overall comprehensive performance of each frame is presented on a percentile scale to obtain the comprehensive score value sequence.

[0129] The comprehensive score value is the unified quantitative result reflecting the multi-dimensional performance of sperm quality in a single-frame image. Its value integrates five key indicators: normalized motility, density, forward motility, trajectory linearity, and motility-morphology consistency, reflecting the weighted contributions and mutual constraints of each parameter in the quality evaluation system for each frame. This score not only considers static characteristics such as the steady-state levels of morphology and density but also integrates dynamic parameters such as the change trends of trajectory stability and motility. At the same time, a consistency index is introduced as a regulatory term to evaluate the coordination among multiple parameters. Therefore, the comprehensive score value can be regarded as a numerical reflection of the overall biological quality of sperm in the current spatio-temporal state and is the core basic indicator for subsequent grading evaluation, state judgment, and diagnostic support.

[0130] The operation logic design of the formula aims to achieve the integrated scoring of multi-dimensional sperm quality indicators. Among them, the normalized motility value, density value, and forward motility value are positive evaluation indicators, and it is beneficial to the quality when they increase in the same direction. Therefore, these three parameters are weighted and added together, with weights of 0.4, 0.3, and 0.3 respectively, to reflect their relative importance in the comprehensive evaluation. This weighted sum structure ensures that each indicator can participate in the superposition of the score value according to the set proportion. At the same time, as an independent dynamic indicator, the larger the value of trajectory linearity, the more stable the movement trajectory. To maintain its independent influence and suppress the pulling effect of its overly large value, the processing of taking the square root and then the absolute value is introduced to achieve scale compression and smooth influence, and then it is deducted from the weighted value in the final score to express the negative correction effect of trajectory deviation on quality. In addition, as a signal term reflecting the co-variation of motility and morphology, the farther the consistency index deviates from the theoretical neutral value of 0.5, the stronger its deviation degree. Therefore, a denominator structure is constructed as the absolute value of the difference between it and 0.5 plus 1, so that the closer this term is to 0.5, the smaller the overall influence, and vice versa, the farther it is, the lower the final score will be. This normalization factor effectively reflects the fluctuation degree of motility-morphology consistency, thus jointly completing the correlation modeling among multi-modal parameters and the quality index scoring.

[0131] S513: Based on the comprehensive score value sequence, set the four-level interval boundaries according to the WHO 2021 semen quality grade standard, judge the intervals to which all score values belong, mark the grade labels, and arrange them in time series to obtain the multi-modal sperm quality assessment grade results;

[0132] According to the comprehensive score value sequence and the WHO 2021 four-level quality standard, the grading rule is set as a score greater than 75 is "normal", 50 to 75 is "critical", 25 to 50 is "abnormal", and less than 25 is "severely abnormal". The scores of each frame are mapped to their corresponding grades in turn. For example, F001 is 133.6 and is classified as the "normal" grade. The complete grading results are shown in Table 11:

[0133] Table 11 Multi-modal quality assessment grade table

[0134] As shown in Table 11, all image frame evaluations are classified into the normal level, a hierarchical mapping structure under multi-modal metrics is established, and the hierarchical results of multi-modal sperm quality evaluation are established.

[0135] A sperm quality evaluation system based on multi-modal fusion, comprising:

[0136] The timing alignment module obtains the real-time sperm movement trajectory image sequence, collects the records of sperm density, total motility percentage, and forward movement ratio parameters, retains the data pairs with consistent time, and generates a spatio-temporal alignment data set;

[0137] The trajectory extraction module extracts all sperm trajectory paths in the image frame by frame based on the spatio-temporal alignment data set, calculates the linearity index of the movement trajectory and locates the active area, annotates the pixel heat of the area, and generates a movement feature heat map;

[0138] The parameter fluctuation module obtains the density, motility, and forward movement data in the spatio-temporal alignment data set, calculates the parameter difference and the change rate gradient, determines whether it exceeds the WHO threshold and marks the interval, and generates a parameter abnormal time window;

[0139] The morphology analysis module extracts the corresponding sperm morphology images according to the image frame numbers with consistent time in the movement feature heat map and the parameter abnormal time window, evaluates the head malformation rate, and combines the movement trajectory linearity index and parameter changes to generate a multi-modal feature matrix;

[0140] The grade scoring module calculates the consistency index based on the multi-modal feature matrix, uses the scoring weight assignment to divide the evaluation grades, and generates the multi-modal sperm quality evaluation result.

[0141] The above is only the preferred embodiment of the present invention, and does not limit the present invention in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes and apply them to other fields. However, as long as it does not depart from the technical solution content of the present invention, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still belong to the protection scope of the technical solution of the present invention.

Claims

1. A sperm quality assessment method based on multimodal fusion, characterized in that, It includes the following steps: S1: Obtain the real-time sperm movement trajectory image sequence, collect the records of sperm density, total motility percentage, and progressive motility ratio parameters, combine the image sequence with the time tags in the parameter records, judge whether the adjacent time error is less than the time synchronization threshold, and generate a spatio-temporal alignment dataset; S2: Based on the spatio-temporal alignment dataset, extract all sperm trajectory paths in the image frame by frame, calculate the linearity index of the movement trajectory, mark the distribution area, and generate a heat map of movement characteristics; S3: Obtain the density, motility, and progressive motility data in the spatio-temporal alignment dataset, calculate the parameter difference and the change rate gradient, judge whether the change amplitude is greater than the corresponding parameter fluctuation threshold, and generate a parameter anomaly time window; S4: According to the image frame numbers with the same time in the heat map of movement characteristics and the parameter anomaly time window, extract the sperm morphology images with the same numbers, evaluate the head deformity rate, and combine the linearity index of the movement trajectory and the parameter change to generate a multi-modal feature matrix; S5: Based on the multi-modal feature matrix, calculate the motility-morphology consistency index, allocate scoring weights in combination with the trajectory continuity index, divide the evaluation levels, and generate a multi-modal sperm quality evaluation result.

2. The sperm quality assessment method based on multimodal fusion according to claim 1, wherein The spatio-temporal alignment dataset includes the image frame sequence number, the synchronous label mapping relationship, and the correspondence table of density and movement parameters. The heat map of movement characteristics includes the linearity distribution map of the trajectory, the regional density map, and the movement direction vector map. The parameter anomaly time window includes the density fluctuation interval, the motility anomaly section, and the movement rate deviation interval. The multi-modal feature matrix includes the morphological anomaly label set, the trajectory stability index group, and the parameter synchronization anomaly mark set. The multi-modal sperm quality evaluation result includes the motility-morphology consistency index score, the trajectory continuity evaluation level, and the sperm quality classification result.

3. The sperm quality assessment method based on multi-modal fusion according to claim 1, wherein The specific steps for obtaining the spatio-temporal alignment dataset are as follows: S111: Obtain the real-time sperm movement trajectory image sequence and the corresponding time tags, collect the three parameters of sperm density, total motility percentage, and progressive motility ratio recorded by the semen analyzer, extract the corresponding time tags respectively, calculate the time difference between adjacent ones, and obtain the time difference set of images and parameters; S112: Based on the time difference set of images and parameters, perform a judgment operation on each time difference and the time synchronization threshold, screen all time tag pairs with time differences less than the time synchronization threshold, and obtain the time consistency data pair set; S113: According to the time consistency data pair set, associate the corresponding image sequence with the three parameters of sperm density, total motility percentage, and progressive motility ratio, match the data pairs with the same time tags, perform consistency verification and validity marking on each group of data, and establish a spatio-temporal alignment dataset.

4. The sperm quality assessment method based on multimodal fusion according to claim 1, wherein The specific steps for obtaining the heat map of movement characteristics are as follows: S211: Based on the spatio-temporal alignment dataset, extract the sperm trajectory paths in the image frame by frame, identify the start and end positions of the trajectory, record the sperm head position coordinates in all consecutive frames on the path, record the Euclidean distance between the start and end points of each path and the total trajectory path length, and obtain the path structure parameter set; S212: Calculate the linearity index of the sperm path trajectory according to the path structure parameter set and the time interval of the trajectory frame segments, and map it to the corresponding image frame to obtain the linearity parameter set of the sperm movement; S213: Based on the sperm motion linearity parameter set and the corresponding image frame coordinate system, the grid units are divided according to the image area, and the distribution density of trajectory points in each cell that meet the trajectory linearity index greater than the judgment threshold is counted, and the color intensity in each grid is proportionally assigned to the image thermal color scale to generate a motion feature heat map.

5. The sperm quality assessment method based on multimodal fusion according to claim 1, characterized in that, The steps for obtaining the parameter abnormal time window are specifically as follows: S311: obtaining sperm density, motility value and forward motion ratio data in the spatiotemporal aligned data set, setting a sliding time window, traversing all continuous time points in the data set in chronological order, recording density changes, motility changes and forward motion ratio changes in each window, and obtaining a parameter change difference sequence; S312: Calculate the gradient of the rate of change per unit time according to the parameter change difference sequence, perform normalization processing, make item-by-item judgments according to the corresponding parameter fluctuation threshold reference value, complete the screening and marking processing of each gradient over-limit state in the continuous window, and obtain a change gradient judgment value sequence; S313: Based on the change gradient judgment value sequence, the time periods that continuously exceed the parameter fluctuation threshold are screened according to the continuity of the time axis, the start time and the end time of the time interval that continuously meets the abnormal condition are extracted, and the parameter abnormal time window is established.

6. The sperm quality assessment method based on multi-modal fusion according to claim 1, wherein The steps of obtaining the multimodal feature matrix are specifically as follows: S411: According to the motion feature heat map and the time information in the parameter abnormality time window, the image frame number in the time overlapping interval is retrieved, all sperm target areas in the corresponding image are extracted according to the frame number, the target sperm head area is marked, and a sperm image group of the time overlapping frame is obtained; S412: According to the sperm image group of overlapping time frames, the edge of the sperm head structure in the image is identified, each sperm is marked for morphological determination, judged according to the standard of normal sperm head, the sperm head deformity rate is calculated, and the head deformity rate sequence is obtained; S413: According to the head deformity rate sequence, the motion trajectory linearity parameter set under the corresponding image frame number is combined with the data at the same number position in the parameter change difference sequence, the data structure is unified, data fusion and format standardization are performed, and a multimodal feature matrix is established.

7. The sperm quality assessment method based on multimodal fusion according to claim 1, wherein The steps for obtaining the multimodal sperm quality assessment result are specifically as follows: S511: Based on the multimodal feature matrix, the deformity rate and vitality value corresponding to each frame image are extracted row by row, whether the synchronization is in a deviated state in the same frame is determined, the consistency frequency of the relative change direction is calculated, and the uniform output is in the form of percentage to obtain the vitality morphology consistency index; S512: reading the trajectory linearity index in the motion trajectory linearity parameter set under the corresponding frame image according to the vitality morphology consistency index, extracting the density, vitality and forward motion parameter change trends of the aforementioned frames, and calculating to obtain a comprehensive score value sequence; S513: Based on the comprehensive score value sequence, judge the intervals to which all score values belong according to the semen quality grade standard, mark the grade labels, and arrange them in time series to obtain the multi-modal sperm quality assessment grade result.

8. A sperm quality assessment system based on multimodal fusion, characterized in that, The system is used to implement the multi-modal fusion-based sperm quality assessment method according to any one of claims 1-7, and includes: The time series alignment module obtains the sperm real-time movement trajectory image sequence, collects the records of sperm density, total motility percentage, and forward movement ratio parameters, retains the data pairs with consistent time, and generates a spatio-temporal alignment data set; The trajectory extraction module extracts all sperm trajectory paths in the image frame by frame based on the spatio-temporal alignment data set, calculates the linearity index of the movement trajectory and locates the active area, annotates the pixel heat of the area, and generates a movement feature heat map; The parameter fluctuation module obtains the density, motility, and forward movement data in the spatio-temporal alignment data set, calculates the parameter difference and the change rate gradient, judges whether it exceeds the WHO threshold and marks the interval, and generates a parameter abnormal time window; The morphology analysis module extracts the corresponding sperm morphology images according to the image frame numbers with consistent time in the movement feature heat map and the parameter abnormal time window, evaluates the head malformation rate, and combines the movement trajectory linearity index and parameter changes to generate a multi-modal feature matrix; The grade scoring module calculates the consistency index based on the multi-modal feature matrix, uses the scoring weight assignment to divide the evaluation grades, and generates the multi-modal sperm quality assessment result.

Images