Gynecological tumor radiotherapy effect prediction method based on multi-modal data

Abstract

The application discloses a gynecological tumor radiotherapy effect prediction method based on multi-modal data and belongs to the technical field of gynecological tumor radiotherapy effect multi-modal prediction. The method acquires medical images, sign parameters and laboratory examination time series before and during radiotherapy and time aligns; performs intra-modal time-varying feature extraction on each sequence to obtain image time-varying vectors, sign time-varying vectors and examination time-varying vectors; according to the three-channel vector superposition at the acquisition time, a multi-modal time-varying state matrix is formed; the singular value decay spectrum of each time matrix is calculated, and a radiotherapy response time sequence description factor is constructed by using the inflection point ordinal number; the description factor is taken as a search item to perform time sequence similarity matching in a pre-constructed reference atlas library, and the effect classification corresponding to the highest matching degree reference atlas is taken as a prediction result output. The application realizes dynamic multi-modal prediction of gynecological tumor radiotherapy effect.

Description

Technical Field

[0001] This invention relates to the field of multimodal prediction technology for the radiotherapy effect of gynecological tumors, specifically a method for predicting the radiotherapy effect of gynecological tumors based on multimodal data. Background Technology

[0002] Predicting the efficacy of radiotherapy for gynecological tumors typically relies on single-modal data from medical imaging, vital signs, or laboratory tests, or on feature extraction and concatenation of multiple modalities at a single time point before inputting them into a classification model for prediction. Medical imaging often employs radiomics methods to extract static features such as texture and shape, while vital signs and laboratory tests are mostly described using statistical indicators. This approach underutilizes the temporal information acquired multiple times during treatment, lacks uniform alignment in acquisition time across different modalities, and extracts primarily static vectors that fail to reflect the dynamic changes in radiotherapy response throughout the treatment process. Instantaneous fluctuations or absences in a single modality can directly alter the prediction input, causing the prediction results to drift between adjacent assessment time points. The temporal correlation structure of multimodal data is not effectively represented, and there is a lack of a unified state matrix representation for changes in imaging sub-regions, rates of change in vital signs, and differences in examination items.

[0003] Existing methods typically concatenate features from different modalities into a single vector, ignoring the temporal relationships between acquisition times and synchronous changes between modalities. Radiotherapy efficacy is influenced by the combined effects of tumor regression rate, normal tissue response, and trends in laboratory indicators; static concatenation struggles to capture these dynamic processes. Due to the lack of analysis of the spectral characteristics of the multimodal state matrix, it is impossible to effectively extract temporal descriptors reflecting the complexity of the radiotherapy response. Conventional models rely on fixed classification boundaries obtained through training for prediction output, failing to fully utilize the temporal patterns of historically similar cases. The method of similarity matching and grading based on historical case temporal patterns has not yet been applied in predicting the efficacy of radiotherapy for gynecological tumors. Summary of the Invention

[0004] To address the issues of insufficient fusion of multimodal temporal information and unrobust dynamic feature expression in existing methods for predicting the efficacy of gynecological tumor radiotherapy, this invention provides a method for predicting the efficacy of gynecological tumor radiotherapy based on multimodal data. This method extracts intramodal time-varying features from Greenwich Mean Time (GMT) aligned medical image time series, vital sign parameter time series, and laboratory examination time series. A multimodal time-varying state matrix is ​​formed by superimposing the three-channel vectors at the acquisition time. A radiotherapy response temporal descriptor factor is constructed using the inflection point ordinal number of the singular value attenuation spectrum. Temporal similarity matching is performed in a radiotherapy efficacy reference map library, and the efficacy grade corresponding to the reference map with the highest matching degree is output as the current predicted efficacy.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a method for predicting the radiotherapy effect of gynecological tumors based on multimodal data, comprising: acquiring periodic Greenwich Mean Time (GMT) aligned medical image time series, vital sign parameter time series, and laboratory examination time series of the target patient before and during radiotherapy planning; performing intramodal time-varying feature extraction on the medical image time series, vital sign parameter time series, and laboratory examination time series respectively to obtain image time-varying vector, vital sign time-varying vector, and examination time-varying vector; superimposing the image time-varying vector, vital sign time-varying vector, and examination time-varying vector into a three-channel vector according to the acquisition time to form a multimodal time-varying state matrix; calculating the matrix singular value attenuation spectrum at each acquisition time according to the multimodal time-varying state matrix, and constructing a radiotherapy response time-varying descriptor factor using the inflection point ordinal number of the attenuation spectrum; using the radiotherapy response time-varying descriptor factor as a search term, performing time-series similarity matching in a pre-constructed radiotherapy effect reference atlas library, and outputting the effect grade corresponding to the reference atlas with the highest matching degree as the current predicted effect. The above method aligns multi-source time series data with Greenwich Mean Time (GMT), eliminates differences in time references between different acquisition devices, and uses a multimodal time-varying state matrix to uniformly express radiotherapy response information in images, physical signs, and laboratory examinations, thereby improving the temporal consistency and accuracy of predicting the efficacy of radiotherapy for gynecological tumors.

[0006] Preferably, when performing intramodal time-varying feature extraction on the medical image time series, vital sign parameter time series, and laboratory examination time series, each frame of the medical image time series is divided into multiple sub-regions according to anatomical structure. The gray-level co-occurrence matrix contrast statistics of each sub-region are calculated. The contrast statistics of the same sub-region are sorted according to the acquisition time to form an image time-varying sub-vector. The image time-varying sub-vectors of all sub-regions are merged to obtain the image time-varying vector. The vital sign parameter time series is subjected to differential operation according to parameter type to obtain the vital sign change rate sequence of each parameter type. The vital sign change rate sequences of all parameter types are merged to obtain the vital sign time-varying vector. The laboratory examination time series is subjected to normalized difference calculation according to examination items to obtain the examination difference sequence of each examination item. The examination difference sequences of all examination items are merged to obtain the examination time-varying vector. This method can extract features reflecting the change of radiotherapy response over time from different modalities and suppress the interference of single-modal noise on the prediction results.

[0007] Preferably, when dividing the image into multiple sub-regions according to anatomical structure, the cervix, uterine body, vagina, and pelvic lymph node regions are each divided into independent sub-regions based on the standard gynecological tumor target delineation guidelines; when performing differential operations on the time-series of the vital signs parameters according to parameter type, first-order forward differencing is used to calculate the changes in vital signs parameters at adjacent acquisition times. This approach makes the sub-regional division of images and the characterization of vital signs more consistent with the anatomical regions and dynamic changes that are of clinical concern in gynecological tumor radiotherapy.

[0008] Furthermore, when the image time-varying vector, vital sign time-varying vector, and examination time-varying vector are superimposed using three channels according to the acquisition time, the acquisition time is used as the row index, and the feature dimension of each sub-region in the image time-varying vector is used as the first channel column, the parameter type dimension of each vital sign time-varying vector is used as the second channel column, and the examination item dimension of each examination time-varying vector is used as the third channel column to construct image channel sub-matrices, vital sign channel sub-matrices, and examination channel sub-matrices, respectively. The row vectors of the image channel sub-matrices, vital sign channel sub-matrices, and examination channel sub-matrices at the same acquisition time are concatenated along the column direction to form the multimodal state row vector at that acquisition time. The multimodal state row vectors of all acquisition times are stacked in chronological order to obtain the multimodal time-varying state matrix. Through this three-channel superposition method, heterogeneous multimodal data are unified into a state representation that can be used for matrix analysis under the same time index, which facilitates subsequent singular value analysis and temporal matching.

[0009] Preferably, when calculating the singular value attenuation spectrum of the matrix at each acquisition time based on the multimodal time-varying state matrix, and constructing a radiotherapy response time-series descriptor using the attenuation spectrum inflection point ordinal numbers, singular value decomposition is performed on the row vectors corresponding to each acquisition time of the multimodal time-varying state matrix to obtain the singular value sequence at that time. The singular value sequence is then arranged in descending order, and the ratio of adjacent singular values ​​is calculated to obtain a ratio sequence. The ordinal number of the first occurrence of a ratio less than a preset ratio threshold in the ratio sequence is taken as the attenuation spectrum inflection point ordinal number for that acquisition time. The attenuation spectrum inflection point ordinal numbers for each acquisition time are arranged in chronological order, and a sliding window variance calculation is performed on the arranged inflection point ordinal number sequence. The variance calculation result is used as the radiotherapy response time-series descriptor. Constructing a descriptor using singular value attenuation spectrum inflection point ordinal numbers can characterize the changing trends of the main information components in the multimodal state matrix as radiotherapy progresses, and reduce the impact of instantaneous fluctuations on the prediction results.

[0010] Furthermore, the inflection point numbers of the attenuation spectrum at each acquisition time are arranged in chronological order. When performing sliding window variance calculation on the arranged inflection point number sequence, the sliding window length and sliding step size are set. Starting from the beginning of the inflection point number sequence, inflection point number subsequences of length equal to the sliding window length are successively extracted according to the sliding step size. The variance value of each inflection point number subsequence is calculated to obtain a variance value sequence. Each variance value in the variance value sequence is aligned according to the corresponding window termination time to form the radiotherapy response time-series descriptor. This method can stably reflect the fluctuation degree of the inflection point numbers at different radiotherapy stages, making the descriptor temporally continuous and comparable.

[0011] Preferably, when performing temporal similarity matching in a pre-constructed radiotherapy effect reference atlas library using the radiotherapy response temporal descriptor factor as the search term, the absolute value of the difference between each descriptor value in the radiotherapy response temporal descriptor factor and the reference descriptor value of each reference atlas in the radiotherapy effect reference atlas library is calculated to obtain the difference vector of each reference atlas; the difference vector is then normalized to obtain a normalized similarity score; the reference atlas with the highest normalized similarity score is selected as the matching reference atlas, and the pre-labeled radiotherapy effect grade of this matching reference atlas is extracted as the current predicted effect output. This method uses the normalized similarity score to quantitatively match reference atlases, which can reduce the influence of different dimensions and numerical ranges on similarity calculation and improve the accuracy of effect grade matching.

[0012] Furthermore, when selecting the reference atlas with the highest normalized similarity score as the matching reference atlas, if there are multiple reference atlases with the highest normalized similarity scores, the radiotherapy efficacy classification of each highest similarity reference atlas is extracted. The frequency of each radiotherapy efficacy classification in the multiple highest similarity reference atlases is counted, and the radiotherapy efficacy classification with the highest frequency is used as the current predicted efficacy output. A multi-reference atlas voting method is used to handle cases with tied similarities, improving the stability of the prediction results.

[0013] Preferably, when acquiring the time-series medical images, vital sign parameters, and laboratory examinations of a target patient before and during radiotherapy planning and treatment, aligned to Greenwich Mean Time (GMT), the medical imaging time-series is obtained by extracting frames from the image data acquired by the medical imaging equipment at preset acquisition intervals, using GMT as the time reference; vital sign parameter records are extracted from the patient monitoring equipment using the same GMT and acquisition intervals, obtaining the vital sign parameter time-series; and laboratory examination result records are extracted from the hospital information system using the same GMT and acquisition intervals, obtaining the laboratory examination time-series. This method enables data from different sources to be aligned in the same time coordinate system, reducing intermodal state misalignment caused by acquisition time deviations.

[0014] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0015] Intramodal time-varying feature extraction was performed on the time-series sequences of medical images, vital signs, and laboratory examinations to obtain time-varying vectors for images, vital signs, and examinations, respectively. These vectors were then superimposed according to the acquisition time to form a multimodal time-varying state matrix. Singular value decomposition was performed on the row vectors of this matrix at each acquisition time. The ordinal number of the first occurrence of a ratio less than a preset ratio in the singular value decay spectrum was used as the inflection point ordinal number of the decay spectrum. The inflection point ordinal numbers at each time point were then arranged chronologically, and the sliding window variance was calculated to obtain the radiotherapy response time-series descriptor. This processing method places the change information of different modalities in the state vector at the same time point, preserving the temporal synchronization of multimodal data and reflecting the state evolution during radiotherapy better than static feature stitching. The inflection point ordinal number of the singular value decay spectrum describes the degree of structural concentration within the state row vector and is insensitive to amplitude fluctuations in a single modality and dimensional differences between modalities. The sliding window variance further extracts dynamic fluctuation features from the inflection point ordinal number sequence, reducing the interference of occasional fluctuations on the prediction input. The inflection point sequence of the attenuation spectrum forms a dynamic sequence at different acquisition times. The sliding window variance incorporates the fluctuation of the local inflection point sequence into the descriptive factor, which can characterize the continuous change of the radiotherapy response state over time.

[0016] Using the radiotherapy response time-series descriptor as the search term, the absolute value of the difference between each reference map and the current descriptor is calculated in a pre-built radiotherapy effect reference map library. The difference vector is then normalized to obtain a normalized similarity score. The radiotherapy effect grade corresponding to the reference map with the highest score is selected as the prediction output. This method utilizes the temporal patterns of historically labeled cases for similarity matching, replacing the direct output of grades from fixed classification boundaries. This constrains the prediction results with similar temporal patterns, reducing the impact of generalization bias of a single model on different patient groups. The normalized similarity score uses the difference vector magnitude as a similarity measure, ensuring that comparisons between different reference maps are not affected by the absolute magnitude of the descriptor. When multiple reference maps with the highest similarity exist, the frequency of occurrence of each effect grade is counted, and the grade with the highest frequency is output, reducing the impact of single reference map annotation noise on the final prediction result. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a flowchart of a method for predicting the efficacy of radiotherapy for gynecological tumors based on multimodal data;

[0019] Figure 2This is a flowchart for generating radiotherapy response time-series descriptor factors based on singular value decay spectrum inflection points;

[0020] Figure 3 This is a flowchart for predicting radiotherapy effects based on reference atlas matching;

[0021] Figure 4 It is a multimodal time-varying state matrix heatmap;

[0022] Figure 5 It is a time-series graph showing the inflection point sequence of the attenuation spectrum and the variance of the sliding window. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] See Figure 1 This invention provides a method for predicting the efficacy of radiotherapy for gynecological tumors based on multimodal data, including:

[0025] This study acquires periodic Greenwich Mean Time (GMT) aligned medical image time series, vital sign parameter time series, and laboratory test time series for target patients before and during radiotherapy planning. The medical image time series is formed from image frames acquired by medical imaging equipment at preset acquisition intervals. The vital sign parameter time series is formed from vital sign parameter records acquired by patient monitoring equipment at the same time points. The laboratory test time series is formed from laboratory test results extracted by the hospital information system at the same time points. The timestamps for all three types of sequences are expressed in Greenwich Mean Time.

[0026] Modal time-varying feature extraction was performed on the aforementioned medical image time series, vital sign parameter time series, and laboratory examination time series to obtain image time-varying vectors, vital sign time-varying vectors, and examination time-varying vectors. The image time-varying vector is formed by arranging the gray-level co-occurrence matrix contrast statistics of each anatomical structure sub-region in the medical image time series according to time. The vital sign time-varying vector is formed by merging the difference sequences of each vital sign parameter type. The examination time-varying vector is formed by merging the normalized difference sequences of each laboratory examination item.

[0027] The image time-varying vector, vital sign time-varying vector, and examination time-varying vector are superimposed using three channels according to the acquisition time to form a multimodal time-varying state matrix. Using the acquisition time as the row index, the image feature dimension as the first channel column, the vital sign parameter type dimension as the second channel column, and the examination item dimension as the third channel column, the row vectors of the three channels at the same acquisition time are concatenated along the column direction to form a multimodal state row vector. All multimodal state row vectors from all acquisition times are stacked in chronological order.

[0028] Based on the multimodal time-varying state matrix, the singular value attenuation spectrum of the matrix at each acquisition time is calculated, and the attenuation spectrum inflection point ordinal number is used to construct a radiotherapy response time-series descriptor. Singular value decomposition is performed on the row vectors at each acquisition time, the singular values ​​are arranged in descending order, the ratio of adjacent singular values ​​is calculated, and the ordinal number of the first occurrence of a ratio less than a preset ratio threshold is taken as the attenuation spectrum inflection point ordinal number for that time. The attenuation spectrum inflection point ordinal numbers at each time are arranged in chronological order, and a sliding window variance calculation is performed. The variance calculation result is used as the radiotherapy response time-series descriptor.

[0029] Using the radiotherapy response time-series descriptor as the search term, time-series similarity matching is performed in a pre-constructed radiotherapy effect reference atlas library, and the effect grade corresponding to the reference atlas with the highest matching degree is used as the current predicted effect output.

[0030] In practice, when performing intramodal time-varying feature extraction on medical image time-series sequences, each frame of the medical image time-series is divided into multiple sub-regions according to anatomical structure. The anatomical structure partitioning follows the standard gynecological tumor target delineation guidelines, dividing the cervix, uterine body, vagina, and pelvic lymph node regions into independent sub-regions. Each image frame contains independent sub-regions for the cervix, uterine body, vagina, and pelvic lymph nodes.

[0031] In practice, a gray-level co-occurrence matrix (GLCM) is established for each independent sub-region in each frame of image. When establishing the GLCM, the frequency of occurrence of gray-level pixel pairs is statistically analyzed within the image range of each independent sub-region according to preset gray-level quantization levels, pixel distances, and pixel directions to obtain the GLCM. Contrast statistics are calculated based on the GLCM, reflecting the degree of gray-level variation within the independent sub-region image. The contrast statistics of the cervical independent sub-region at each acquisition time are arranged in chronological order to form a time-varying sub-vector for the cervical image. The contrast statistics of the uterine body independent sub-region at each acquisition time are arranged in chronological order to form a time-varying sub-vector for the uterine body image. The contrast statistics of the vaginal independent sub-region at each acquisition time are arranged in chronological order to form a time-varying sub-vector for the vaginal image. The contrast statistics of the pelvic lymph node independent sub-region at each acquisition time are arranged in chronological order to form a time-varying sub-vector for the pelvic lymph node image. When merging the time-varying subvectors of all subregions, the time-varying subvectors of the cervix, uterus, vagina, and pelvic lymph nodes are spliced ​​in sequence to obtain the time-varying subvectors of the images.

[0032] In practice, the time series of vital signs parameters is differentially analyzed according to parameter type. The time series of vital signs parameters contains recorded values ​​of various parameter types. For the first... For individual vital signs, the changes in vital signs between adjacent acquisition times are calculated using first-order forward differencing. The first-order forward differencing formula is:

[0033]

[0034] in, Indicates the ordinal number of the vital sign parameter type. Take a positive integer. The range of values ​​is to , This represents the total number of vital sign parameter types. Indicates the sequence number of the acquisition time. Take a positive integer. The range of values ​​is to , This represents the total number of data collection moments. Indicates the first Individual trait parameter types in the first Recorded values ​​of vital signs parameters at each acquisition time; Indicates the first Individual trait parameter types in the first Recorded values ​​of vital signs parameters at each acquisition time; Indicates the first Individual trait parameter types in the first The rate of change of vital signs at each acquisition time.

[0035] In specific implementation, the first Individual trait parameter types obtained at each adjacent acquisition time Arranged in chronological order of collection time, forming the [number]th... Individual symptom parameter type symptom change rate sequence. When merging symptom change rate sequences of all parameter types, they are concatenated sequentially according to the order of the symptom parameter type in the symptom parameter time series to obtain the symptom time-varying vector.

[0036] In practice, the laboratory examination time series is normalized by calculating the difference for each examination item. The laboratory examination time series contains the results of multiple examination items. For each examination item, the difference between the results at adjacent acquisition times is calculated, and this difference is divided by the baseline result value to obtain the normalized difference. The baseline result value is the result value of each examination item at the first acquisition time before the radiotherapy plan is executed. The normalized differences for the same examination item are arranged in order of acquisition time to form an examination difference sequence. When merging the examination difference sequences of all examination items, they are concatenated sequentially according to the order of their arrangement in the laboratory examination time series to obtain the examination time-varying vector.

[0037] In practice, the time-varying vectors of images, vital signs, and examinations are superimposed in three channels according to the acquisition time to form a multimodal time-varying state matrix.

[0038] In practice, the data acquisition time is used as the row index. The feature dimensions of each sub-region in the time-varying image vector include the time-varying sub-vector dimensions of the cervix, uterus, vagina, and pelvic lymph nodes. These feature dimensions of each sub-region in the time-varying image vector serve as the first channel column. The parameter type dimensions in the physical examination time-varying vector serve as the second channel column. The examination item dimensions in the examination time-varying vector serve as the third channel column.

[0039] In practice, image channel sub-matrices, vital sign channel sub-matrices, and examination channel sub-matrices are constructed separately. The image channel sub-matrices use the acquisition time as the row index and the feature dimension of each sub-region in the image time-varying vector as the column index. The number of rows in the image channel sub-matrices equals the number of acquisition times, and the number of columns equals the total number of feature dimensions of each sub-region in the image time-varying vector. Each row vector in the image channel sub-matrices corresponds to an acquisition time, and each element in a row vector corresponds to the value of a feature dimension of an image sub-region at the corresponding acquisition time.

[0040] In practice, the vital sign channel submatrix uses the acquisition time as the row index and the dimension of each parameter type in the vital sign time-varying vector as the column index. The number of rows in the vital sign channel submatrix equals the number of acquisition times, and the number of columns equals the total number of dimensions of each parameter type in the vital sign time-varying vector. Each row vector in the vital sign channel submatrix corresponds to an acquisition time, and each element in the row vector corresponds to the rate of change value of a vital sign parameter type at the corresponding acquisition time.

[0041] In practice, the channel submatrix is ​​inspected using the acquisition time as the row index and the dimensions of each inspection item in the time-varying vector as the column index. The number of rows in the channel submatrix equals the number of acquisition times, and the number of columns equals the total number of dimensions of each inspection item in the time-varying vector. Each row vector in the channel submatrix corresponds to an acquisition time, and each element in the row vector corresponds to the laboratory inspection normalized difference of an inspection item at the corresponding acquisition time.

[0042] In practice, the row vectors of the image channel submatrix, vital sign channel submatrix, and examination channel submatrix at the same acquisition time are concatenated along the column direction. During concatenation, the image channel submatrix row vector is first placed on the left, then the vital sign channel submatrix row vector is placed to the right of the image channel submatrix row vector, and finally the examination channel submatrix row vector is placed to the right of the vital sign channel submatrix row vector, forming a multimodal state row vector at the same acquisition time. The dimension of the multimodal state row vector is equal to the sum of the number of columns in the image channel submatrix, the number of columns in the vital sign channel submatrix, and the number of columns in the examination channel submatrix. The multimodal state row vector contains the image sub-region texture contrast statistics, vital sign parameter change rate, and laboratory examination normalized difference at the same acquisition time.

[0043] In practice, the multimodal state row vectors from all acquisition moments are stacked in chronological order. When stacking in chronological order, the order of acquisition moments determines the row order of the multimodal state row vectors in the matrix. The multimodal state row vector from the earliest acquisition moment is located in the first row of the multimodal time-varying state matrix, and the multimodal state row vectors from subsequent acquisition moments are located in the subsequent rows of the multimodal time-varying state matrix. The number of rows in the multimodal time-varying state matrix equals the number of acquisition moments, and the number of columns in the multimodal time-varying state matrix equals the sum of the number of columns in the first channel, the second channel, and the third channel.

[0044] See Figure 4The figure shows a heatmap of a multimodal time-varying state matrix constructed based on multimodal data. The horizontal axis represents the ordinal dimension of the multimodal state row vectors, and the vertical axis represents the ordinal number of the acquisition time, which increases from 0, reflecting the changing trend of periodic acquisition data before and during radiotherapy planning. The horizontal axis is divided into three regions, corresponding to the image channel submatrix (ordinal numbers 1 to 4, red), the vital signs channel submatrix (ordinal numbers 5 to 7, green), and the examination channel submatrix (ordinal numbers 8 to 10, blue). The number of columns in each channel submatrix is ​​consistent with the number of columns in the multimodal state row vectors it describes.

[0045] The vertical axis, arranged sequentially by acquisition time sequence, reflects the evolution of the multimodal state row vector over time. The image channel sub-matrix region exhibits a gradual change in red intensity from top to bottom. The red is deeper in the early stages (lower acquisition time sequences), indicating higher statistical contrast in the image sub-region. It gradually lightens in later stages, showing changes in image features as radiotherapy progresses, potentially reflecting dynamic changes in tumor tissue structure. The vital signs channel sub-matrix region is a moderate green, with a deeper green in the middle acquisition time sequence (approximately acquisition sequences 5 to 15), indicating significant fluctuations in the rate of change of vital signs parameters during this stage, reflecting the dynamic response of the patient's physiological state. The examination channel sub-matrix region is generally light blue, with localized areas showing a moderate blue in the early acquisition times (sequences 0 to 5), gradually lightening in color. This indicates that the normalized difference in laboratory examinations is more significant in the early stages of treatment and tends to stabilize later.

[0046] In specific implementation, please refer to Figure 2 Singular value decomposition (SVD) is performed on the row vectors corresponding to each acquisition time of the multimodal time-varying state matrix. Each row vector of the multimodal time-varying state matrix represents the multimodal state row vector at an acquisition time. After performing SVD on the multimodal state row vectors, a sequence of singular values ​​for the corresponding acquisition time is obtained. The singular value sequence is then arranged in descending order to form the matrix singular value decay spectrum for the corresponding acquisition time. The matrix singular value decay spectrum reflects the decreasing trend of singular values ​​in the multimodal state row vectors from large to small at the corresponding acquisition time.

[0047] In practice, the ratios of adjacent singular values ​​are calculated for the descendingly ordered sequence of singular values ​​to obtain a ratio sequence. The ratio of adjacent singular values ​​is calculated by dividing the singular value at the beginning of the descending sequence by the singular value at the end. All adjacent singular value ratios are then arranged in order of their corresponding singular value positions to form a ratio sequence.

[0048] In practice, the ordinal number of the first occurrence of a ratio less than a preset ratio threshold in the ratio sequence is used as the inflection point ordinal number of the attenuation spectrum at the corresponding acquisition time. The preset ratio threshold is set to 1.05, and its value range is a real number greater than 1. The basis for setting the preset ratio threshold to 1.05 is that when the ratio of adjacent singular values ​​is less than 1.05, the difference between adjacent singular values ​​is small, and the attenuation trend of singular values ​​changes from a rapid decline to a gradual decline. At this time, the corresponding ordinal number can identify the inflection point position of the attenuation spectrum. The inflection point ordinal number of the attenuation spectrum is a positive integer, representing the position ordinal number of the first occurrence of an adjacent singular value ratio less than the preset ratio threshold in the descendingly sorted singular value sequence.

[0049] In practice, the inflection point numbers of the attenuation spectrum at each acquisition time are arranged in chronological order to form an inflection point number sequence. Each inflection point number in the inflection point number sequence corresponds to an acquisition time, and the order of the inflection point number sequence is consistent with the chronological order of the acquisition times.

[0050] In the specific implementation, the sliding window length and sliding step size are set. The sliding window length is set to 5, and the value of the sliding window length is a positive integer. The reason for setting the sliding window length to 5 is that using 5 collection times as a local statistical window can preserve the local fluctuation characteristics of the inflection point ordinal numbers. The sliding step size is set to 1, and the value of the sliding step size is a positive integer. The reason for setting the sliding step size to 1 is that using 1 collection time as the moving interval can ensure continuity between adjacent sliding windows. Starting from the beginning position of the inflection point ordinal number sequence, inflection point ordinal number subsequences with a length equal to the sliding window length are successively extracted according to the sliding step size.

[0051] In specific implementation, for the first The variance of the inflection point ordinal subsequence is calculated using the following formula:

[0052]

[0053] in, Indicates the ordinal number of the sliding window. Take a positive integer. The range of values ​​is to , This represents the total number of sliding windows; Indicates the length of the sliding window. Set to 5; This represents the ordinal number of the inflection point data within the sliding window. Take a positive integer. The range of values ​​is to ; Indicates the first Within the first sliding window Number of inflection points in the attenuation spectrum; Indicates the first The average of the inflection point numbers of all attenuation spectra within a sliding window; Indicates the first The variance of each sliding window.

[0054] In practice, after calculating the variance of each inflection point ordinal subsequence, a variance value sequence is obtained. Each variance value in the variance value sequence corresponds to a sliding window. Each variance value in the variance value sequence is aligned according to the corresponding window termination time. The window termination time is the latest acquisition time within the sliding window. After mapping each variance value to a window termination time, they are arranged in chronological order according to the window termination times to form the radiotherapy response time-series descriptor. Each descriptor value in the radiotherapy response time-series descriptor is the variance of an inflection point ordinal subsequence within a sliding window.

[0055] See Figure 5 In the figure, the horizontal axis represents the acquisition time sequence, indicating the sequence number of the multimodal time-varying state data acquisition time points obtained sequentially according to the preset acquisition cycle interval during radiotherapy. The left vertical axis represents the attenuation spectrum inflection point sequence, reflecting the position number of the first occurrence of the ratio of adjacent singular values ​​less than the preset threshold of 1.05 in the singular value attenuation spectrum of the multimodal time-varying state matrix at the corresponding acquisition time. The right vertical axis represents the sliding window variance value, which represents the variance calculated from a subsequence locally truncated from the attenuation spectrum inflection point sequence with a sliding window length of 5 and a sliding step size of 1, and serves as a radiotherapy response time-series descriptor.

[0056] The blue dashed inflection line represents the trend of the attenuation spectrum inflection point ordinal number with the acquisition time. In the initial stage (approximately acquisition time 0 to 50), the attenuation spectrum inflection point ordinal number shows a significant downward trend, gradually decreasing from a maximum value of 10 to approximately 4, reflecting the transition from a sharp drop to a relatively gradual decrease in the rapid decay characteristics of the singular values ​​of the multimodal state matrix. In the intermediate stage (approximately acquisition time 50 to 150), the attenuation spectrum inflection point ordinal number tends to stabilize, mainly fluctuating between 2 and 4, indicating that the multimodal state characteristics are relatively stable and the changes are slowing down. In the late stage (approximately acquisition time 150 to 300), the attenuation spectrum inflection point ordinal number further tends to the low value range of 0 to 3, and there are many short-term fluctuations, indicating that the radiotherapy response state has entered a relatively stable or slowly changing stage.

[0057] The red dashed inflection line represents the change in the variance of the corresponding sliding window with the acquisition time. Overall, the variance of the sliding window remains at a low level in the early stage (approximately 0.2 to 0.4), reflecting small local fluctuations in the inflection point ordinal number of the attenuation spectrum. In the middle and late stages, the variance of the sliding window shows several significant increases, reaching a maximum of approximately 1.2, indicating that the fluctuations in the inflection point ordinal number of the attenuation spectrum intensify within the corresponding acquisition time window, reflecting a phased dynamic change in the time-series characteristics of the radiotherapy response. The periodic peak-and-trough trend of the sliding window variance corresponds to the fluctuations in the inflection point ordinal number of the attenuation spectrum, revealing the temporal dynamic characteristics of the radiotherapy effect response.

[0058] In specific implementation, please refer to Figure 3 A pre-constructed reference atlas library for radiotherapy efficacy has been completed. The library consists of multiple reference atlases, each containing a reference descriptive factor and a radiotherapy efficacy grade. The reference descriptive factor is generated in the same manner as the radiotherapy response time-series descriptive factor, and the radiotherapy efficacy grade is labeled by physicians based on follow-up results of historical cases. Each reference atlas in the library has a unique reference atlas ordinal number.

[0059] In practice, the absolute value of the difference between each descriptor value in the radiotherapy response time-series descriptor factors and the reference descriptor value of each reference atlas in the radiotherapy effect reference atlas library is calculated to obtain the difference vector of each reference atlas. Each descriptor value in the radiotherapy response time-series descriptor factors corresponds to a descriptor value ordinal number, which starts from 1 and increases sequentially. The reference descriptor values ​​and the descriptor values ​​have the same ordinal correspondence. For the ... The first reference atlas will be the first of the radiotherapy response time descriptor factors. The value of the first descriptor and the value of the second descriptor The first reference map The absolute value of the difference between the first and second reference descriptor values ​​is taken to obtain the third... The reference map in the first... The difference component at the ordinal value of the descriptor. The difference components of the reference spectrum at all descriptor values ​​ordinal numbers are arranged according to the descriptor value ordinal numbers to obtain the first reference spectrum. The difference vector of each reference map.

[0060] In practice, the difference vectors are normalized to obtain a normalized similarity score. The normalized similarity score is calculated using the following formula:

[0061]

[0062] in, Indicates the reference atlas ordinal number. Take a positive integer. The range of values ​​is to , This represents the total number of reference atlases in the radiotherapy efficacy reference atlas library; Indicates the ordinal number of the descriptor value. Take a positive integer. The range of values ​​is to , This represents the total number of descriptor values ​​in the radiotherapy response time-series descriptor. This represents the first of the time-series descriptors of radiotherapy response. One descriptor value; Indicates the first The first reference map One reference descriptive factor value; Indicates the first The value of the first descriptor and the value of the second descriptor The first reference map The absolute value of the difference between the values ​​of the reference descriptor factors; Indicates all The sum of the squares of the absolute values ​​of the corresponding differences; Indicates the first The magnitude of the difference vector of each reference map; Indicates the first Normalized similarity scores of reference maps.

[0063] The value in the denominator This is the normalized smoothing constant. The normalized smoothing constant is set to... The basis is: when the normalized smoothing constant is At that time, normalized similarity score The range of values ​​is arrive And when the magnitude of the difference vector is The denominator is not The normalized similarity score reaches its maximum value. As the magnitude of the difference vector increases, the normalized similarity score gradually decreases, and the change process is smooth.

[0064] In practice, a normalized similarity score is calculated for each reference atlas in the radiotherapy effect reference atlas library according to the above formula, resulting in a set of normalized similarity scores for all reference atlases. The maximum value in the set of normalized similarity scores is taken as the highest normalized similarity score. If the highest normalized similarity score corresponds to only one reference atlas, that reference atlas is taken as the matching reference atlas, and the pre-labeled radiotherapy effect grade of the matching reference atlas is extracted as the current predicted effect output.

[0065] In practice, if the highest normalized similarity score corresponds to multiple reference atlases, the radiotherapy efficacy classification for each reference atlas corresponding to the highest normalized similarity score is extracted. The frequency of each radiotherapy efficacy classification in the reference atlases corresponding to the multiple highest normalized similarity scores is then counted. The radiotherapy efficacy classification with the highest frequency is taken as the current predicted efficacy output.

[0066] In practice, Greenwich Mean Time (GMT) is used as the time reference to obtain the periodic medical imaging time series, vital sign parameter time series, and laboratory test time series of the target patient before and during the radiotherapy plan.

[0067] In practice, when acquiring the time-series medical images, frames are extracted from the image data acquired by the medical imaging equipment according to a preset acquisition period. The medical imaging equipment acquires image data of the target patient before and during radiotherapy planning. This data contains image frames corresponding to multiple acquisition time points. Image frames are extracted from the medical image data according to the preset acquisition period, and these extracted frames are arranged in chronological order of acquisition time to form the time-series medical images. Each image frame in the time-series medical images corresponds to an acquisition time, expressed in Greenwich Mean Time (GMT).

[0068] In practice, when acquiring the time-series sequence of vital signs parameters, records are extracted from the patient monitoring equipment using the same Greenwich Mean Time (GMT) and the same preset acquisition interval. The patient monitoring equipment collects vital sign parameter records from the target patient before and during radiotherapy planning. These records contain values ​​for vital signs parameters corresponding to multiple acquisition time points. Based on the acquisition time corresponding to the medical image time-series, vital sign parameter records corresponding to each acquisition time are extracted from the patient monitoring equipment. The extracted records are arranged in chronological order of acquisition time to form the time-series sequence of vital signs parameters. Each record in the time-series sequence corresponds to one acquisition time, expressed in Greenwich Mean Time.

[0069] In practice, when acquiring the laboratory examination time sequence, laboratory examination result records are extracted from the hospital information system using the same Greenwich Mean Time (GMT) and the same preset acquisition interval. The hospital information system stores laboratory examination result records for the target patient before and during radiotherapy planning. These records contain examination result values ​​corresponding to multiple acquisition time points. Based on the acquisition time corresponding to the medical image time sequence, laboratory examination result records corresponding to each acquisition time are extracted from the hospital information system. The extracted laboratory examination result records are arranged in chronological order of acquisition time to form the laboratory examination time sequence. Each laboratory examination result record in the time sequence corresponds to one acquisition time, expressed in Greenwich Mean Time.

[0070] In practice, the data acquisition time is calculated using the following formula:

[0071]

[0072] in, Indicates the first The Greenwich Mean Time corresponding to each acquisition time; This indicates the Greenwich Mean Time (GMT) corresponding to the baseline acquisition time prior to the radiotherapy plan. The start time will be determined according to the radiotherapy plan. The time unit is consistent with the Greenwich Mean Time (GMT) time unit. Indicates the preset data collection period interval. The time unit is consistent with the Greenwich Mean Time (GMT) time unit. Indicates the sequence number of the acquisition time. Take non-negative integers. The range of values ​​is to , The total number of data collection times. Take a positive integer. The number of times the sample is collected during the radiotherapy treatment is determined.

[0073] In practice, a preset data collection period interval is used. Set to 7 days. Preset data collection interval. The 7-day period was set as follows: In clinical practice, the evaluation of the radiotherapy effect of gynecologic tumors involves the periodic collection of medical images, vital signs, and laboratory tests once a week, which can cover the baseline status before the radiotherapy plan is implemented and the changes in radiotherapy response at different stages during the treatment process.

[0074] In practice, the timestamps for all three types of sequences are expressed in Greenwich Mean Time (GMT). The timestamps for each image frame in the medical imaging time series, each vital sign parameter record in the vital sign parameter time series, and each laboratory test result record in the laboratory test time series are all calculated according to the formula... The calculated acquisition time markers. Image frames, vital sign parameter records, and laboratory test result records at the same acquisition time are aligned with each other.

[0075] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for predicting the efficacy of radiotherapy for gynecological tumors based on multimodal data, characterized in that, include: Acquire time-series medical images, vital signs parameters, and laboratory tests of the target patient at periodic Greenwich Mean Time alignment before and during radiotherapy planning. Modal time-varying feature extraction is performed on the medical image time sequence, vital sign parameter time sequence, and laboratory examination time sequence to obtain image time-varying vector, vital sign time-varying vector, and examination time-varying vector, respectively. The image time-varying vector, vital sign time-varying vector, and examination time-varying vector are superimposed in three channels according to the acquisition time to form a multimodal time-varying state matrix; Based on the multimodal time-varying state matrix, the matrix singular value attenuation spectrum at each acquisition time is calculated, and the radiotherapy response time-series descriptor is constructed using the attenuation spectrum inflection point ordinal number. Using the radiotherapy response time-series descriptor as the search term, time-series similarity matching is performed in a pre-constructed radiotherapy effect reference atlas library, and the effect grade corresponding to the reference atlas with the highest matching degree is used as the current predicted effect output.

2. The method for predicting the efficacy of gynecological tumor radiotherapy based on multimodal data according to claim 1, characterized in that, Modal time-varying feature extraction is performed on the time-series medical images, vital sign parameters, and laboratory examinations to obtain time-varying vectors for images, vital signs, and examinations, including: Each frame of the medical image time sequence is divided into multiple sub-regions according to anatomical structure. The gray-level co-occurrence matrix contrast statistics of each sub-region are calculated. The contrast statistics of the same sub-region are sorted according to the acquisition time to form the image time-varying sub-vector. The image time-varying sub-vectors of all sub-regions are merged to obtain the image time-varying vector. The time series of vital signs parameters are subjected to differential operation according to parameter type to obtain the vital signs change rate sequence of each parameter type. The vital signs change rate sequences of all parameter types are merged to obtain the time-varying vector of vital signs. The laboratory examination time series is normalized and the difference is calculated according to the examination items to obtain the examination difference series of each examination item. The examination difference series of all examination items are merged to obtain the examination time-varying vector.

3. The method for predicting the efficacy of gynecological tumor radiotherapy based on multimodal data according to claim 1, characterized in that, The image time-varying vector, vital sign time-varying vector, and examination time-varying vector are superimposed using three channels according to the acquisition time to form a multimodal time-varying state matrix, including: Using the acquisition time as the row index, and the feature dimension of each sub-region in the image time-varying vector as the first channel column, the parameter type dimension in the vital signs time-varying vector as the second channel column, and the examination item dimension in the examination time-varying vector as the third channel column, image channel sub-matrix, vital signs channel sub-matrix and examination channel sub-matrix are constructed respectively. The row vectors of the image channel submatrix, the row vectors of the vital signs channel submatrix, and the row vectors of the examination channel submatrix at the same acquisition time are concatenated along the column direction to form the multimodal state row vector at that acquisition time. The multimodal state row vectors at all acquisition times are stacked in chronological order to obtain the multimodal time-varying state matrix.

4. The method for predicting the efficacy of gynecological tumor radiotherapy based on multimodal data according to claim 1, characterized in that, Based on the multimodal time-varying state matrix, the singular value attenuation spectrum of the matrix is ​​calculated at each acquisition time, and the radiotherapy response time-series descriptor is constructed using the inflection point ordinal number of the attenuation spectrum, including: Singular value decomposition is performed on the row vectors corresponding to the multimodal time-varying state matrix at each acquisition time to obtain the singular value sequence at that time. The singular value sequence is then arranged in descending order, and the ratio of adjacent singular values ​​is calculated to obtain the ratio sequence. The ordinal number of the first occurrence of a ratio less than a preset ratio threshold in the ratio sequence is taken as the ordinal number of the attenuation spectrum inflection point at that acquisition time; The inflection point numbers of the attenuation spectrum at each acquisition time are arranged in chronological order, and the variance of the arranged inflection point number sequence is calculated using a sliding window. The variance calculation result is used as the time-series descriptor of the radiotherapy response.

5. The method for predicting the efficacy of gynecological tumor radiotherapy based on multimodal data according to claim 1, characterized in that, Using the aforementioned radiotherapy response time-series descriptor as the search term, time-series similarity matching is performed in a pre-constructed radiotherapy effect reference atlas library. The effect grade corresponding to the reference atlas with the highest matching degree is used as the current predicted effect output, including: The absolute value of the difference between each descriptor value in the radiotherapy response time-series descriptor factors and the reference descriptor value of each reference atlas in the radiotherapy effect reference atlas library is calculated to obtain the difference vector of each reference atlas. The difference vectors are normalized to obtain normalized similarity scores. The reference map with the highest normalized similarity score is selected as the matching reference map, and the radiotherapy effect grade pre-annotated in the matching reference map is extracted as the current predicted effect output.

6. The method for predicting the efficacy of gynecological tumor radiotherapy based on multimodal data according to claim 1, characterized in that, Obtain periodic, Greenwich Mean Time-aligned medical imaging time sequences, vital sign parameter time sequences, and laboratory test time sequences for the target patient before and during radiotherapy planning, including: Using Greenwich Mean Time as the time reference, frames are extracted from the image data acquired by the medical imaging equipment according to the preset acquisition cycle interval to obtain the medical image time sequence. Using the same Greenwich Mean Time (GMT) and acquisition cycle interval, vital signs parameter records were extracted from patient monitoring equipment to obtain a time series sequence of vital signs parameters. Using the same Greenwich Mean Time (GMT) and data collection interval, laboratory test result records were extracted from the hospital information system to obtain the laboratory test time sequence.

7. The method for predicting the efficacy of gynecological tumor radiotherapy based on multimodal data according to claim 4, characterized in that, The inflection point numbers of the attenuation spectrum at each acquisition time are arranged in chronological order, and a sliding window variance calculation is performed on the arranged inflection point number sequence. The variance calculation result is used as the radiotherapy response time-series descriptor, including: Set the sliding window length and sliding step size, and starting from the beginning position of the inflection point ordinal sequence, extract inflection point ordinal subsequences with a length equal to the sliding window length according to the sliding step size; Calculate the variance of the ordinal subsequence at each inflection point to obtain the variance value sequence; Each variance value in the variance value sequence is aligned according to the corresponding window termination time to form the radiotherapy response time-series descriptor.

8. The method for predicting the efficacy of gynecological tumor radiotherapy based on multimodal data according to claim 5, characterized in that, The reference atlas with the highest normalized similarity score is selected as the matching reference atlas. The pre-annotated radiotherapy effect grade of this matching reference atlas is extracted as the current predicted effect output, including: If there are multiple reference maps with the highest normalized similarity scores, the radiotherapy efficacy classification of each reference map with the highest similarity score will be extracted separately. The frequency of each radiotherapy effect grade in the multiple highest similarity reference maps is counted, and the radiotherapy effect grade with the highest frequency is used as the current predicted effect output.

9. The method for predicting the efficacy of gynecological tumor radiotherapy based on multimodal data according to claim 2, characterized in that, The anatomical division into multiple sub-regions includes dividing the cervix, uterine body, vagina, and pelvic lymph node regions into independent sub-regions based on the standard gynecological tumor target delineation guidelines.

10. The method for predicting the efficacy of gynecological tumor radiotherapy based on multimodal data according to claim 2, characterized in that, The time series of vital signs parameters are subjected to differential operations according to parameter type, and the changes in vital signs parameters at adjacent acquisition times are calculated using first-order forward differential.

Images