Method and System for Processing and Analyzing Multiplex Immunohistochemistry Images

Abstract

The present invention relates to the field of image processing, and particularly to a method and system for processing and analyzing multi - immunohistochemistry images. The method includes the following steps: obtaining multi - immunohistochemistry images at multiple time points and performing temporal sequence fitting to obtain an immunohistochemistry image sequence; performing deep visual semantic recognition and frame - by - frame segmentation and reconstruction on the immunohistochemistry image sequence to construct an image frame sequence for each marker; calculating the change in cell spatial distance and mining the spatial expression correlation according to the image frame sequence of each marker to generate the temporal behavior characteristics of each marker; performing frame - by - frame tissue structure analysis on the image frame sequence of each marker to obtain a three - dimensional tissue structure model for each region. The present invention realizes accurate and efficient tissue structure evaluation and provides personalized quantitative diagnosis results.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and particularly to a method and system for processing and analyzing multiplex immunohistochemistry images. Background Art

[0002] Multiplex Immunohistochemistry (mIHC) technology is an advanced analytical method widely used in the biomedical field. By simultaneously detecting multiple biomarkers in tissue samples, it provides richer biological information than traditional single-marker immunohistochemistry technology. In cancer research, immunology, pathology, and other biomedical fields, mIHC technology has become an important tool for studying complex diseases, revealing cell heterogeneity, and the immune microenvironment due to its advantages of high throughput and multiple targets.

[0003] However, with the continuous improvement of research requirements and the increase in sample complexity, traditional methods for processing and analyzing multiplex immunohistochemistry images face many challenges. Traditional methods often rely on manual or semi-automated analysis means, which are easily affected by the experience of operators, sample quality, and limitations of analysis tools, resulting in low analysis efficiency and inaccurate results. In addition, with the increase in the types of biomarkers, the problems of cross-interference between signals of different biomarkers and background noise in images are more prominent, making image processing and analysis more complex and difficult.

[0004] Current analysis methods mostly adopt image segmentation-based techniques. Although they can separate and quantify biomarkers to a certain extent, they still cannot effectively solve the problems caused by overlapping signals between different biomarkers, background noise, and image quality differences in complex samples. At the same time, the computational amount of image analysis is huge, requiring fast processing speed and high accuracy. In the process of data processing and analysis, the flexibility and intelligence of traditional methods are still insufficient. Especially when facing large-scale data sets, the analysis efficiency and accuracy often cannot be guaranteed. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a method and system for processing and analyzing multiplex immunohistochemistry images to solve at least one of the above technical problems.

[0006] To achieve the above object, the present invention provides a method for processing and analyzing multiplex immunohistochemistry images, including the following steps:

[0007] Step S1: Obtain multiplex immunohistochemistry images at multiple time points and perform temporal serialization fitting to obtain an immunohistochemistry image sequence;

[0008] Step S2: Perform in-depth visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemical image sequence to construct an image frame sequence for each marker;

[0009] Step S3: Calculate the change in cell spatial distance and mine the spatial expression correlation based on the image frame sequence of each marker to generate the temporal behavior characteristics of each marker;

[0010] Step S4: Analyze the frame-by-frame tissue structure of the image frame sequence of each marker to obtain a three-dimensional tissue structure model for each region;

[0011] Step S5: Predict the tissue structure evolution trend of the three-dimensional tissue structure model of each region based on the temporal behavior characteristics of each marker, thereby constructing a tissue structure trend prediction model for each region;

[0012] Step S6: Perform personalized quantitative diagnosis according to the tissue structure trend prediction model of each region, and conduct a comprehensive evaluation of the tissue structure to obtain a comprehensive tissue structure evaluation report.

[0013] By obtaining immunohistochemical images at multiple time points, the present invention can capture the changes of cells and tissues at different time points, thus providing a more dynamic perspective. The fitting of this temporal image sequence provides a basis for subsequent temporal analysis, making the processing of immunohistochemical images no longer static, but enabling in-depth analysis of important information such as cell behavior and dynamic changes of markers. The temporal processing helps to identify the evolution law of cells or tissues in the time dimension, thus providing more accurate data for subsequent analysis. Through deep visual semantic recognition, automatically identify the features of different regions and markers in the image, accurately segment and extract relevant cell or tissue structures. This not only improves the accuracy and efficiency of analysis, but also ensures the fine distinction of each marker at different time points through frame-by-frame segmentation and reconstruction, making the subsequent analysis more accurately track the temporal changes of markers in tissues. By constructing an image frame sequence for each marker, detailed tracking of the cell or tissue state is carried out at different time points, providing key data for further spatial and behavioral analysis. Accurately quantify the changes in cell spatial layout and analyze the dynamic changes of cells or tissues on the time axis. By calculating the changes in cell spatial distance and its spatial expression correlation with other markers, the interaction, positional relationship and dynamic behavior between cells can be deeply understood. This analysis method reveals the activity characteristics of cells and tissues at different time points, such as changes in cell migration, division or other biological processes, thus providing valuable information for disease progression, drug effects, etc. Through frame-by-frame analysis of tissue structure, three-dimensional structure information is extracted from the two-dimensional image sequence to form a more intuitive and accurate tissue structure model. This step can deeply explore the three-dimensional spatial distribution of each marker in different regions, thus providing a reliable basis for subsequent spatial analysis and model construction. The construction of the three-dimensional structure model makes the spatial relationship of cells or tissues no longer limited to the plane, providing a more comprehensive perspective, thus better reflecting the complexity of tissues. By combining temporal behavior characteristics with the three-dimensional structure model, predict the evolution trend of tissue structure. This not only helps to understand the dynamic changes of markers, but also can predict the change trend of cells or tissues at future time points. This predictive ability is particularly applicable to the analysis of disease progression, such as the expansion of cancer cells and the change of tumor microenvironment. By establishing a tissue structure trend prediction model, scientific basis can be provided for early diagnosis, drug development, disease prevention, etc. Integrate the previous analysis results, and generate a personalized quantitative diagnosis report through comprehensive evaluation of tissue structure. This enables the analysis results of each sample to be accurately evaluated according to its unique structure and behavior characteristics, providing highly personalized information for clinical or scientific research. The comprehensive evaluation report can help doctors or researchers more intuitively understand the state of the sample.

[0014] In this specification, a multiple immunohistochemistry image processing and analysis system is provided for performing the multiple immunohistochemistry image processing and analysis method as described above, including:

[0015] An image processing module for acquiring multiple immunohistochemistry images at multiple time points and performing temporal sequence serialization fitting to obtain an immunohistochemistry image sequence;

[0016] A visual recognition module for performing deep visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemistry image sequence to construct an image frame sequence for each marker;

[0017] A cell feature analysis module for calculating the change in cell spatial distance and mining the spatial expression correlation based on the image frame sequence of each marker to generate the temporal behavior feature of each marker;

[0018] An organizational structure analysis module for performing frame-by-frame organizational structure analysis on the image frame sequence of each marker to obtain a three-dimensional organizational structure model for each region;

[0019] A situation prediction module for predicting the organizational structure evolution situation of the three-dimensional organizational structure model of each region based on the temporal behavior feature of each marker, thereby constructing an organizational structure situation prediction model for each region;

[0020] A quantitative diagnosis module for performing personalized quantitative diagnosis based on the organizational structure situation prediction model of each region and conducting a comprehensive evaluation of the organizational structure to obtain a comprehensive organizational structure evaluation report.

[0021] By acquiring immunohistochemical images at different time points and subjecting them to chronological processing, the present invention can capture the dynamic changes of cells or tissues at different times. This is crucial for studying time-dependent changes in biological processes, especially the dynamic evolution of processes such as cell growth, proliferation, and migration. Through the fitting process of the image sequence, the influence brought by experimental errors can be reduced, and a more accurate time series of immunohistochemical images can be obtained, making subsequent analysis more accurate and reliable. Using deep learning technology for semantic recognition of images can effectively identify and label important information and target regions (such as cell and tissue regions, etc.) in each image, so as to accurately extract the features of multiple markers and reduce human errors. Frame-by-frame segmentation and reconstruction can refine the spatial features and morphology of each marker, ensuring that image details are not lost, and laying a solid foundation for subsequent spatial analysis and structure recognition. The image frame sequence of each marker helps to individually track and analyze the spatio-temporal behavior of different markers, ensuring that the analysis between different markers does not interfere with each other, and improving the diversity and accuracy of multiple immunohistochemical image analysis. By calculating the change in spatial distance between cells or markers, the movement trajectory, aggregation and diffusion processes of cells in tissues, as well as their interactions can be revealed. This is very important for understanding cell behaviors (such as migration, differentiation, etc.). By analyzing the spatial expression relationship of markers, the interaction, co-variation and other laws between different markers in tissues can be deeply understood. This analysis helps to reveal the cooperative relationship between biomarker and their roles in pathological or biological processes. The temporal behavior characteristics of markers can help researchers observe the dynamic patterns of markers changing over time, and combine the behavior patterns of cells to speculate their functions in tissues and their potential roles in diseases. Frame-by-frame analysis can provide fine tissue structure information at each time point, enabling researchers to deeply observe the spatial distribution and morphological changes of different markers at each moment. Through three-dimensional reconstruction, the distribution and structural characteristics of markers in three-dimensional space in immunohistochemical images are comprehensively displayed, overcoming the limitations brought by two-dimensional image analysis. The three-dimensional model can more intuitively reveal the complex spatial layout of cells and tissues, providing strong support for further analysis. Through the three-dimensional structure model, the spatial structure of tissues can be more accurately classified, calibrated and analyzed, which helps to better understand the structural changes and biological mechanisms in diseased tissues. By combining and analyzing the temporal behavior characteristics, the evolution trends of each marker and tissue region can be predicted, revealing the change trends that occur in tissues at different time nodes (such as tumor growth, changes in immune responses, etc.). This provides a powerful tool for studying the progression of diseases or the dynamic changes of cells / tissues. By establishing a situation prediction model of tissue structure, the future change trends can be more scientifically predicted, and then early prediction and intervention of diseases can be carried out. This prediction ability is especially applicable to fields such as oncology and immunology, and has important clinical value.Based on the aforementioned temporal behavior characteristics and three-dimensional structure model, personalized analysis is carried out according to the specific situation of each patient. Brief Description of the Drawings

[0022] Figure 1 It is a schematic flowchart of the steps of a method for processing and analyzing multiple immunohistochemical images of the present invention;

[0023] Figure 2 It is a schematic flowchart of the detailed implementation steps of step S1;

[0024] Figure 3 It is a schematic flowchart of the detailed implementation steps of step S2;

[0025] Figure 4 It is a schematic flowchart of the detailed implementation steps of step S3. Detailed Embodiments

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0027] The embodiments of the present application provide a method and system for processing and analyzing multiple immunohistochemical images. The execution subjects of the method and system for processing and analyzing multiple immunohistochemical images include but are not limited to: mechanical equipment, data processing platforms, cloud server nodes, network upload devices, etc. that carry the system, which can be regarded as general computing nodes of the present application. The data processing platform includes but is not limited to at least one of: audio and image management systems, information management systems, and cloud data management systems.

[0028] Please refer to Figures 1 to 4 , the present invention provides a method for processing and analyzing multiple immunohistochemical images, and the method for processing and analyzing multiple immunohistochemical images includes the following steps:

[0029] Step S1: Obtain multiple immunohistochemical images at multiple time points and perform temporal sequence fitting to obtain an immunohistochemical image sequence;

[0030] Step S2: Perform deep visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemical image sequence to construct an image frame sequence of each marker;

[0031] Step S3: Calculate the change in cell spatial distance and mine the spatial expression correlation according to the image frame sequence of each marker to generate the temporal behavior characteristics of each marker;

[0032] Step S4: Perform frame-by-frame tissue structure analysis on the image frame sequence of each marker to obtain a three-dimensional tissue structure model of each region;

[0033] Step S5: Predict the organizational structure evolution trend of the three-dimensional organizational structure model of each region according to the temporal behavior characteristics of each marker, so as to construct the organizational structure trend prediction model of each region;

[0034] Step S6: Conduct personalized quantitative diagnosis according to the organizational structure trend prediction model of each region, and conduct comprehensive evaluation of the organizational structure to obtain a comprehensive evaluation report on the organizational structure.

[0035] By acquiring immunohistochemical images at multiple time points, the present invention can capture the changes of cells and tissues at different time points, thereby providing a more dynamic perspective. The fitting of this time-series image sequence provides a basis for subsequent time-series analysis, making the processing of immunohistochemical images no longer static, but enabling in-depth analysis of important information such as cell behavior and dynamic changes of markers. The time-series processing helps to identify the evolution laws of cells or tissues in the time dimension, thus providing more accurate data for subsequent analysis. Through deep visual semantic recognition, automatically identify the features of different regions and markers in the image, accurately segment and extract relevant cell or tissue structures. This not only improves the accuracy and efficiency of analysis, but also ensures the fine distinction of each marker at different time points through frame-by-frame segmentation and reconstruction, making subsequent analysis more accurately track the time-series changes of markers in tissues. By constructing an image frame sequence for each marker, detailed tracking of the cell or tissue state is carried out at different time points, providing key data for further spatial and behavioral analysis. Accurately quantify the changes in cell spatial layout and analyze the dynamic changes of cells or tissues on the time axis. By calculating the changes in cell spatial distance and its spatial expression correlation with other markers, the interactions, positional relationships and dynamic behaviors between cells can be deeply understood. This analysis method reveals the activity characteristics of cells and tissues at different time points, such as changes in cell migration, division or other biological processes, thus providing valuable information for disease progression, drug effects, etc. Through frame-by-frame analysis of tissue structure, three-dimensional structure information is extracted from the two-dimensional image sequence to form a more intuitive and accurate tissue structure model. This step can deeply explore the three-dimensional spatial distribution of each marker in different regions, thus providing a reliable basis for subsequent spatial analysis and model construction. The construction of the three-dimensional structure model makes the spatial relationship of cells or tissues no longer limited to the plane, providing a more comprehensive perspective, thus better reflecting the complexity of tissues. By combining time-series behavior characteristics with the three-dimensional structure model, predict the evolution trend of tissue structure. This not only helps to understand the dynamic changes of markers, but also predicts the change trends of cells or tissues at future time points. This predictive ability is particularly applicable to the analysis of disease progression, such as the expansion of cancer cells and the change of tumor microenvironment. By establishing a tissue structure trend prediction model, scientific basis can be provided for early diagnosis, drug development, disease prevention, etc. Integrate the previous analysis results, and generate a personalized quantitative diagnosis report through comprehensive evaluation of tissue structure. This enables the analysis results of each sample to be accurately evaluated according to its unique structure and behavior characteristics, providing highly personalized information for clinical or scientific research. The comprehensive evaluation report can help doctors or researchers more intuitively understand the state of the sample, thus formulating more targeted analysis plans or scientific research strategies.

[0036] In the embodiments of the present invention, refer to Figure 1, which is a schematic diagram of the step process of a method for processing and analyzing multiple immunohistochemical images of the present invention. In this example, the steps of the method for processing and analyzing multiple immunohistochemical images include:

[0037] Step S1: Obtain multiple immunohistochemical images at multiple time points and perform time-series serialization fitting to obtain an immunohistochemical image sequence;

[0038] In this embodiment, appropriate biological samples are collected, ensuring that the samples are processed under the same experimental conditions to avoid batch effects. The samples should be approved by the ethics committee and comply with biosafety and ethical requirements. The samples are processed using a standard immunohistochemical staining procedure. The general steps include dewaxing, antigen retrieval, blocking, antibody incubation, and color development. In this step, appropriate primary and secondary antibodies are selected to ensure the specificity and sensitivity of the antibodies. The primary antibody dilution is 1:200, and the incubation time is 1 hour to ensure the uniformity and effectiveness of the staining. A high-resolution fluorescence microscope or optical microscope is used to ensure that image information at the cellular level can be captured. The device should be equipped with image acquisition software that can set appropriate exposure times and gains to obtain the best image quality. According to the nature of the tissue section and the fluorescence characteristics of the markers, the exposure time (e.g., 100 ms), gain (e.g., set to 50%), and magnification (e.g., 40x) of the microscope are adjusted. Multiple images are acquired at different time points to ensure that the image quality at each time point is consistent. Images of the same tissue section are acquired at set time points (e.g., 0 hours, 6 hours, 12 hours, 24 hours, etc.). Ensure that the images at each time point are acquired under the same lighting and background conditions to reduce the impact of environmental changes on the results. The acquired images are preprocessed, including denoising, contrast enhancement, and image registration. Image processing software (such as ImageJ or FIJI) is used for image processing to ensure that the images at each time point can be spatially aligned. Feature point matching or mutual information-based registration algorithms are used to register the images at different time points. Ensure that the images of the same cell correspond at different time points to prepare for time-series analysis. The SIFT feature point matching algorithm is used for registration to achieve a matching accuracy of 95%. The registered images are organized into a time-series sequence in chronological order. Programming tools (such as Python or MATLAB) are used to write scripts to integrate the image data at each time point into a time-series data structure. The image at each time point should contain corresponding metadata, such as time stamps, sample IDs, etc., for subsequent analysis. The integrated time-series sequence is verified to ensure that the data at each time point is complete and usable. Check the alignment quality and image clarity to ensure that no important information is lost. The final immunohistochemical image sequence is stored in a standard data format (such as TIFF or HDF5) for subsequent analysis and access.

[0039] Step S2: Perform in-depth visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemical image sequence to construct an image frame sequence for each marker;

[0040] In this embodiment, representative images are selected for annotation. When constructing the dataset, ensure the diversity of samples, covering different time points, sample types, and markers. These samples should have different cell morphologies and tissue structures so that the model can learn rich features. Use professional image annotation tools (such as Label box or VGG ImageAnnotator) to annotate the images. The annotation content includes cell regions, tissue regions, and specific marker regions. Usually, it is carried out by biological experts to ensure the accuracy of annotation. Divide the annotated dataset into a training set (70%), a validation set (20%), and a test set (10%) to ensure the generalization ability of the model. The training set is used for model training, the validation set is used to adjust hyperparameters, and the test set is used to finally evaluate the model performance. Select a suitable deep learning model, such as U-Net, MaskR-CNN, or DeepLabV3+, which perform excellently in image segmentation and semantic recognition tasks. For the complex morphologies of cells and tissues, U-Net can effectively capture detailed information due to its symmetric structure and skip connections. Use deep learning frameworks such as TensorFlow or PyTorch for model training. Set hyperparameters, including the learning rate (usually set to 0.001), batch size (such as 16), and number of training epochs (such as 50 epochs). Utilize data augmentation techniques (such as rotation, scaling, and flipping) to improve the robustness of the model. Evaluate the model performance on the validation set, and use metrics such as the intersection over union (IoU) and Dice coefficient to measure the segmentation effect. Adjust the model according to the evaluation results, such as changing the learning rate or increasing the number of training epochs, to improve the segmentation accuracy. Use the trained deep learning model to perform frame-by-frame segmentation on the immunohistochemical image sequence. Input each frame of the image into the model to obtain the segmentation masks for each cell and tissue. Ensure that the size of the input image meets the requirements of the model, usually by scaling (such as resizing to 256x256 pixels). Post-process the segmentation masks to remove noise and small-area artifacts. Use morphological operations (such as dilation and erosion) to smooth the segmentation results, eliminate small artifacts, and ensure the connectivity of the segmented regions. At this time, ensure the integrity and accuracy of each cell region. Recombine the segmentation results of each frame to generate an image frame sequence for each marker. Each box should contain the position information of the cells and the distribution of the markers. Generate a statistical data table containing the number of cells, cell area, and cell morphological features for each time point.

[0041] Step S3: Calculate the changes in cell spatial distance and mine the spatial expression correlation based on the image frame sequence of each marker to generate the temporal behavior features of each marker;

[0042] In this embodiment, the time points for analysis are determined. For example, in each time-series image, a sequence of image frames at 0 hours, 6 hours, 12 hours, and 24 hours is selected to ensure that there is sufficient data for analysis at each time point. The Euclidean distance is used as the calculation method for the distance between cells. For each pair of cells, the distance between their centroids is calculated. Frame-by-frame distance calculation: The sequence of image frames at each time point is traversed, the distance between cells is calculated, and the results are stored in a matrix. In this way, for each time point, the distance matrix between cells will record the spatial distances between all cell pairs. If there are 5 cells at a certain time point, a 5x5 distance matrix is generated, and each element in the matrix represents the distance between a cell pair. The results of the distance between cells at each time point are recorded as structured data, including the IDs of cell pairs, distance values, and time information. This data will provide a basis for generating subsequent behavioral characteristics. The Pearson correlation coefficient or Spearman rank correlation coefficient is used to quantify the spatial expression correlation between cells. The selected correlation analysis method should be able to handle continuous variables and rank variables to adapt to different cell expression data. Correlation analysis is performed on the cell expression data (such as the expression intensity of specific markers) at each time point. First, a cell expression matrix is constructed, where the rows represent cells and the columns represent different time points or different markers. Then, the correlation between cell expressions is calculated. The calculated results of spatial expression correlation are recorded as structured data, including the correlation coefficients of each pair of cells and the corresponding time point information. These results will provide an important basis for generating subsequent time-series behavioral characteristics. Based on the changes in the distance between cells and the spatial expression correlation, time-series behavioral characteristics are extracted. Calculate the average value, standard deviation, change rate, etc. of the distance between cells at each time point, and at the same time consider the changes in the correlation coefficients. The extracted time-series behavioral characteristics are integrated into a comprehensive data table, which includes information such as time points, cell pair IDs, spatial distance characteristics, and expression correlation characteristics. In this way, it is convenient for subsequent analysis and visualization. The finally generated time-series behavioral characteristics are recorded in a structured data format (such as CSV or JSON) for subsequent analysis and visualization. This data will provide a basis for subsequent dynamic behavior analysis.

[0043] Step S4: Perform frame-by-frame tissue structure analysis on the image frame sequence of each marker to obtain a three-dimensional tissue structure model of each region;

[0044] In this embodiment, preprocessing is performed on each image frame, including denoising, contrast enhancement, and normalization. Image processing software (such as ImageJ or OpenCV) is used for processing. Gaussian filtering is used for denoising, and histogram equalization can be used to enhance the contrast. Ensure that the images of each frame are at the same scale and contrast to improve the accuracy of subsequent analysis. Select a suitable image segmentation algorithm, such as U-Net or K-means clustering, to extract the tissue structure. U-Net performs well in biomedical image segmentation and can effectively capture tissue features. Set the input size of the model (such as 256x256 pixels), and use the previously trained model to segment each frame. Manually verify the segmentation results to ensure the accurate segmentation of cell and tissue regions. Make adjustments if necessary to enhance the precision of the model. If there are mis-segmented cell regions in the segmentation results, correct them by manually adjusting the segmentation mask. Extract features for each segmented region, including region area, perimeter, shape moments, centroid coordinates, etc. Use image processing tools (such as OpenCV) to extract these features and store the results in a structured data format. Record the area (in pixels) and perimeter (calculated using a formula) of each region. Select a suitable 3D reconstruction algorithm, such as Marching Cubes or voxel reconstruction method. These methods can generate 3D models based on the frame-by-frame analysis results. For the complexity of the tissue structure, the voxel reconstruction method provides better flexibility. Convert the segmentation results of each frame into a 3D voxel model. Adopt 3D mesh reconstruction technology to convert the segmentation results of each cell region into 3D voxels. If the Marching Cubes algorithm is used, first a 3D scalar field needs to be constructed, and then a 3D surface model is generated through the algorithm. Refine the generated 3D model, delete redundant voxels and noise, and smooth the model surface. Use 3D graphics processing software (such as MeshLab or Blender) for processing to ensure the smoothness and realism of the model. Save the final 3D tissue structure model in a standard 3D file format (such as STL, OBJ, or PLY) to ensure compatibility and ease of subsequent analysis. Record the relevant parameters of the model, such as volume, surface area, and cell density, etc. Back up all the generated 3D models and the original data to ensure the security and traceability of the data. Use a database or file system for effective management for subsequent analysis and research.

[0045] Step S5: Predict the organizational structure evolution trend of the 3D organizational structure model of each region according to the temporal behavior characteristics of each marker, so as to construct an organizational structure trend prediction model for each region;

[0046] In this embodiment, the extracted temporal behavior features are standardized to eliminate the dimensional differences between different features. Usually, the Z-score standardization method is adopted to adjust the mean of each feature to 0 and the standard deviation to 1. This helps to improve the training efficiency and accuracy of the model. According to the data characteristics and objectives, a suitable prediction model is selected. Common choices include time series analysis models (such as ARIMA), machine learning models (such as random forest, support vector machine), and deep learning models (such as LSTM). For biological data with temporal characteristics, LSTM is widely used because of its excellent time series modeling ability. The standardized temporal behavior feature data is divided into a training set and a test set, usually in a ratio of 80% for training and 20% for testing. The training set is used for model training, and the test set is used to evaluate the generalization ability of the model. Deep learning frameworks such as Keras or TensorFlow in Python are used to train the LSTM model. Set the hyperparameters of the model, including the input sequence length (select the first 5 time points as the input), the number of hidden layer units (such as 64), and the learning rate (such as 0.001). Evaluate the model performance on the test set, and use metrics such as mean squared error (MSE) and coefficient of determination (R²) to measure the prediction effect. Adjust the model parameters according to the evaluation results, such as increasing the number of LSTM layers or units, to improve the accuracy of the model. Use the trained model to predict the future organizational structure state. Input the feature data of the last few known time points, and the model will output the organizational structure features of the next time point. Predict the changes in cell density and its spatial distribution within the next 6 hours.

[0047] Step S6: Perform personalized quantitative diagnosis according to the organizational structure situation prediction model of each region, and conduct a comprehensive evaluation of the organizational structure to obtain a comprehensive evaluation report on the organizational structure.

[0048] In this embodiment, the prediction results of each region are standardized to eliminate the dimensional differences caused by different regions or markers. Z-score standardization is adopted to adjust the mean of each feature to 0 and the standard deviation to 1. This helps to improve the accuracy of subsequent quantitative diagnosis. According to the biological significance of the tissue structure, personalized quantitative diagnosis criteria are constructed. Thresholds such as cell density threshold and cell spacing threshold are set to judge the health status of the tissue. Referring to existing literature or clinical guidelines and combining experimental data, corresponding criteria are formulated. Statistical analysis software (such as the Scipy library in R or Python) is used to analyze the standardized prediction data. The mean and standard deviation of the cell density in each region are calculated and compared with the set thresholds to make a diagnostic decision. If the cell density in a certain region is significantly higher than the normal range, it indicates a pathological state such as tumor or inflammation. The quantitative diagnosis results of each region are organized into a structured format, including region ID, predicted value, diagnosis result (such as normal, abnormal, malignant) and clinical suggestions. Ensure that the information in the report is complete and easy to understand. Appropriate comprehensive evaluation methods, such as weighted scoring method or fuzzy logic system, are selected to generate comprehensive evaluation results according to different prediction features. These methods can comprehensively consider the influence of multiple factors and form a comprehensive evaluation result. Weights are assigned to each prediction feature according to its importance and clinical relevance. The cell density has a greater impact on the tissue health status than the cell spacing, so a higher weight is set for the cell density. The weight values are determined through expert discussion or literature research. Combining the prediction features of each region and the set weights, the comprehensive evaluation score is calculated through weighted average or fuzzy logic model. The quantitative diagnosis results and the comprehensive evaluation score are integrated into the report, accompanied by necessary charts and data support. Data visualization tools (such as Matplotlib or Tableau) are used to generate relevant charts to enhance the readability and intuitiveness of the report.

[0049] In this embodiment, refer to Figure 2 , which is a schematic diagram of the detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include:

[0050] Step S11: Obtain multiplex immunohistochemical images at multiple time points;

[0051] Step S12: Perform global brightness optimization on the multiplex immunohistochemical images to obtain a brightness-enhanced image;

[0052] Step S13: Perform adaptive local background elimination on the brightness-enhanced image to obtain a locally background-corrected image;

[0053] Step S14: Perform dynamic filtering for noise reduction and time-series serialization fitting based on the locally background-corrected image to obtain an immunohistochemical image sequence.

[0054] In this embodiment, the microscope equipment and related reagents used are determined to ensure that the equipment can support the imaging of multiplex immunohistochemistry staining. A fluorescence microscope and multiple specific antibodies are used for sample preparation. The samples need to be properly fixed and sectioned for subsequent imaging. An experiment is designed to perform imaging at different time points (e.g., 0 hours, 1 hour, 2 hours, 4 hours, 8 hours). An appropriate time interval is selected to capture the dynamic process of cell response, ensuring that clear images can be obtained at each time point. At each time point, the samples are imaged using a fluorescence microscope, ensuring that the images are taken under the same exposure conditions for subsequent processing. The light intensity, exposure time, and image resolution at each time point are recorded, and the image resolution of each image is 1024x1024 pixels. The collected images are classified and stored according to the time points, ensuring that each image has a clear identifier for subsequent processing. Finally, images at 5 time points are obtained, and the sample images at each time point are 10MB. An appropriate global brightness optimization algorithm is selected, such as histogram equalization or adaptive histogram equalization (CLAHE). CLAHE can effectively enhance the local contrast of the image while avoiding artifacts caused by over-enhancement. The global brightness of the multiplex immunohistochemistry images at each time point is optimized, and the processing steps include: calculating the histogram of the image and analyzing the brightness distribution. The selected algorithm is applied for brightness adjustment to enhance the visualization effect of the image. The images before and after processing are compared to ensure that the brightness enhancement effect is obvious and the cell structure and markers are clearer. After optimization, the contrast of the image is increased by 30%, making the cell boundaries more obvious. An adaptive background elimination technique, such as Gaussian filtering or median filtering, is adopted. These methods can effectively remove the background noise in the image while retaining the structural features of the cells. Local background elimination is performed on each brightness-enhanced image, and the specific steps include: selecting an appropriate filtering window size, usually adjusted according to the size of the cells in the image, and the selected window size is 15x15 pixels. The image is filtered to generate a background image, and the background is subtracted from the original image. The corrected image is checked to ensure that the cell contours are clear and the background noise is significantly reduced. The background noise is reduced by 50%, and the visualization effect of the cell structure is significantly improved. Dynamic filtering and noise reduction are performed on the images after local background correction, and the specific steps include: setting filtering parameters, such as filtering intensity and window size, and the window size is 5x5 pixels. The selected filtering algorithm is applied to process the image to generate a noise-reduced image. The noise-reduced images are serialized in chronological order to construct time-series data. Analysis is performed using image processing software or dedicated analysis software to record the changes of cells over time. The generated immunohistochemistry image sequence is stored to ensure that the images at each time point can be accurately reflected in the sequence for subsequent analysis and comparison.

[0055] In this embodiment, the specific steps of step S13 are as follows:

[0056] Perform non-target marker background recognition on the brightness-enhanced image and mark the background area of the image;

[0057] Perform regional texture recognition on the background area of the image to generate background area texture features;

[0058] Estimate the color features of the background area of the image;

[0059] Perform low-dimensional visual expression quantization based on the background area texture features and the color features, so as to extract the low-dimensional visual expression vector of the background area;

[0060] Perform adaptive local background elimination on the brightness-enhanced image based on the low-dimensional visual expression vector of the background area, so as to obtain a locally background-corrected image.

[0061] In this embodiment, an image processed by brightness enhancement is obtained to ensure that the image contrast and brightness are suitable for subsequent analysis. This step includes using techniques such as histogram equalization or adaptive histogram equalization (CLAHE) to improve the brightness and contrast of the image, making the distinction between the background and the target object more obvious. Apply color- and texture-based segmentation algorithms, such as K-means clustering or Grab Cut algorithm, to identify the background region in the image. These algorithms distinguish the background and foreground by analyzing the color features of pixels and the relationship between adjacent pixels. In K-means clustering, the color space of the image (such as RGB or HSV) is subjected to clustering analysis to determine the category of each pixel and label it as background or foreground. The labeled image will show the boundaries of the background region, making subsequent processing more accurate. Use the gray-level co-occurrence matrix (GLCM) or local binary pattern (LBP) to extract the texture features of the background region. These techniques can effectively capture the local texture information of the image and are of great significance for identifying the details of the background. For the gray-level co-occurrence matrix, first calculate the gray matrix of the background region, and then extract relevant features, such as contrast, entropy, correlation, etc. These features reflect the texture complexity and pattern of the background and can provide important data for subsequent analysis. Store the extracted texture features in a feature vector for subsequent use. The extracted features include indicators such as contrast, correlation, and energy. Select the HSV or Lab color space for statistical analysis of the color features of the background region. These spaces can reflect the color characteristics of human visual perception better than the traditional RGB space. Perform color value statistics on each pixel in the background region to calculate features such as the mean value, standard deviation, and color histogram. The calculated color mean is (180, 100, 150), and the standard deviation is (20, 15, 25), indicating that the colors in this region are concentrated within a specific range. Record the color features as a feature vector, ensuring that it includes the mean and standard deviation of each channel to form a complete color feature description. Use dimensionality reduction techniques such as principal component analysis (PCA) or t-SNE to fuse the texture features and color features of the background region and extract a low-dimensional visual expression vector. Combine the recorded texture features and color features into a high-dimensional feature vector, and then apply PCA for dimensionality reduction to extract the first few principal components as the low-dimensional visual expression vector. After PCA processing, the extracted low-dimensional vector is (0.75, -0.80). Select an adaptive background modeling technique, such as Gaussian mixture model (GMM) or background subtraction algorithm, to perform background elimination based on the extracted low-dimensional visual expression vector. In the incoming brightness-enhanced image, perform local background elimination on each pixel according to the extracted low-dimensional visual expression vector. By calculating the difference between the current pixel and the background model, the target object can be highlighted. Generate a local background correction image and record the image quality after elimination and the clarity of the target object. The final image shows the details of the target object, and the background part is effectively removed.

[0062] In this embodiment, the specific steps of step S14 are as follows:

[0063] Perform pixel-by-pixel detection on the locally background-corrected image to identify abnormal image noise points;

[0064] Conduct noise distribution analysis on the abnormal image noise points to generate noise distribution data in the image;

[0065] Perform dynamic filtering and noise reduction based on the noise distribution data in the image, thereby constructing a filtered and noise-reduced image;

[0066] Traverse all images to obtain filtered and noise-reduced images at multiple time points;

[0067] Extract the timestamps of the filtered and noise-reduced images at the multiple time points;

[0068] Perform temporal sequence fitting on the filtered and noise-reduced images based on the timestamps to obtain an immunohistochemical image sequence.

[0069] In this embodiment, local contrast analysis and threshold segmentation methods are adopted to perform pixel-by-pixel detection to identify abnormal noise points in the image. It is achieved by calculating the luminance difference between each pixel and its neighboring pixels. A threshold is set, and when the luminance of a certain pixel exceeds this threshold, it is marked as a noise point. The image after local background correction is traversed, and the difference between the luminance value of each pixel and the average value of its surrounding 8 neighboring pixels is calculated. The set threshold is 20. If the difference exceeds this threshold, the pixel is marked as a noise point. All detected abnormal noise points are recorded in a list, marking their coordinate positions and luminance values for subsequent analysis. The luminance value of the pixel with the noise point position of (45, 67) is recorded as 255. Distribution analysis of the identified noise points is carried out to count the distribution of noise points in the image, including the number of noise points, position distribution density, luminance distribution, etc. It is achieved by using histogram analysis and heat map generation techniques. The number and distribution density of all noise points are counted. A total of 150 noise points are detected in an image, distributed in different regions. By generating a heat map, the concentrated areas of noise points are visually displayed. The noise point distribution data is recorded in a data structure, including information such as the total number of noise points, coordinate distribution, and luminance range, for subsequent filtering processing. Dynamic median filtering or Gaussian filtering algorithms are adopted to perform noise reduction processing on the noise points. Median filtering can effectively remove salt-and-pepper noise, while Gaussian filtering can preserve edge information while smoothing the image. According to the noise point distribution data, the filtering algorithm is applied to each noise point position in the image. For each noise point, the median or weighted average value of the pixels within a certain radius around it is calculated, and the value of the noise point pixel is replaced with this value. The selected filtering window is 3x3. The denoised image after filtering is generated, and the comparison results with the original image are recorded, including the amount of noise reduction and image quality metrics (such as PSNR, SSIM, etc.), to evaluate the filtering effect. The PSNR value of the denoised image is increased to 35 dB. The images at all time points are traversed, and the above denoising process is repeated to ensure that the images at each time point go through the same denoising process. The processing status of each image is recorded for subsequent analysis. The denoised images at each time point are recorded and saved as time series data. The denoising result of the image at the 1st time point is recorded as "Denoised Image 1", and the result at the 2nd time point is "Denoised Image 2". Timestamp information is extracted from the metadata of each image to ensure the integrity of the time series information of all images. The timestamp should include the specific time of image acquisition for subsequent serialization fitting. The extracted timestamps are stored in a data structure in a unified format to ensure that the timestamps correspond to the corresponding filtered and denoised images. The recorded timestamp format is "YYYY-MM-DD HH:MM:SS". Algorithms such as linear regression or spline interpolation are selected to perform time series serialization fitting on the extracted timestamps and the corresponding filtered and denoised images. This can help establish the relationship between time and image changes. The timestamps and image data are input into the fitting model to generate the fitting results of the time series images.Through linear regression analysis, the generated fitting curve can represent the image change trend at different time points. Record the fitting results and save the fitted image sequence as a new image sequence for subsequent analysis and visualization. Save it as the "immunohistochemistry image sequence".

[0070] In this embodiment, refer to Figure 3 , which is a schematic diagram of the detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include:

[0071] Step S21: Perform in-depth visual semantic recognition on the immunohistochemistry image sequence to obtain multiple image biomarkers;

[0072] Step S22: Analyze the sample types of multiple image biomarkers to obtain the type characteristics of each biomarker;

[0073] Step S23: Adjust the dynamic segmentation parameters based on the type characteristics of each biomarker to obtain the segmentation parameters of each biomarker;

[0074] Step S24: Perform adaptive image frame segmentation on multiple image biomarkers according to the segmentation parameters of each biomarker to obtain the bounding box of each biomarker;

[0075] Step S25: Perform frame-by-frame segmentation and reconstruction on the immunohistochemistry image sequence according to the bounding box of each biomarker to construct the image frame sequence of each biomarker.

[0076] In this embodiment, a deep learning model suitable for image biomarker recognition is selected, such as a convolutional neural network (CNN) architecture, especially architectures based on ResNet or U-Net, which are suitable for medical image segmentation and recognition tasks. The immunohistochemical image sequence is normalized, including image scaling, normalization, and enhancement, etc., to improve the accuracy of model recognition. The images are uniformly adjusted to 256x256 pixels and appropriately rotated and flipped. The preprocessed image sequence is input into the trained deep learning model for biomarker recognition. The model output includes the biomarkers recognized in each image and their probability values. The recognition results are stored in a database, recording the biomarker information of each image, such as the biomarker name, location, and its confidence. The recognized biomarkers include "cell nucleus", "cytoplasm", etc. According to the characteristics of the biomarkers and the recognition results, feature engineering methods are used to analyze the type of each biomarker. Clustering analysis (such as K-means) or support vector machine (SVM) can be used for type classification. Shape features, color features, and texture features, etc., are extracted from the recognized biomarkers. By calculating the area, perimeter, mean color, and texture metrics (such as LBP), etc., a description vector of the biomarker is obtained. The type features of each biomarker are stored in a structured form for subsequent analysis. The recorded features include "type: cell nucleus", "area: 500 square pixels", "average color: (255, 200, 150)". A suitable image segmentation algorithm is selected, such as the threshold method, region growing, or a graph model-based segmentation method (such as Graph Cut), and the corresponding segmentation parameters are configured according to the type features of the biomarker. The segmentation parameters are dynamically adjusted according to the characteristics of the biomarker. For biomarkers with a larger area, a higher threshold needs to be set to avoid mis-segmentation, while for fine structures, the threshold needs to be reduced. The segmentation parameter settings of each biomarker are recorded, including the threshold, structural element size, etc., for subsequent segmentation use. Record "cell nucleus segmentation parameters: threshold 0.5, structural element size 3". According to the segmentation parameters of each biomarker, the image is adaptively box-segmented. The previously set segmentation algorithm is used to process the image to generate the bounding box of the biomarker. The bounding box of each biomarker is marked in the image, and the coordinate information of the bounding box (the upper left and lower right coordinates) is recorded. Record "cell nucleus bounding box: (x1, y1) = (30, 40), (x2, y2) = (80, 100)". For each frame in the immunohistochemical image sequence, the generated bounding box is applied for reconstruction. By extracting the image region within each bounding box, an image box sequence of each biomarker is generated. The image box sequence of each biomarker is stored as an independent image file or structured data for subsequent analysis and use. Record "cell nucleus image box sequence: image1.png, image2.png".

[0077] In this embodiment, refer to Figure 4, which is a schematic diagram of the detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include:

[0078] Step S31: Perform quantitative analysis on the intercellular relative relationship of the image frame sequence of each marker to generate the intercellular relative relationship data of each image frame;

[0079] Step S32: Calculate the change in the spatial distance of cells over multiple time periods based on the intercellular relative relationship data of each image frame to obtain the multi-time period change data of the cell spatial distance;

[0080] Step S33: Identify the cell spatial layout based on the image frame sequence of each marker and extract the cell spatial layout features;

[0081] Step S34: Identify the dynamic behavior between cells based on the cell spatial layout features and detect the dynamic behavior data between cells in each image frame;

[0082] Step S35: Mine the spatial expression correlation of the dynamic behavior data between cells in each image frame to obtain the cell spatial expression correlation;

[0083] Step S36: Evolve the temporal behavior characteristics of each marker based on the multi-time period change data of the cell spatial distance and the cell spatial expression correlation to generate the temporal behavior characteristics of each marker.

[0084] In this embodiment, first, the image frame sequence of each marker is processed to extract the cell information in each image frame. Through image segmentation technology, the contour of each cell is identified and its centroid position is calculated. Classic image processing functions, such as Canny edge detection or threshold segmentation, are used to ensure that the edges of the cells are clearly distinguishable. At this time, the coordinates, area, and contour information of each cell are stored for subsequent analysis. For each pair of cells, their relative positions are calculated. Using the Euclidean distance formula, d = , where (x1, y1) and (x2, y2) are the centroid coordinates of two cells. The calculated distances are recorded in a matrix, and each element in the matrix represents the distance between cell pairs. The relative relationship data of cells in each image frame are stored as structured data, including the distances between cell pairs and relative orientation information (left, right, up, down). Such a data structure facilitates subsequent analysis and visualization. Determine to use time series analysis methods to evaluate the spatial distance changes of cells at different time points. The selected algorithm needs to be able to handle multi-dimensional data, and usually dynamic time warping (DTW) or simple distance change calculation is used. Analysis of cell distance changes: Calculate the relative distances between each cell pair at different time points. For each time point, extract the distance data between cells and calculate the relative changes. Select a suitable clustering algorithm, such as DBSCAN or K-means, for analyzing the spatial distribution of cells. The basis for selecting the clustering algorithm is the distribution nature of the cell layout: if the cells are unevenly distributed, select DBSCAN. Conduct spatial layout analysis on cells and extract layout features. First, use clustering analysis to identify the aggregation regions of cells. The clustering results will assign a class label to each cell to identify the cluster it belongs to. Then, calculate the center point of the cluster, the degree of aggregation (such as the number of cells within the cluster), and the distribution shape (such as elliptical shape, regularity). Record the extracted cell spatial layout features as structured data, including feature data such as cluster centers, cluster numbers, and layout shapes. These features will help analyze the spatial distribution and interrelationships of cells. According to the movement trajectories and relative position relationships of cells, select methods for dynamic behavior analysis. Use time series-based dynamic monitoring methods to monitor the interactions between cells. Implementation of behavior detection: Conduct dynamic behavior detection on the cells in each image frame and record the movements and interactions between cells. Monitor the movement patterns, contact, and separation behaviors of cells. Analyze the distance changes between cells. When the relative distance between cells decreases to a certain threshold, record it as a "contact" behavior; when the relative distance increases, record it as a "separation" behavior. Record the dynamic behavior data between cells in each image frame as structured information, including the behavior types, frequencies, and durations between cells, etc. This information provides a basis for subsequent spatial expression correlation analysis. Select suitable statistical methods, such as Pearson correlation coefficient or Spearman rank correlation coefficient, to explore the spatial expression correlation between cells. These methods can quantify the relationship between cell behaviors and their spatial layouts. Spatial expression correlation analysis: Based on the dynamic behavior data between cells, analyze the expression correlation between cells. By calculating the behavior data between different cell pairs, identify the correlation between cell behaviors and spatial layouts. Calculate the correlation between the mutual contact frequency and spatial distance between cell A and cell B over a period of time. Record the obtained cell spatial expression correlation data in the database to ensure that the mutual influence between cells can be traced and provide support for subsequent analysis.Select suitable time series analysis methods, such as Dynamic Time Warping (DTW) or ARIMA model, to analyze the evolution of cell behavior characteristics. These methods can effectively capture patterns and trends in time series data. Based on the multi-period change data of cell spatial distance and the cell spatial expression correlation, perform the evolutionary analysis of the time series behavior characteristics for each marker one by one. By establishing the time series of each cell behavior, identify the evolutionary trends and patterns of cell behavior. Analyze the change of cell contact frequency over time and explore the relationship between it and the change of cell layout. Record the time series behavior characteristics of each marker as time series data to ensure that the data can be used for subsequent analysis and research. This data structure will provide support for understanding the dynamic relationships and behavioral evolution among cells.

[0085] In this embodiment, step S4 includes the following steps:

[0086] Step S41: Parse the organizational structure of each frame of the image frame sequence of each marker to obtain the organizational structure data of each marker region;

[0087] Step S42: Perform geometric morphology depth recognition on the organizational structure data of each marker region to generate the tissue morphology characteristics of each region;

[0088] Step S43: Perform tissue topology connection analysis based on the organizational structure data of each marker region to generate the component topology connection characteristics of each region;

[0089] Step S44: Perform three-dimensional tissue topology reconstruction on the tissue morphology characteristics of each region based on the component topology connection characteristics of each region to obtain the three-dimensional organizational structure model of each region.

[0090] In this embodiment, the image frame sequences of each marker are sorted to ensure that the order and timestamps of each frame of the image are consistent. The images should undergo necessary preprocessing, such as denoising, enhancing contrast, and standardizing the size (e.g., resizing to 512x512 pixels), to improve the accuracy of subsequent analysis. Image segmentation techniques are employed, using image thresholding segmentation or deep learning-based image segmentation models (such as U-Net) to extract tissue structures. By analyzing the pixel information of each frame, the boundaries and shapes of the tissue structures are identified. Specifically, the Otsu method is used for threshold selection to calculate the optimal threshold for each frame of the image, so as to effectively separate the tissue region from the background. The tissue structure data of each marker region obtained by parsing is stored as structured data, including information such as the shape, position, and area of the region. Record the coordinates, area (e.g., 500 square pixels), and shape features (such as contour, boundary length) of each tissue region. Select suitable geometric shape analysis methods, usually using shape descriptors (such as Hu moments, Zernike moments) to extract tissue shape features. These descriptors can effectively capture the shape features of the tissue and generate high-dimensional feature vectors. Calculate the geometric features for each extracted tissue region. Calculate the rotation-invariant features of each tissue region, and extract the first seven Hu moment values, which can reflect the shape features of the tissue. By writing a feature extraction algorithm, loop through each region and summarize the calculation results. Record the tissue shape features of each generated region as structured data, including the specific values of the shape features (such as Hu moment values), and store them in a data format that can be used for subsequent analysis. Select topological data analysis methods (such as Persistent Homology) or graph theory methods to analyze the connection relationships between tissue regions. Construct an adjacency matrix based on the geometric shape data to represent the connection relationships between regions. Topological analysis process: According to the geometric shapes and positions of each region, determine their topological connection relationships. By calculating the contact situation between every two regions and the distance between them, construct a topological feature map of tissue connections. If region A and region B are in contact, mark the corresponding connection in the adjacency matrix. Record the component topological connection features of each region, including the region numbers of the connections, connection types (such as adjacent, overlapping), and their corresponding distance information. This information will provide the necessary basic data for subsequent 3D reconstruction. Select a suitable 3D reconstruction method, usually using voxel reconstruction methods or mesh-based reconstruction methods (such as the Marching Cubes algorithm). These methods can generate 3D models based on the 2D tissue structure data. Construct a 3D model according to the component topological connection features of each region. Utilize the geometric shape features and topological connection information that have been extracted to construct a 3D voxel map. For each tissue region, determine its volume and shape in 3D space, and smooth the model surface through an interpolation algorithm (such as cubic spline interpolation).Store the generated three-dimensional tissue structure model in a standard three-dimensional file format (such as STL or OBJ), and record the relevant parameters of the model (such as volume, surface area, etc.). This model will provide important information for subsequent analysis and visualization.

[0091] In this embodiment, the specific steps of step S5 are as follows:

[0092] Step S51: Identify the change trend of each marker's temporal behavior characteristics at different time points to generate the change trend characteristics of each marker.

[0093] Step S52: Mine the dynamic cell expression situation of the change trend characteristics of each marker to generate a cell expression trend spectrum.

[0094] Step S53: Analyze the dynamic evolution law of the cell expression trend spectrum to obtain the dynamic evolution law of each marker region.

[0095] Step S54: Predict the organizational structure evolution situation of each region's three-dimensional tissue structure model based on the dynamic evolution law of each marker region, so as to construct an organizational structure situation prediction model for each region.

[0096] In this embodiment, select a reasonable time interval (for example, collect images every 24 hours or 48 hours) to ensure that sufficient data points are captured during the change process. Resolution: The resolution of each image is set to 1024x1024 pixels to ensure that the details and spatial distribution of the markers in the tissue can be accurately captured. Marker concentration: According to the experimental design, the concentration of the marker needs to be kept constant or changed appropriately to ensure the accuracy of the change trend. Obtain immunohistochemical image data at multiple time points from the experiment. The acquisition time points are T0, T1, T2,..., Tn, and the time interval is 12 hours, 24 hours or 48 hours. Set the acquisition frequency according to the dynamic characteristics of the research object. Each time point involves multiple sample images. Preprocess the images. First, use a Gaussian filter (sigma = 1.5) to remove the high-frequency noise in the images to ensure the smoothness of the images. Then apply histogram equalization to improve the image contrast to make it easier to identify the markers in the images. Use a convolutional neural network (CNN) or ResNet (deep residual network) to extract the spatial characteristics of each marker at different time points. These networks can automatically extract the position information, morphological characteristics, etc. of the cell or tissue markers from the images through multiple convolutional operations.

[0097] The features extracted at each time point include information such as the concentration, shape, distribution, and location of the biomarker. For the change trend of the biomarker, time series analysis methods, such as dynamic time warping (DTW) and autoregressive moving average model (ARMA), are used to model and compare the expression intensity of the biomarker at each time point, so as to identify its change trend. Through time series analysis, the features of each biomarker at each time point are compared to obtain the change trend features of each biomarker. The expression intensity of the biomarker gradually increases in the initial stage and gradually decreases after reaching the peak, and this trend will be extracted as features such as the change rate, peak time point, and change curve type. The change trend features can be dimensionally reduced by principal component analysis (PCA) or factor analysis method for the features at multiple time points to obtain more representative trend features. Select an appropriate time window (such as 48 hours or 72 hours) to ensure that the whole picture of the cell cycle or tissue development process can be captured. Use a batch size of 32 or 64, set the learning rate to 0.001 when training the LSTM network, and use the Adam optimizer for parameter update. Take the change trend features of the biomarker obtained from step S51 as input data and continue with deep feature extraction. The data includes features such as the concentration, location, and morphology of the biomarker at each time point. Use long short-term memory network (LSTM) or bidirectional LSTM to model the time series data of the biomarker. The LSTM network is particularly suitable for processing time series data and can effectively capture the change trend of biomarker expression over time. The memory units in the LSTM network can maintain long-term dependencies, enabling the model to accurately predict the dynamic behavior of the biomarker during the learning process. Combine with a convolutional neural network (CNN), extract the spatial features in the cell image through multiple layers of convolution and pooling operations, and then combine the time series features with the spatial features to generate the dynamic cell expression trend of each biomarker. Use wavelet transform or Fourier transform to perform frequency domain analysis on the time series signal of the biomarker to reveal the periodic changes of the biomarker. According to the expression profile features of the biomarker, construct a cell expression trend spectrum, which represents the dynamic pattern of cell behavior at different time points. The cell expression trend spectrum contains indicators of biological processes such as proliferation pattern, differentiation pattern, and death pattern.

[0098] The proliferation trend of cells is manifested as a gradual increase in expression intensity, and the differentiation trend is manifested as a decrease in expression level. Set the modeling time window in the experiment, 72 hours or 96 hours, to ensure the full capture of cell behavior. The number of samples collected should be at least 10 - 20 to ensure that the analysis results are statistically significant. Model the cell expression trend spectrum through dynamic system modeling, such as using state - space models or Markov processes. These methods can reveal the temporal variation law of biomarker expression and further infer the biological processes of cells or tissues. Use regression analysis (such as linear regression, polynomial regression) or machine - learning algorithms (such as random forest, support vector machine) to further analyze the evolution law of cell behavior, explore the mutual relationship and temporal influence between different biomarkers. Compare the biomarker data in different regions, extract its dynamic evolution law, such as the proliferation rate, differentiation pattern, interaction, etc. of biomarkers in tissues. Through these laws, reveal the dynamic change trend of tissues or cells at different stages. For cancer cells, the dynamic evolution law reveals the malignant progression of cells, such as the rapid proliferation, migration, and invasion characteristics of cancer cells.

[0099] Set the predicted time window to 48 hours, 72 hours, or 96 hours to ensure the capture of key changes in tissue evolution. To ensure the accuracy of the model, at least 30 biomarker data from different regions need to be collected and analyzed in combination with the tissue structure. The dynamic evolution law obtained from step S53 will be used to guide the establishment of a three - dimensional tissue structure model. According to the spatial distribution of biomarkers at different time points, generate a three - dimensional tissue structure model for each region through a three - dimensional image reconstruction algorithm (such as volume rendering algorithm). These models can reflect the distribution of cells or tissues in three - dimensional space. Based on the evolution law obtained from step S53, use machine - learning methods (such as support vector regression (SVR), random forest regression) to predict the changes in the three - dimensional structure model. By analyzing the dynamic changes of biomarkers and combining their spatial distribution, predict the evolution trend of the tissue at future time points. Biomarkers in some regions show strong proliferation signals, and the model will predict that the cells in these regions will further expand to form new tissue structures. On the contrary, regions with weakened biomarker expression will experience tissue atrophy or cell death. According to the prediction results of the evolution trend, construct an organizational structure trend prediction model for each region to provide a reference for subsequent tissue progression and disease analysis.

[0100] In this embodiment, the specific steps of step S6 are as follows:

[0101] Step S61: Conduct personalized quantitative diagnosis according to the organizational structure trend prediction model of each region to generate a diagnostic analysis result;

[0102] Step S62: Conduct a pathological prediction for the future period of the diagnostic analysis result to generate a pathological prediction curve;

[0103] Step S63: Comprehensively evaluate the pathological prediction curve and the cell expression trend spectrum for tissue structure to obtain a comprehensive tissue structure evaluation report.

[0104] In this embodiment, the organizational structure trend prediction models obtained from step S54 are used. These models are constructed based on the dynamic evolution laws of each marker region. The models provide prediction results of cell behaviors (proliferation, differentiation, death, etc.) at different time points for each tissue region. Through these models, we can predict the changes in the organizational structure at future time points. The input data includes the three-dimensional structure model of the region, the temporal behavior characteristics, spatial distribution characteristics of the markers, and the predicted organizational evolution trend. For each region, personalized quantitative diagnosis is performed by combining the patient's pathological data, clinical manifestations, and genomic data. Using a quantitative calculation model (such as a regression model or neural network), a personalized diagnosis score is generated based on the output generated by the organizational structure trend prediction model. The support vector machine (SVM) or random forest algorithm is used to quantitatively evaluate the tissue state of each region, evaluate the expression pattern of the markers and the dynamic evolution of cell behaviors, and generate a diagnostic analysis result. The analysis result will include the proliferation rate of each region, the degree of cell differentiation, the marker concentration in the lesion area, etc. According to the results of the personalized quantitative analysis, a specific diagnostic analysis report is generated. The report includes information such as the cell behavior status of different regions, the predicted disease development stage (such as tumor progression, tissue damage, etc.), and the comparative analysis with healthy tissues. Visualization tools, such as heat maps, 3D visualization graphs, bar graphs, etc., are used to visually display the pathological states of different regions. Through these visualization means, it helps clinicians intuitively understand the pathological state and development trend of the tissue. At least 10 - 15 tissue data from different regions are collected to ensure the accuracy of the diagnosis. The model learning rate is set to 0.001, the batch size is 32, and the Adam optimizer is used for training to ensure the robustness and generalization ability of the model. The diagnostic analysis results obtained in step S61 are used, which include the organizational structure trend prediction models of each region, the change trends of the markers, cell behavior characteristics, etc. Specifically, the current state, change rate, and future change direction of each region are obtained. For pathological prediction, other external information such as the patient's clinical data and gene data is also required, and these information will further refine the prediction results of the model. Based on the organizational structure trend prediction model of each region, time series prediction methods, such as LSTM (long short-term memory network) or GRU (gated recurrent unit), are used to perform pathological prediction for future time periods. These methods are good at processing time series data and can predict the change trends of cells in the future for a period of time from the current cell behaviors. By training the model, using the current tissue behavior characteristics and historical data, the tissue pathological state at a future time period (such as 72 hours later, 96 hours later, etc.) is predicted. The tumor tissue will continue to proliferate, and there is no significant change in the healthy tissue. The prediction results will reflect the state differences between tumor development and healthy regions. According to the output of the model, a pathological prediction curve is constructed, which represents the tissue change trends of different regions in the future time period.The curve shows the acceleration phase of cell proliferation, the stable phase of differentiation, the period of tissue decline, etc. Each curve corresponds to a region and shows the pathological changes in that region over a future time period. The cell expression trend spectrum describes the change pattern of markers, while the pathological prediction curve describes the changes in future pathological states. The combination of these two provides a more comprehensive understanding of the tissue structure. After data fusion, it will provide the dynamic changes at different time points in each region, the concentration of markers, the cell state, the tissue evolution process, etc. An integrated evaluation model (such as the weighted scoring method, fuzzy logic reasoning method, etc.) is used to combine the pathological prediction curve and the cell expression trend spectrum to comprehensively evaluate the health status and pathological changes of the tissue. The evaluation indicators include the expression changes of markers, tissue hyperplasia, necrosis, fibrosis, etc. The evaluation model can give a comprehensive health score for each region based on the temporal change trends of different markers. If the expression of markers in the tumor region increases and the pathological prediction curve shows accelerated proliferation, the integrated evaluation model will conclude that the disease deterioration risk in this region is relatively high. Based on the integrated evaluation results, a detailed comprehensive evaluation report of the tissue structure is generated. The report will include the health scores of each region, the change trends of cell behaviors, the predicted pathological changes, and the future evolution trends of different regions. Through visualization charts (such as 3D charts, heat maps, change trend curves, etc.), the report will display the evolution process of the tissue structure. The evaluation report for each region not only includes the current state but also provides early warnings of future pathological development trends.

[0105] In this embodiment, a multiple immunohistochemistry image processing and analysis system is provided for performing the multiple immunohistochemistry image processing and analysis method as described above, including:

[0106] An image processing module for acquiring multiple immunohistochemistry images at multiple time points and performing time-series serialization fitting to obtain an immunohistochemistry image sequence;

[0107] A visual recognition module for performing deep visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemistry image sequence to construct an image frame sequence for each marker;

[0108] A cell feature analysis module for calculating the change in cell spatial distance and mining the spatial expression correlation based on the image frame sequence of each marker to generate the temporal behavior feature of each marker;

[0109] An organizational structure analysis module for performing frame-by-frame organizational structure analysis on the image frame sequence of each marker to obtain a three-dimensional organizational structure model for each region;

[0110] A situation prediction module for predicting the organizational structure evolution situation of the three-dimensional organizational structure model of each region based on the temporal behavior feature of each marker, thereby constructing an organizational structure situation prediction model for each region;

[0111] A quantitative diagnosis module, which is used to perform personalized quantitative diagnosis according to the organizational structure situation prediction model of each region, and conduct a comprehensive evaluation of the organizational structure to obtain a comprehensive evaluation report of the organizational structure.

[0112] By acquiring immunohistochemical images at different time points and subjecting them to chronological processing, the present invention can capture the dynamic changes of cells or tissues at different times. This is crucial for studying time-dependent changes in biological processes, especially the dynamic evolution of processes such as cell growth, proliferation, and migration. Through the fitting process of the image sequence, the influence brought by experimental errors can be reduced, and a more accurate time series of immunohistochemical images can be obtained, making subsequent analysis more accurate and reliable. Using deep learning technology for semantic recognition of images can effectively identify and label important information and target regions (such as cell and tissue regions, etc.) in each image, so as to accurately extract the features of multiple markers and reduce human errors. Frame-by-frame segmentation and reconstruction can refine the spatial features and morphology of each marker, ensuring that image details are not lost, and laying a solid foundation for subsequent spatial analysis and structure recognition. The image frame sequence of each marker helps to individually track and analyze the spatio-temporal behavior of different markers, ensuring that the analysis between different markers does not interfere with each other and improving the diversity and accuracy of multiple immunohistochemical image analysis. By calculating the change in the spatial distance between cells or markers, the movement trajectories, aggregation and diffusion processes of cells in tissues, and their interactions can be revealed. This is very important for understanding cell behaviors (such as migration, differentiation, etc.). By analyzing the spatial expression relationship of markers, the laws of the interaction and co-variation of different markers in tissues can be deeply understood. This analysis helps to reveal the cooperative relationship between biomarker and their roles in pathological or biological processes. The chronological behavior characteristics of markers can help researchers observe the dynamic patterns of markers changing over time, and combine the behavior patterns of cells to infer their functions in tissues and their potential roles in diseases. Frame-by-frame analysis can provide fine tissue structure information at each time point, enabling researchers to deeply observe the spatial distribution and morphological changes of different markers at each moment. Through three-dimensional reconstruction, the distribution and structural characteristics of markers in three-dimensional space in immunohistochemical images can be comprehensively displayed, overcoming the limitations brought by two-dimensional image analysis. The three-dimensional model can more intuitively reveal the complex spatial layout of cells and tissues, providing strong support for further analysis. Through the three-dimensional structure model, the spatial structure of tissues can be more accurately classified, calibrated, and analyzed, helping to better understand the structural changes and biological mechanisms in diseased tissues. Through the combined analysis of chronological behavior characteristics, the evolution trends of each marker and tissue region can be predicted, revealing the change trends that occur in tissues at different time nodes (such as tumor growth, changes in immune responses, etc.). This provides a powerful tool for studying the progression of diseases or the dynamic changes of cells / tissues. By establishing a situation prediction model of tissue structure, the future change trends can be more scientifically predicted, and then early prediction and intervention of diseases can be carried out to optimize the analysis plan. This prediction ability is especially applicable to fields such as oncology and immunology and has important clinical value.Based on the aforementioned timing behavior characteristics and three-dimensional structure model, personalized analysis is carried out according to the specific situation of each patient.

[0113] Therefore, in any regard, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Thus, all changes falling within the meaning and scope of the equivalent elements of the application documents are intended to be encompassed within the present invention.

[0114] As described above, these are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather will conform to the broadest scope consistent with the principles and novel features invented herein.

Claims

1. A method for processing and analyzing multi - immunohistochemistry image, characterized in that, It includes the following steps: Step S1: Obtain multiplex immunohistochemistry images at multiple time points and perform temporal sequence fitting to obtain an immunohistochemistry image sequence; Step S2: Perform deep visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemistry image sequence to construct an image frame sequence for each biomarker; Step S3: Calculate the change in cell spatial distance and mine the spatial expression correlation based on the image frame sequence of each biomarker to generate the temporal behavior characteristics of each biomarker; Step S4: Perform frame-by-frame tissue structure analysis on the image frame sequence of each biomarker to obtain a three-dimensional tissue structure model for each region; Step S5: Predict the tissue structure evolution trend of the three-dimensional tissue structure model for each region based on the temporal behavior characteristics of each biomarker, thereby constructing a tissue structure trend prediction model for each region; Step S6: Perform personalized quantitative diagnosis according to the tissue structure trend prediction model for each region and conduct comprehensive tissue structure evaluation to obtain a comprehensive tissue structure evaluation report.

2. The multi-immunohistochemical image processing and analysis method according to claim 1, wherein The specific steps of Step S1 are: Step S11: Obtain multiplex immunohistochemistry images at multiple time points; Step S12: Optimize the global brightness of the multiplex immunohistochemistry images to obtain a brightness-enhanced image; Step S13: Perform adaptive local background elimination on the brightness-enhanced image to obtain a locally background-corrected image; Step S14: Perform dynamic filtering and noise reduction and temporal sequence fitting based on the locally background-corrected image to obtain an immunohistochemistry image sequence.

3. The multiple immunohistochemical image processing and analysis method according to claim 2, wherein The specific steps of Step S13 are: Perform non-target biomarker background recognition on the brightness-enhanced image and mark the image background area; Perform regional texture recognition on the image background area to generate background area texture features; Estimate the color features of the image background area; Perform low-dimensional visual expression quantization based on the background area texture features and the color features to extract the low-dimensional visual expression vector of the background area; Perform adaptive local background elimination on the brightness-enhanced image based on the low-dimensional visual expression vector of the background area to obtain a locally background-corrected image.

4. The multi-immunohistochemical image processing and analysis method according to claim 2, wherein The specific steps of Step S14 are: Perform pixel-by-pixel detection on the locally background-corrected image to identify abnormal image noise points; Perform noise point distribution analysis on the abnormal image noise points to generate image noise point distribution data; Perform dynamic filtering and noise reduction based on the image noise point distribution data in the image to construct a filtered and noise-reduced image; Traverse all images to obtain filtered and noise-reduced images at multiple time points; Extract the timestamps of the filtered and noise-reduced images at the multiple time points; Perform temporal sequence fitting on the filtered and noise-reduced images based on the timestamps to obtain an immunohistochemistry image sequence.

5. The multiple immunohistochemical image processing and analysis method according to claim 1, characterized in that, The specific steps of Step S2 are: Step S21: Perform deep visual semantic recognition on the immunohistochemistry image sequence to obtain multiple image biomarkers; Step S22: Perform sample type analysis on the multiple image biomarkers to obtain the type characteristics of each biomarker; Step S23: Adjust the dynamic segmentation parameters based on the type characteristics of each biomarker to obtain the segmentation parameters of each biomarker; Step S24: Perform adaptive image frame segmentation on multiple image biomarkers according to the segmentation parameters of each biomarker to obtain the bounding box of each biomarker; Step S25: Perform frame-by-frame segmentation and reconstruction on the immunohistochemistry image sequence according to the bounding box of each biomarker to construct the image frame sequence of each biomarker.

6. The multi-immunohistochemical image processing and analysis method according to claim 1, wherein The specific steps of Step S3 are as follows: Step S31: Perform quantitative analysis of the relative relationship between cells on the image frame sequence of each biomarker to generate the cell relative relationship data of each image frame; Step S32: Calculate the change in cell spatial distance over multiple time periods according to the cell relative relationship data of each image frame to obtain the multi-period change data of cell spatial distance; Step S33: Identify the cell spatial layout based on the image frame sequence of each biomarker and extract the cell spatial layout features; Step S34: Identify the dynamic behavior between cells based on the cell spatial layout features and detect the dynamic behavior data between cells in each image frame; Step S35: Mine the spatial expression correlation of the dynamic behavior data between cells in each image frame to obtain the cell spatial expression correlation; Step S36: Evolve the temporal behavior characteristics of each biomarker one by one based on the multi-period change data of cell spatial distance and the cell spatial expression correlation to generate the temporal behavior characteristics of each biomarker.

7. The multi-immunohistochemical image processing and analysis method according to claim 1, wherein The specific steps of Step S4 are as follows: Step S41: Analyze the tissue structure frame by frame on the image frame sequence of each biomarker to obtain the tissue structure data of each biomarker region; Step S42: Perform in-depth geometric morphology recognition on the tissue structure data of each biomarker region to generate the tissue morphology characteristics of each region; Step S43: Perform tissue topology connection analysis according to the tissue structure data of each biomarker region to generate the component topology connection characteristics of each region; Step S44: Perform three-dimensional tissue topology reconstruction on the tissue morphology characteristics of each region based on the component topology connection characteristics of each region to obtain the three-dimensional tissue structure model of each region.

8. The method for processing and analyzing multi-immunohistochemical images according to claim 1, wherein The specific steps of Step S5 are as follows: Step S51: Identify the change trend at different time points of the temporal behavior characteristics of each biomarker to generate the change trend characteristics of each biomarker; Step S52: Mine the dynamic cell expression trend of the change trend characteristics of each biomarker to generate the cell expression trend spectrum; Step S53: Analyze the dynamic evolution law of the cell expression trend spectrum to obtain the dynamic evolution law of each biomarker region; Step S54: Predict the organizational structure evolution trend of the three-dimensional tissue structure model of each region based on the dynamic evolution law of each biomarker region, thereby constructing the organizational structure trend prediction model of each region.

9. The multi-immunohistochemistry image processing and analysis method according to claim 1, characterized in that The specific steps of Step S6 are as follows: Step S61: Perform personalized quantitative diagnosis according to the organizational structure trend prediction model of each region to generate the diagnostic analysis result; Step S62: Perform pathological prediction for future time periods on the diagnostic analysis result to generate the pathological prediction curve; Step S63: Conduct a comprehensive assessment of the pathological prediction curve and the cell expression trend spectrum for tissue structure to obtain a comprehensive tissue structure assessment report.

10. A multiple immunohistochemistry image processing and analysis system, characterized in that, For implementing the multi-immunohistochemistry image processing and analysis method as claimed in claim 1, including: An image processing module, configured to obtain multi-immunohistochemistry images at multiple time points and perform time-series serialization fitting to obtain an immunohistochemistry image sequence; A visual recognition module, configured to perform deep visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemistry image sequence to construct an image frame sequence for each marker; A cell feature analysis module, configured to calculate the change in cell spatial distance and mine the spatial expression correlation based on the image frame sequence of each marker to generate the time-series behavior feature of each marker; A tissue structure analysis module, configured to perform frame-by-frame tissue structure analysis on the image frame sequence of each marker to obtain a three-dimensional tissue structure model for each region; A situation prediction module, configured to predict the tissue structure evolution situation of the three-dimensional tissue structure model for each region based on the time-series behavior feature of each marker, thereby constructing a tissue structure situation prediction model for each region; A quantitative diagnosis module, configured to perform personalized quantitative diagnosis based on the tissue structure situation prediction model for each region and conduct a comprehensive assessment of the tissue structure to obtain a comprehensive tissue structure assessment report.

Images