Multi-immunohistochemical image processing and analyzing method and system

Abstract

The invention relates to the field of image processing, in particular to a multi-immunohistochemical image processing and analyzing method and system. The method comprises the following steps: acquiring multiple immunohistochemical images at a plurality of time points, and performing sequential serialization fitting to obtain an immunohistochemical image sequence; performing depth visual semantic recognition and frame-by-frame segmentation reconstruction on the immunohistochemical image sequence, and constructing an image frame sequence of each marker; performing cell spatial distance change calculation and spatial expression correlation mining according to the image frame sequence of each marker to generate a time sequence behavior feature of each marker; and carrying out frame-by-frame organization structure analysis on the image frame sequence of each marker to obtain a three-dimensional organization structure model of each region. According to the method, accurate and efficient tissue structure evaluation is realized, and a personalized quantitative diagnosis result is provided.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a multiple immunohistochemical image processing and analysis method and system. Background Art

[0002] Multiplex Immunohistochemistry (mIHC) is an advanced analytical method widely used in the biomedical field. It provides richer biological information than traditional single marker immunohistochemistry by simultaneously detecting multiple biomarkers in tissue samples. In cancer research, immunology, pathology and other biomedical fields, mIHC technology has become an important tool for studying complex diseases, revealing cell heterogeneity and immune microenvironment due to its high throughput and multi-target advantages.

[0003] However, with the continuous improvement of research needs and the increase in sample complexity, traditional multiplex immunohistochemistry image processing and analysis methods face many challenges. Traditional methods often rely on manual or semi-automated analysis methods, which are easily affected by operator experience, sample quality and analysis tool limitations, resulting in low analysis efficiency and inaccurate results. In addition, with the increase in the number of markers, the cross-interference and background noise of different marker signals in the image become more prominent, making image processing and analysis more complex and difficult.

[0004] Most current analysis methods use techniques based on image segmentation. Although they can achieve the separation and quantification of markers to a certain extent, they still cannot effectively solve the problems caused by overlapping signals, background noise and image quality differences between different markers in complex samples. At the same time, image analysis requires a huge amount of calculations, requiring fast processing speed and high accuracy. In the process of data processing and analysis, the flexibility and intelligence of traditional methods are still insufficient, especially when faced with large-scale data sets, the analysis efficiency and accuracy are often not guaranteed. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a multiple immunohistochemical image processing and analysis method and system to solve at least one of the above technical problems.

[0006] To achieve the above object, the present invention provides a multiplex immunohistochemical image processing and analysis method, comprising the following steps: Step S1: acquiring multiple immunohistochemical images at multiple time points, and performing time series fitting to obtain an immunohistochemical image sequence; Step S2: Perform deep visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemical image sequence to construct an image frame sequence for each marker; Step S3: Calculate the cell space distance change and mine the spatial expression correlation according to the image frame sequence of each marker to generate the temporal behavior characteristics of each marker; Step S4: performing frame-by-frame tissue structure analysis on the image frame sequence of each marker to obtain a three-dimensional tissue structure model of each region; Step S5: predicting the organizational structure evolution situation of the three-dimensional organizational structure model of each region according to the temporal behavior characteristics of each marker, thereby constructing an organizational structure situation prediction model for each region; Step S6: Perform personalized quantitative diagnosis based on the organizational structure situation prediction model of each region, and conduct a comprehensive evaluation of the organizational structure to obtain a comprehensive evaluation report of the organizational structure.

[0007] The present invention can capture the changes of cells and tissues at different time points by acquiring immunohistochemical images at multiple time points, thereby providing a more dynamic perspective. The fitting of this time-series image sequence provides a basis for subsequent time-series analysis, so that the processing of immunohistochemical images is no longer static, but can deeply analyze important information such as cell behavior and dynamic changes of markers. Time-series processing helps to identify the evolution law of cells or tissues in the time dimension, thereby providing more accurate data for subsequent analysis. Through deep visual semantic recognition, the features of different regions and markers in the image are automatically identified, and the relevant cell or tissue structures are accurately segmented and extracted. This not only improves the accuracy and efficiency of the analysis, but also ensures the fine distinction of each marker at different time points through frame-by-frame segmentation and reconstruction, so that subsequent analysis can more accurately track the time-series changes of markers in tissues. By constructing an image frame sequence for each marker, the state of cells or tissues is tracked in detail at different time points, providing key data for further spatial and behavioral analysis. The changes in the spatial layout of cells are accurately quantified, and the dynamic changes of cells or tissues on the timeline are analyzed. By calculating the change in cell spatial distance and its correlation with the spatial expression of other markers, we can gain a deep understanding of the interaction, positional relationship and dynamic behavior between cells. This analysis method reveals the activity characteristics of cells and tissues at different time points, such as changes in cell migration, division or other biological processes, thereby providing valuable information for disease progression, drug effects, etc. By analyzing the tissue structure frame by frame, three-dimensional structural information is extracted from the two-dimensional image sequence to form a more intuitive and accurate tissue structure model. This step can deeply explore the three-dimensional spatial distribution of each marker in different regions, thereby providing a reliable basis for subsequent spatial analysis and model construction. The construction of a three-dimensional structural model makes the spatial relationship of cells or tissues no longer limited to a plane, and can provide a more comprehensive perspective, thereby better reflecting the complexity of the tissue. By combining the temporal behavior characteristics with the three-dimensional structural model, the evolutionary trend of the tissue structure is predicted. This not only helps to understand the dynamic changes of markers, but also predicts the changing trend of cells or tissues at future time points. This predictive ability is particularly suitable for disease process analysis, such as the expansion of cancer cells, changes in the tumor microenvironment, etc. By establishing a tissue structure situation prediction model, it can provide a scientific basis for early diagnosis, drug development, disease prevention, etc. The previous analysis results are integrated and a personalized quantitative diagnostic report is generated through a comprehensive evaluation of the tissue structure. This allows the analysis results of each sample to be accurately evaluated based on its unique structural and behavioral characteristics, providing highly personalized information for clinical or scientific research. The comprehensive evaluation report can help doctors or researchers understand the status of the sample more intuitively.

[0008] In this specification, a multiple immunohistochemical image processing and analysis system is provided, which is used to execute the multiple immunohistochemical image processing and analysis method as described above, comprising: An image processing module is used to obtain multiple immunohistochemical images at multiple time points and perform time series fitting to obtain an immunohistochemical image sequence; The visual recognition module is used to perform deep visual semantic recognition and frame-by-frame segmentation and reconstruction of the immunohistochemistry image sequence to construct an image frame sequence for each marker; The cell feature analysis module is used to calculate the cell spatial distance change and mine the spatial expression correlation according to the image frame sequence of each marker, so as to generate the temporal behavior characteristics of each marker; The tissue structure analysis module is used to analyze the tissue structure of each marker image frame sequence frame by frame to obtain a three-dimensional tissue structure model of each area; A situation prediction module is used to predict the organizational structure evolution situation of the three-dimensional organizational structure model of each region according to the temporal behavior characteristics of each marker, thereby constructing an organizational structure situation prediction model for each region; The quantitative diagnosis module is used to perform personalized quantitative diagnosis based on the organizational structure situation prediction model of each area, and conduct a comprehensive evaluation of the organizational structure to obtain a comprehensive evaluation report of the organizational structure.

[0009] The present invention can capture the dynamic changes of cells or tissues at different times by acquiring immunohistochemical images at different time points and processing them in a time series. This is crucial for studying the time-dependent changes in biological processes, especially the dynamic evolution of processes such as cell growth, proliferation, and migration. By fitting the image sequence, the impact of experimental errors can be reduced, and a more accurate immunohistochemical image time series can be obtained, making subsequent analysis more accurate and reliable. Deep learning technology is used to perform semantic recognition of images, effectively identify and annotate important information and target areas (such as cells, tissue areas, etc.) in each image, so that the features of multiple markers can be accurately extracted to reduce human errors. Frame-by-frame segmentation and reconstruction can refine the spatial features and morphology of each marker, ensure that image details are not lost, and lay a solid foundation for subsequent spatial analysis and structural recognition. The image frame sequence of each marker helps to track and analyze the spatiotemporal behavior of different markers separately, ensure that the analysis between different markers does not interfere with each other, and improve the diversity and accuracy of multiple immunohistochemical image analysis. By calculating the changes in the spatial distance between cells or markers, the movement trajectory, aggregation and diffusion process of cells in tissues, and their interactions are revealed. This is very important for understanding cell behaviors (such as migration, differentiation, etc.). By analyzing the spatial expression relationship of markers, we can gain a deeper understanding of the interaction and coordinated changes of different markers in tissues. This analysis helps to reveal the cooperative relationship between biomarkers and their role in pathological or biological processes. The temporal behavior characteristics of markers can help researchers observe the dynamic patterns of marker changes over time, and combine the behavior patterns of cells to infer their functions in tissues and their potential role in diseases. Frame-by-frame analysis can provide detailed tissue structure information at each time point, allowing researchers to deeply observe the spatial distribution and morphological changes of different markers at each moment. Through three-dimensional reconstruction, the distribution and structural characteristics of markers in immunohistochemical images in three-dimensional space are fully displayed, overcoming the limitations of two-dimensional image analysis. The three-dimensional model can more intuitively reveal the complex spatial layout of cells and tissues, providing strong support for further analysis. Through the three-dimensional structural model, the spatial structure of the tissue can be more accurately classified, calibrated and analyzed, which helps to better understand the structural changes and biological mechanisms in diseased tissues. By combining and analyzing the characteristics of temporal behavior, the evolution trend of each marker and tissue region can be predicted, revealing the changing trends of tissues at different time points (such as tumor growth, changes in immune response, etc.). This provides a powerful tool for studying the progression of diseases or the dynamic changes of cells / tissues. By establishing a situation prediction model for tissue structure, future trends can be foreseen more scientifically, and early prediction and intervention of diseases can be carried out. This prediction capability is particularly applicable to fields such as oncology and immunology, and has important clinical value.Based on the aforementioned temporal behavioral characteristics and three-dimensional structural model, personalized analysis is performed according to the specific situation of each patient. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of the steps of a multiple immunohistochemical image processing and analysis method of the present invention; Figure 2 Detailed implementation flow chart of step S1; Figure 3 Detailed implementation flow chart of step S2; Figure 4 Detailed implementation flow chart of step S3. DETAILED DESCRIPTION

[0011] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0012] The present application example provides a multiple immunohistochemical image processing and analysis method and system. The execution subject of the multiple immunohistochemical image processing and analysis method and system includes but is not limited to: mechanical equipment, data processing platform, cloud server node, network upload device, etc. equipped with the system can be regarded as the general computing node of the present application, and the data processing platform includes but is not limited to: at least one of an audio image management system, an information management system, and a cloud data management system.

[0013] See also Figures 1 to 4 The present invention provides a multiple immunohistochemical image processing and analysis method, which comprises the following steps: Step S1: acquiring multiple immunohistochemical images at multiple time points, and performing time series fitting to obtain an immunohistochemical image sequence; Step S2: Perform deep visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemical image sequence to construct an image frame sequence for each marker; Step S3: Calculate the cell space distance change and mine the spatial expression correlation according to the image frame sequence of each marker to generate the temporal behavior characteristics of each marker; Step S4: performing frame-by-frame tissue structure analysis on the image frame sequence of each marker to obtain a three-dimensional tissue structure model of each region; Step S5: predicting the organizational structure evolution situation of the three-dimensional organizational structure model of each region according to the temporal behavior characteristics of each marker, thereby constructing an organizational structure situation prediction model for each region; Step S6: Perform personalized quantitative diagnosis based on the organizational structure situation prediction model of each region, and conduct a comprehensive evaluation of the organizational structure to obtain a comprehensive evaluation report of the organizational structure.

[0014] The present invention can capture the changes of cells and tissues at different time points by acquiring immunohistochemical images at multiple time points, thereby providing a more dynamic perspective. The fitting of this time-series image sequence provides a basis for subsequent time-series analysis, so that the processing of immunohistochemical images is no longer static, but can deeply analyze important information such as cell behavior and dynamic changes of markers. Time-series processing helps to identify the evolution law of cells or tissues in the time dimension, thereby providing more accurate data for subsequent analysis. Through deep visual semantic recognition, the features of different regions and markers in the image are automatically identified, and the relevant cell or tissue structures are accurately segmented and extracted. This not only improves the accuracy and efficiency of the analysis, but also ensures the fine distinction of each marker at different time points through frame-by-frame segmentation and reconstruction, so that subsequent analysis can more accurately track the time-series changes of markers in tissues. By constructing an image frame sequence for each marker, the state of cells or tissues is tracked in detail at different time points, providing key data for further spatial and behavioral analysis. The changes in the spatial layout of cells are accurately quantified, and the dynamic changes of cells or tissues on the timeline are analyzed. By calculating the change in cell spatial distance and its correlation with the spatial expression of other markers, we can gain a deep understanding of the interaction, positional relationship and dynamic behavior between cells. This analysis method reveals the activity characteristics of cells and tissues at different time points, such as changes in cell migration, division or other biological processes, thereby providing valuable information for disease progression, drug effects, etc. By analyzing the tissue structure frame by frame, three-dimensional structural information is extracted from the two-dimensional image sequence to form a more intuitive and accurate tissue structure model. This step can deeply explore the three-dimensional spatial distribution of each marker in different regions, thereby providing a reliable basis for subsequent spatial analysis and model construction. The construction of a three-dimensional structural model makes the spatial relationship of cells or tissues no longer limited to a plane, and can provide a more comprehensive perspective, thereby better reflecting the complexity of the tissue. By combining the temporal behavior characteristics with the three-dimensional structural model, the evolutionary trend of the tissue structure is predicted. This not only helps to understand the dynamic changes of markers, but also predicts the changing trend of cells or tissues at future time points. This predictive ability is particularly suitable for disease process analysis, such as the expansion of cancer cells, changes in the tumor microenvironment, etc. By establishing a tissue structure situation prediction model, it can provide a scientific basis for early diagnosis, drug development, disease prevention, etc. The previous analysis results are integrated and a personalized quantitative diagnostic report is generated through a comprehensive evaluation of the tissue structure. This allows the analysis results of each sample to be accurately evaluated based on its unique structural and behavioral characteristics, providing highly personalized information for clinical or scientific research. The comprehensive evaluation report can help doctors or researchers understand the status of the sample more intuitively, thereby formulating a more targeted analysis plan or scientific research strategy.

[0015] In the embodiment of the present invention, refer to Figure 1, is a schematic flow chart of the steps of a multiple immunohistochemical image processing and analysis method of the present invention. In this example, the steps of the multiple immunohistochemical image processing and analysis method include: Step S1: acquiring multiple immunohistochemical images at multiple time points, and performing time series fitting to obtain an immunohistochemical image sequence; In this embodiment, appropriate biological samples are collected to ensure that the samples are processed under the same experimental conditions to avoid batch effects. The samples should be approved by the ethics committee and meet biosafety and ethical requirements. The samples are processed using a standard immunohistochemical staining process. The general steps include dewaxing, antigen repair, blocking, antibody incubation and color development. In this step, appropriate primary and secondary antibodies are selected to ensure the specificity and sensitivity of the antibodies. The primary antibody dilution is 1:200 and the incubation time is 1 hour to ensure the uniformity and effectiveness of the staining. A high-resolution fluorescence microscope or optical microscope is used to ensure that the image information at the cellular level can be captured. The equipment should have image acquisition software that can set the appropriate exposure time and gain to obtain the best image quality. According to the nature of the tissue section and the fluorescent characteristics of the marker, adjust the exposure time (such as 100ms), gain (such as set to 50%) and magnification (such as 40x) of the microscope. Multiple image acquisition is performed at different time points to ensure that the image quality at each time point is consistent. Image acquisition is performed on the same tissue section at a set time point (such as 0 hours, 6 hours, 12 hours, 24 hours, etc.). Ensure that images at each time point are acquired under the same lighting and background conditions to reduce the impact of environmental changes on the results. Preprocess the acquired images, including denoising, contrast enhancement, and image registration. Use image processing software (such as ImageJ or FIJI) to process the images and ensure that the images at each time point can be spatially aligned. Use feature point matching or mutual information-based registration algorithms to register images at different time points. Ensure that images of the same cell at different time points correspond to each other in preparation for time series analysis. Use the SIFT feature point matching algorithm for registration, achieving a matching accuracy of 95%. Organize the registered images into a time series in chronological order. Use programming tools (such as Python or MATLAB) to write scripts to integrate the image data at each time point into a time series data structure. The images at each time point should contain corresponding metadata, such as timestamps, sample IDs, etc., for subsequent analysis. Verify the integrated time series to ensure that the data at each time point is complete and usable. Check the alignment quality and image clarity to ensure that no important information is lost. Store the final immunohistochemistry image series in a standard data format (such as TIFF or HDF5) to facilitate subsequent analysis and access.

[0016] Step S2: Perform deep visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemical image sequence to construct an image frame sequence for each marker; In this embodiment, representative images are selected for annotation. When constructing a data set, ensure the diversity of samples, covering different time points, sample types and markers. These samples should have different cell morphology and tissue structure so that the model can learn rich features. Use professional image annotation tools (such as Label box or VGG ImageAnnotator) to annotate the images. The annotation content includes cell area, tissue area and specific marker area. Usually performed by biological experts to ensure the accuracy of annotation. The annotated data set is divided into 70% training set, 20% validation set and 10% test set to ensure the generalization ability of the model. The training set is used for model training, the validation set is used to adjust the hyperparameters, and the test set is used to finally evaluate the model performance. Select suitable deep learning models, such as U-Net, MaskR-CNN or DeepLabV3+, which perform well in image segmentation and semantic recognition tasks. For the complex morphology of cells and tissues, U-Net can effectively capture detailed information due to its symmetrical structure and jump connection. Use deep learning frameworks such as TensorFlow or Py Torch for model training. Set hyperparameters, including learning rate (usually set to 0.001), batch size (e.g., 16), and number of training rounds (e.g., 50 rounds). Use data augmentation techniques (e.g., rotation, scaling, and flipping) to improve the robustness of the model. Evaluate model performance on the validation set, using metrics such as intersection over union (IoU) and Dice coefficient to measure segmentation results. Adjust the model based on the evaluation results, such as changing the learning rate or increasing the number of training rounds, to improve segmentation accuracy. Use the trained deep learning model to segment the immunohistochemistry image sequence frame by frame. Input each frame of the image into the model to obtain the segmentation mask for each cell and tissue. To ensure that the size of the input image meets the requirements of the model, it is usually necessary to scale it (e.g., adjust it to 256x256 pixels). Post-process the segmentation mask to remove noise and small artifacts. Use morphological operations (e.g., dilation and erosion) to smooth the segmentation results, eliminate small artifacts, and ensure the connectivity of the segmented regions. At this point, ensure the integrity and accuracy of each cell region. Reassemble the segmentation results of each frame to generate a sequence of image frames for each marker. Each frame should contain the location information of the cell and the distribution of the marker. A statistical data table containing cell number, cell area, and cell morphological characteristics was generated for each time point.

[0017] Step S3: Calculate the cell space distance change and mine the spatial expression correlation according to the image frame sequence of each marker to generate the temporal behavior characteristics of each marker; In this embodiment, the time point of analysis is determined, for example, the image frame sequence of 0 hours, 6 hours, 12 hours and 24 hours is selected in each time series image to ensure that there is enough data for analysis at each time point. The Euclidean distance is used as the calculation method of the distance between cells. For each pair of cells, the distance between their centroids is calculated, and the frame-by-frame distance calculation: the image frame sequence at each time point is traversed, the distance between cells is calculated, and the result is stored in a matrix. In this way, for each time point, the distance matrix between cells will record the spatial distance between all cell pairs. If there are 5 cells at a certain time point, a 5x5 distance matrix is ​​generated, and each element in the matrix represents the distance between the cell pairs. The intercellular distance results at each time point are recorded as structured data, including the ID, distance value and time information of the cell pair. This data will provide a basis for the subsequent generation of behavioral features. The Pearson correlation coefficient or the Spearman rank correlation coefficient is used to quantify the spatial expression correlation between cells. The selected correlation analysis method should be able to handle continuous variables and rank variables to adapt to different cell expression data. Correlation analysis is performed on the cell expression data (such as the expression intensity of a specific marker) at each time point. First, a cell expression matrix is ​​constructed, in which rows represent cells and columns represent different time points or different markers. Then, the correlation between cell expressions is calculated. The calculated spatial expression correlation results are recorded as structured data, including the correlation coefficient of each pair of cells and the corresponding time point information. These results will provide an important basis for the subsequent generation of temporal behavior features. Temporal behavior features are extracted based on the distance changes between cells and the spatial expression correlation. The mean, standard deviation, and rate of change of the distance between cells at each time point are calculated, while considering the change in the correlation coefficient. The extracted temporal behavior features are integrated into a comprehensive data table, which includes information such as time points, cell pair IDs, spatial distance features, and expression correlation features. This facilitates subsequent analysis and visualization. The final generated temporal behavior features are recorded in a structured data format (such as CSV or JSON) for subsequent analysis and visualization. This data will provide a basis for subsequent dynamic behavior analysis.

[0018] Step S4: performing frame-by-frame tissue structure analysis on the image frame sequence of each marker to obtain a three-dimensional tissue structure model of each region; In this embodiment, each image frame is preprocessed, including denoising, contrast enhancement and standardization. Image processing software (such as ImageJ or OpenCV) is used for processing. Gaussian filtering is used for denoising, and histogram equalization can be used for contrast enhancement. Ensure that the images of each frame are at the same scale and contrast to improve the accuracy of subsequent analysis. Select a suitable image segmentation algorithm, such as U-Net or K-means clustering, to extract tissue structure. U-Net performs well in biomedical image segmentation and can effectively capture tissue features. Set the input size of the model (such as 256x256 pixels) and use the previously trained model to segment each frame. The segmentation results are manually verified to ensure that the segmentation of cells and tissue areas is accurate. Adjust if necessary to enhance the accuracy of the model. If there is a wrongly segmented cell area in the segmentation result, it is corrected by manually adjusting the segmentation mask. Feature extraction is performed on each segmented area, including area, perimeter, shape moment, centroid coordinates, etc. Use image processing tools (such as OpenCV) to extract these features and store the results in a structured data format. Record the area (in pixels) and perimeter (calculated using the formula) of each region. Select an appropriate 3D reconstruction algorithm, such as Marching Cubes or voxel reconstruction methods. These methods can generate 3D models based on frame-by-frame analysis results. Voxel reconstruction methods provide better flexibility for the complexity of tissue structures. Convert the segmentation results of each frame into a 3D voxel model. Use 3D mesh reconstruction technology to convert the segmentation results of each cell region into 3D voxels. If the Marching Cubes algorithm is used, it is necessary to first construct a 3D scalar field, and then generate a 3D surface model through the algorithm. Refine the generated 3D model, remove redundant voxels and noise, and smooth the model surface. Use 3D graphics processing software (such as MeshLab or Blender) to process to ensure the smoothness and realism of the model. Save the final 3D tissue structure model in a standard 3D file format (such as STL, OBJ, or PLY) to ensure compatibility and ease of subsequent analysis. Record relevant parameters of the model, such as volume, surface area, and cell density. Back up all generated 3D models and original data to ensure data security and traceability. Use database or file system for effective management to facilitate subsequent analysis and research.

[0019] Step S5: predicting the organizational structure evolution situation of the three-dimensional organizational structure model of each region according to the temporal behavior characteristics of each marker, thereby constructing an organizational structure situation prediction model for each region; In this embodiment, the extracted temporal behavior features are standardized to eliminate the dimensional differences between different features. The Z-score standardization method is usually used to adjust the mean of each feature to 0 and the standard deviation to 1. It helps to improve the training efficiency and accuracy of the model. According to the characteristics and goals of the data, select a suitable prediction model. Common choices include time series analysis models (such as ARIMA), machine learning models (such as random forests, support vector machines), and deep learning models (such as LSTM). For biological data with temporal characteristics, LSTM is widely used due to its excellent time series modeling capabilities. The standardized temporal behavior feature data is divided into a training set and a test set, usually with a ratio of 80% training and 20% testing. The training set is used for model training, and the test set is used to evaluate the generalization ability of the model. Use deep learning frameworks such as Keras or TensorFlow in Python to train the LSTM model. Set the hyperparameters of the model, including the input sequence length (select the first 5 time points as input), the number of hidden layer units (such as 64), and the learning rate (such as 0.001). Evaluate the model performance on the test set, using indicators such as mean square error (MSE) and coefficient of determination (R²) to measure the prediction effect. Adjust the model parameters based on the evaluation results, such as increasing the number of LSTM layers or units, to improve the accuracy of the model. Use the trained model to predict the future state of tissue structure. Input the feature data of the last few known time points, and the model will output the tissue structure features at the next time point. Predict the changes in cell density and its spatial distribution in the next 6 hours.

[0020] Step S6: Perform personalized quantitative diagnosis based on the organizational structure situation prediction model of each region, and conduct a comprehensive evaluation of the organizational structure to obtain a comprehensive evaluation report of the organizational structure.

[0021] In this embodiment, the prediction results of each region are standardized to eliminate the dimensional differences caused by different regions or markers. Z-score standardization is used to adjust the mean of each feature to 0 and the standard deviation to 1. It is helpful to improve the accuracy of subsequent quantitative diagnosis. According to the biological significance of tissue structure, a personalized quantitative diagnostic standard is constructed. Cell density thresholds, cell spacing thresholds, etc. are set to judge the health status of the tissue. Reference existing literature or clinical guidelines, and corresponding standards are formulated in combination with experimental data. Statistical analysis software (such as R or Python's Scipy library) is used to analyze the standardized prediction data. The mean and standard deviation of the cell density in each region are calculated and compared with the set threshold to make a diagnostic decision. If the cell density in a certain area is significantly higher than the normal range, it indicates a pathological state such as a tumor or inflammation. The quantitative diagnostic results of each region are organized into a structured format, including region ID, predicted value, diagnostic results (such as normal, abnormal, malignant) and clinical recommendations. Ensure that the information in the report is complete and easy to understand. Select a suitable comprehensive evaluation method, such as a weighted scoring method or a fuzzy logic system, to generate a comprehensive evaluation result based on different prediction features. These methods can comprehensively consider the influence of multiple factors to form a comprehensive evaluation result. Assign weights to each predictive feature based on its importance and clinical relevance. Cell density has a greater impact on tissue health than intercellular spacing, so a higher weight is set for cell density. Determine the weight value through expert discussion or literature research. Combine the predictive features of each region with the set weights and calculate the comprehensive evaluation score through weighted average or fuzzy logic model. Integrate the quantitative diagnostic results and comprehensive evaluation scores into a report with necessary charts and data support. Generate relevant charts using data visualization tools (such as Matplotlib or Tableau) to enhance the readability and intuitiveness of the report.

[0022] In this embodiment, refer to Figure 2 , is a flowchart of detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Step S11: acquiring multiple immunohistochemical images at multiple time points; Step S12: performing global brightness optimization on the multiple immunohistochemical images to obtain brightness enhanced images; Step S13: performing adaptive local background removal on the brightness enhanced image to obtain a local background corrected image; Step S14: performing dynamic filtering noise reduction and time series fitting according to the local background correction image to obtain an immunohistochemical image sequence.

[0023] In this embodiment, the microscope equipment and related reagents used are determined to ensure that the equipment can support the imaging of multiple immunohistochemical staining. Samples are prepared using a fluorescence microscope and a variety of specific antibodies. The samples need to be properly fixed and sliced ​​for subsequent imaging. The experiment is designed to image at different time points (e.g., 0 hours, 1 hour, 2 hours, 4 hours, and 8 hours). Select appropriate time intervals to capture the dynamic process of cell response and ensure that clear images can be obtained at each time point. At each time point, the sample is imaged using a fluorescence microscope to ensure that it is photographed under the same exposure conditions for subsequent processing. The light intensity, exposure time, and image resolution at each time point are recorded, and the resolution of each image is 1024x1024 pixels. The collected images are classified and stored by time point to ensure that each image has a clear mark for subsequent processing. Finally, images of 5 time points are obtained, and the sample images at each time point are 10MB. Select a suitable global brightness optimization algorithm, such as histogram equalization or adaptive histogram equalization (CLAHE). CLAHE can effectively enhance the local contrast of the image while avoiding artifacts caused by over-enhancement. The multiple immunohistochemistry images at each time point were optimized for global brightness. The processing steps included: calculating the histogram of the image and analyzing the brightness distribution. Applying the selected algorithm to adjust the brightness and enhance the visualization of the image. Comparing the images before and after processing to ensure that the brightness enhancement effect is obvious and the cell structure and markers are clearer. After optimization, the contrast of the image was improved by 30%, making the cell boundaries more obvious. Adaptive background removal techniques such as Gaussian filtering or median filtering were used. These methods can effectively remove background noise in the image while preserving the structural characteristics of the cell. Local background removal was performed on each brightness-enhanced image. The specific steps included: selecting an appropriate filter window size, which is usually adjusted according to the size of the cells in the image. The selected window size is 15x15 pixels. The image is filtered to generate a background image and the background is subtracted from the original image. The corrected image is checked to ensure that the cell outline is clear and the background noise is significantly reduced. The background noise is reduced by 50%, and the visualization of the cell structure is significantly improved. The images after local background correction are subjected to dynamic filtering and noise reduction. The specific steps include: setting the filter parameters, such as filter strength and window size, and the window size is 5x5 pixels. Apply the selected filtering algorithm to process the image and generate a denoised image. Serialize the denoised image in chronological order to construct time series data. Analyze with image processing software or dedicated analysis software to record the changes of cells over time. Store the generated immunohistochemical image sequence to ensure that the image at each time point can be accurately reflected in the sequence for subsequent analysis and comparison.

[0024] In this embodiment, the specific steps of step S13 are: Perform background recognition of non-target markers on the brightness enhanced image and mark the image background area; Performing regional texture recognition on the background area of ​​the image to generate background area texture features; Estimate the color characteristics of the background area of ​​the image; Quantifying the low-dimensional visual expression according to the texture features of the background area and the color features, thereby extracting the low-dimensional visual expression vector of the background area; Based on the low-dimensional visual expression vector of the background area, the brightness enhanced image is adaptively localized for background removal, thereby obtaining a local background corrected image.

[0025] In this embodiment, an image that has been subjected to brightness enhancement processing is obtained to ensure that the image contrast and brightness are suitable for subsequent analysis. This step includes using techniques such as histogram equalization or adaptive histogram equalization (CLAHE) to improve the brightness and contrast of the image, so that the distinction between the background and the target object is more obvious. Color and texture-based segmentation algorithms, such as K-means clustering or Grab Cut algorithm, are applied to identify the background area in the image. These algorithms distinguish the background and foreground by analyzing the color features of pixels and the relationship between adjacent pixels. In K-means clustering, the color space of the image (such as RGB or HSV) is clustered to determine the category of each pixel and mark it as background or foreground. The marked image will show the boundary of the background area, making subsequent processing more accurate. Gray level co-occurrence matrix (GLCM) or local binary pattern (LBP) is used to extract the texture features of the background area. These techniques can effectively capture the local texture information of the image, which is of great significance for identifying the details of the background. For the gray level co-occurrence matrix, the gray level matrix of the background area is first calculated, and then relevant features such as contrast, entropy, correlation, etc. are extracted. These features reflect the complexity and pattern of the background texture and can provide important data for subsequent analysis. The extracted texture features are stored in a feature vector for subsequent use. The extracted features include indicators such as contrast, correlation, and energy. Select HSV or Lab color space for statistical analysis of the color features of the background area. These spaces can better reflect the color characteristics of human visual perception than the traditional RGB space. The color value of each pixel in the background area is counted, and features such as mean, standard deviation, and color histogram are calculated. The calculated color mean is (180, 100, 150) and the standard deviation is (20, 15, 25), indicating that the colors in this area are concentrated in a specific range. The color features are recorded as a feature vector, ensuring that the mean and standard deviation of each channel are included to form a complete color feature description. Using dimensionality reduction techniques such as principal component analysis (PCA) or t-SNE, the texture features and color features of the background area are fused to extract a low-dimensional visual expression vector. The recorded texture features and color features are merged into a high-dimensional feature vector, and then PCA is applied for dimensionality reduction to extract the first few principal components as a low-dimensional visual expression vector. After PCA processing, the extracted low-dimensional vector is (0.75, -0.80). Select adaptive background modeling technology, such as Gaussian mixture model (GMM) or background subtraction algorithm, to perform background removal based on the extracted low-dimensional visual expression vector. In the incoming brightness-enhanced image, local background removal is performed on each pixel according to the extracted low-dimensional visual expression vector. By calculating the difference between the current pixel and the background model, the target object is highlighted. Generate a local background correction image and record the image quality after elimination and the clarity of the target object. The final image shows the details of the target object and the background is effectively removed.

[0026] In this embodiment, the specific steps of step S14 are: Perform pixel-by-pixel detection on the local background correction image to identify abnormal image noise; Perform noise distribution analysis on abnormal image noise points to generate noise distribution data in the image; Dynamic filtering and denoising is performed according to the noise distribution data in the image, thereby constructing a filtered denoised image; Traverse all images to obtain filtered denoised images at multiple time points; Extracting timestamps of the filtered and denoised images at the multiple time points; The filtered and denoised images are subjected to time series fitting based on the timestamp to obtain an immunohistochemical image sequence.

[0027] In this embodiment, local contrast analysis and threshold segmentation methods are used to perform pixel-by-pixel detection and identify abnormal noise points in the image. This is achieved by calculating the brightness difference between each pixel and its neighboring pixels, setting a threshold, and marking it as a noise point when the brightness of a pixel exceeds the threshold. The image after local background correction is traversed to calculate the difference between the brightness value of each pixel and the average value of the 8 neighboring pixels around it. The threshold is set to 20, and if the difference exceeds the threshold, the pixel is marked as a noise point. All detected abnormal noise points are recorded in a list, and their coordinate positions and brightness values ​​are marked for subsequent analysis. The pixel brightness value recorded as (45,67) for the noise point position is 255. The distribution analysis of the identified noise points is performed, and the distribution of the noise points in the image is counted, including the number of noise points, position distribution density, and brightness distribution. This is achieved by using histogram analysis and heat map generation technology. The number and distribution density of all noise points are counted. A total of 150 noise points are detected in an image, distributed in different areas. By generating a heat map, the concentrated area of ​​the noise points is intuitively displayed. The noise distribution data is recorded in a data structure, including the total number of noise points, coordinate distribution, and brightness range, for subsequent filtering. Dynamic median filtering or Gaussian filtering algorithms are used to reduce noise. Median filtering can effectively remove salt and pepper noise, while Gaussian filtering can retain edge information while smoothing the image. According to the noise distribution data, the filtering algorithm is applied to each noise point position in the image. For each noise point, the median or weighted average of the pixels within a certain radius around it is calculated, and the value of the noise pixel is replaced with this value. The selected filtering window is 3x3. Generate the filtered denoised image and record the comparison results with the original image, including the noise reduction amount and image quality indicators (such as PSNR, SSIM, etc.) to evaluate the filtering effect. The PSNR value of the denoised image is increased to 35dB. Traverse the images at all time points and repeat the above denoising process to ensure that the images at each time point go through the same denoising process. Record the processing status of each image for subsequent analysis. Record the denoised image at each time point and save it as time series data. The image denoising result recorded at the first time point is "denoised image 1", and the result at the second time point is "denoised image 2". Extract the timestamp information from the metadata of each image to ensure that the time series information of all images is complete. The timestamp should include the specific time of image acquisition for subsequent serialization fitting. Store the extracted timestamps in a data structure in a unified format to ensure that the timestamps correspond to the corresponding filtered denoised images. The recorded timestamp format is "YYYY-MM-DDHH:MM:SS". Use algorithms such as linear regression or spline interpolation to perform time series serialization fitting on the extracted timestamps and the corresponding filtered denoised images. This can help establish the relationship between time and image changes. Input the timestamps and image data into the fitting model to generate the fitting results of the time series image.Through linear regression analysis, the generated fitting curve can represent the image change trend at different time points. Record the fitting results and save the fitted image sequence as a new image sequence for subsequent analysis and visualization. Save as "immunohistochemistry image sequence".

[0028] In this embodiment, refer to Figure 3 , is a flowchart of detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include: Step S21: performing deep visual semantic recognition on the immunohistochemistry image sequence to obtain multiple image biomarkers; Step S22: performing sample type analysis on multiple image biomarkers to obtain type characteristics of each marker; Step S23: dynamically adjusting the segmentation parameters based on the type characteristics of each marker to obtain the segmentation parameters of each marker; Step S24: performing adaptive image frame segmentation on the multiple image biomarkers according to the segmentation parameters of each marker to obtain a bounding box of each marker; Step S25: segmenting and reconstructing the immunohistochemical image sequence frame by frame according to the boundary frame of each marker, and constructing an image frame sequence of each marker.

[0029] In this embodiment, a deep learning model suitable for image biomarker recognition is selected, such as a convolutional neural network (CNN) architecture, especially based on ResNet or U-Net architecture, which is suitable for medical image segmentation and recognition tasks. The immunohistochemistry image sequence is standardized, including image scaling, normalization and enhancement, to improve the accuracy of model recognition. The image is uniformly adjusted to 256x256 pixels and rotated and flipped appropriately. The preprocessed image sequence is input into the trained deep learning model for biomarker recognition. The model output includes the biomarkers identified in each image and their probability values. The recognition results are stored in a database, and the biomarker information of each image is recorded, such as the name, location and confidence of the marker. The identified markers include "nucleus", "cytoplasm" and the like. According to the characteristics of the biomarker and the recognition results, the type of each marker is analyzed by feature engineering methods. Cluster analysis (such as K-means) or support vector machine (SVM) can be used for type classification. Shape features, color features, texture features, etc. are extracted from the identified markers. The description vector of the marker is obtained by calculating the area, perimeter, mean color and texture metrics (such as LBP). The type features of each marker are stored in a structured form for subsequent analysis. The recorded features include "Type: Cell nucleus", "Area: 500 square pixels", and "Average color: (255, 200, 150)". Select a suitable image segmentation algorithm, such as thresholding, region growing, or a segmentation method based on a graphical model (such as Graph Cut), and configure the corresponding segmentation parameters according to the type features of the marker. Dynamically adjust the segmentation parameters according to the characteristics of the marker. For markers with larger areas, a higher threshold needs to be set to avoid mis-segmentation, while for small structures, the threshold needs to be reduced. Record the segmentation parameter settings of each marker, including threshold, structure element size, etc., for subsequent segmentation. Record "Cell nucleus segmentation parameters: threshold 0.5, structure element size 3". According to the segmentation parameters of each marker, perform adaptive box segmentation on the image. Use the previously set segmentation algorithm to process the image and generate the bounding box of the marker. Mark the bounding box of each biomarker in the image and record the coordinate information of the bounding box (upper left corner and lower right corner coordinates). Record "Nucleus bounding box: (x1, y1) = (30, 40), (x2, y2) = (80, 100)". For each frame in the immunohistochemistry image sequence, apply the generated bounding box for reconstruction. Generate an image frame sequence for each marker by extracting the image area within each bounding box. Store the image frame sequence for each marker as an independent image file or structured data for subsequent analysis and use. Record "Nucleus image frame sequence: image1.png, image2.png".

[0030] In this embodiment, refer to Figure 4, is a flowchart of detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include: Step S31: performing a quantitative analysis of the relative relationship between cells on the image frame sequence of each marker, and generating the relative relationship data of cells in each image frame; Step S32: Calculating the cell space distance change in multiple time periods according to the cell relative relationship data of each image frame to obtain the cell space distance change data in multiple time periods; Step S33: performing cell space layout recognition according to the image frame sequence of each marker, and extracting cell space layout features; Step S34: performing intercellular dynamic behavior recognition based on the cell spatial layout characteristics, and detecting intercellular dynamic behavior data of each image frame; Step S35: mining the spatial expression correlation of the dynamic behavior data between cells in each image frame to obtain the spatial expression correlation of cells; Step S36: Based on the multi-period variation data of cell space distance and the correlation of cell space expression, the temporal behavior characteristics of each marker are evolved to generate the temporal behavior characteristics of each marker.

[0031] In this embodiment, the image frame sequence of each marker is first processed to extract the cell information in each image frame. Through image segmentation technology, the outline of each cell is identified and its center of mass position is calculated. Classic image processing functions such as Canny edge detection or threshold segmentation are used to ensure that the edges of the cells are clearly distinguishable. At this point, the coordinates, area and outline information of each cell are stored for subsequent analysis. For each pair of cells, the relative position between them is calculated. Using the Euclidean distance formula, d = , where (x1, y1) and (x2, y2) are the centroid coordinates of the two cells. The calculated distances are recorded in a matrix, where each element in the matrix represents the distance between the cell pairs. The cell relative relationship data of each image frame is stored as structured data, including the distance and relative orientation information (left, right, top, bottom) between the cell pairs. Such a data structure facilitates subsequent analysis and visualization. Determine the use of time series analysis methods to evaluate the changes in the spatial distance of cells at different time points. The selected algorithm needs to be able to handle multidimensional data, usually using dynamic time warping (DTW) or simple distance change calculation. Analysis of changes in inter-cell distances: Calculate the relative distance of each cell pair at different time points. For each time point, extract the distance data between cells and calculate the relative change. Select a suitable clustering algorithm, such as DBSCAN or K-means, to analyze the spatial distribution of cells. The basis for selecting a clustering algorithm is the distribution nature of the cell layout: if the cells are unevenly distributed, select DBSCAN. Perform spatial layout analysis on the cells and extract layout features. First, use clustering analysis to identify the clustering areas of cells. The clustering result will assign a category label to each cell to identify the cluster to which it belongs. Then, the center point of the cluster, the degree of clustering (such as the number of cells in the cluster), and the distribution shape (such as elliptical shape, regularity) are calculated. The extracted cell spatial layout features are recorded as structured data, including characteristic data such as cluster center, number of clusters, and layout shape. These features will help analyze the spatial distribution and mutual relationship of cells. According to the movement trajectory and relative position relationship of the cells, the method of dynamic behavior analysis is selected. The interaction between cells is monitored using a dynamic monitoring method based on time series. Behavior detection implementation: Dynamic behavior detection is performed on cells in each image frame to record the movement and interaction between cells. The movement mode, contact and separation behavior of cells are monitored. The distance changes between cells are analyzed. When the relative distance of cells decreases to a certain threshold, it is recorded as "contact" behavior; when the relative distance increases, it is recorded as "separation" behavior. The dynamic behavior data between cells in each image frame are recorded as structured information, including the type, frequency and duration of behavior between cells. This information provides a basis for subsequent spatial expression correlation analysis. Select appropriate statistical methods, such as Pearson correlation coefficient or Spearman rank correlation coefficient, to explore the spatial expression correlation between cells. These methods can quantify the relationship between cell behavior and its spatial layout. Spatial expression correlation analysis: Analyze the expression correlation between cells based on the dynamic behavior data between cells. Identify the correlation between cell behavior and spatial layout by calculating the behavior data between different cell pairs. Calculate the correlation between the frequency of mutual contact between cell A and cell B over a period of time and their spatial distance. Record the obtained cell spatial expression correlation data in the database to ensure that the mutual influence between cells can be tracked and provide support for subsequent analysis.Select appropriate time series analysis methods, such as dynamic time warping (DTW) or ARIMA models, to analyze the evolution of cell behavior characteristics. These methods effectively capture patterns and trends in time series data. Based on the multi-period change data of cell spatial distance and the correlation of cell spatial expression, the evolution analysis of time series behavior characteristics is performed marker by marker. By establishing a time series of each cell behavior, the evolution trend and pattern of cell behavior are identified. Analyze the changes in the contact frequency between cells over time and explore its relationship with changes in cell layout. Record the time series behavior characteristics of each marker as time series data to ensure that the data can be used for subsequent analysis and research. This data structure will provide support for understanding the dynamic relationship between cells and the evolution of behavior.

[0032] In this embodiment, step S4 includes the following steps: Step S41: performing frame-by-frame tissue structure analysis on the image frame sequence of each marker to obtain tissue structure data of each marker region; Step S42: performing geometric morphology depth recognition on the tissue structure data of each marker region to generate tissue morphology features of each region; Step S43: performing tissue topology connection analysis based on the tissue structure data of each marker region to generate component topology connection features of each region; Step S44: performing three-dimensional tissue topology reconstruction on the tissue morphology characteristics of each region based on the component topological connection characteristics of each region to obtain a three-dimensional tissue structure model of each region.

[0033] In this embodiment, the image frame sequence of each marker is sorted to ensure that the order and timestamp of each frame image are consistent. The image should undergo necessary preprocessing, such as denoising, contrast enhancement and standardized size (such as adjusting to 512x512 pixels) to improve the accuracy of subsequent analysis. Image segmentation technology is used to extract tissue structure using image threshold segmentation or image segmentation model based on deep learning (such as U-Net). By analyzing the pixel information of each frame, the boundary and shape of the tissue structure are identified. Specifically, the Otsu method is used for threshold selection to calculate the optimal threshold for each frame image so as to effectively separate the tissue area from the background. The tissue structure data of each marker area obtained by analysis is stored as structured data, including information such as the shape, position and area of ​​the area. The coordinates, area (such as 500 square pixels) and shape features (such as contour, boundary length) of each tissue area are recorded. A suitable geometric morphological analysis method is selected, and shape descriptors (such as Hu moments and Zernike moments) are usually used to extract tissue morphological features. These descriptors can effectively capture the shape features of the tissue and generate high-dimensional feature vectors. The geometric features of each extracted tissue area are calculated. Calculate the rotational invariance features of each tissue region and extract the first seven Hu moment values, which can reflect the shape characteristics of the tissue. By writing a feature extraction algorithm, loop through each region and summarize the calculation results. Record the generated tissue morphological features of each region as structured data, including the specific values ​​of the shape features (such as Hu moment values), and store them in a data format that can be used for subsequent analysis. Select a topological data analysis method (such as Persistent Homology) or a graph theory method to analyze the connection relationship between tissue regions. Construct an adjacency matrix based on geometric morphological data to represent the connection relationship between each region. Topological analysis process: Determine the topological connection relationship between each region based on the geometric shape and position of each region. Construct a topological feature map of tissue connection by calculating the contact between each two regions and the distance between them. If region A and region B are in contact, mark the corresponding connection in the adjacency matrix. Record the component topological connection features of each region, including the connected region number, connection type (such as adjacent, overlapping) and its corresponding distance information. This information will provide the necessary basic data for subsequent three-dimensional reconstruction. Select a suitable three-dimensional reconstruction method, usually using voxel reconstruction method or grid-based reconstruction method (such as MarchingCubes algorithm). These methods can generate a 3D model based on 2D tissue structure data. The 3D model is constructed based on the topological connectivity features of the components in each region. A 3D voxel map is constructed using the extracted geometric features and topological connectivity information. For each tissue region, its volume and shape in 3D space are determined, and the model surface is smoothed using an interpolation algorithm (such as cubic spline interpolation).The generated 3D tissue structure model is stored in a standard 3D file format (such as STL or OBJ), and the relevant parameters of the model (such as volume, surface area, etc.) are recorded. This model will provide important information for subsequent analysis and visualization.

[0034] In this embodiment, the specific steps of step S5 are: Step S51: identifying the changing trend of the temporal behavior characteristics of each marker at different time points to generate the changing trend characteristics of each marker; Step S52: performing dynamic cell expression situation mining on the change trend characteristics of each marker to generate a cell expression trend spectrum; Step S53: Analyze the dynamic evolution law of the cell expression trend spectrum, so as to obtain the dynamic evolution law of each marker region; Step S54: Based on the dynamic evolution law of each marker region, the three-dimensional tissue structure model of each region is predicted to have a tissue structure evolution state, thereby constructing a tissue structure state prediction model for each region.

[0035] In this embodiment, a reasonable time interval is selected (for example, an image is collected every 24 hours or 48 hours) to ensure that enough data points are captured during the change process. Resolution: The resolution of each image is set to 1024x1024 pixels to ensure that the details and spatial distribution of the marker in the tissue can be accurately captured. Marker concentration: According to the experimental design, the concentration of the marker needs to be kept constant or appropriately changed to ensure the accuracy of the change trend. Immunohistochemical image data at multiple time points are obtained from the experiment. The acquisition time points are T0, T1, T2,..., Tn, and the time interval is 12 hours, 24 hours or 48 hours. The acquisition frequency is set according to the dynamic characteristics of the research object. Each time point involves multiple sample images. The image is preprocessed. First, a Gaussian filter (sigma=1.5) is used to remove high-frequency noise in the image to ensure the smoothness of the image. Then histogram equalization is applied to increase the image contrast so that the markers in the image can be more easily identified. Convolutional neural network (CNN) or ResNet (deep residual network) is used to extract the spatial features of each marker at different time points. Through multi-layer convolution operations, these networks can automatically extract the location information, morphological features, etc. of cell or tissue markers from images.

[0036] The features extracted at each time point include information such as the concentration, shape, distribution and location of the marker. For the change trend of the marker, time series analysis methods such as dynamic time warping (DTW) and autoregressive moving average model (ARMA) are used to model and compare the expression intensity of the marker at each time point to identify its change trend. Through time series analysis, the features of each marker at each time point are compared to obtain the change trend characteristics of each marker. The expression intensity of the marker gradually increases in the early stage and gradually decreases after reaching the peak. This trend will be extracted as features such as change rate, peak time point and change curve type. The change trend characteristics can be reduced by the principal component analysis (PCA) or factor analysis method to obtain more representative trend characteristics. Select an appropriate time window (such as 48 hours or 72 hours) to ensure that the full picture of the cell cycle or tissue development process can be captured. Use a batch size of 32 or 64, set the learning rate to 0.001 when training the LSTM network, and use the Adam optimizer for parameter update. The change trend characteristics of the marker obtained in step S51 are used as input data to continue deep feature extraction. The data includes characteristics such as marker concentration, position and morphology at each time point. The time series data of the marker is modeled using a long short-term memory network (LSTM) or a bidirectional LSTM. The LSTM network is particularly suitable for processing time series data and effectively captures the changing trend of marker expression over time. The memory units in the LSTM network can maintain long-term dependencies, so that the model can accurately predict the dynamic behavior of the marker during the learning process. Combined with a convolutional neural network (CNN), the spatial features in the cell image are extracted through multi-layer convolution and pooling operations, and then the time series features are combined with the spatial features to generate the dynamic cell expression status of each marker. The time series signal of the marker is analyzed in the frequency domain using wavelet transform or Fourier transform to reveal the periodic changes of the marker. According to the expression spectrum characteristics of the marker, a cell expression trend spectrum is constructed, which represents the dynamic pattern of cell behavior at different time points. The cell expression trend spectrum contains indications of biological processes such as proliferation mode, differentiation mode, and death mode.

[0037] The proliferation trend of cells is manifested as a gradual increase in expression intensity, and the differentiation trend is manifested as a decrease in expression level. Set the modeling time window in the experiment to 72 hours or 96 hours to ensure that the cell behavior is captured throughout the process. The number of samples collected should be at least 10-20 to ensure that the analysis results are statistically significant. Model the cell expression trend spectrum through dynamic system modeling, such as using state space models or Markov processes. These methods can reveal the temporal changes in marker expression and further infer the biological processes of cells or tissues. Use regression analysis (such as linear regression, polynomial regression) or machine learning algorithms (such as random forests, support vector machines) to further analyze the evolution of cell behavior and explore the relationship and temporal influence between different markers. Compare the marker data in different regions to extract their dynamic evolution laws, such as the proliferation rate, differentiation pattern, and interaction of markers in tissues. Through these laws, the dynamic change trends of tissues or cells at different stages are revealed. For cancer cells, the dynamic evolution laws reveal the malignant progression of cells, such as the rapid proliferation, migration, and invasion of cancer cells.

[0038] The prediction time window is set to 48 hours, 72 hours or 96 hours to ensure that the key changes in tissue evolution are captured. In order to ensure the accuracy of the model, at least 30 marker data from different regions need to be collected and analyzed in combination with tissue structure. The dynamic evolution law obtained from step S53 will be used to guide the establishment of a three-dimensional tissue structure model. According to the spatial distribution of markers at different time points, a three-dimensional tissue structure model of each region is generated by a three-dimensional image reconstruction algorithm (such as a volume rendering algorithm). These models can reflect the distribution of cells or tissues in three-dimensional space. Based on the evolution law obtained from step S53, a machine learning method (such as support vector regression (SVR) and random forest regression) is used to predict the changes in the three-dimensional structure model. By analyzing the dynamic changes of markers and combining their spatial distribution, the evolution trend of the tissue at future time points is predicted. The markers in some regions show strong proliferation signals, and the model will predict that the cells in these regions will further expand and form new tissue structures. On the contrary, areas with weakened marker expression will experience tissue atrophy or cell death. According to the prediction results of the evolution trend, a tissue structure trend prediction model for each region is constructed to provide a reference for subsequent tissue progression and disease analysis.

[0039] In this embodiment, the specific steps of step S6 are: Step S61: Performing personalized quantitative diagnosis based on the organizational structure situation prediction model of each region to generate diagnostic analysis results; Step S62: performing pathological prediction for a future period on the diagnostic analysis result to generate a pathological prediction curve; Step S63: Perform a comprehensive tissue structure evaluation on the pathological prediction curve and the cell expression trend spectrum to obtain a comprehensive tissue structure evaluation report.

[0040] In this embodiment, the tissue structure status prediction models obtained from step S54 are used, and these models are constructed based on the dynamic evolution law of each marker area. The model provides prediction results of cell behavior (proliferation, differentiation, death, etc.) at different time points for each tissue area. Through these models, we can predict the changes of tissue structure at future time points. The input data includes the three-dimensional structural model of the region, the temporal behavior characteristics of the markers, the spatial distribution characteristics, and the predicted tissue evolution trend. For each region, personalized quantitative diagnosis is performed in combination with the patient's pathological data, clinical manifestations, and genomic data. A quantitative calculation model (such as a regression model or a neural network) is used to perform personalized diagnostic scoring based on the output generated by the tissue structure status prediction model. A support vector machine (SVM) or a random forest algorithm is used to quantitatively evaluate the tissue state of each region, evaluate the expression pattern of the markers and the dynamic evolution of cell behavior, and generate diagnostic analysis results. The analysis results will include the proliferation rate of each region, the degree of cell differentiation, the marker concentration in the lesion area, etc. Based on the results of the personalized quantitative analysis, a specific diagnostic analysis report is generated. The report includes information such as the cell behavior state of different regions, the predicted disease development stage (such as tumor progression, tissue damage, etc.), and comparative analysis with healthy tissues. Use visualization tools, such as heat maps, 3D visualizations, bar charts, etc., to visualize the pathological status of different regions. These visualization methods help clinicians intuitively understand the pathological status and development trend of tissues. Collect tissue data from at least 10-15 different regions to ensure the accuracy of diagnosis. Use a model learning rate of 0.001, a batch size of 32, and an optimizer trained with Adam to ensure the robustness and generalization of the model. Use the diagnostic analysis results obtained in step S61, which include the tissue structure status prediction model, marker change trend, cell behavior characteristics, etc. of each region. In particular, obtain the current state, change rate, and future change direction of each region. In order to make pathological predictions, other external information such as the patient's clinical data and genetic data is also required, which will further refine the prediction results of the model. Based on the tissue structure status prediction model of each region, use time series prediction methods, such as LSTM (long short-term memory network) or GRU (gated recurrent unit) to make pathological predictions for future time periods. These methods are good at processing time series data and can predict the change trend of cells in the future period from the current cell behavior. By training the model, using the current tissue behavior characteristics and historical data, the pathological state of the tissue in a certain period of time in the future (such as 72 hours later, 96 hours later, etc.) is predicted. Tumor tissue will continue to proliferate, and healthy tissue will not change significantly. The prediction results will reflect the difference in the state of tumor development and healthy areas. Based on the output of the model, a pathological prediction curve is constructed, which represents the trend of tissue changes in different areas in the future period.The curve shows the accelerated period of cell proliferation, the stable period of differentiation, the period of tissue decline, etc. Each curve corresponds to a region, showing the pathological changes in the region in the future period. The cell expression trend spectrum describes the change pattern of the marker, while the pathological prediction curve describes the change of the future pathological state. The combination of the two provides a more comprehensive understanding of the tissue structure. After data fusion, the dynamic changes of each region at different time points, the concentration of markers, the cell state, the tissue evolution process, etc. will be provided. A comprehensive evaluation model (such as weighted scoring method, fuzzy logic reasoning method, etc.) is used to combine the pathological prediction curve and the cell expression trend spectrum to comprehensively evaluate the health status and pathological changes of the tissue. The evaluation indicators include changes in marker expression, tissue hyperplasia, necrosis or fibrosis, etc. The evaluation model can give a comprehensive health score for each region based on the temporal change trend of different markers. If the marker expression in the tumor area increases and the pathological prediction curve shows accelerated proliferation, the comprehensive evaluation model will conclude that the risk of disease deterioration in the area is high. Based on the comprehensive evaluation results, a detailed comprehensive evaluation report of the tissue structure is generated. The report will include the health score of each area, the changing trend of cell behavior, the predicted pathological changes, and the future evolution of different areas. Through visual charts (such as 3D charts, heat maps, trend curves, etc.), the report will show the evolution of tissue structure. The evaluation report of each area not only includes the current status, but also provides early warning of future pathological development trends.

[0041] In this embodiment, a multiple immunohistochemical image processing and analysis system is provided, which is used to execute the multiple immunohistochemical image processing and analysis method as described above, including: An image processing module is used to obtain multiple immunohistochemical images at multiple time points and perform time series fitting to obtain an immunohistochemical image sequence; The visual recognition module is used to perform deep visual semantic recognition and frame-by-frame segmentation and reconstruction of the immunohistochemistry image sequence to construct an image frame sequence for each marker; The cell feature analysis module is used to calculate the cell spatial distance change and mine the spatial expression correlation according to the image frame sequence of each marker, so as to generate the temporal behavior characteristics of each marker; The tissue structure analysis module is used to analyze the tissue structure of each marker image frame sequence frame by frame to obtain a three-dimensional tissue structure model of each area; A situation prediction module is used to predict the organizational structure evolution situation of the three-dimensional organizational structure model of each region according to the temporal behavior characteristics of each marker, thereby constructing an organizational structure situation prediction model for each region; The quantitative diagnosis module is used to perform personalized quantitative diagnosis based on the organizational structure situation prediction model of each area, and conduct a comprehensive evaluation of the organizational structure to obtain a comprehensive evaluation report of the organizational structure.

[0042] The present invention can capture the dynamic changes of cells or tissues at different times by acquiring immunohistochemical images at different time points and processing them in a time series. This is crucial for studying the time-dependent changes in biological processes, especially the dynamic evolution of processes such as cell growth, proliferation, and migration. By fitting the image sequence, the impact of experimental errors can be reduced, and a more accurate immunohistochemical image time series can be obtained, making subsequent analysis more accurate and reliable. Deep learning technology is used to perform semantic recognition of images, effectively identify and annotate important information and target areas (such as cells, tissue areas, etc.) in each image, so that the features of multiple markers can be accurately extracted to reduce human errors. Frame-by-frame segmentation and reconstruction can refine the spatial features and morphology of each marker, ensure that image details are not lost, and lay a solid foundation for subsequent spatial analysis and structural recognition. The image frame sequence of each marker helps to track and analyze the spatiotemporal behavior of different markers separately, ensure that the analysis between different markers does not interfere with each other, and improve the diversity and accuracy of multiple immunohistochemical image analysis. By calculating the changes in the spatial distance between cells or markers, the movement trajectory, aggregation and diffusion process of cells in tissues, and their interactions are revealed. This is very important for understanding cell behaviors (such as migration, differentiation, etc.). By analyzing the spatial expression relationship of markers, we can gain a deeper understanding of the interaction and coordinated changes of different markers in tissues. This analysis helps to reveal the cooperative relationship between biomarkers and their role in pathological or biological processes. The temporal behavior characteristics of markers can help researchers observe the dynamic patterns of marker changes over time, and combine the behavior patterns of cells to infer their functions in tissues and their potential role in diseases. Frame-by-frame analysis can provide detailed tissue structure information at each time point, allowing researchers to deeply observe the spatial distribution and morphological changes of different markers at each moment. Through three-dimensional reconstruction, the distribution and structural characteristics of markers in immunohistochemical images in three-dimensional space are fully displayed, overcoming the limitations of two-dimensional image analysis. The three-dimensional model can more intuitively reveal the complex spatial layout of cells and tissues, providing strong support for further analysis. Through the three-dimensional structural model, the spatial structure of the tissue can be more accurately classified, calibrated and analyzed, which helps to better understand the structural changes and biological mechanisms in diseased tissues. By combining and analyzing the characteristics of temporal behavior, the evolution trend of each marker and tissue region can be predicted, revealing the changing trends of tissues at different time points (such as tumor growth, changes in immune response, etc.). This provides a powerful tool for studying the progression of diseases or the dynamic changes of cells / tissues. By establishing a situation prediction model for tissue structure, future trends can be foreseen more scientifically, and diseases can be predicted and intervened early, and analysis plans can be optimized. This predictive ability is particularly applicable to fields such as oncology and immunology, and has important clinical value.Based on the aforementioned temporal behavioral characteristics and three-dimensional structural model, personalized analysis is performed according to the specific situation of each patient.

[0043] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0044] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A multiplex immunohistochemical image processing and analysis method, characterized in that: The following steps are involved: Step S1: acquiring multiple immunohistochemical images at multiple time points, and performing time series fitting to obtain an immunohistochemical image sequence; Step S2: Perform deep visual semantic recognition and frame-by-frame segmentation and reconstruction on the immunohistochemical image sequence to construct an image frame sequence for each marker; Step S3: Calculate the cell space distance change and mine the spatial expression correlation according to the image frame sequence of each marker to generate the temporal behavior characteristics of each marker; Step S4: performing frame-by-frame tissue structure analysis on the image frame sequence of each marker to obtain a three-dimensional tissue structure model of each region; Step S5: predicting the organizational structure evolution situation of the three-dimensional organizational structure model of each region according to the temporal behavior characteristics of each marker, thereby constructing an organizational structure situation prediction model for each region; Step S6: Perform personalized quantitative diagnosis based on the organizational structure situation prediction model of each region, and conduct a comprehensive evaluation of the organizational structure to obtain a comprehensive evaluation report of the organizational structure.

2. The multiplex immunohistochemistry image processing and analysis method according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: acquiring multiple immunohistochemical images at multiple time points; Step S12: performing global brightness optimization on the multiple immunohistochemical images to obtain brightness enhanced images; Step S13: performing adaptive local background removal on the brightness enhanced image to obtain a local background corrected image; Step S14: performing dynamic filtering noise reduction and time series fitting according to the local background correction image to obtain an immunohistochemical image sequence.

3. The multiplex immunohistochemistry image processing and analysis method according to claim 2, characterized in that: The specific steps of step S13 are: Perform background recognition of non-target markers on the brightness enhanced image and mark the image background area; Performing regional texture recognition on the background area of ​​the image to generate background area texture features; Estimate the color characteristics of the background area of ​​the image; Quantifying the low-dimensional visual expression according to the texture features of the background area and the color features, thereby extracting the low-dimensional visual expression vector of the background area; Based on the low-dimensional visual expression vector of the background area, the brightness enhanced image is adaptively localized for background removal, thereby obtaining a local background corrected image.

4. The multiplex immunohistochemistry image processing and analysis method according to claim 2, characterized in that: The specific steps of step S14 are: Perform pixel-by-pixel detection on the local background correction image to identify abnormal image noise; Perform noise distribution analysis on abnormal image noise points to generate noise distribution data in the image; Dynamic filtering and denoising is performed according to the noise distribution data in the image, thereby constructing a filtered denoised image; Traverse all images to obtain filtered denoised images at multiple time points; Extracting timestamps of the filtered and denoised images at the multiple time points; The filtered and denoised images are subjected to time series fitting based on the timestamp to obtain an immunohistochemical image sequence.

5. The multiplex immunohistochemistry image processing and analysis method according to claim 1, characterized in that: The specific steps of step S2 are: Step S21: performing deep visual semantic recognition on the immunohistochemistry image sequence to obtain multiple image biomarkers; Step S22: performing sample type analysis on multiple image biomarkers to obtain type characteristics of each marker; Step S23: dynamically adjusting the segmentation parameters based on the type characteristics of each marker to obtain the segmentation parameters of each marker; Step S24: performing adaptive image frame segmentation on the multiple image biomarkers according to the segmentation parameters of each marker to obtain a bounding box of each marker; Step S25: segmenting and reconstructing the immunohistochemical image sequence frame by frame according to the boundary frame of each marker, and constructing an image frame sequence of each marker.

6. The multiplex immunohistochemistry image processing and analysis method according to claim 1, characterized in that: The specific steps of step S3 are: Step S31: performing a quantitative analysis of the relative relationship between cells on the image frame sequence of each marker, and generating the relative relationship data of cells in each image frame; Step S32: Calculating the cell space distance change in multiple time periods according to the cell relative relationship data of each image frame to obtain the cell space distance change data in multiple time periods; Step S33: performing cell space layout recognition according to the image frame sequence of each marker, and extracting cell space layout features; Step S34: performing intercellular dynamic behavior recognition based on the cell spatial layout characteristics, and detecting intercellular dynamic behavior data of each image frame; Step S35: mining the spatial expression correlation of the dynamic behavior data between cells in each image frame to obtain the spatial expression correlation of cells; Step S36: Based on the multi-period variation data of cell space distance and the correlation of cell space expression, the temporal behavior characteristics of each marker are evolved to generate the temporal behavior characteristics of each marker.

7. The multiplex immunohistochemistry image processing and analysis method according to claim 1, characterized in that: The specific steps of step S4 are: Step S41: performing frame-by-frame tissue structure analysis on the image frame sequence of each marker to obtain tissue structure data of each marker region; Step S42: performing geometric morphology depth recognition on the tissue structure data of each marker region to generate tissue morphology features of each region; Step S43: performing tissue topology connection analysis based on the tissue structure data of each marker region to generate component topology connection features of each region; Step S44: performing three-dimensional tissue topology reconstruction on the tissue morphology characteristics of each region based on the component topological connection characteristics of each region to obtain a three-dimensional tissue structure model of each region.

8. The multiplex immunohistochemistry image processing and analysis method according to claim 1, characterized in that: The specific steps of step S5 are: Step S51: identifying the changing trend of the temporal behavior characteristics of each marker at different time points to generate the changing trend characteristics of each marker; Step S52: performing dynamic cell expression situation mining on the change trend characteristics of each marker to generate a cell expression trend spectrum; Step S53: Analyze the dynamic evolution law of the cell expression trend spectrum, so as to obtain the dynamic evolution law of each marker region; Step S54: Based on the dynamic evolution law of each marker region, the three-dimensional tissue structure model of each region is predicted to have a tissue structure evolution state, thereby constructing a tissue structure state prediction model for each region.

9. The multiplex immunohistochemistry image processing and analysis method according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: Performing personalized quantitative diagnosis based on the organizational structure situation prediction model of each region to generate diagnostic analysis results; Step S62: performing pathological prediction for future time periods on the diagnostic analysis results to generate a pathological prediction curve; Step S63: Perform a comprehensive tissue structure evaluation on the pathological prediction curve and the cell expression trend spectrum to obtain a comprehensive tissue structure evaluation report.

10. A multiplex immunohistochemistry image processing and analysis system, characterized in that: Used to perform the multiple immunohistochemical image processing and analysis method as claimed in claim 1, comprising: An image processing module is used to obtain multiple immunohistochemical images at multiple time points and perform time series fitting to obtain an immunohistochemical image sequence; The visual recognition module is used to perform deep visual semantic recognition and frame-by-frame segmentation and reconstruction of the immunohistochemistry image sequence to construct an image frame sequence for each marker; The cell feature analysis module is used to calculate the cell spatial distance change and mine the spatial expression correlation according to the image frame sequence of each marker, so as to generate the temporal behavior characteristics of each marker; The tissue structure analysis module is used to analyze the tissue structure of each marker image frame sequence frame by frame to obtain a three-dimensional tissue structure model of each area; A situation prediction module is used to predict the organizational structure evolution situation of the three-dimensional organizational structure model of each region according to the temporal behavior characteristics of each marker, thereby constructing an organizational structure situation prediction model for each region; The quantitative diagnosis module is used to perform personalized quantitative diagnosis based on the organizational structure situation prediction model of each area, and conduct a comprehensive evaluation of the organizational structure to obtain a comprehensive evaluation report of the organizational structure.

Images