Method and system for constructing raman spectral dataset based on pathological image spatial registration

Abstract

The application relates to the technical field of medical image processing, and discloses a Raman spectrum data set construction method and system based on pathological image space registration, which acquires a macroscopic tissue image, an immunohistochemical image, a Raman measurable region and scale conversion information, completes tissue segmentation, contour extraction and target molecular marker positive mask extraction; constructs an initial space transformation according to the tissue contour, realizes common coordinate space mapping of multi-modal images and masks, screens a target deformation field based on a distance transformation graph non-rigid registration, corrects to obtain a corrected target molecular marker positive mask, fuses multiple masks to construct a common effective region, corrodes to obtain a safe sampling region, arranges Raman sampling points in combination with scale information, generates labels and confidence according to positive area proportion of a spot neighborhood, screens the Raman sampling points and outputs a Raman spectrum data set. The application provides technical support for Raman spectrum data set construction.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically, to a method and system for constructing Raman spectroscopy datasets based on spatial registration of pathological images. Background Technology

[0002] Raman spectroscopy can acquire molecular vibrational characteristics of substances such as proteins, lipids, and nucleic acids within biological tissues, and has significant application value in the field of histopathological analysis. A core prerequisite in constructing a Raman spectroscopy dataset with clearly defined pathological and molecular biological annotations is to achieve a precise spatial mapping between the Raman sampling locations on the surface of ex vivo tissues and the expression regions of target molecular markers within the immunohistochemical images of tissue sections, enabling each Raman spectrum to be associated with a corresponding molecular marker label.

[0003] However, gross images of ex vivo tissues and immunohistochemical images belong to different imaging modalities. These two types of images commonly suffer from practical problems such as inconsistencies in imaging scale, sample orientation shifts, tissue shrinkage due to sectioning, local stretching deformation, tissue edge defects, and differences in stained background. Existing registration schemes rely directly on the color and texture features of the original image for registration calculations, making them highly susceptible to interference from staining depth, lighting conditions, and imaging background, leading to registration failures. Methods such as manual visual calibration, single-point mapping of sample point center pixels, and single rigid transformations for establishing spatial correspondence also struggle to output stable and reliable point-to-point labels.

[0004] Furthermore, most existing non-rigid registration schemes rely solely on the overall overlap of tissues as the evaluation criterion for registration results, unilaterally pursuing high overlap indicators. This can easily lead to physically unreasonable transformation fields such as local image folding and excessive tissue deformation, thereby disrupting the real spatial correspondence between modalities.

[0005] In addition, Raman spectral signals originate from the physically sized laser spot coverage area of ​​the instrument, not individual pixels in the image. When the Raman sampling location is close to tissue edges, positioning holes, areas with section defects, or when local registration deviations exist, the laser spot can easily cover positive tissue, negative tissue, and the background outside the tissue simultaneously, ultimately causing spectral label mixing and distortion. To obtain a high-quality labeled dataset, it is necessary to comprehensively consider multiple constraints under a unified common coordinate space, including registration quality, molecular marker region transformation and transfer, the actual measurable range of Raman, sampling safety boundaries, and the effective coverage ratio of the laser spot.

[0006] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0007] To address the problems in related technologies, this invention proposes a method and system for constructing Raman spectroscopy datasets based on spatial registration of pathological images, in order to overcome the aforementioned technical problems existing in existing related technologies.

[0008] Therefore, the specific technical solution adopted by the present invention is as follows:

[0009] According to one aspect of the present invention, a method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images is provided, the method comprising: S1. Obtain gross tissue images, immunohistochemical images, Raman-measurable regions, and scale transformation information of the same ex vivo tissue sample; S2. Perform tissue segmentation and contour extraction on gross tissue images and immunohistochemical images respectively to obtain gross tissue mask, immunohistochemical tissue mask, gross tissue contour, and immunohistochemical tissue contour; extract the positive mask of target molecular markers based on the immunohistochemical image. S3. Determine the initial spatial transformation based on the gross tissue outline and immunohistochemical tissue outline, and map the immunohistochemical image, immunohistochemical tissue mask and target molecular marker positive mask to the common coordinate space according to the initial spatial transformation. S4. Convert the gross tissue mask and immunohistochemical tissue mask in the common coordinate space into distance transformation maps respectively. Perform non-rigid registration based on the distance transformation maps to generate multiple sets of candidate deformation fields. Evaluate the registration quality of each candidate deformation field and select the target deformation field. S5. Simultaneously transfer the initial spatial transformation and the target deformation field to the target molecular marker positive mask to obtain the modified target molecular marker positive mask; S6. Construct a common effective region based on the macroscopic tissue mask, the modified target molecular marker positive mask, and the Raman measurable region. Then, perform an etching operation on the common effective region according to the Raman effective spot size and registration error to obtain a safe sampling region. S7. Calculate the sampling step size using scale transformation information, generate Raman sampling points within the safe sampling area, and for each Raman sampling point, correct the positive area ratio of the positive mask of the target molecular marker within the neighborhood of the effective Raman spot size, generate molecular marker labels and label confidence, and filter sampling points to output the Raman spectroscopy dataset.

[0010] According to another aspect of the present invention, a Raman spectroscopy dataset construction system based on spatial registration of pathological images is also provided, the system comprising: The sample data acquisition module is used to acquire gross tissue images, immunohistochemical images, Raman-measurable regions, and scale transformation information of the same ex vivo tissue sample. The image segmentation mask extraction module is used to perform tissue segmentation and contour extraction on gross tissue images and immunohistochemical images respectively, to obtain gross tissue masks, immunohistochemical tissue masks, gross tissue contours and immunohistochemical tissue contours; and to extract positive masks for target molecular markers based on immunohistochemical images. The initial spatial transformation mapping module is used to determine the initial spatial transformation based on the gross tissue contour and immunohistochemical tissue contour, and to map the immunohistochemical image, immunohistochemical tissue mask and target molecular marker positive mask to the common coordinate space according to the initial spatial transformation. The non-rigid registration and screening module is used to convert the gross tissue mask and immunohistochemical tissue mask in the common coordinate space into distance transformation maps, respectively. Based on the distance transformation maps, non-rigid registration is performed to generate multiple sets of candidate deformation fields. The registration quality of each candidate deformation field is evaluated and the target deformation field is obtained through screening. The mask deformation correction module is used to synchronously transfer the initial spatial transformation and the target deformation field to the target molecular marker positive mask to obtain the corrected target molecular marker positive mask; The sampling region generation module is used to construct a common effective region based on the macroscopic tissue mask, the modified target molecular marker positive mask, and the Raman measurable region, and to perform an etching operation on the common effective region according to the Raman effective spot size and registration error to obtain a safe sampling region. The Raman spectroscopy dataset generation module is used to calculate the sampling step size using scale transformation information, generate Raman sampling points within a safe sampling area, and for each Raman sampling point, correct the positive area ratio of the positive mask of the target molecular marker within the neighborhood that matches the effective Raman spot size, generate molecular marker labels and label confidence, and filter sampling points to output the Raman spectroscopy dataset.

[0011] The beneficial effects of this invention are as follows: 1. This invention extracts tissue masks and tissue contours from gross images and immunohistochemical images of ex vivo tissues, and uses the contour shape that does not contain the original color and texture information as the basis for spatial registration. This allows the registration process to be mainly based on the position of the tissue boundary, the overall shape, and the distance relationship from the tissue interior to the boundary, thereby reducing the influence of cross-modal color and texture differences on spatial transformation estimation and improving the stability of the registration process between different imaging modalities.

[0012] 2. This invention converts the tissue mask into a distance transformation map and then performs non-rigid registration, so that each pixel inside the tissue can participate in the deformation field calculation through its distance to the nearest tissue boundary. At the same time, it preserves the continuous spatial information of the tissue outer boundary and the tissue interior, which can provide a more continuous optimization target for non-rigid registration. This is beneficial for correcting non-rigid spatial differences such as slice shrinkage, local stretching, and edge deformation, and enables the tissue region in the immunohistochemical image to be mapped to the common coordinate space more stably.

[0013] 3. For each candidate deformation field, this invention simultaneously calculates the Dice coefficient, bidirectional average nearest contour distance, bidirectional maximum nearest contour distance, normalized mutual information, and the proportion of pixels with Jacobian determinant less than zero. By jointly evaluating and screening deformation fields through multiple indicators, this invention avoids selecting registration results based on only a single indicator. This improves the overall correspondence of the tissue while reducing local folding, excessive stretching, and unreasonable deformation.

[0014] 4. This invention synchronously transmits the initial spatial transformation and target deformation field used for tissue registration to the positive mask of the target molecular marker in the same transformation order, so that the target molecular marker region always maintains a consistent spatial relationship with the corresponding tissue region, avoids spatial misalignment caused by separate registration, and improves the traceability and consistency of the spatial distribution mapping of the target molecular marker.

[0015] 5. This invention constructs a common effective region by the intersection of a gross tissue mask, an immunohistochemical tissue mask after being subjected to a target deformation field, and a Raman measurable region. Furthermore, it removes positioning holes, tissue defects, forbidden areas, and fragmented areas. Combined with the effective Raman spot radius and registration error, the boundary is shrunken to construct a safe sampling area, reducing the impact of tissue edges, holes, section defects, and local high-error areas on Raman spectral acquisition and label generation results.

[0016] 6. This invention constructs a neighborhood corresponding to the actual effective Raman spot shape and size with each Raman sampling point as the center, and calculates the area ratio of positive pixels of target molecular markers to effective tissue pixels in the neighborhood, so that the label reflects the distribution of molecular markers within the actual coverage area of ​​the Raman spot, reducing the influence of tissue heterogeneity and positive-negative region boundaries on point-level labels.

[0017] 7. This invention uses the number of pixels belonging to the common effective region within the Raman spot neighborhood as the denominator of the positive area ratio, limiting the calculation range to areas with effective tissue correspondence, avoiding unreasonable reduction of the positive area ratio due to background outside the tissue, positioning holes, or slice defects; at the same time, the sampling points are screened or marked with low confidence by the coverage ratio of the common effective region, thereby improving the reliability of the label data.

[0018] 8. This invention generates a unique label record for each Raman sampling point, which includes the sampling point identifier, pixel coordinates, physical coordinates, region status, coverage ratio, positive area ratio, label, and confidence level. By establishing a one-to-one correspondence between the sampling point identifier and the Raman spectrum, the traceability of the Raman spectrum dataset is improved, which facilitates subsequent anomaly point verification, label threshold adjustment, spectral modeling, and pathological region correspondence verification.

[0019] 9. This invention can visualize the tissue contour, common effective area, safe sampling area and Raman sampling points of different label states by using contour lines, filled areas and different symbol markers. It can intuitively display the contour changes before and after registration, the sampling space range and the correspondence between sampling points and labels without displaying the color and texture of the original image. This is helpful to check whether the registration results, sampling range and label generation process are reasonable.

[0020] 10. This invention forms a continuous processing flow from multimodal tissue image registration to Raman sampling point-level label generation by registering tissue contours and distance transformation maps, screening multi-index candidate deformation fields, transferring the same spatial transformation, constructing common effective regions and safe sampling regions, and calculating the positive area ratio based on the actual Raman spot neighborhood. This establishes a stable and traceable one-to-one correspondence between Raman sampling points, tissue spatial locations, molecular marker labels, and Raman spectra, providing a technical foundation for constructing Raman spectral datasets with clear molecular biological meaning. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a Raman spectroscopy dataset construction system based on spatial registration of pathological images according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the tissue contour extraction and registration results of gross tissue images and immunohistochemical images according to an embodiment of the present invention, wherein... Figure 3 (a) in the figure is a general tissue outline. Figure 3 (b) in the image is an immunohistochemical tissue outline. Figure 3 (c) in the diagram represents the contour relationship after the initial spatial transformation. Figure 3(d) in the figure represents the contour relationship diagram after the target deformation field has been applied. Figure 4 This is a flowchart of candidate deformation field generation and comprehensive quality evaluation screening based on distance transformation map according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the common effective region, safe sampling region, and point-to-point correspondence between Raman sampling points and molecular marker tags according to an embodiment of the present invention. Figure 5 (a) in the diagram represents the common effective area and the safe sampling area. Figure 5 (b) in the diagram is a point-to-point correspondence between Raman sampling points and molecular marker labels.

[0023] Figure 2 middle: 1. Sample data acquisition module; 2. Image segmentation mask extraction module; 3. Initial spatial transformation mapping module; 4. Non-rigid registration screening module; 5. Mask deformation correction module; 6. Sampling region generation module; 7. Raman spectroscopy dataset generation module. Detailed Implementation

[0024] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0025] According to embodiments of the present invention, a method and system for constructing Raman spectroscopy datasets based on spatial registration of pathological images are provided.

[0026] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 As shown, according to an embodiment of the present invention, a method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images includes: Step S1: Obtain gross tissue images, immunohistochemical images, Raman-measurable regions, and scale transformation information of the same ex vivo tissue sample; It should be explained that gross tissue images are used as fixed-modality images, while immunohistochemical (IHC) images are used as moving-modality images. The Raman-measurable region can be determined based on the actual movement range of the Raman scanning platform, the preset scanning trajectory, manually marked areas, or the effective window of the sample-supporting structure. Scale conversion information can be obtained from the scale lines on the sample-supporting structure, known size frames, or positioning marks, and is used to establish the conversion relationship between image pixel coordinates and actual physical coordinates. By simultaneously acquiring image information, the Raman-measurable region, and scale conversion information, the subsequently generated Raman sampling points not only have image coordinates but also correspond to the physical location on the actual sample surface. In the current program implementation, if a separate Raman-measurable region mask is not provided, the effective area of ​​gross tissue in the common coordinate space is used as the Raman-measurable region; if a scale calibration value is provided, such as pixels per millimeter, the sampling size and physical coordinates are converted using millimeter parameters; otherwise, pixel parameters are directly used for processing, and the pixel coordinates are retained in the data table.

[0027] Step S2: Perform tissue segmentation and contour extraction on the gross tissue image and immunohistochemical image respectively to obtain the gross tissue mask, immunohistochemical tissue mask, gross tissue contour, and immunohistochemical tissue contour; extract the positive mask of target molecular markers based on the immunohistochemical image; In this optional embodiment, tissue segmentation and contour extraction are performed on the gross tissue image and the immunohistochemical image respectively to obtain a gross tissue mask, an immunohistochemical tissue mask, a gross tissue contour, and an immunohistochemical tissue contour, including: For gross tissue images and immunohistochemical images, grayscale conversion and filtering denoising processes are performed respectively to obtain denoised grayscale images. Threshold segmentation is then applied to the denoised grayscale images to obtain initial binary regions. After correcting the foreground-background polarity of the initial binary region, and performing connected component filtering after morphological processing to remove small noise regions, macroscopic tissue masks and immunohistochemical tissue masks are obtained respectively. Boundary tracing was performed on the gross tissue mask and the immunohistochemical tissue mask to extract the outer contour, thus obtaining the gross tissue contour and the immunohistochemical tissue contour respectively.

[0028] In this optional embodiment, extracting a positive mask for target molecular markers based on immunohistochemical images includes: The immunohistochemical images were converted to the hue-saturation-lightness color space while preserving the original red-green-blue color space of the immunohistochemical images; In the hue-saturation-lightness color space, the first candidate region is constructed based on the value range of each channel of hue, saturation and lightness; In the red-green-blue color space, a second candidate region is constructed based on the relative intensity relationship between the red, green, and blue channels; Construct exclusion regions based on the color characteristics of the re-dyed regions; The union of the first and second candidate regions is taken and the excluded regions are removed to obtain the positive candidate regions. The positive candidate regions are then limited to the immunohistochemical tissue mask area and filtered by morphological filtering and connected region area to obtain the positive mask of the target molecular marker.

[0029] It should be explained that after obtaining the two modal images, tissue region extraction is performed on the gross tissue image and the immunohistochemical image respectively. Specifically, the input image is converted into a grayscale image and filtered. Global thresholding is used to obtain an initial binary region. The foreground polarity of the tissue region is determined based on the proportion of foreground pixels. When a tissue region is incorrectly represented as background, the binary image is inverted. Subsequently, morphological closing operations are performed on the binary region to fill small holes inside the tissue and connect adjacent regions. Then, the outer contour in the image is extracted, and the tissue region with the largest area or multiple effective tissue regions with an area greater than a preset threshold are retained to form gross tissue masks and immunohistochemical tissue masks respectively. When global thresholding fails to obtain an effective tissue contour, adaptive thresholding can be used as a backup processing path. The two modal images use the same or corresponding tissue region extraction process to reduce contour deviations introduced by different segmentation rules. In this embodiment, the Gaussian filter kernel size is [missing information]. Pixel, morphological closing operation uses The pixel elliptical structural element has a minimum area threshold of 300 pixels for connected components. This area threshold is mainly used to remove small connected components formed by background noise, holes, or fragmented parts. It is determined based on the input image resolution and the minimum effective tissue block size to be retained. When the image resolution changes, it can be adjusted synchronously according to the pixel area ratio. If physical scale calibration has been completed, the physical area of ​​the minimum effective tissue block can also be converted into the corresponding pixel area before setting. This embodiment uses the method of retaining the largest connected component; for samples with multiple independent effective tissue blocks, it can be changed to retain multiple connected components with an area not less than the threshold. The above values ​​are implementation parameters of the current embodiment and do not constitute a limitation on the scope of protection.

[0030] In terms of tissue region extraction, this invention preferably obtains a tissue mask through grayscale conversion, filtering, global thresholding, foreground polarity correction, morphological processing, and connected component filtering. Alternatively, the above processing flow can be replaced by adaptive thresholding, region growing, watershed segmentation, active contour models, graph cut algorithms, clustering segmentation, or semantic segmentation methods based on convolutional neural networks. For images with complex tissue backgrounds, uneven lighting, or multiple discrete blocks within the tissue region, tissue regions can also be extracted based on color space, texture features, edge features, or pre-trained segmentation models. As long as the alternative method can obtain an effective tissue mask that maintains a spatial correspondence with the original image, it will not affect subsequent tissue contour extraction, spatial registration, and Raman sampling point label generation.

[0031] Based on the tissue mask, a boundary tracking method is used to extract the gross tissue contour and the immunohistochemical tissue contour. The tissue contour can be represented as a set of contour points arranged in image coordinate order, or as a black-and-white binary contour image. It only retains the position, shape, and topological relationships of the tissue boundaries, without preserving the color, grayscale, and texture content of the original gross tissue image or immunohistochemical image. For example... Figure 3 As shown, the fixed modal contour can be represented by a solid black line, and the moving modal contour can be represented by a dashed black line. The contour overlay visually displays the degree of correspondence between tissue boundaries before and after initial registration and before and after non-rigid registration. Since registration mainly relies on tissue contour and distance information, rather than directly depending on the color or texture of the two physical images, it can reduce the influence of lighting conditions, staining depth, and cross-modal texture differences on the registration results.

[0032] Regarding tissue contour representation, this invention preferably extracts the outer boundary of the tissue from a binary tissue mask and represents it using a contour point set or a black-and-white binary contour map. Alternatively, the tissue contour can be replaced by a tissue edge map, contour distance map, signed distance function, skeleton map, boundary probability map, or a feature set composed of tissue shape descriptors. Furthermore, the initial spatial correspondence between the fixed and moving modes, in addition to centroid translation, can be determined by rotation, translation, scaling, or affine transformation parameters based on the contour principal axis direction, the tissue's minimum bounding rectangle, convex hull, positioning holes, manually set corresponding points, or positioning marks on the sample's supporting structure.

[0033] Simultaneously, positive regions for target molecular markers are extracted from immunohistochemical images. Taking DAB (3,3'-diaminobenzidine) stained immunohistochemical images as an example, the image is first converted to a hue-saturation-value color space. A first candidate region is constructed based on the hue, saturation, and value ranges of the target stained region. Simultaneously, a second candidate region is constructed based on the relative intensity relationships between the red, green, and blue channels in the red-green-blue color space. Exclusion regions are constructed based on the color characteristics of hematoxylin and other blue-purple counterstaining regions. The counterstaining exclusion regions are removed from the union of the first and second candidate regions, and the resulting regions (i.e., positive candidate regions) are confined within the immunohistochemical tissue mask. Then, morphological opening and closing operations and connected region area filtering are used to remove isolated noise and excessively small regions, resulting in a positive mask for the target molecular markers.

[0034] The system can also perform weighted fusion based on target color intensity, darkness, and staining saturation to generate a continuous color intensity map of the target molecular marker, used to represent the relative color intensity at different locations. In the current DAB color development embodiment, the hue candidate region has a hue range of 0-65, a lower limit of saturation of 8, and a lightness range of 15-255; the red, green, and blue candidate regions satisfy the following conditions: red channel intensity greater than 1.03 times that of the blue channel, green channel intensity greater than 0.95 times that of the blue channel, and red channel intensity greater than 35. For blue-violet restaining regions, exclusion is based on a hue of 75-145 and a saturation not less than 15, or a blue channel intensity greater than 1.03 times that of both red and green channels. The positive mask uses... Pixel opening operation and Pixel closing operations are performed, and positive connected components with an area less than 50 pixels are removed. The continuous color intensity map is formed by normalizing the target brownness, darkness, and saturation and then weighting them with weights of 0.60, 0.25, and 0.15, respectively. The above thresholds can be adjusted according to the dyeing batch, imaging exposure, and color calibration results.

[0035] Regarding the extraction of positive regions for target molecular markers, this invention uses DAB-developed immunohistochemical images as an example, obtaining a positive mask through HSV color range (i.e., hue-saturation-brightness color space), RGB channel relationships (i.e., red-green-blue color space), counterstaining exclusion, and morphological processing. As an alternative, this method can be replaced by color deconvolution, optical density analysis, color clustering, supervised classification, deep learning segmentation, or positive region identification methods based on pathological image analysis software. The target molecular marker is not limited to EGFR and can be Ki-67, PD-L1, HER2, p53, or other molecular markers capable of forming spatial distribution maps. The corresponding images are not limited to DAB immunohistochemical images and can also be immunofluorescence images, in situ hybridization images, specially stained images, or other images that can provide molecular spatial information.

[0036] Step S3: Determine the initial spatial transformation based on the gross tissue outline and the immunohistochemical tissue outline, and map the immunohistochemical image, immunohistochemical tissue mask and the positive mask of the target molecular marker to the common coordinate space according to the initial spatial transformation; In this optional embodiment, determining the initial spatial transformation based on the gross tissue contour and the immunohistochemical tissue contour, and mapping the immunohistochemical image, immunohistochemical tissue mask, and target molecular marker positive mask to a common coordinate space according to the initial spatial transformation includes: Contour point sampling was performed on the gross tissue contour and the immunohistochemical tissue contour to obtain a fixed contour sampling point set and a moving contour sampling point set. The centroids of the two sampling point sets were calculated to obtain the centroid coordinates of the fixed contour and the moving contour. The translation parameters are determined by the positional deviation between the centroid coordinates of the fixed contour and the centroid coordinates of the moving contour, and the initial spatial transformation is constructed. According to the initial spatial transformation, the immunohistochemical images, immunohistochemical tissue masks, and positive masks of target molecular markers are mapped to the common coordinate space.

[0037] It should be explained that, in the initial spatial alignment stage, fixed contour sampling point sets and moving contour sampling point sets are obtained from the gross tissue contour and the immunohistochemical tissue contour, respectively. The contour sampling points can be selected at equal intervals along the contour arc length, or a random sampling method with a fixed random seed can be used to ensure consistent sampling results across multiple runs. The centroids of the two sets of contour sampling points are calculated, and the translation parameters are determined based on the lateral and longitudinal coordinate differences between the two centroids, thus constructing the initial spatial transformation. Based on this, the width, height, and coordinate offset of the common coordinate space are determined jointly by the boundaries of the translated immunohistochemical image and the gross tissue image. Then, the immunohistochemical image, the immunohistochemical tissue mask, and the target molecular marker positive mask are mapped to the common coordinate space according to the same initial spatial transformation. Furthermore, when there are still significant rotation or scale differences between the two images, rotation and scale parameters can be estimated using orientation markers, positioning holes, or contour principal axes. In the current embodiment, the program samples 800 contour points at equal intervals along the maximum outer contour arc length, calculates the centroids of the sampling points of the fixed contour and the moving contour respectively, and uses the difference between the coordinates of the two centroids as the two-dimensional translation amount; when establishing the common coordinate space, a 20-pixel boundary is reserved on the basis of surrounding the transformed range of the two images to avoid the transformed tissue edges being cropped.

[0038] Regarding the initial registration method, this invention preferably uses the centroid difference between the two tissue contours to determine the translation parameters. Alternatively, rigid registration, similarity transformation, affine registration, iterative nearest-point algorithm, contour feature point matching, template matching, or coordinate transformation based on positioning markers can be used to complete the initial alignment. Furthermore, for samples with clear directional markers or positioning holes, a common coordinate space can be directly established using the positioning markers; for samples with rotation, mirroring, or scale changes, rotation correction, mirroring judgment, and scale normalization steps can be added before the translation transformation.

[0039] Step S4: Convert the gross tissue mask and immunohistochemical tissue mask in the common coordinate space into distance transformation maps respectively. Perform non-rigid registration based on the distance transformation maps to generate multiple sets of candidate deformation fields. Evaluate the registration quality of each candidate deformation field and select the target deformation field. In this optional embodiment, the gross tissue mask and immunohistochemical tissue mask in the common coordinate space are converted into distance transformation maps, and non-rigid registration is performed based on the distance transformation maps to generate multiple sets of candidate deformation fields, including: The gross tissue mask and immunohistochemical tissue mask in the common coordinate space are converted into Euclidean distance transformation maps and normalized to obtain normalized gross tissue mask distance transformation maps and normalized immunohistochemical tissue mask distance transformation maps. Set the normalized gross tissue mask distance transformation map as a fixed image and the normalized immunohistochemical tissue mask distance transformation map as a moving image. Multiple sets of differentiated non-rigid registration parameter combinations are configured. Fixed and moving images are input into the non-rigid registration algorithm, and multiple sets of candidate deformation fields and corresponding candidate registration tissue masks are output.

[0040] In this optional embodiment, the registration quality evaluation and selection of the target deformation field for each candidate deformation field includes: For each candidate deformation field, calculate the Dice coefficient between the candidate registration tissue mask and the gross tissue mask corresponding to that candidate deformation field; Calculate the bidirectional average nearest contour distance and the bidirectional maximum nearest contour distance between the gross tissue contour and the immunohistochemical tissue contour after transformation by the candidate deformation field. Find the overlapping region between the gross tissue image and the immunohistochemical image after the candidate deformation field transformation, and calculate the normalized mutual information within the overlapping region. Solve for the Jacobian determinant of the candidate deformation field and count the proportion of pixels whose Jacobian determinant is less than zero; Based on the Dice coefficient, bidirectional average nearest contour distance, bidirectional maximum nearest contour distance, normalized mutual information, and the proportion of pixels with a Jacobian determinant less than zero, a comprehensive quality score is calculated for each candidate deformation field. The candidate deformation field that meets the validity criteria and has the highest comprehensive quality score is selected as the target deformation field.

[0041] It should be explained that after initial alignment, the gross tissue mask and immunohistochemical tissue mask in the common coordinate space are converted into Euclidean distance transform maps, and normalized using a common scale. Each pixel value in the distance transform map represents the distance from that pixel to the nearest tissue boundary, thus containing information about both the outer boundary of the tissue and the spatial relationships of different locations within the tissue relative to the boundary. Based on this, the normalized gross tissue distance transform map is used as a fixed image, and the normalized immunohistochemical tissue distance transform map is used as a moving image input to the non-rigid registration unit. By changing the combination of parameters such as the number of iterations and the displacement field smoothing parameter, multiple sets of candidate deformation fields and corresponding candidate registration tissue masks are generated. The non-rigid registration algorithm can use the symmetric force Demons algorithm, or other non-rigid registration algorithms that can output dense deformation fields. By performing non-rigid registration directly on the distance transform map instead of the original cross-modal intensity map, the interference of color and texture differences between different modal images on deformation field estimation can be reduced. In this embodiment, L2 Euclidean distance transformation is adopted, the distance transformation mask size parameter is 5, and the maximum distance value in the fixed distance map and the moving distance map is used as a common scale to normalize the two distance maps to a unified numerical range. Non-rigid registration adopts the Fast Symmetric ForcesDemons algorithm, with the candidate set of iteration times being 10, 15, 20, 25, and 30, and the candidate set of displacement field smoothing parameters being 1.0, 1.5, 2.0, 2.5, and 3.0. The two sets of parameters are combined in pairs to form 25 sets of candidate deformation fields.

[0042] For each candidate deformation field, the Dice coefficient between the candidate registration tissue mask and the gross tissue mask is calculated to evaluate the overall overlap between the two tissue regions. The bidirectional average nearest contour distance and the bidirectional maximum nearest contour distance between the gross tissue contour and the immunohistochemical tissue contour transformed by the candidate deformation field are calculated to evaluate the average registration error and the local maximum error. The corresponding candidate deformation field is applied to the immunohistochemical image, and the normalized mutual information is calculated in the overlapping area between the transformed immunohistochemical image and the gross tissue image to evaluate the correspondence between the images after the candidate deformation field is applied. At the same time, the Jacobian determinant is calculated based on the first-order spatial gradient of the candidate deformation field in the horizontal and vertical directions, and the proportion of pixels with a Jacobian determinant less than zero is counted to determine whether the candidate deformation field has local folding or unreasonable deformation. Figure 4 This shows the number of iterations based on multiple sets (i.e. Figure 4 The candidate set of the number of iterations) and the smoothing parameter (i.e. Figure 4 The candidate elastic deformation field is generated by combining the candidate set of smooth parameters (i.e., the candidate set of smooth parameters). Figure 4 Candidate set of elastic deformation fields), and through Dice coefficients, average profile error, maximum profile error, and normalized mutual information (i.e. Figure 4The flowchart illustrates the process of comprehensively evaluating the quality of deformation fields based on the proportion of negative Jacobian values ​​(NMI) and the overall quality, ultimately selecting the deformation field with the highest comprehensive score as the target deformation field. Specifically, the macroscopic tissue mask is denoted as A, the candidate registration tissue mask is denoted as B, and the Dice coefficient D is determined according to... Calculation, where and These represent the number of foreground pixels in the two binary masks, respectively. This represents the number of foreground pixels at the intersection of the two images. Contour error is calculated using bidirectional nearest distance: the nearest distance from each point of the gross tissue contour to the immunohistochemical tissue contour transformed by the candidate deformation field is calculated, as is the nearest distance from each point of the immunohistochemical tissue contour transformed by the candidate deformation field to the gross tissue contour; the bidirectional average nearest contour distance is the arithmetic mean of the average distances in both directions (i.e., average contour error), and the bidirectional maximum nearest contour distance is the larger of the two maximum nearest distances (i.e., maximum contour error). Normalized mutual information (NMI) is calculated within the overlapping region of the gross tissue image and the immunohistochemical image transformed by the candidate deformation field.

[0043] This embodiment uses 64 grayscale histogram intervals, and follows... Normalization is performed; when the number of overlapping pixels is less than 100, this term is recorded as 0. Here, F represents the fixed image (i.e., the gross tissue mask distance transformation map), M represents the moving image (i.e., the immunohistochemical tissue mask distance transformation map), I(F,M) represents the mutual information between the fixed and moving images, H(F) and H(M) represent the information entropy of the fixed and moving images, respectively, and NMI represents the normalized mutual information. The displacement field is denoted as... At that time, the Jacobian determinant was calculated according to... Calculation; this embodiment will or Positions larger than 0.10 pixels are considered valid deformation areas, and the proportion of pixels with J < 0 within these areas is counted. Where (x,y) represents the image pixel coordinates, u and v represent the displacement components of the pixel in the horizontal (x-direction) and vertical (y-direction) directions, respectively, and J represents the Jacobian determinant. This represents the proportion of pixels within the effective deformation region whose Jacobian determinant is less than zero.

[0044] Furthermore, candidate deformation fields are screened by first determining their validity, then calculating a comprehensive quality score and ranking them. Specifically, the normalized average contour error is obtained by dividing the bidirectional average nearest contour distance and the bidirectional maximum nearest contour distance by the length of the image diagonal in the common coordinate space. and normalized maximum profile error To eliminate the influence of image size on scoring; the Dice coefficient is denoted as D, the normalized mutual information in the overlapping region is denoted as NMI, and the proportion of pixels with a Jacobian determinant less than zero is denoted as... The overall quality score of the candidate deformation field is determined according to... Calculation. Where w1, w2, w3, w4, and w5 are the weighting coefficients of each evaluation index, used to adjust the contribution of different indicators to the overall quality score; a larger Q value indicates a higher overall quality of the candidate deformation field. In this embodiment, , , , , ,Right now .

[0045] At the same time, A value no greater than 0.01 is used as the validity condition for candidate deformation fields; candidate results exceeding this value are not included in the final preferred set. Among the candidate deformation fields that meet the validity condition, they are sorted from high to low according to their Q-values, and the one with the highest Q-value is selected as the target deformation field. The above weights involve overall overlap, contour error, cross-modal image similarity, and local folding penalty in the selection process. A larger penalty weight is applied to the proportion of negative Jacobian values ​​to avoid obtaining high overlap on the surface through non-physical folding. It should be noted that the above weights and validity thresholds can be adjusted according to different samples and validation data; the current values ​​are only specific implementations of this embodiment.

[0046] Furthermore, regarding non-rigid registration, this invention preferably converts the tissue mask into a distance transformation map and uses the Demons algorithm to generate candidate deformation fields. However, the Demons algorithm can also be replaced by B-spline free deformation, thin-plate splines, elastic registration, optical flow registration, differential homeomorphic registration, finite element-based biomechanical registration, Gaussian process-based deformation estimation, or other non-rigid registration methods capable of outputting local displacement fields. The distance transformation map can also be replaced by a signed distance map, edge probability map, contour distance field, or tissue shape feature map. As long as the method used can generate a corresponding spatial transformation based on the tissue shape differences between the two modes and can transfer this spatial transformation to the target molecular marker region, the main technical objective of this invention can be achieved.

[0047] Similarly, in the selection of candidate deformation fields, this invention preferably uses the Dice coefficient, bidirectional average nearest profile distance, bidirectional maximum nearest profile distance, normalized mutual information, and the proportion of negative Jacobian determinants for comprehensive evaluation. However, the above indicators can also be partially or completely replaced by the Jaccard coefficient, Hausdorff distance, average surface distance, profile overlap rate, normalized correlation coefficient, structural similarity index, gradient correlation, cross-correlation coefficient, local deformation energy, displacement field smoothness, degree of Jacobian determinant deviation from 1, or inverse consistency error. In addition to using weighted comprehensive scoring, the selection of candidate deformation fields can also employ hierarchical screening, constrained optimization, multi-objective ranking, Pareto optimal selection, or a method of first excluding invalid deformation fields and then ranking them according to the main evaluation indicators.

[0048] Step S5: Simultaneously transfer the initial spatial transformation and the target deformation field to the target molecular marker positive mask to obtain the modified target molecular marker positive mask; It should be explained that after determining the target deformation field, the initial spatial transformation and the target deformation field are synchronously transferred to the target molecular marker positive mask in the exact same transformation order as the immunohistochemical tissue mask. For the binary target molecular marker positive mask, nearest neighbor interpolation is used to avoid generating new non-integer class values ​​after interpolation. After completing the spatial transformation, positive regions outside the gross tissue mask are set to zero. Because the gross tissue mask and the target molecular marker positive mask use the same initial spatial transformation and the same target deformation field, spatial misalignment caused by separate registration of different regions can be avoided.

[0049] Furthermore, regarding spatial transformation propagation, this invention preferably employs nearest-neighbor interpolation for the binary positive mask, but label-preserving interpolation, distance-field interpolation followed by re-thresholding, or local majority voting can also be used as alternatives. As long as the tissue mask and the target molecular marker positive mask adopt the same or traceable consistent spatial transformation relationship, additional spatial misalignment between the tissue contour and the molecular marker region can be avoided.

[0050] Step S6: Construct a common effective region based on the macroscopic tissue mask, the modified target molecular marker positive mask, and the Raman measurable region, and perform an etching operation on the common effective region according to the Raman effective spot size and registration error to obtain a safe sampling region; In this optional embodiment, constructing a common effective region based on a macroscopic tissue mask, a modified target molecular marker positive mask, and a Raman-detectable region includes: In the common coordinate space, pixel-level intersection operations are performed on the gross tissue mask, the modified target molecular marker positive mask, and the Raman measurable region to obtain the initial common region; The common effective region is obtained by excluding the positioning hole region, tissue defect region, forbidden region, and fragment region with an area smaller than the preset threshold from the initial common region.

[0051] In this optional embodiment, the common effective region is etched according to the Raman effective spot size and registration error to obtain a safe sampling region including: Determine the safety boundary distance based on the effective Raman spot size and registration error; Morphological structural elements are constructed with the safe boundary distance as the radius. Erosion operations are performed on the common effective region, so that the boundary of the eroded region shrinks inward by the safe boundary distance compared with the boundary of the common effective region, thus obtaining the safe sampling region.

[0052] It should be explained that, in the common coordinate space, pixel-level intersection operations are performed on the gross tissue mask, the immunohistochemical tissue mask after being subjected to the target deformation field (i.e., the corrected target molecular marker positive mask), and the Raman-detectable region to obtain an initial common region. Subsequently, positioning hole regions, tissue defect regions, forbidden regions, and small fragment regions are removed from the initial common region to obtain a common effective region. This invention can set a minimum connected region area threshold of 100 pixels to remove isolated small fragment regions generated after multi-mask intersection; this threshold is determined based on the image resolution and the minimum usable sampling area size, and can be synchronously adjusted according to the pixel area ratio when the image resolution changes.

[0053] The safety boundary distance is a configurable parameter, set based on the sum of the effective Raman spot radius (i.e., the effective Raman spot size), registration error, and edge uncertainty reserve. In this embodiment, where millimeter-scale calibration has not been performed, the Raman spot radius is set to 10 pixels, and the safety boundary distance is set to 15 pixels, meaning an additional 5 pixels of edge margin and registration error reserve are reserved on top of the spot radius. The program uses a size of... The elliptical structural elements of pixels perform a single erosion operation on the common effective area to obtain the safe sampling area. If physical scale calibration has been completed, the program can use a spot radius of 0.25mm and a safety boundary distance of 0.30mm as the initial configuration, and can complete parameter recalibration according to the actual spot size of the instrument and the registration error. For manually marked positioning hole areas and prohibited areas, the area can be removed according to the preset center coordinates and exclusion radius. This embodiment supports exclusion radius parameters of 0.35mm or 18 pixels.

[0054] At the level of constructing the common effective region and the safe sampling region, this invention preferably calculates the intersection of the gross tissue mask, the registered tissue slice mask (i.e., the corrected positive mask for the target molecular marker), and the Raman-measurable region, and combines the effective Raman spot size and registration error to complete the region boundary shrinkage processing. As an alternative, the safe sampling region can also be constructed using distance transformation threshold constraints, local confidence maps, registration error heatmaps, uncertainty probability maps, or manually set exclusion regions. The safe boundary distance can be set with fixed pixel values ​​and fixed physical dimensions, or it can be dynamically and adaptively adjusted according to the local registration error of different regions. For regions with large local registration errors, a larger boundary shrinkage distance can be configured; for regions with high registration confidence, the shrinkage distance can be appropriately reduced, thereby constructing a safe sampling region with spatial adaptive characteristics.

[0055] Step S7: Calculate the sampling step size using scale conversion information, generate Raman sampling points within the safe sampling area, and for each Raman sampling point, correct the positive area ratio of the positive mask of the target molecular marker within the neighborhood that matches the effective Raman spot size, generate molecular marker labels and label confidence, and filter sampling points to output the Raman spectroscopy dataset.

[0056] In this optional embodiment, the sampling step size is calculated using scale transformation information, and Raman sampling points are generated within a safe sampling area. For each Raman sampling point, the positive area ratio of the positive mask of the target molecular marker is corrected based on the neighborhood of the effective Raman spot size. Molecular marker labels and label confidence levels are generated, and sampling points are screened. The output Raman spectral dataset includes: The preset physical sampling step size is converted into a pixel sampling step size using scale conversion information. Grid points are generated within the outer range of the safe sampling area. Grid points located within the safe sampling area and not within the exclusion area are retained as Raman candidate sampling points. For each Raman candidate sampling point, a neighborhood matching the size of the effective Raman spot is constructed with the Raman candidate sampling point as the center. The number of pixels in the neighborhood that simultaneously belong to the positive mask of the modified target molecular marker and the common effective region is counted. The positive area ratio is obtained by dividing the number of pixels in the neighborhood that belong to the common effective region. The proportion of pixels belonging to the common valid region within the neighborhood is counted to obtain the neighborhood organization pixel ratio. When the neighborhood organization pixel ratio is lower than a preset threshold, the sampling point is discarded or marked as low confidence. Molecular marker labels are generated based on the percentage of positive areas; label confidence is calculated based on the percentage of pixels in neighboring tissues and the distance from the percentage of positive areas to the nearest label threshold. For each retained Raman sampling point, generate a record containing the sampling point identifier, pixel coordinates, physical coordinates, positive area percentage, neighboring tissue pixel percentage, molecular marker label, and label confidence, and output a Raman spectroscopy dataset.

[0057] It should be explained that when the proportion of pixels in the common effective region within the Raman spot's neighborhood to the total number of pixels in the neighborhood (i.e., the neighborhood organization pixel ratio) is lower than a preset threshold, the sampling point is discarded, or it is marked as low confidence. Neighborhood organization pixel ratio The value range is 0-1. In the current embodiment, the minimum effective coverage rate is set to 0.80, which means that at least 80% of the pixels in a sampling spot belong to the common effective area. The principle for determining this threshold is to retain as many usable sampling points as possible while avoiding tissue edges, slice defects or registration uncertain areas from occupying the main part of the spot. If higher label purity is required, the threshold can be increased. If the sample area is small and more sampling points need to be retained, it can be appropriately reduced.

[0058] For sampling points that meet the requirement that the proportion of neighboring tissue pixels is not less than a preset threshold, the current binary classification implementation sets the positive area ratio threshold to 0.20. That is, when the proportion of positive pixels of the target molecular marker within the effective area of ​​the light spot is not less than 20%, it is marked as "positive"; otherwise, it is marked as "negative." A four-level labeling mode is reserved, with default thresholds of 0.05, 0.20, and 0.50, representing negative, weakly positive, moderately positive, and strongly positive, respectively. Based on this, the label confidence is determined by the distance from both the proportion of neighboring tissue pixels and the positive area ratio to the nearest judgment threshold: during binary classification, the distance is first calculated... Where f is the positive area ratio and T is the positive threshold, then... Normalization yields ;in, This indicates a truncation operation, restricting the calculation result within the parentheses to between 0 and 1: if less than 0, take 0; if greater than 1, take 1; otherwise, retain the original value. For four-level classification, d is replaced with the distance from f to the nearest threshold among the three level thresholds. Each Raman sampling point generates one and only one point-to-point label record. This label record includes at least the sampling point identifier, pixel coordinates, physical coordinates, common effective region status, safe sampling region status, effective spot coverage ratio (i.e., the proportion of neighboring tissue pixels), target molecular marker positive area ratio (i.e., positive area ratio), molecular marker label, and label confidence level. Sampling point numbers are generated sequentially in the form of P001, P002, etc., and can be associated with corresponding Raman spectra one-to-one through the sampling point number, sampling order, or spectral file identifier.

[0059] like Figure 5As shown, the common effective region can be represented by a solid black line, the safe sampling region by a dashed black line, and the positive region of the target molecular marker (i.e., the modified positive mask of the target molecular marker) by a diagonal line. Positive sampling points can be represented by solid circles, negative or weakly positive sampling points by hollow circles, and low-confidence sampling points by cross symbols. For any sampling point, a circular neighborhood corresponding to the Raman effective spot can also be drawn, and the corresponding label record can be displayed in the form of "sampling point identifier-coordinates-positive area ratio-label". Through the above visualization method, the tissue contour registration, common effective region, safe sampling region, and point-to-point correspondence between Raman sampling points and molecular marker labels (i.e., Raman spectroscopy dataset) can be clearly displayed without displaying the original gross tissue photographs and the color texture of the immunohistochemical images.

[0060] Furthermore, regarding Raman sampling point generation, this invention preferably generates regular grid points within a safe sampling area according to a preset step size. However, sampling points can also be generated using hexagonal grids, random sampling, layered sampling, Poisson disk sampling, path-optimized sampling, adaptive density sampling, or based on the spatial variation of the target molecular marker (i.e., a modified positive mask for the target molecular marker). For the boundary between positive and negative samples, the sampling density can be increased; for relatively homogeneous areas within the tissue, the sampling density can be decreased. Sampling points can also be reordered according to the movement path of the Raman displacement platform to reduce the platform movement distance and sampling time. The current embodiment uses a 25-pixel step size; in the implementation with scale calibration, the default physical step size of the program is 0.50 mm. The preset step size is not a fixed constant; it is determined based on the effective Raman spot diameter, the minimum positioning interval of the displacement platform, the desired overlap rate of adjacent spots, and the overall sampling time, and can be adjusted between dense spatial imaging and sampling efficiency.

[0061] Regarding Raman sampling point label generation, this invention preferably generates labels based on the proportion of positive pixels of the target molecular marker (i.e., positive pixels in the modified target molecular marker positive mask) within the neighborhood of the light spot to the pixels in the common effective region. This label generation method can be replaced by generating labels based on the average, median, integral, maximum, weighted proportion of positive pixels, or combined results of multiple molecular markers for continuous colorimetric intensity. Labels can be binary labels for positive and negative, or multi-level labels for negative, weakly positive, moderately positive, and strongly positive. Alternatively, the continuous positive proportion or continuous colorimetric intensity value can be directly output. Besides using a circular shape, the Raman light spot neighborhood can also be elliptical, rectangular, Gaussian weighted regions, or weighted neighborhoods determined based on the diffusion function of the measured points, depending on the actual shape of the light spot in the optical system.

[0062] In calculating label confidence, it can be determined based on the effective tissue coverage ratio (i.e., the proportion of neighboring tissue pixels), the distance from the sampling point to the tissue boundary, the local registration error, the distance between the positive area ratio and the label threshold, or a weighted average of the above indicators. Low-confidence sampling points can be directly deleted, retained with a low-confidence label, or have repeated sampling and manual verification arranged.

[0063] See Figure 5 Each sampling point (e.g., positive sampling point P006) consists of a quadruple ( ) characterization, in which These are the two-dimensional spatial coordinates of the sampling point in the tissue slice image. This represents the proportion of locally positive areas. The quadruple serves as the tissue category label (e.g., tumor or normal tissue) corresponding to the sampling point, and is used as input for subsequent classification or confidence calculation.

[0064] For point-to-point data (i.e., Raman spectral datasets) output, the output format can be a CSV table, spreadsheet, database record, JSON file, Raman platform control file, or a data package saved together with the spectral file. The sampling points and Raman spectra are distinguished by the sampling point identifier (…). Besides association via sampling order, association can also be established through spectral file names, timestamps, platform coordinates, QR codes, sample numbers, or database primary keys. As long as a one-to-one correspondence can be guaranteed between each Raman sampling point, its corresponding spatial location, molecular marker label, and Raman spectrum, the data association purpose of this invention can be achieved.

[0065] This invention is not limited to any specific image segmentation algorithm, non-rigid registration algorithm, evaluation index, positive region extraction method, sampling point arrangement method, or label threshold. The above-mentioned technical means can be replaced or combined according to sample type, image quality, staining method, Raman instrument parameters, and sampling requirements, but their common technical essence remains the same: to complete the registration of multimodal ex vivo tissue images in a unified coordinate space, to consistently transfer the registration transformation to the spatial information of the target molecular markers, and to generate molecular marker labels (i.e., Raman spectroscopy datasets) for sampling points that can be associated one-to-one with Raman spectra based on the actual Raman spot coverage area.

[0066] This application has been developed into an executable software embodiment. The main parameters and results of a single complete run are shown in Table 1.

[0067] Table 1: Parameter Execution Table

[0068] For different tissue types, image resolutions, staining batches, and Raman spot sizes, the tissue area threshold, safety boundary, sampling step size, effective coverage threshold, and label threshold can all be recalibrated according to the aforementioned determination principles.

[0069] According to one embodiment of the present invention, such as Figure 2 As shown, a Raman spectroscopy dataset construction system based on pathological image spatial registration is also provided. This system includes: Sample data acquisition module 1 is used to acquire gross tissue images, immunohistochemical images, Raman-measurable regions, and scale transformation information of the same ex vivo tissue sample; Image segmentation mask extraction module 2 is used to perform tissue segmentation and contour extraction on gross tissue images and immunohistochemical images respectively, to obtain gross tissue mask, immunohistochemical tissue mask, gross tissue contour and immunohistochemical tissue contour; and to extract the positive mask of target molecular markers based on the immunohistochemical image. Initial spatial transformation mapping module 3 is used to determine the initial spatial transformation based on the gross tissue contour and immunohistochemical tissue contour, and to map the immunohistochemical image, immunohistochemical tissue mask and target molecular marker positive mask to the common coordinate space according to the initial spatial transformation. The non-rigid registration and screening module 4 is used to convert the gross tissue mask and immunohistochemical tissue mask in the common coordinate space into distance transformation maps, perform non-rigid registration based on the distance transformation maps to generate multiple sets of candidate deformation fields, evaluate the registration quality of each candidate deformation field, and screen to obtain the target deformation field. The mask deformation correction module 5 is used to synchronously transfer the initial spatial transformation and the target deformation field to the target molecular marker positive mask to obtain the corrected target molecular marker positive mask; The sampling area generation module 6 is used to construct a common effective area based on the macroscopic tissue mask, the modified target molecular marker positive mask, and the Raman measurable area, and to perform an etching operation on the common effective area according to the Raman effective spot size and registration error to obtain a safe sampling area. The Raman spectroscopy dataset generation module 7 is used to calculate the sampling step size using scale transformation information, generate Raman sampling points within the safe sampling area, and for each Raman sampling point, correct the positive area ratio of the positive mask of the target molecular marker within the neighborhood that matches the effective Raman spot size, generate molecular marker labels and label confidence, and perform sampling point screening to output the Raman spectroscopy dataset.

[0070] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images, characterized in that, The method includes: S1. Obtain gross tissue images, immunohistochemical images, Raman-measurable regions, and scale transformation information of the same ex vivo tissue sample; S2. Perform tissue segmentation and contour extraction on gross tissue images and immunohistochemical images respectively to obtain gross tissue mask, immunohistochemical tissue mask, gross tissue contour, and immunohistochemical tissue contour; extract the positive mask of target molecular markers based on the immunohistochemical image. S3. Determine the initial spatial transformation based on the gross tissue outline and immunohistochemical tissue outline, and map the immunohistochemical image, immunohistochemical tissue mask and target molecular marker positive mask to the common coordinate space according to the initial spatial transformation. S4. Convert the gross tissue mask and immunohistochemical tissue mask in the common coordinate space into distance transformation maps respectively. Perform non-rigid registration based on the distance transformation maps to generate multiple sets of candidate deformation fields. Evaluate the registration quality of each candidate deformation field and select the target deformation field. S5. Simultaneously transfer the initial spatial transformation and the target deformation field to the target molecular marker positive mask to obtain the modified target molecular marker positive mask; S6. Construct a common effective region based on the macroscopic tissue mask, the modified target molecular marker positive mask, and the Raman measurable region. Then, perform an etching operation on the common effective region according to the Raman effective spot size and registration error to obtain a safe sampling region. S7. Calculate the sampling step size using scale transformation information, generate Raman sampling points within the safe sampling area, and for each Raman sampling point, correct the positive area ratio of the positive mask of the target molecular marker within the neighborhood of the effective Raman spot size, generate molecular marker labels and label confidence, and filter sampling points to output the Raman spectroscopy dataset.

2. The method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images according to claim 1, characterized in that, The process of performing tissue segmentation and contour extraction on gross tissue images and immunohistochemical images respectively to obtain gross tissue masks, immunohistochemical tissue masks, gross tissue contours, and immunohistochemical tissue contours includes: For gross tissue images and immunohistochemical images, grayscale conversion and filtering denoising processes are performed respectively to obtain denoised grayscale images. Threshold segmentation is then applied to the denoised grayscale images to obtain initial binary regions. After correcting the foreground-background polarity of the initial binary region, and performing connected component filtering after morphological processing to remove small noise regions, macroscopic tissue masks and immunohistochemical tissue masks are obtained respectively. Boundary tracing was performed on the gross tissue mask and the immunohistochemical tissue mask to extract the outer contour, thus obtaining the gross tissue contour and the immunohistochemical tissue contour respectively.

3. The method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images according to claim 1, characterized in that, The positive mask for extracting target molecular markers based on immunohistochemical images includes: The immunohistochemical images were converted to the hue-saturation-lightness color space while preserving the original red-green-blue color space of the immunohistochemical images; In the hue-saturation-lightness color space, the first candidate region is constructed based on the value range of each channel of hue, saturation and lightness; In the red-green-blue color space, a second candidate region is constructed based on the relative intensity relationship between the red, green, and blue channels; Construct exclusion regions based on the color characteristics of the re-dyed regions; The union of the first and second candidate regions is taken and the excluded regions are removed to obtain the positive candidate regions. The positive candidate regions are then limited to the immunohistochemical tissue mask area and filtered by morphological filtering and connected region area to obtain the positive mask of the target molecular marker.

4. The method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images according to claim 1, characterized in that, The initial spatial transformation determined based on gross tissue contours and immunohistochemical tissue contours, mapping the immunohistochemical image, immunohistochemical tissue mask, and target molecular marker positive mask to a common coordinate space according to the initial spatial transformation, includes: Contour point sampling was performed on the gross tissue contour and the immunohistochemical tissue contour to obtain a fixed contour sampling point set and a moving contour sampling point set. The centroids of the two sampling point sets were calculated to obtain the centroid coordinates of the fixed contour and the moving contour. The translation parameters are determined by the positional deviation between the centroid coordinates of the fixed contour and the centroid coordinates of the moving contour, and the initial spatial transformation is constructed. According to the initial spatial transformation, the immunohistochemical images, immunohistochemical tissue masks, and positive masks of target molecular markers are mapped to the common coordinate space.

5. The method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images according to claim 1, characterized in that, The step of converting the gross tissue mask and immunohistochemical tissue mask in the common coordinate space into distance transformation maps, and performing non-rigid registration based on the distance transformation maps to generate multiple sets of candidate deformation fields includes: The gross tissue mask and immunohistochemical tissue mask in the common coordinate space are converted into Euclidean distance transformation maps and normalized to obtain normalized gross tissue mask distance transformation maps and normalized immunohistochemical tissue mask distance transformation maps. Set the normalized gross tissue mask distance transformation map as a fixed image and the normalized immunohistochemical tissue mask distance transformation map as a moving image. Multiple sets of differentiated non-rigid registration parameter combinations are configured. Fixed and moving images are input into the non-rigid registration algorithm, and multiple sets of candidate deformation fields and corresponding candidate registration tissue masks are output.

6. The method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images according to claim 5, characterized in that, The process of evaluating the registration quality of each candidate deformation field and selecting the target deformation field includes: For each candidate deformation field, calculate the Dice coefficient between the candidate registration tissue mask and the gross tissue mask corresponding to that candidate deformation field; Calculate the bidirectional average nearest contour distance and the bidirectional maximum nearest contour distance between the gross tissue contour and the immunohistochemical tissue contour after transformation by the candidate deformation field. Find the overlapping region between the gross tissue image and the immunohistochemical image after the candidate deformation field transformation, and calculate the normalized mutual information within the overlapping region. Solve for the Jacobian determinant of the candidate deformation field and count the proportion of pixels whose Jacobian determinant is less than zero; Based on the Dice coefficient, bidirectional average nearest contour distance, bidirectional maximum nearest contour distance, normalized mutual information, and the proportion of pixels with a Jacobian determinant less than zero, a comprehensive quality score is calculated for each candidate deformation field. The candidate deformation field that meets the validity criteria and has the highest comprehensive quality score is selected as the target deformation field.

7. The method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images according to claim 1, characterized in that, The construction of a common effective region based on macroscopic tissue mask, modified target molecular marker positive mask, and Raman-detectable region includes: In the common coordinate space, pixel-level intersection operations are performed on the gross tissue mask, the modified target molecular marker positive mask, and the Raman measurable region to obtain the initial common region; The common effective region is obtained by excluding the positioning hole region, tissue defect region, forbidden region, and fragment region with an area smaller than the preset threshold from the initial common region.

8. The method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images according to claim 1, characterized in that, The etching operation performed on the common effective region based on the Raman effective spot size and registration error to obtain the safe sampling region includes: Determine the safety boundary distance based on the effective Raman spot size and registration error; Morphological structural elements are constructed with the safe boundary distance as the radius. Erosion operations are performed on the common effective region, so that the boundary of the eroded region shrinks inward by the safe boundary distance compared with the boundary of the common effective region, thus obtaining the safe sampling region.

9. The method for constructing a Raman spectroscopy dataset based on spatial registration of pathological images according to claim 1, characterized in that, The sampling step size is calculated using scale transformation information, and Raman sampling points are generated within a safe sampling area. For each Raman sampling point, the positive area ratio of the positive mask of the target molecular marker is corrected based on the neighborhood of the effective Raman spot size. Molecular marker labels and label confidence are generated, and sampling points are screened. The output Raman spectral dataset includes: The preset physical sampling step size is converted into a pixel sampling step size using scale conversion information. Grid points are generated within the outer range of the safe sampling area. Grid points located within the safe sampling area and not within the exclusion area are retained as Raman candidate sampling points. For each Raman candidate sampling point, a neighborhood matching the size of the effective Raman spot is constructed with the Raman candidate sampling point as the center. The number of pixels in the neighborhood that simultaneously belong to the positive mask of the modified target molecular marker and the common effective region is counted. The positive area ratio is obtained by dividing the number of pixels in the neighborhood that belong to the common effective region. The proportion of pixels belonging to the common valid region within the neighborhood is counted to obtain the neighborhood organization pixel ratio. When the neighborhood organization pixel ratio is lower than a preset threshold, the sampling point is discarded or marked as low confidence. Molecular marker labels are generated based on the percentage of positive areas; label confidence is calculated based on the percentage of pixels in neighboring tissues and the distance from the percentage of positive areas to the nearest label threshold. For each retained Raman sampling point, generate a record containing the sampling point identifier, pixel coordinates, physical coordinates, positive area percentage, neighboring tissue pixel percentage, molecular marker label, and label confidence, and output a Raman spectroscopy dataset.

10. A Raman spectroscopy dataset construction system based on pathological image spatial registration, used to implement the Raman spectroscopy dataset construction method based on pathological image spatial registration as described in any one of claims 1-9, characterized in that, The system includes: The sample data acquisition module is used to acquire gross tissue images, immunohistochemical images, Raman-measurable regions, and scale transformation information of the same ex vivo tissue sample. The image segmentation mask extraction module is used to perform tissue segmentation and contour extraction on gross tissue images and immunohistochemical images respectively, to obtain gross tissue masks, immunohistochemical tissue masks, gross tissue contours and immunohistochemical tissue contours; and to extract positive masks for target molecular markers based on immunohistochemical images. The initial spatial transformation mapping module is used to determine the initial spatial transformation based on the gross tissue contour and immunohistochemical tissue contour, and to map the immunohistochemical image, immunohistochemical tissue mask and target molecular marker positive mask to the common coordinate space according to the initial spatial transformation. The non-rigid registration and screening module is used to convert the gross tissue mask and immunohistochemical tissue mask in the common coordinate space into distance transformation maps, respectively. Based on the distance transformation maps, non-rigid registration is performed to generate multiple sets of candidate deformation fields. The registration quality of each candidate deformation field is evaluated and the target deformation field is obtained through screening. The mask deformation correction module is used to synchronously transfer the initial spatial transformation and the target deformation field to the target molecular marker positive mask to obtain the corrected target molecular marker positive mask; The sampling region generation module is used to construct a common effective region based on the macroscopic tissue mask, the modified target molecular marker positive mask, and the Raman measurable region, and to perform an etching operation on the common effective region according to the Raman effective spot size and registration error to obtain a safe sampling region. The Raman spectroscopy dataset generation module is used to calculate the sampling step size using scale transformation information, generate Raman sampling points within a safe sampling area, and for each Raman sampling point, correct the positive area ratio of the positive mask of the target molecular marker within the neighborhood that matches the effective Raman spot size, generate molecular marker labels and label confidence, and filter sampling points to output the Raman spectroscopy dataset.

Images