Breast FISH recognition interpretation HER2 score method and application thereof

Abstract

The application provides a breast FISH recognition and interpretation HER2 scoring method based on multi-staining slice registration and application thereof, and aims to solve the problems of low multi-slice registration accuracy and disconnection between target area positioning and detection. The application obtains HE, HER2-IHC and immunofluorescence full slice images of breast tissue; tumor segmentation is performed on the HE image to obtain a mask; the HE and IHC images are registered in the first stage and the mask is mapped; the target area of the target score invasive carcinoma is segmented based on a model on the IHC image; the IHC and immunofluorescence images are registered in the second stage and the target area is mapped; the independent counting unit is detected in the target area of the immunofluorescence image to obtain a cell nucleus, and the counting unit is counted to determine the HER2 state. The application is mainly used for realizing high-precision breast cancer cell-level FISH targeted detection and auxiliary diagnosis.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence analysis technology for pathological images, and more specifically, to a method for identifying and interpreting HER2 scores in breast FISH based on multi-stained slide registration and its application. Background Technology

[0002] In precision medicine for breast cancer, the status of human epidermal growth factor receptor 2 (HER2) is crucial. Immunohistochemistry (IHC) is typically used first to detect HER2 protein expression levels. Based on pathological interpretation criteria, a score of 2+ is considered indeterminate, defined as ">10% of invasive cancer cells showing weak to moderate intensity of intact cell membrane staining or ≤10% of invasive cancer cells showing strong and intact cell membrane staining." For such indeterminate cases, fluorescence in situ hybridization (FISH) is further used to confirm HER2 amplification at the genetic level, thereby determining the subsequent targeted therapy strategy.

[0003] Existing automated FISH interpretation workflows typically include the following steps: first, the tumor region boundary is delineated on hematoxylin-eosin (HE) stained sections; then, the roughly corresponding region is marked on the FISH section, and cell nucleus and probe signal counts are performed within this region. However, in this process, due to the inconsistent placement orientation of tissue blocks during section preparation, multiple independent tissue blocks may exist on a single section, each with different rotation or mirror flip states, leading to significant deviations in the overall registration of HE, IHC, and FISH images. Furthermore, because existing workflows lack a clear quantitative method for accurately locating "2+ invasive cancer regions" on IHC images, they often rely on subjective human judgment, resulting in misalignment between the cell regions detected on the FISH image and the actual target area to be detected. This often leads to the inclusion of non-invasive cancer cells or cells from non-2+ regions in the statistics, causing result bias.

[0004] Therefore, there is an urgent need for a breast FISH identification and interpretation HER2 scoring method based on multi-stained slide registration and its application to solve the problems existing in the current technology. Summary of the Invention

[0005] This invention provides a method for identifying and interpreting HER2 scores in breast cancer based on multi-stain section registration and its application. It addresses the problem that existing pathological image analysis processes do not establish a precise three-level spatial mapping mechanism from the HE tumor region to the IHC target region and then to the FISH detection region, which leads to misalignment between the FISH detection region and the invasive carcinoma region with an IHC score of 2+, seriously affecting the reliability of HER2 status interpretation.

[0006] The core technology of this invention is to construct a full-link spatial alignment and interpretation method that includes "HE whole-slice tumor segmentation - HE to IHC adaptive local registration - IHC targeted scoring region segmentation based on deep learning model - IHC to immunofluorescence image re-registration mapping - precise intracellular probe counting". It achieves accurate mapping across stained slices through independent weighted matching and optimized geometric transformation for multiple tissue blocks.

[0007] In a first aspect, the present invention provides a method for identifying and interpreting HER2 scores in breast cancer based on multi-stained slide registration using FISH, the method comprising the following steps:

[0008] Obtain whole-section images of HE, HER2-IHC, and immunofluorescence-enhanced breast tissue. Tumor regions were segmented from the whole-section HE image to obtain the HE tumor region mask; The HE whole slice image and the HER2-IHC whole slice image were registered in the first stage to establish a spatial mapping from HE image to IHC image, and the HE tumor region mask was mapped onto the HER2-IHC whole slice image. On a HER2-IHC whole-slice image mapped with a HE tumor region mask, the invasive cancer region with the target score is segmented based on a prediction model to obtain the target region mask. A second-stage registration was performed between the HER2-IHC whole-section image and the immunofluorescence whole-section image to establish a spatial mapping from the IHC image to the immunofluorescence image, and the target region mask was mapped onto the immunofluorescence whole-section image. Cell nuclei were detected within a mask of the target region mapped onto the immunofluorescence whole-section image to obtain cell-level counting units. Probe signals were counted for each counting unit, and the HER2 status was determined based on the counting results.

[0009] Furthermore, the HE whole-slice image and the HER2-IHC whole-slice image are registered in the first stage to establish a spatial mapping from the HE image to the IHC image, including: Tissue region segmentation was performed on the HE whole slice image and the HER2-IHC whole slice image respectively, and at least one independent tissue region contained in each was extracted; A multi-feature weighted scoring matching algorithm was used to establish region pair matching relationships between independent tissue regions in HE whole slice images and independent tissue regions in HER2-IHC whole slice images; For each matched region pair, perform multi-transformation combination registration, and evaluate the registration quality based on the relative target registration error and distance improvement rate after registration, so as to select the best registration transformation matrix; The best registration transformation matrices of all matching region pairs are fused and mapped to the global coordinate system to complete the first stage of registration.

[0010] Furthermore, a multi-feature weighted scoring matching algorithm is employed to establish region pair matching relationships between independent tissue regions in the HE whole-slice image and independent tissue regions in the HER2-IHC whole-slice image, specifically including: Shape features, contour features, texture features, color distribution features, and feature point matching features of each independent tissue region in the HE whole slice image and the HER2-IHC whole slice image were extracted respectively for similarity calculation. The similarity scores of each dimension are weighted and summed to obtain a comprehensive similarity score, in which the feature point matching feature has the highest weight. Based on the preset scoring threshold and greedy strategy, establish corresponding region pair matching relationships.

[0011] Furthermore, a second-stage registration was performed between the HER2-IHC whole-section images and the immunofluorescence whole-section images to establish a spatial mapping from IHC images to immunofluorescence images, including: Using the immunofluorescence whole-section image as the target, a pre-defined registration algorithm is used to map the target region mask onto the corresponding region in the immunofluorescence whole-section image.

[0012] Furthermore, based on the prediction model, the invasive cancer region with the target score is segmented to obtain a target region mask, including: The whole-slice image of HER2-IHC with a mask of HE tumor region was processed into blocks; The segmented image is input into a deep learning segmentation model that has been pre-trained with consistency test annotations. The model predicts and outputs regions with moderate cell membrane staining, which serve as target region masks with a score of 2+.

[0013] Furthermore, cell nucleus detection was performed within a mask of the target region mapped onto the immunofluorescence whole-section image to obtain cell-level counting units. Probe signals were counted for each counting unit, and the HER2 status was determined based on the counting results, including: Within the mapped target region mask, load the DAPI channel of the immunofluorescence whole-section image; Cell nuclei were detected within the DAPI channel using a cell segmentation model, with each detected cell nucleus mask being treated as an independent counting unit. Using a signal recognition model, the number of HER2 probe fluorescence signals and the number of CEP17 probe fluorescence signals in each counting unit were identified and counted. Calculate the average HER2 signal count and the ratio of HER2 to CEP17 signal for all detected cells, compare them with the preset diagnostic threshold, and output the HER2 status.

[0014] Furthermore, the invasive cancer region for the target score is the invasive cancer region with an immunohistochemical score of 2+.

[0015] Secondly, the present invention provides a breast FISH identification and interpretation HER2 scoring device based on multi-stained slide registration, comprising: The acquisition module is used to acquire whole-section images of HE, HER2-IHC, and immunofluorescence of breast tissue. The first segmentation module is used to segment the tumor region of the HE whole slice image to obtain the HE tumor region mask; The first mapping module is used to perform a first-stage registration between the HE whole slice image and the HER2-IHC whole slice image, establish a spatial mapping from the HE image to the IHC image, and map the HE tumor region mask onto the HER2-IHC whole slice image. The second segmentation module is used to segment the invasive cancer region with the target score on the HER2-IHC whole slice image mapped with HE tumor region mask, based on the prediction model, and obtain the target region mask. The second mapping module is used to perform a second-stage registration between the HER2-IHC whole-slice image and the immunofluorescence whole-slice image, establish a spatial mapping from the IHC image to the immunofluorescence image, and map the target region mask onto the immunofluorescence whole-slice image. The interpretation module is used to detect cell nuclei within a mask of the target region mapped to the immunofluorescence whole-section image, obtain cell-level counting units, count probe signals in each counting unit, and determine the HER2 status based on the counting results.

[0016] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to execute the above-described method for breast FISH identification and interpretation of HER2 scoring based on multi-stained slide registration.

[0017] Fourthly, the present invention provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the above-described method for interpreting HER2 scores based on multi-stain slide registration for breast FISH identification.

[0018] The main contributions and innovations of this invention are as follows: 1. By introducing segmentation of independent local tissue regions in the whole slice, multi-feature weighted matching, and multi-transformation optimal registration, the problem of global inaccurate alignment caused by local flipping and rotation during the preparation process of multi-tissue slices is effectively solved, significantly improving the spatial registration accuracy of HE and IHC images, so that the tumor region mask can be accurately mapped to the corresponding stained area.

[0019] 2. Based on a deep learning model validated by consistent annotations from pathology experts, this study achieves for the first time automated and highly repeatable quantitative localization of a specific staining intensity micro-pattern of “score 2+” on digital pathology images. This eliminates subjective discrimination differences and ensures that the target area for FISH detection is strictly limited to the “uncertain” invasive cancer area defined in the guidelines.

[0020] 3. By using a two-step cascade mapping mechanism from HE to IHC and then from IHC to immunofluorescence, a three-level spatial consistency bridge of "morphological localization region - protein expression discrimination region - gene detection and counting region" is constructed, which completely eliminates the hidden danger of confusion in non-target region cell counting caused by simple registration of slices directly in the existing technology.

[0021] 4. In the final interpretation stage, the cell segmentation model is used to accurately define the boundary of a single cell nucleus within the DAPI channel as an independent counting unit, and the dual probe signals are counted separately accordingly. This avoids interference errors from non-whole cells or overlapping signals, and comprehensively improves the objectivity of FISH test result interpretation and the effectiveness of diagnostic decision support.

[0022] Details of one or more embodiments of the present invention are set forth in the following drawings and description, so that other features, objects and advantages of the invention will be more readily understood. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of a breast FISH identification and interpretation HER2 scoring method based on multi-stained slide registration according to an embodiment of the present invention; Figure 2 These are the HE full slice image (left), IHC full slice image (middle), and registration images of each region (right) according to embodiments of the present invention. Figure 3 The images shown are a full HE slice (left) and a tumor segmentation result (red represents the tumor area) according to an embodiment of the present invention. Figure 4The image shows a full slice of HER2-IHC (left image) and a result image of an invasive carcinoma region with a score of 2+ (right image, yellow represents the invasive carcinoma region with a score of 2+) according to an embodiment of the present invention. Figure 5 The image shown is an immunofluorescence whole-section image (left image) and a registration image of an invasive carcinoma region with a score of 2+ (right image, purple represents the invasive carcinoma region) according to an embodiment of the present invention. Figure 6 The image shows the FISH detection results according to an embodiment of the present invention (green outline represents cell nuclei, R represents HER2 fluorescence signal, G represents CEP17 fluorescence signal, and the value represents the number of detected signals). Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0025] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0026] Example 1 This embodiment provides a breast FISH identification and interpretation method based on multi-stained section registration, primarily used for fully automated FISH-assisted interpretation of invasive breast cancer sections with a HER2-IHC score of 2+ in clinical pathological diagnosis. This method achieves precise spatial mapping from the tissue level to the cellular level through whole-section HE tumor segmentation, multi-stage multimodal image registration, invasive cancer target region localization, and single-cell level signal statistics.

[0027] See Figure 1 This illustrates the overall flow of the method of the present invention. The specific execution flow of this embodiment includes the following steps: Step 1: Segmentation of the tumor region in a whole slice of HE image.

[0028] First, the acquired whole-section breast tissue images obtained via hematologic resection (HE) were preprocessed. The original HE whole-section images with a scan resolution of, for example, 0.5 μm / pixel were loaded, and Gaussian filtering with a standard deviation σ = 1.2 was used to remove image noise. Contrast-limited adaptive histogram equalization (CLAHE) was then applied to enhance image contrast, with a clip limit of 2.0 to ensure that the grayscale difference between tumor tissue and normal glandular and adipose tissue was significantly amplified (e.g., grayscale difference ≥ 30), making the tumor region features more prominent.

[0029] After preprocessing, a first semantic segmentation model (e.g., the TIA Toolbox semantic segmentation model) is invoked to perform inference on the whole slice. The model's prediction results retain only the tumor tissue region, directly outputting a binary segmentation mask of the tumor region in the HE whole slice. See [link to documentation]. Figure 3 The left image shows an example of a whole-section HE slice, while the red area in the right image marks the segmented tumor region mask. This mask will serve as the spatial reference for subsequent multi-stain section registration.

[0030] Step 2: First-stage registration of HE whole-slice image with HER2-IHC whole-slice image.

[0031] This step aims to establish a spatial mapping from HE images to IHC images and accurately map the HE tumor region mask onto the IHC image. It is a crucial step in resolving global misalignment caused by local flipping and rotation of multiple tissue slices. Specifically, it includes the following sub-steps: (1) Region Segmentation and Extraction: Tissue region segmentation was performed on the HE full-slice image and the HER2-IHC full-slice image, respectively. For the IHC image, the Mask2Former deep learning model (e.g., configured as maskformer2_swin_large_IN21k_384_bs16_90k.yaml) was used for semantic segmentation, dividing the slice content into background, foreground tissue, and contrast tissue. For the HE image, the same deep learning model was used for semantic segmentation to extract the foreground tissue region. Noise fragment regions with an area of ​​less than 5000 pixels were filtered out in both images, and the bounding boxes, region identifiers, and location information of each independent foreground tissue region were recorded, thereby extracting at least one independent tissue region contained in each.

[0032] (2) Region Matching: Since the placement and angle of tissue blocks are random during slice preparation, this embodiment does not rely on spatial coordinates, but instead uses a multi-feature weighted scoring matching algorithm. Shape features of local regions in the HE image and the foreground tissue region of the IHC are extracted using the ORB feature detector (nfeatures=2000). Contour features Texture features S_texture, color distribution features and feature point matching features .

[0033] Calculate the overall similarity score:

[0034] The weight configuration is as follows: =0.15, =0.10, =0.10, =0.15, =0.50. Feature point matching features are assigned the highest weight to ensure the robustness of matching correspondences within the tissue structure. A similarity threshold of 0.8 is set, and a greedy strategy is adopted to match each HE tissue region with the IHC region pair that has the highest comprehensive similarity score and exceeds the threshold.

[0035] (3) Multi-transformation registration: For each matched region pair, since the orientation of the tissue block cannot be predicted, the system automatically traverses multiple candidate geometric transformations, including no transformation, 90° rotation, 180° rotation, 270° rotation, horizontal flip, vertical flip, and their combinations. For the image under each transformation, the VALIS algorithm is used to perform rigid and non-rigid registration. To objectively select the optimal transformation, a comprehensive quality score is calculated:

[0036] in, This represents the relative target registration error for rigid registration. The average distance between the registered images. This represents the original average distance between images before registration. Images are selected based on a quality score ≥ 0.97 and... The optimal registration transformation matrix under the condition <0.02.

[0037] (4) Transformation Fusion Mapping: For local region pairs that have obtained the best registration transformation, their transformation matrices are fused into the global coordinate system. The global transformation matrix is ​​calculated as follows:

[0038] in, This is a translation transformation from global to local HE. The optimal registration matrix for VALIS. To flip the inverse transformation matrix, This involves a translation transformation from local to global IHC. Using the generated coordinate mapping function, the HE tumor region mask obtained in step 1 is precisely mapped onto the HER2-IHC whole-slice image. See [link to relevant documentation]. Figure 2 The images, from left to right, show the independent tissue regions of the HE whole-section image, the independent tissue regions of the IHC whole-section image, and the effect of fusion and registration of the two. It can be seen that each tissue region has been accurately aligned.

[0039] Step 3: Locate the invasive carcinoma region with a HER2-IHC score of 2+.

[0040] IHC images mapping tumor regions were processed into 2048×2048 pixel grid blocks. These block images were then input into a pre-trained second semantic segmentation model (e.g., the PIDNet segmentation model, with PIDNet_M_ImageNet as the baseline). This model was trained by three senior pathologists on a high-quality dataset with an annotation consistency greater than 0.8 (Kappa test), and possesses the ability to recognize the specific microscopic pattern of "moderate-intensity intact cell membrane staining." The model's inference output includes a mask of invasive cancer regions with a score of 2+, which serves as the target region mask, thereby accurately distinguishing invasive cancer regions with a score of 2+ from non-2+ regions. See also... Figure 4 The left image shows a whole slice image of HER2-IHC, and the yellow area in the right image marks the segmented and located invasive carcinoma area with a score of 2+.

[0041] Step 4: Second-stage registration of HER2-IHC whole-section images with immunofluorescence whole-section images.

[0042] This step establishes a spatial mapping from IHC images to immunofluorescence images and remaps the target region mask. Using the immunofluorescence whole-section image (including the DAPI channel) as the target image also faces the challenge of random deformation and shifting of the tissue block during preparation. Therefore, a local registration strategy similar to that in step 2 is adopted, including: The IHC image is segmented to extract independent regions; the DAPI channel of the immunofluorescence image is used as a morphological reference to extract the corresponding tissue region contour; the correspondence between the two regions is established based on multi-feature weighted scoring matching; local registration of each region pair is performed with multiple transformation combinations, and the best transformation matrix is ​​selected according to the registration index; finally, the local transformations are fused into the global transformation to achieve accurate mapping of the target region mask to the immunofluorescence image.

[0043] See Figure 5 The left image shows an example of an immunofluorescence whole-section image, and the purple area in the right image indicates the target region mask mapped onto it.

[0044] Step 5: Immunofluorescence DAPI channel cell detection.

[0045] Within the target region mask mapped in the immunofluorescence image, the area was further segmented, with the segment size adjusted to 512×512 pixels to accommodate the microscale of FISH signal points and balance computational resources. The DAPI channel image was loaded, and cell segmentation was performed using a cell segmentation model. This embodiment uses the Cellpose model, specifically its "nuclei" baseline model, and sets the average diameter parameter of the target cell nucleus to 12 μm based on the nuclear size characteristics of invasive breast cancer cells. The model outputs a precise contour mask for each individual cell nucleus; each detected nucleus mask is defined as an independent FISH counting unit. This step effectively eliminates signal counting interference from cells outside the 2+ score region, ensuring that subsequent analysis is performed within the target invasive cancer cell nucleus. See [link to relevant documentation]. Figure 6 The green outline represents the cell nucleus boundary segmented by the DAPI channel.

[0046] Step 6: FISH diagnosis and HER2 status determination.

[0047] For each counting unit, a signal recognition model (such as the RS-FISH model) was used to identify red fluorescent signals (HER2 gene probe) and green fluorescent signals (CEP17 chromosome probe). The number of HER2 signals in the i-th infiltrating cancer cell was counted. and CEP17 signal number N CEP17,i Calculate the average number of HER2 signals per cell:

[0048] and the ratio of HER2 to CEP17 signal counts (in The average number of CEP17 signals per single invasive cancer cell. ).

[0049] Logical judgment is performed according to a preset dual-probe judgment standard threshold. The judgment rules used in this embodiment include: If R ≥ 2.0 and C ≥ 4.0, it is considered HER2 positive (amplification); If R < 2.0 and C ≥ 6.0, it is considered HER2 positive (amplification); If R < 2.0 and 4.0 ≤ C < 6.0, it is determined to be HER2 negative (no amplification); If R ≥ 2.0 and C < 4.0, it is determined to be HER2 negative (no amplification); If R < 2.0 and C < 4.0, it is determined to be HER2 negative (no amplification).

[0050] Finally, the HER2 status of the sample is automatically output by computer. For cases without amplification, some results can be further classified into the category of "low HER2 expression," providing precise pathological stratification for targeted therapies such as antibody-drug conjugates.

[0051] Example 2 Based on the same inventive concept, this embodiment provides a breast FISH identification and interpretation HER2 scoring device based on multi-stained slide registration. The device includes: The acquisition module is used to acquire whole-section images of breast tissue; The first segmentation module is used to segment the tumor region of the HE whole slice image to obtain the HE tumor region mask; The first mapping module is used to perform the first-stage registration and mask mapping from HE to IHC; The second segmentation module is used to segment the invasive cancer target region mask with a score of 2+ on the IHC image; The second mapping module is used to perform the second-stage registration and target mapping from IHC to immunofluorescence images; The interpretation module is used for cell nucleus segmentation, dual-probe signal counting, and HER2 status logic interpretation within the immunofluorescence target area.

[0052] The functions of each module correspond one-to-one with the steps of the above method embodiments, and will not be repeated here.

[0053] Example 3 This embodiment also provides an electronic device, see reference. Figure 7 It includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0054] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement embodiments of the present invention.

[0055] Memory 404 may include a mass storage device for data or instructions. For example, and not limitingly, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to a data processing device. In a particular embodiment, memory 404 is non-volatile memory. In a particular embodiment, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0056] The memory 404 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 402.

[0057] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any of the breast FISH identification and interpretation HER2 scoring methods based on multi-stained slide registration in the above embodiments.

[0058] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected to the processor 402, and the input / output device 408 is connected to the processor 402.

[0059] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 406 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0060] Input / output device 408 is used to input or output information.

[0061] Example 4 This embodiment also provides a readable storage medium storing a computer program, which includes program code for controlling and executing a process, the process including the breast FISH identification and interpretation HER2 scoring method based on multi-stained slide registration according to Embodiment 1.

[0062] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0063] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0064] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product may include one or more computer-executable components configured to perform the embodiments when the program is run. The one or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted in this respect that, as Figure 1 Any box in the logical flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0065] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] The above embodiments are merely illustrative of several implementations of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the appended claims.

Claims

1. A method for identifying and interpreting HER2 scores in breast cancer using FISH, characterized in that, Includes the following steps: Obtain whole-section images of HE, HER2-IHC, and immunofluorescence-enhanced breast tissue. The HE whole slice image is segmented into tumor regions to obtain an HE tumor region mask; The HE whole slice image and the HER2-IHC whole slice image are registered in the first stage to establish a spatial mapping from the HE image to the IHC image, and the HE tumor region mask is mapped onto the HER2-IHC whole slice image. On the HER2-IHC whole-slice image mapped with the HE tumor region mask, the invasive cancer region with the target score is segmented based on the prediction model to obtain the target region mask; The HER2-IHC whole-section image and the immunofluorescence whole-section image are registered in the second stage to establish a spatial mapping from the IHC image to the immunofluorescence image, and the target region mask is mapped onto the immunofluorescence whole-section image. Cell nucleus detection is performed within the target region mask mapped to the immunofluorescence whole slice image to obtain cell-level counting units, and probe signal counting is performed on each counting unit to determine the HER2 status based on the counting results; The first-stage registration involves matching the HE full-slice image with the HER2-IHC full-slice image to establish a spatial mapping from the HE image to the IHC image, including: The HE whole-slice image and the HER2-IHC whole-slice image are respectively segmented into tissue regions, and at least one independent tissue region contained in each is extracted. A multi-feature weighted scoring matching algorithm is used to establish region pair matching relationships between independent tissue regions in the HE whole slice image and independent tissue regions in the HER2-IHC whole slice image; For each matched region pair, perform multi-transformation combined registration, and evaluate the registration quality based on the relative target registration error and distance improvement rate after registration, so as to select the best registration transformation matrix; The best registration transformation matrices of all matching region pairs are fused and mapped to the global coordinate system to complete the first stage of registration.

2. The breast FISH identification and interpretation HER2 scoring method according to claim 1, characterized in that, A multi-feature weighted scoring matching algorithm is used to establish region pair matching relationships between independent tissue regions in the HE whole-slice image and independent tissue regions in the HER2-IHC whole-slice image, specifically including: The shape features, contour features, texture features, color distribution features, and feature point matching features of each independent tissue region in the HE whole slice image and the HER2-IHC whole slice image are extracted respectively to calculate the similarity. The similarity scores of each dimension are weighted and summed to obtain a comprehensive similarity score, in which the feature point matching feature has the highest weight. Based on the preset scoring threshold and greedy strategy, establish corresponding region pair matching relationships.

3. The breast FISH identification and interpretation HER2 scoring method according to claim 1, characterized in that, The HER2-IHC whole-slice image and the immunofluorescence whole-slice image are registered in a second stage to establish a spatial mapping from the IHC image to the immunofluorescence image, including: Using the immunofluorescence whole-section image as the target, a preset registration algorithm is used to map the target region mask onto the corresponding region in the immunofluorescence whole-section image.

4. The breast FISH identification and interpretation HER2 scoring method according to claim 3, characterized in that, Based on the predictive model, the invasive cancer region with the target score is segmented to obtain a target region mask, including: The HER2-IHC whole-slice image mapped with the HE tumor region mask is segmented; The segmented image is input into a deep learning segmentation model that has been pre-trained with consistency check annotations. The model predicts and outputs regions with moderate cell membrane staining, which serve as masks for the target regions with a score of 2+.

5. The breast FISH identification and interpretation HER2 scoring method according to claim 1, characterized in that, Cell nucleus detection is performed within the target region mask mapped to the immunofluorescence whole-slice image to obtain cell-level counting units. Probe signals are counted for each counting unit, and the HER2 status is determined based on the counting results, including: Within the mapped target region mask, the DAPI channel of the immunofluorescence whole-slice image is loaded; Cell nuclei were detected within the DAPI channel using a cell segmentation model, and each detected cell nucleus mask was treated as an independent counting unit. Using a signal recognition model, the number of HER2 probe fluorescence signals and the number of CEP17 probe fluorescence signals in each counting unit were identified and counted. The average number of HER2 signals and the ratio of HER2 to CEP17 signals of all detected cells are calculated and compared with a preset diagnostic threshold to output the HER2 status.

6. The breast FISH identification and interpretation HER2 scoring method according to any one of claims 1 to 5, characterized in that, The invasive cancer region for the target score is the invasive cancer region with an immunohistochemical score of 2+.

7. A breast FISH identification and interpretation HER2 scoring device, characterized in that, include: The acquisition module is used to acquire whole-section images of HE, HER2-IHC, and immunofluorescence of breast tissue. The first segmentation module is used to segment the tumor region of the HE whole slice image to obtain an HE tumor region mask. The first mapping module is used to perform a first-stage registration between the HE whole-slice image and the HER2-IHC whole-slice image, establishing a spatial mapping from the HE image to the IHC image, and mapping the HE tumor region mask onto the HER2-IHC whole-slice image; wherein, performing the first-stage registration between the HE whole-slice image and the HER2-IHC whole-slice image to establish a spatial mapping from the HE image to the IHC image includes: The HE whole-slice image and the HER2-IHC whole-slice image are respectively segmented into tissue regions, and at least one independent tissue region contained in each is extracted. A multi-feature weighted scoring matching algorithm is used to establish region pair matching relationships between independent tissue regions in the HE whole slice image and independent tissue regions in the HER2-IHC whole slice image; For each matched region pair, perform multi-transformation combined registration, and evaluate the registration quality based on the relative target registration error and distance improvement rate after registration, so as to select the best registration transformation matrix; The best registration transformation matrices of all matching region pairs are fused and mapped to the global coordinate system to complete the first stage of registration. The second segmentation module is used to segment the invasive cancer region with the target score on the HER2-IHC whole slice image mapped with the HE tumor region mask, based on a prediction model, to obtain the target region mask. The second mapping module is used to perform a second-stage registration between the HER2-IHC whole-slice image and the immunofluorescence whole-slice image, establish a spatial mapping from the IHC image to the immunofluorescence image, and map the target region mask onto the immunofluorescence whole-slice image. The interpretation module is used to perform cell nucleus detection within the target region mask mapped to the immunofluorescence whole slice image, obtain cell-level counting units, count probe signals for each counting unit, and determine the HER2 status based on the counting results.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the breast FISH identification and interpretation HER2 scoring method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that, The readable storage medium stores a computer program, the computer program including program code for controlling a process to execute the process, the process including the breast FISH identification and interpretation HER2 scoring method according to any one of claims 1 to 6.

Images