Multiple staining image matching method, apparatus, medium, and device

Abstract

The application discloses a kind of multiple dyeing image matching method, device, medium and equipment, first obtain the multiple dyeing images of multiple pathological sections, wherein each pathological section is obtained after being dyed by a kind of preset dyeing method and is photographed to obtain a corresponding dyeing image, then in each dyeing image, the contour of pathological section is extracted respectively, and the matching feature of corresponding pathological section is identified based on the extracted contour, then the relative position of the dyeing image to be matched and the preset reference dyeing image is adjusted, until the matching feature of the dyeing image to be matched and the matching feature of reference dyeing image satisfy preset matching requirement, finally obtain the matching parameter between the dyeing image to be matched and reference dyeing image, and calculate matching value according to matching parameter, if matching value is greater than preset matching value threshold, then determine that the dyeing image to be matched is matched successfully. Visible, the present application can automatically complete the automatic matching of pathological image of multiple levels different dyeing methods.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a multi-staining image matching method, device, medium and equipment. BACKGROUND

[0002] Pathological examination is widely used in clinical work and scientific research. Through clinical pathological analysis, a large amount of valuable scientific research data can be obtained.

[0003] In pathological examination, different staining operations are often needed on the same wax block, including hematoxylin-eosin staining (HE), immunohistochemistry (IHC) staining, special staining, immunofluorescence (IF), etc. In order to obtain staining pathological images of substantially the same position, the same wax block is continuously sectioned to form multiple layers of pathological sections, which are then stained by different staining schemes. However, after multiple staining on the same wax block at multiple layers, the position and direction of the wax block on different slides will be different, which will cause difficulty in image matching, and manual matching is needed. SUMMARY

[0004] Therefore, it is necessary to provide a multi-staining image matching method, device, medium and equipment to solve the problem of difficulty in matching of the existing staining images and the need for manual matching.

[0005] A multi-staining image matching method, the method comprising:

[0006] Obtaining a plurality of staining images of a plurality of pathological sections; wherein the plurality of pathological sections are obtained by processing the same pathological wax block, and each pathological section is photographed after being stained by a predetermined staining method to obtain a corresponding staining image;

[0007] In each staining image, the contour of the pathological section is extracted, and the matching feature of the corresponding pathological section is identified based on the extracted contour;

[0008] Adjusting the relative position of the to-be-matched staining image and the predetermined reference staining image until the matching features of the to-be-matched staining image and the reference staining image meet the predetermined matching requirement; wherein the to-be-matched staining image is any one of the plurality of staining images except the reference staining image;

[0009] Obtaining a matching parameter between the to-be-matched staining image and the reference staining image, and calculating a matching value according to the matching parameter, and determining that the to-be-matched staining image is matched successfully if the matching value is greater than a preset matching value threshold.

[0010] In one of the embodiments, the plurality of staining images are labeled with numbers, and the numbers of the staining images corresponding to the adjacent pathological sections in the pathological wax block are adjacent, and the matching features of the corresponding pathological sections are identified based on the extracted contour, including:

[0011] The contour in the target staining image is extended in combination with the size of the contour in the adjacent staining image to obtain an extended contour of the target staining image, wherein the target staining image is any one of the plurality of staining images, and the number of the adjacent staining image is adjacent to the number of the target staining image.

[0012] The center point, the major axis and the minor axis of the extended contour are identified, and the center point, the major axis and the minor axis are taken as the matching features, wherein the major axis is the longest axis in the extended contour passing through the center point, and the minor axis is the axis in the extended contour passing through the center point and perpendicular to the major axis.

[0013] In one of the embodiments, the contour in the target staining image is extended in combination with the size of the contour in the adjacent staining image to obtain an extended contour of the target staining image, including:

[0014] The contour in the target staining image is synchronously extended until the contour in the target staining image covers the contour in the adjacent staining image after extension to obtain the extended contour of the target staining image.

[0015] In one of the embodiments, the relative position of the to-be-matched staining image and the preset reference staining image is adjusted until the matching features of the to-be-matched staining image and the matching features of the reference staining image satisfy the preset matching requirement, including:

[0016] The relative position of the to-be-matched staining image and the preset reference staining image is adjusted, and when the center point of the to-be-matched staining image is aligned with the center point of the reference staining image, and the direction of the major axis of the to-be-matched staining image is consistent with the direction of the major axis of the reference staining image, and the direction of the minor axis of the to-be-matched staining image is consistent with the direction of the minor axis of the reference staining image, it is determined that the matching features of the to-be-matched staining image and the matching features of the reference staining image satisfy the preset matching requirement.

[0017] In one of the embodiments, the matching features of the corresponding pathological sections are identified based on the extracted contour, including:

[0018] identify a center point of the contour in the target staining image; wherein the target staining image is any one of the plurality of staining images;

[0019] divide the contour in the target staining image into a plurality of sub-contours based on the center point, and take shapes of the plurality of sub-contours as the matching features; wherein each sub-contour has the same angle formed with the center point.

[0020] In one of the embodiments, the adjusting the relative position of the to-be-matched staining image and the preset reference staining image until the matching features of the to-be-matched staining image and the matching features of the reference staining image satisfy the preset matching requirement comprises:

[0021] adjusting the relative position of the to-be-matched staining image and the preset reference staining image until the center point of the contour of the to-be-matched staining image is aligned with the center point of the contour of the reference staining image, and the number of the matching sub-contours in the to-be-matched staining image is greater than the preset number threshold, to determine that the matching features of the to-be-matched staining image and the matching features of the reference staining image satisfy the preset matching requirement; wherein if the matching degree between two sub-contours of different staining images is greater than the preset matching degree threshold, the two sub-contours are determined as matching sub-contours.

[0022] In one of the embodiments, the calculation formula of the matching value is:

[0023] P = 1 - [(A - C) 2 + (B - C) 2 ] / (A + B)

[0024] In the above formula, P is the matching value, A, B, and C are matching parameters, A indicates the area of the contour in the reference staining image, B indicates the area of the contour in the to-be-matched staining image, and C indicates the area of the overlapping contour of the reference staining image and the to-be-matched staining image.

[0025] A multi-staining image matching device, the device comprising:

[0026] a feature extraction module configured to obtain a plurality of staining images of a plurality of pathological sections; wherein the plurality of pathological sections are obtained from the same pathological wax block, and each pathological section is dyed by a preset staining method and then an image corresponding to the staining image is taken; in each staining image, a contour of the pathological section is extracted, and a matching feature of the corresponding pathological section is identified based on the extracted contour;

[0027] The matching module is configured to adjust the relative position of a to-be-matched staining image and a preset reference staining image until the matching features of the to-be-matched staining image and the matching features of the reference staining image satisfy a preset matching requirement.

[0028] The matching confirmation module is configured to obtain a matching parameter between the to-be-matched staining image and the reference staining image, and calculate a matching value according to the matching parameter, and if the matching value is greater than a preset matching value threshold, it is determined that the to-be-matched staining image is matched successfully.

[0029] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to make the processor execute the steps of the multiple staining image matching method.

[0030] A multiple staining image matching device includes a memory and a processor, and the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the multiple staining image matching method.

[0031] The present application provides a multiple staining image matching method, device, medium and equipment, first, a plurality of pathological sections of multiple staining images are obtained, wherein each pathological section is stained by a preset staining method to obtain a corresponding staining image, then in each staining image, the outline of the pathological section is extracted respectively, and the matching features of the corresponding pathological section are identified based on the extracted outline, then the relative position of the to-be-matched staining image and the preset reference staining image is adjusted until the matching features of the to-be-matched staining image and the matching features of the reference staining image satisfy the preset matching requirement, finally, the matching parameter between the to-be-matched staining image and the reference staining image is obtained, and the matching value is calculated according to the matching parameter, if the matching value is greater than the preset matching value threshold, it is determined that the to-be-matched staining image is matched successfully. It can be seen that the present application can automatically complete the automatic matching of multiple pathological images with different staining methods. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0033] Among them:

[0034] Figure 1 It is a flowchart of the multiple staining image matching method in the first embodiment.

[0035] Figure 2 Fig. 1 is a schematic diagram of processing pathological wax blocks into multiple staining images;

[0036] Figure 3 Fig. 2 is a schematic diagram of extracting the contour of pathological sections within the staining images;

[0037] Figure 4 Fig. 3 is a schematic diagram of the flow of the multiple staining image matching method in the second embodiment;

[0038] Figure 5 Fig. 4 is a schematic diagram of extending the contour within the target staining image and determining the matching features;

[0039] Figure 6 Fig. 5 is a schematic diagram of the to-be-matched staining image before adjusting the relative position;

[0040] Figure 7 Fig. 6 is a schematic diagram of the to-be-matched staining image after adjusting the relative position;

[0041] Figure 8 Fig. 7 is a schematic diagram of the flow of the multiple staining image matching method in the third embodiment;

[0042] Figure 9 Fig. 8 is a schematic diagram of the structure of the multiple staining image matching device in one embodiment;

[0043] Figure 10 Fig. 9 is a structural block diagram of the multiple staining image matching device in one embodiment. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0045] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed or can optionally include other steps or units inherent to the process, method, product or device.

[0046] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0047] like Figure 1 As shown, Figure 1 This is a flowchart illustrating the multiple staining image matching method in the first embodiment. The steps provided by the multiple staining image matching method in this first embodiment include:

[0048] S101: Obtain multiple stained images of multiple pathological sections.

[0049] Specifically, the excised human biological samples are sent to the pathology department for fixation, sampling, dehydration, embedding, sectioning, staining, and other processes to generate pathological paraffin blocks containing multiple pathological sections. These pathological sections can be observed under a microscope or scanned to generate digital pathological images for observation and analysis on a display screen.

[0050] In some cases, the same pathological paraffin block requires multiple staining protocols. For example, breast cancer requires multiple staining methods such as HE staining, IHC staining (ER, PR, HER2, Ki 67, etc.), and IF staining (HER2). Figure 2 As shown, the same pathological paraffin block needs to be sliced ​​into multiple layers to generate multiple unstained white pathological slides. Then, each pathological slide is stained with a preset staining method and a corresponding stained image is obtained.

[0051] S102, In each stained image, the contour of the pathological section is extracted, and the matching features of the corresponding pathological section are identified based on the extracted contour.

[0052] For example, suppose the stained image is an HE image, such as Figure 3 (a) shows the unprocessed HE image. Next, coarse-grained pixelation or black-and-white binarization is performed on the computer to obtain the image shown below. Figure 3 The schematic diagram shown in (b) is finally obtained through edge extraction. Figure 3 (c) shows the region image and edge contours, for example, using the Canny edge detection algorithm or a threshold-based image segmentation algorithm. This method removes any small noise that may be present in the background.

[0053] Further, the matching features corresponding to the pathological section can be identified by a computer vision algorithm, such as the center point in the contour, the extreme point of the x-axis or y-axis, the area of angle mutation, etc. The matching features are identified for subsequent matching between images.

[0054] S103, adjusting the relative position of the to-be-matched staining image and the preset reference staining image until the matching features of the to-be-matched staining image and the matching features of the reference staining image meet the preset matching requirement.

[0055] The to-be-matched staining image is any one of the plurality of staining images except the reference staining image. For example, the plurality of staining images include HE staining image a, HE staining image b, IHC staining image, and IF staining image. If the HE staining image a is taken as the reference staining image, the HE staining image b, the IHC staining image, and the IF staining image are all to-be-matched staining images.

[0056] The preset matching requirement can be that the matching features are aligned, such as the alignment of the contour center points of two staining images or the alignment of the extreme points of the x-axis and y-axis of two staining images, or that the matching degree is greater than a preset matching degree threshold, such as that the matching degree of the angle mutation area of two staining images is greater than a preset matching degree threshold. Of course, it can also be set according to the requirements.

[0057] S104, obtaining the matching parameters between the to-be-matched staining image and the reference staining image, and calculating a matching value according to the matching parameters, and determining that the to-be-matched staining image is matched successfully if the matching value is greater than a preset matching value threshold.

[0058] Optionally, the calculation formula of the matching value is:

[0059] P = 1 - [(A-C) 2 +(B-C) 2 ] / (A+B)

[0060] In the above formula, P is the matching value, A, B, and C are the matching parameters, A indicates the area of the contour in the reference staining image, B indicates the area of the contour in the to-be-matched staining image, and C indicates the area of the overlapping contour of the reference staining image and the to-be-matched staining image.

[0061] In practice, it is recommended to use 0.9 or 0.95 as the matching threshold; that is, a match is considered successful only when the P-value is greater than or equal to 0.9 or 0.95. Furthermore, a lower limit can be set, for example, 0.5. If the P-value is between 0.5 and 0.9, it indicates that there is some similarity between the pathological sections, but they are not a complete match. In this case, a pathologist needs to double-check to determine if there are any issues such as partial detachment or similar outlines caused by different paraffin blocks. If the P-value is below 0.5, it indicates that there is almost no overlap between the two pathological sections, possibly due to incorrect numbering or misattribution. In this case, further checks of section numbering or other possible error factors are needed.

[0062] Of course, you can also adjust the formula according to the actual situation. For example, A, B, and C do not indicate the area, but the number of cells they occupy. You can also consider removing the interfering pixels in A, B, and C.

[0063] Furthermore, the p-value can also be used to evaluate the slide preparation quality of a particular laboratory or pathologist. Each month, N cases from a given laboratory or pathologist can be randomly selected, the matching overlap rate (p-value) calculated, and the results compared to certain standards. If the p-value remains consistently high, it indicates that the laboratory or pathologist has excellent slide preparation skills and high slide quality. Conversely, if the p-value is below a certain level, it suggests potential problems with slide preparation quality and room for improvement. This method allows for quantitative evaluation and comparison of the slide preparation quality of pathology laboratories or pathologists, contributing to improved slide preparation quality and diagnostic accuracy, and ultimately enhancing the quality of medical services.

[0064] The aforementioned multiple staining image matching method first acquires multiple stained images of multiple pathological sections. Each pathological section is stained using a preset staining method, resulting in a corresponding stained image. Next, the contour of the pathological section is extracted from each stained image, and matching features are identified based on the extracted contours. The relative positions of the stained image to be matched and a preset reference stained image are then adjusted until the matching features of the stained image to be matched and the reference stained image meet preset matching requirements. Finally, matching parameters between the stained image to be matched and the reference stained image are obtained, and a matching value is calculated based on these parameters. If the matching value is greater than a preset matching value threshold, the stained image to be matched is considered successfully matched. Therefore, this invention can automatically complete the automatic matching of pathological images with different staining methods at multiple levels.

[0065] like Figure 4 As shown, Figure 4 This is a flowchart illustrating the multiple staining image matching method in the second embodiment. The steps provided by the multiple staining image matching method in this second embodiment include:

[0066] S401, obtaining a plurality of staining images of a plurality of pathological sections.

[0067] In one specific implementation scenario, the S401 is basically the same as the S101 of the multi-staining image matching method in the first embodiment, and will not be described here.

[0068] S402, extracting the contour of each pathological section in each staining image, and extending the contour in the target staining image in combination with the size of the contour in the adjacent staining image to obtain an extended contour of the target staining image.

[0069] The plurality of staining images are labeled with numbers, and the numbers of the staining images corresponding to the adjacent layered pathological sections in the pathological wax block are adjacent, for example, for the staining image numbered 2, the pathological section in the staining image is adjacent to the layered pathological sections in the staining images numbered 1 and 3 in the pathological wax block. At the same time, the target staining image is any one of the plurality of staining images, and the numbers of the adjacent staining images are adjacent to the number of the target staining image, that is, the same extension operation is performed on all staining images, and if the staining image numbered 2 is the target staining image, the staining images numbered 1 and 3 are adjacent staining images.

[0070] Optionally, in one specific embodiment, the contour in the target staining image is extended synchronously until the contour in the target staining image after extension covers the contour in the adjacent staining image to obtain the extended contour of the target staining image. That is, if the contour numbered 2 is extended, the pathological sections in the staining images numbered 1 and 3 need to be covered. Exemplarily, the final result is as shown in the schematic diagram of (a). Figure 5

[0071] In actual operation, it is generally necessary to extend 5 μm in width to both sides, and the basic extension range can cover the pathological images of 1-2 image layers before and after. If more layers of pathological images need to be covered, further appropriate extension can be made in proportion. It is recommended to extend 5-25 μm to both sides, and the extension distances of the two sides can also be unequal. The larger the value, the greater the fault tolerance, and the smaller the value, the smaller the fault tolerance.

[0072] Through the above processing, the differences between the layers can be reduced, and the accuracy of subsequent matching can be improved.

[0073] S403, identifying the center point, long axis and short axis of the extended contour, and taking the center point, long axis and short axis as matching features.

[0074] The long axis is the longest axis passing through the center point in the extended contour, and the short axis is the axis passing through the center point and perpendicular to the long axis in the extended contour.

[0075] ​The specific process of this step is: (1) find the center point of the extended contour. The center point of the extended contour can be automatically found by computer, and marked with a yellow dot. (2) find the long axis of the extended contour. Starting from the center point of the extended contour, scan in each direction of the contour, find the longest line segment passing through the center point as the long axis, and mark it with a green dashed line. (3) find the short axis of the extended contour. Generate the short axis through the center point of the extended contour, perpendicular to the long axis, and mark it with a gold dashed line. Exemplarily, the final result is as shown in Figure 5 (b) shows a schematic diagram.

[0076] S404, adjust the relative position of the to-be-matched staining image and the preset reference staining image, and when the center point of the to-be-matched staining image is aligned with the center point of the reference staining image, and the long axis direction of the to-be-matched staining image is consistent with the long axis direction of the reference staining image, and the short axis direction of the to-be-matched staining image is consistent with the short axis direction of the reference staining image, it is determined that the matching features of the to-be-matched staining image and the reference staining image meet the preset matching requirement.

[0077] Exemplarily, as Figure 6 (a), (b), and (c) are three to-be-matched staining images with unadjusted relative positions. Then, through the adjustment of this step, the to-be-matched staining images after adjustment are obtained as shown in Figure 7 (a), (b), and (c) show the to-be-matched staining images after adjustment, at this time it is determined that the matching features of the to-be-matched staining image and the reference staining image meet the preset matching requirement.

[0078] S405, obtain the matching parameters between the to-be-matched staining image and the reference staining image, and calculate the matching value according to the matching parameters, and if the matching value is greater than the preset matching value threshold, it is determined that the to-be-matched staining image matches successfully.

[0079] In one specific implementation scenario, the present S405 is basically the same as S104 of the multi-staining image matching method in the first embodiment, which will not be described here.

[0080] As Figure 8 shown, Figure 8 is a flowchart of the multi-staining image matching method in the third embodiment. The steps provided by the multi-staining image matching method in the third embodiment include:

[0081] S801, obtain a plurality of staining images of a plurality of pathological sections.

[0082] In one specific implementation scenario, the present S801 is basically the same as S101 of the multi-staining image matching method in the first embodiment, which will not be described here.

[0083] S802, extract the contour of the pathological section in each stained image, and identify the center point of the contour in the target stained image.

[0084] The target stained image is any one of the plurality of stained images, that is, the same operation is performed on all the stained images.

[0085] S803, divide the contour in the target stained image into a plurality of sub-contours based on the center point, and take the shape of the plurality of sub-contours as the matching feature.

[0086] Each sub-contour forms an angle with the center point. For example, if the angle is set to 60°, six sub-contours will be obtained. If the angle is set to 30°, twelve sub-contours will be obtained.

[0087] S804, adjust the relative position of the to-be-matched stained image and the preset reference stained image until the center points of the contours of the to-be-matched stained image and the reference stained image are aligned, and the number of matching sub-contours in the to-be-matched stained image is greater than the preset number threshold, and determine that the matching features of the to-be-matched stained image and the reference stained image meet the preset matching requirement.

[0088] If the matching degree between two sub-contours of different stained images is greater than the preset matching degree threshold, the two sub-contours are determined to be matching sub-contours.

[0089] For example, the matching degree can be calculated by Euclidean distance: in contour matching, each point in the two contours can be regarded as a point, and then the Euclidean distance between the points is calculated. The average of these distances is taken as the matching degree between the two contours.

[0090] Alternatively, if the shapes of the two contours are very similar but have slight differences, a shape matching algorithm can be used to calculate the matching degree between the two contours: the shape matching algorithm maps one contour onto another and tries to find the best matching way. One of the shape matching algorithms is Hu invariant moment, which represents the contour as a set of invariants and can be used to identify and match contours.

[0091] The above matching method is more accurate than the multiple stained image matching method of the second embodiment, and can be refined to the matching of each sub-contour.

[0092] S805, obtain the matching parameters between the to-be-matched stained image and the reference stained image, and calculate the matching value according to the matching parameters. If the matching value is greater than the preset matching value threshold, it is determined that the to-be-matched stained image is matched successfully.

[0093] In a specific implementation scenario, the S805 is basically the same as the S104 of the multi-staining image matching method in the first embodiment, and thus will not be described here.

[0094] Of course, the multi-staining image matching method of the second embodiment and the multi-staining image matching method of the third embodiment can be executed according to actual conditions, or can be executed synchronously, and then the two results are summarized and evaluated comprehensively.

[0095] In one embodiment, as shown in FIG. 8, a multi-staining image matching device is provided, which comprises: Figure 9

[0096] The feature extraction module 901 is configured to obtain a plurality of staining images of a plurality of pathological sections, wherein the plurality of pathological sections are obtained from the same pathological paraffin block, and each pathological section is stained by a predetermined staining method and then photographed to obtain a corresponding staining image. In each staining image, the contour of the pathological section is extracted, and the matching features of the corresponding pathological section are identified based on the extracted contour.

[0097] The matching module 902 is configured to adjust the relative position of the to-be-matched staining image and the preset reference staining image until the matching features of the to-be-matched staining image and the reference staining image satisfy the preset matching requirement. The to-be-matched staining image is any one of the plurality of staining images except the reference staining image.

[0098] The matching confirmation module 903 is configured to obtain the matching parameters between the to-be-matched staining image and the reference staining image, and calculate a matching value according to the matching parameters. If the matching value is greater than a preset matching value threshold, it is determined that the to-be-matched staining image is matched successfully.

[0099] In one embodiment, the plurality of staining images are labeled with numbers, and the numbers of the staining images corresponding to the adjacent layered pathological sections in the pathological paraffin block are adjacent. The feature extraction module 901 is specifically configured to: extend the contour in the target staining image in combination with the size of the contour in the adjacent staining image to obtain an extended contour of the target staining image. The target staining image is any one of the plurality of staining images, and the number of the adjacent staining image is adjacent to the number of the target staining image. The center point, the major axis and the minor axis of the extended contour are identified as the matching features. The major axis is the longest axis in the extended contour passing through the center point, and the minor axis is the axis in the extended contour passing through the center point and perpendicular to the major axis.

[0100] In one embodiment, the feature extraction module 901 is specifically configured to: extend the contour in the target staining image synchronously until the contour in the target staining image after extension covers the contour in the adjacent staining image to obtain the extended contour of the target staining image. ​

[0101] In one of the embodiments, the matching module 902 is configured to: adjust the relative position of the to-be-matched staining image and the preset reference staining image, and determine that the matching features of the to-be-matched staining image and the matching features of the reference staining image satisfy the preset matching requirement when the center point of the to-be-matched staining image is aligned with the center point of the reference staining image, the long axis direction of the to-be-matched staining image is consistent with the long axis direction of the reference staining image, and the short axis direction of the to-be-matched staining image is consistent with the short axis direction of the reference staining image.

[0102] In one of the embodiments, the feature extraction module 901 is configured to: identify the center point of the contour in the target staining image; wherein the target staining image is any one of the plurality of staining images; divide the contour in the target staining image into a plurality of sub-contours based on the center point, and take the shape of the plurality of sub-contours as the matching features; wherein each sub-contour forms the same angle with the center point.

[0103] In one of the embodiments, the matching module 902 is specifically configured to: adjust the relative position of the to-be-matched staining image and the preset reference staining image until the center point of the contour of the to-be-matched staining image is aligned with the center point of the contour of the reference staining image, and the number of the matching sub-contours in the to-be-matched staining image is greater than the preset number threshold, and determine that the matching features of the to-be-matched staining image and the matching features of the reference staining image satisfy the preset matching requirement; wherein if the matching degree between two sub-contours of different staining images is greater than the preset matching degree threshold, the two sub-contours are determined as the matching sub-contours.

[0104] In one of the embodiments, the matching confirmation module 903 is specifically configured to calculate:

[0105] P = 1 - [(A - C) 2 + (B - C) 2 ] / (A + B)

[0106] In the above formula, P is the matching value, A, B, and C are matching parameters, A indicates the area of the contour in the reference staining image, B indicates the area of the contour in the to-be-matched staining image, and C indicates the area of the overlapping contour of the reference staining image and the to-be-matched staining image.

[0107] Figure 10 An internal structure diagram of the multiple staining image matching device in one of the embodiments is shown. As shown in FIG. 2, the multiple staining image matching device includes a feature extraction module 901, a matching module 902, and a matching confirmation module 903. Figure 10As shown, the multiple staining image matching device includes a processor, a memory and a network interface connected through a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the multiple staining image matching device stores an operating system, and can also store a computer program which, when executed by the processor, enables the processor to implement the multiple staining image matching method. The internal memory can also store a computer program which, when executed by the processor, enables the processor to execute the multiple staining image matching method. Those skilled in the art can understand that Figure 10 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the multiple staining image matching device to which the scheme of the present application is applied. The specific multiple staining image matching device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0108] A computer readable storage medium stores a computer program which, when executed by a processor, implements the following steps: obtaining multiple staining images of multiple pathological sections; in each staining image, respectively extracting the contour of the pathological section, and identifying the matching feature of the corresponding pathological section based on the extracted contour; adjusting the relative position of the to-be-matched staining image and the preset reference staining image until the matching features of the to-be-matched staining image and the reference staining image satisfy a preset matching requirement; obtaining the matching parameter between the to-be-matched staining image and the reference staining image, and calculating a matching value according to the matching parameter, and if the matching value is greater than a preset matching value threshold, determining that the to-be-matched staining image is successfully matched.

[0109] A multiple staining image matching device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented: obtaining multiple staining images of multiple pathological sections; in each staining image, respectively extracting the contour of the pathological section, and identifying the matching feature of the corresponding pathological section based on the extracted contour; adjusting the relative position of the to-be-matched staining image and the preset reference staining image until the matching features of the to-be-matched staining image and the reference staining image satisfy a preset matching requirement; obtaining the matching parameter between the to-be-matched staining image and the reference staining image, and calculating a matching value according to the matching parameter, and if the matching value is greater than a preset matching value threshold, determining that the to-be-matched staining image is successfully matched.

[0110] It should be noted that the above multiple staining image matching method, device, equipment and computer readable storage medium belong to one overall inventive concept, and the contents in the multiple staining image matching method, device, equipment and computer readable storage medium embodiments can be mutually applicable.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0112] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0113] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A multiple-stained image matching method, characterized by, The method comprises: obtaining a plurality of staining images of a plurality of pathological sections; wherein the plurality of pathological sections are obtained from the same pathological paraffin block, and each pathological section is photographed to obtain a corresponding staining image after being dyed by a preset staining method; in each staining image, the contour of the pathological section is extracted respectively, and the matching feature of the corresponding pathological section is identified based on the extracted contour; adjusting the relative position of the to-be-matched staining image and the preset reference staining image until the matching features of the to-be-matched staining image and the reference staining image meet the preset matching requirement; wherein the to-be-matched staining image is any one of the plurality of staining images except the reference staining image; obtaining the matching parameters between the to-be-matched staining image and the reference staining image, and calculating a matching value according to the matching parameters, if the matching value is greater than a preset matching value threshold, it is determined that the to-be-matched staining image is matched successfully; The calculation formula of the matching value is: P = 1 - [(A - C) 2 + (B - C) 2 ] / (A + B) In the above formula, P is the matching value, A, B and C are the matching parameters, A indicates the area of the contour in the reference staining image, B indicates the area of the contour in the to-be-matched staining image, and C indicates the area of the overlapping contour of the reference staining image and the to-be-matched staining image.

2. The method of claim 1, wherein, The plurality of staining images are labeled with numbers, and the numbers of the staining images corresponding to the adjacent layered pathological sections in the pathological paraffin block are adjacent, and the matching feature of the corresponding pathological section is identified based on the extracted contour, comprising: extending the contour in the target staining image in combination with the size of the contour in the adjacent staining image to obtain the extended contour of the target staining image; wherein the target staining image is any one of the plurality of staining images, and the number of the adjacent staining image is adjacent to the number of the target staining image; identifying the center point, the long axis and the short axis of the extended contour, and taking the center point, the long axis and the short axis as the matching feature; wherein the long axis is the longest axis in the extended contour passing through the center point, and the short axis is the axis in the extended contour passing through the center point and perpendicular to the long axis.

3. The method of claim 2, wherein, The combination of the size of the contour in the adjacent staining image and the extension of the contour in the target staining image to obtain the extended contour of the target staining image comprises: synchronously extending the contour in the target staining image until the contour in the target staining image after extension covers the contour in the adjacent staining image to obtain the extended contour of the target staining image.

4. The method of claim 2, wherein, The adjustment of the relative position of the to-be-matched staining image and the preset reference staining image until the matching features of the to-be-matched staining image and the reference staining image meet the preset matching requirement comprises: The relative positions of the to-be-matched staining image and the preset reference staining image are adjusted, and when a center point of the to-be-matched staining image is aligned with a center point of the reference staining image, a long axis direction of the to-be-matched staining image is consistent with a long axis direction of the reference staining image, and a short axis direction of the to-be-matched staining image is consistent with a short axis direction of the reference staining image, it is determined that the matching features of the to-be-matched staining image and the reference staining image satisfy the preset matching requirement.

5. The method of claim 1, wherein, The matching features of the corresponding pathological sections are identified based on the extracted contours, including: identifying a center point of the contour in the target staining image; wherein the target staining image is any one of the plurality of staining images; dividing the contour in the target staining image into a plurality of sub-contours with the center point as a reference, and taking the shapes of the plurality of sub-contours as the matching features; wherein each sub-contour has the same angle formed with the center point.

6. The method of claim 5, wherein, The relative positions of the to-be-matched staining image and the preset reference staining image are adjusted until the matching features of the to-be-matched staining image and the reference staining image satisfy the preset matching requirement, including: The relative positions of the to-be-matched staining image and the preset reference staining image are adjusted until the center point of the contour of the to-be-matched staining image is aligned with the center point of the contour of the reference staining image, and the number of matching sub-contours in the to-be-matched staining image is greater than a preset number threshold, it is determined that the matching features of the to-be-matched staining image and the reference staining image satisfy the preset matching requirement; wherein if the matching degree between two sub-contours of different staining images is greater than a preset matching degree threshold, the two sub-contours are determined as matching sub-contours.

7. A multiple-stained image matching apparatus characterized by comprising: The device includes: The feature extraction module is configured to obtain a plurality of staining images of a plurality of pathological sections; wherein the plurality of pathological sections are obtained from the same pathological wax block, and each pathological section is stained by a preset staining method to obtain a corresponding staining image; in each staining image, the contour of the pathological section is extracted, and the matching features of the corresponding pathological section are identified based on the extracted contour; The matching module is configured to adjust the relative positions of the to-be-matched staining image and the preset reference staining image until the matching features of the to-be-matched staining image and the reference staining image satisfy the preset matching requirement; wherein the to-be-matched staining image is any one of the plurality of staining images except the reference staining image; The matching confirmation module is configured to obtain matching parameters between the to-be-matched staining image and the reference staining image, and calculate a matching value according to the matching parameters; if the matching value is greater than a preset matching value threshold, it is determined that the to-be-matched staining image is matched successfully; The formula for calculating the matching value is: P = 1 - [(A - C) 2 + (B - C) 2 ] / (A + B) In the above formula, P is the matching value, A, B, and C are matching parameters, A indicates the area of the contour in the reference staining image, B indicates the area of the contour in the to-be-matched staining image, and C indicates the area of the overlapping contour of the reference staining image and the to-be-matched staining image.

8. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method of any one of claims 1 to 6.

9. A multiplexed staining image matching device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the method of any one of claims 1 to 6.

Images