Image segmentation method, device, electronic device and storage medium

Abstract

The present application provides an image segmentation method, device, electronic device and storage medium, which relate to the field of image processing. The method includes: obtaining a target image of a sample; performing a rough segmentation on the target image to determine the core tissue area in the target image; performing a fine segmentation on the target image using a target threshold to determine the soft tissue area and tissue cavity area in the target image; the target threshold is calculated based on the grayscale value of each pixel in the target image; based on the core tissue area, soft tissue area and tissue cavity area in the target image, the segmentation result of the sample is obtained. The present application solves the problem of low accuracy of image segmentation in related technologies.

Description

Technical Field

[0001] The present application relates to the field of image processing, and more specifically, to an image segmentation method, device, electronic device and storage medium. Background Art

[0002] As pathology digitization rapidly advances, digital pathology scanners have become an indispensable tool in the field. These devices digitize tissue samples by placing them into a machine and taking a picture of the slice using a high-resolution image sensor.

[0003] In the process of digitizing tissue samples, a normal camera is first used to capture the sample placed on the scanner platform to assist the scanner in intelligently identifying the tissue area to be scanned. This recognition process usually relies on deep learning image segmentation methods to determine the target area and then complete the focusing process.

[0004] However, due to the diversity of tissues, including differences in color and shape, scanners have difficulty distinguishing soft tissue areas, which interferes with the focus of pathological sections and reduces the scanning quality, thus affecting the effectiveness of medical panoramic imaging in subsequent AI-assisted diagnosis.

[0005] In order to improve the positioning accuracy of the image segmentation method, the soft tissue area can be located by manual marking. However, this process is time-consuming and laborious. Given the diversity of tissues, a large number of soft tissue area positioning maps are required to achieve effective soft tissue differentiation.

[0006] From the above, we can see that the problem of low accuracy in image segmentation of samples still needs to be solved. Summary of the invention

[0007] The present application provides an image segmentation method, device, electronic device and storage medium, which can solve the problem of low accuracy of image segmentation in related technologies. The technical solution is as follows:

[0008] According to one aspect of the present application, an image segmentation method includes: acquiring a target image of a sample; performing coarse segmentation on the target image to determine a core tissue area in the target image; performing fine segmentation on the target image using a target threshold to determine a soft tissue area and a tissue cavity area in the target image; the target threshold is calculated based on the grayscale value of each pixel in the target image; and obtaining a segmentation result of the sample based on the core tissue area, soft tissue area, and tissue cavity area in the target image.

[0009] According to one aspect of the present application, an image segmentation device is characterized in that it includes: an image acquisition module, used to acquire a target image of a sample; a coarse segmentation module, used to perform coarse segmentation on the target image and determine the core tissue area in the target image; a fine segmentation module, used to perform fine segmentation on the target image using a target threshold and determine the soft tissue area and tissue cavity area in the target image; the target threshold is calculated based on the grayscale value of each pixel in the target image; and an image segmentation module, used to obtain the segmentation result of the sample based on the core tissue area, soft tissue area and tissue cavity area in the target image.

[0010] In an exemplary embodiment, the image segmentation device also includes that the target image is obtained by performing light intensity correction on the original image of the sample using a light intensity correction weight matrix map; the light intensity correction weight matrix map is used to correct the light intensity differences in various areas of the original image.

[0011] In an exemplary embodiment, the image segmentation device is also used to obtain a light intensity template map, which is obtained by photographing a stage without the sample placed thereon; and perform light intensity correction calculation on each pixel in the light intensity template map according to a first set value to obtain the light intensity correction weight matrix map.

[0012] In an exemplary embodiment, the image segmentation device is also used to obtain the first grayscale value, the second grayscale value and the third grayscale value of each pixel in the target image; determine whether there is a soft tissue area in the target image according to the third grayscale value, and if so, obtain the target threshold based on the first grayscale value and the second grayscale value.

[0013] In an exemplary embodiment, the third grayscale value refers to the average grayscale pixel value of the background area in the target image; the image segmentation device is also used to use the third grayscale value as a first threshold, compare each pixel in the target image based on the first threshold, and obtain a first tissue mask image; reduce the first threshold according to a second set value to obtain a second threshold; compare each pixel in the target image based on the second threshold to obtain a second tissue mask image; perform an intersection-and-union (IoU) calculation on the first tissue mask image and the second tissue mask image to obtain a first IoU result. If the first IoU result satisfies a first set condition, there is a soft tissue area in the target image.

[0014] In an exemplary embodiment, the first grayscale value refers to the local maximum grayscale value of the target image; the second grayscale value refers to the local minimum grayscale value of the target image; the image segmentation device is also used to obtain an adjustment interval based on the first grayscale value and the second grayscale value; one of the grayscale values ​​in the adjustment interval is selected as the adjustment threshold, and each pixel of the target image is compared based on the adjustment threshold to obtain a tissue mask image corresponding to the adjustment threshold; if the second intersection-and-union result between the tissue mask image and the target tissue mask image meets the second set condition, the adjustment threshold is the target threshold; otherwise, the next grayscale value in the adjustment interval is selected as the adjustment threshold until a tissue mask image that meets the second set condition is obtained.

[0015] In an exemplary embodiment, the coarse segmentation module is also used to obtain the cumulative probability corresponding to each pixel in the target image; perform variance calculation based on each of the cumulative probability distributions to obtain multiple inter-class variances; select the largest inter-class variance from each of the inter-class variances, and the threshold corresponding to the largest inter-class variance is the coarse segmentation threshold; and perform coarse segmentation on the target image according to the coarse segmentation threshold to obtain the core tissue area of ​​the target image.

[0016] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores computer-readable instructions; the computer-readable instructions are executed by one or more of the processors, so that the electronic device implements the image segmentation method as described above.

[0017] According to one aspect of the present application, a storage medium stores computer-readable instructions thereon, and the computer-readable instructions are executed by one or more processors to implement the image segmentation method as described above.

[0018] According to one aspect of the present application, a computer program product includes computer-readable instructions, which are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the image segmentation method as described above.

[0019] The beneficial effects of the technical solution provided by this application are:

[0020] In the above technical scheme, first, the target image of the sample is coarsely segmented to obtain the core tissue area, and the target image is finely segmented using the target threshold to obtain the soft tissue area and the tissue cavity area, and the segmentation result of the sample is obtained based on the core tissue area, soft tissue area and tissue cavity area of ​​the target image; wherein, the target threshold is calculated based on the grayscale value of each pixel in the target image, and does not need to be set manually, which can reduce the manual processing steps of image segmentation and improve the efficiency of image segmentation; at the same time, the target threshold can effectively distinguish the soft tissue area and the tissue cavity area in the target image, thereby effectively solving the problem of low accuracy of image segmentation existing in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0022] Figure 1 It is a schematic diagram of the implementation environment involved in this application;

[0023] Figure 2 is a flow chart of an image segmentation method according to an exemplary embodiment;

[0024] Figure 3 for Figure 2 The original image obtained by photographing the sample after the sample is stained with eosin in the corresponding embodiment;

[0025] Figure 4 for Figure 2 The segmentation result obtained by positioning and identifying the original image in the corresponding embodiment;

[0026] Figure 5 for Figure 2 A target tissue mask image corresponding to the target image in the corresponding embodiment;

[0027] Figure 6 yes Figure 2 A flowchart of step 310 in one embodiment of the corresponding embodiment;

[0028] Figure 7 for Figure 6 A schematic diagram of a light intensity template diagram corresponding to the embodiment;

[0029] Figure 8 for Figure 6 A light intensity correction weight matrix diagram corresponding to the embodiment;

[0030] Fig. 9for Figure 6 A corresponding embodiment is a heat map corresponding to an original image;

[0031] Fig.10 for Figure 6 A corresponding embodiment is a heat map corresponding to a target image;

[0032] Fig.11 yes Figure 2 A flowchart of step 330 in one embodiment corresponding to the embodiment;

[0033] Fig.12 for Fig.11 A corresponding embodiment is a tissue mask map of a core tissue region;

[0034] Fig.13a yes Figure 2 A flowchart of step 350 in one embodiment corresponding to the embodiment;

[0035] Fig.13b yes Fig.13a A grayscale pixel frequency diagram of a target image in the corresponding embodiment;

[0036] Fig.14 yes Fig.13a A flowchart of step 353 in one embodiment corresponding to the embodiment;

[0037] Fig.15 yes Fig.13a A flowchart of step 353 in another embodiment corresponding to the embodiment;

[0038] Fig.16 yes Figure 2 A tissue mask image in which tissue cavity areas are eliminated in the corresponding embodiment;

[0039] Fig.17 yes Figure 2 A tissue mask image corresponding to a soft tissue area in a corresponding embodiment;

[0040] Fig.18a It is a schematic diagram of a specific implementation of an image segmentation method in an application scenario;

[0041] Fig.18b yes Fig.18a A schematic diagram of a target image of a pathological tissue in a corresponding application scenario;

[0042] Fig.19 is a structural block diagram of an image segmentation device according to an exemplary embodiment;

[0043] Fig. 20 is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0044] Fig.21 The present invention is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0045] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0046] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0047] As mentioned above, in order to improve the accuracy of the deep learning-based image segmentation method, the target image of the sample can be manually labeled to obtain the soft tissue area in the target image.

[0048] Due to the diversity of samples, the above image segmentation method requires a large number of soft tissue localization maps for learning in order to achieve effective soft tissue differentiation. Secondly, due to the production problems of samples, there will be cavities in the tissue area, and the cavities vary in size. The image segmentation method based on deep learning cannot accurately identify smaller tissue cavity areas, which will also affect the subsequent focus, thereby affecting the effect of medical panoramic imaging in subsequent artificial intelligence-assisted diagnosis.

[0049] It can be seen from the above that the related art still has the defect of low accuracy in image segmentation of samples.

[0050] To this end, the image segmentation method provided in the present application can effectively improve the accuracy of image segmentation. Accordingly, the image segmentation method is suitable for an image segmentation device, which can be deployed in an electronic device, and the electronic device can be a computer device configured with a von Neumann architecture, for example, the computer device includes a desktop computer, a laptop computer, a server, etc.

[0051] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0052] Figure 1 It is a schematic diagram of an implementation environment involved in an image processing method. It should be noted that this implementation environment is only an example adapted to the present invention and cannot be considered as providing any limitation on the scope of use of the present invention.

[0053] The implementation environment includes a collection end 110 and a service end 130 .

[0054] Specifically, the acquisition end 110 can also be considered as an image acquisition device, including but not limited to electronic devices with a shooting function such as a camera, a still camera, and a camcorder. For example, the acquisition end 110 is a pathology digital scanner.

[0055] The server 130 may be an electronic device such as a desktop computer, a laptop computer, a server, etc., or may be a computer cluster composed of multiple servers, or even a cloud computing center composed of multiple servers. The server 130 is used to provide background services, for example, background services include but are not limited to image segmentation services, etc.

[0056] The server 130 and the acquisition terminal 110 establish a network communication connection in advance through wired or wireless means, and data transmission between the server 130 and the acquisition terminal 110 is realized through the network communication connection. The transmitted data includes but is not limited to: the target image of the sample, etc.

[0057] In an application scenario, through the interaction between the acquisition end 110 and the server end 130, the acquisition end 110 captures and acquires a target image for a sample, and uploads the target image to the server end 130 to request the server end 130 to provide an image segmentation service.

[0058] For the server 130, after receiving the target image uploaded by the acquisition end 110, it calls the image segmentation service, performs coarse segmentation on the target image to obtain the core tissue area in the target image, and uses the target threshold to perform fine segmentation on the target image to determine the soft tissue area and tissue cavity area in the target image, thereby obtaining the segmentation result of the target image, thereby solving the problem of low accuracy of image segmentation of samples existing in related technologies.

[0059] See also Figure 2 The present application embodiment provides an image segmentation method, which is applicable to an electronic device, which may be Figure 1 A server 130 in an implementation environment is shown.

[0060] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0061] like Figure 2 As shown, the method may include the following steps:

[0062] Step 310, obtaining a target image of the sample.

[0063] The sample refers to biological tissue, such as animal organ tissue, etc., which is not limited here.

[0064] The target image is an image obtained by performing light intensity correction on the original image of the sample using the light intensity correction weight matrix.

[0065] The original image of the sample is obtained by photographing and collecting the sample through an image acquisition device, wherein the image acquisition device can be an electronic device with an image acquisition function, such as a pathology digital scanner and the like.

[0066] Regarding the shooting of the sample, since the sample is a biological tissue, the tissue needs to be stained before shooting so as to distinguish various regions of the tissue later. In a possible implementation, the sample can be stained with eosin staining.

[0067] Figure 3 This is the original image obtained after the sample is stained with eosin. Figure 3 As shown, the sample includes a dark pink area and a light pink area. The dark pink area ( Figure 3 The medium-dark gray area is the core tissue area of ​​the tissue sample, and the light pink area ( Figure 3 The light gray area in the middle is the soft tissue area of ​​the sample. In the core tissue area, there are small holes caused by the preparation of the film. These small holes are the tissue cavity areas.

[0068] Regarding the acquisition of the original image, the original image can come from the original image captured and collected by the original image acquisition device in real time, or it can be the original image captured and collected by the original image acquisition device in a historical time period pre-stored in the electronic device. Then, for the electronic device, after the original image acquisition device captures and collects the original image, the original image can be processed in real time, or it can be pre-stored and processed, for example, the original image can be processed when the CPU of the electronic device is low, or the original image can be processed according to the instructions of the staff. Therefore, the target tracking in this embodiment can be for the original image acquired in real time, or for the original image acquired in a historical time period, which is not specifically limited here.

[0069] It should be noted that before the original image is subjected to light intensity correction, since the original image includes the area where the sample is located (target image) and the area where the sample does not exist, in one possible implementation method, the original image is located and identified. Specifically, the machine learning model can be trained by deep learning methods to establish an image localization model, and the area where the sample is located can be segmented by the image localization model.

[0070] The target image also includes a tissue region (eg, a soft tissue region, a core tissue region, and a tissue cavity region) and a background region, and the background region may be a blank region in a glass slide where the sample is placed.

[0071] For example, the original image is positioned and recognized through the image segmentation model, such as Figure 4 As shown, Figure 4 The segmentation result is obtained by positioning and identifying the original image. The box part is the target image, the dark part in the box is the tissue area of ​​the sample, and the blank part in the box is the background area of ​​the target image. Figure 4 Output tissue mask image, that is, the pixels of the tissue area of ​​the sample are 1, and the pixels of the background area are 0, such as Figure 5 As shown, Figure 5 is the target tissue mask corresponding to the target image, Figure 5 The white area is where the sample is located, and the black area is the background area.

[0072] It should be noted that due to the difference in light transmittance of subdivided tissues, the light transmittance of different tissue regions of the sample is different. Specifically, the light transmittance of the core tissue region is the worst, the light transmittance of the soft tissue region is better, and the light transmittance of the tissue cavity region is the best. Therefore, when segmenting each tissue region of the sample, the difference in light transmittance of the subdivided tissues can be used for image segmentation.

[0073] However, since the sample is placed on the stage during shooting, the shooting will be affected by the aperture, that is, the light intensity gradually weakens from the center of the stage to the surrounding areas, and due to the uncontrollable nature of the production, the position of the sample on the pick is also unknown. In other words, the original image will definitely be affected by uneven lighting, resulting in the failure of the difference in light transmittance of the subdivided tissue, thereby affecting the image segmentation effect of the sample.

[0074] Based on this, the original image of the sample needs to be processed to reproduce the characteristics of the light transmittance difference of its subdivided tissue. In a possible implementation method, the original image of the sample is subjected to light intensity correction using a light intensity correction weight matrix map, such as Figure 6 As shown, the light intensity correction weight matrix diagram can be obtained by the following steps:

[0075] Step 311, obtaining a light intensity template map.

[0076] The light intensity template image is obtained by photographing the stage without a sample. Since there is no sample on the stage, the aperture will not be disturbed during the downward irradiation and will directly irradiate the stage. Therefore, the light intensity template image can truly reflect the light intensity distribution of the aperture under normal irradiation (without a sample).

[0077] Figure 7 A schematic diagram of a light intensity template diagram is shown, such as Figure 7 As shown, the light intensity in each area of ​​the light intensity template image is obviously unevenly distributed, with strong light intensity in the central area and weak light intensity in the edge area.

[0078] Step 313, performing light intensity correction calculation on each pixel in the light intensity template image according to the first set value to obtain a light intensity correction weight matrix image.

[0079] First of all, it should be noted that the grayscale value, that is, the pixel, can reflect the light intensity. It can be understood that if there is no influence of light intensity, the grayscale values ​​of each area of ​​the light intensity template map are the same. In other words, the grayscale values ​​of each area of ​​the light intensity template map can reflect the light intensity distribution of different areas in the image.

[0080] The first setting value is the maximum grayscale value, which can represent the brightest pixel, that is, corresponds to the maximum light intensity. It should be understood that when the stage is illuminated during shooting, without the influence of the aperture, the light intensity of each area is consistent, that is, the pixel value of each area of ​​the image obtained at this time is the maximum grayscale value (maximum light intensity).

[0081] Based on this, in a possible implementation, the light intensity template image can be added to the light intensity correction weight matrix image to achieve the maximum grayscale value of 255 for each pixel after addition. Based on this, the light intensity correction weight matrix image can be obtained by reverse solution, that is, the new image obtained by subtracting each pixel of the light intensity template image from the maximum grayscale value 255 is the light intensity correction weight matrix image.

[0082] Then, by using the illumination correction weight matrix to correct the original image of the sample, the illumination intensity of each area of ​​the original image can be made uniform, the aperture effect can be eliminated, and the characteristics of the light transmittance differences of the sample's subdivided tissues can be reproduced.

[0083] Figure 8 A light intensity correction weight matrix diagram is shown, as shown in Figure 8. The light intensity correction weight matrix diagram can reflect that the grayscale value that needs to be increased in the area at the center of the aperture is lower, while the grayscale value that needs to be increased around is higher, which indirectly shows the problem of uneven lighting of the aperture.

[0084] Based on this, the light intensity correction weight matrix is ​​used to correct the light intensity difference of each area of ​​the original image. In a possible implementation, the light intensity correction weight matrix is ​​added to the original image of the sample to obtain the target image.

[0085] Fig. 9 is the heat map corresponding to the original image, such as Fig. 9 As shown in the figure, the original image without light intensity correction is affected by the aperture, and the light intensity in each area of ​​the original image is uneven; Fig.10 is the heat map corresponding to the target image, such as Fig.10 As shown, the target image after light intensity correction eliminates the aperture effect, and the light intensity of each area of ​​the target image is more balanced.

[0086] Therefore, the target image that has completed light intensity correction can be segmented by utilizing the characteristics of the light transmittance differences of the subdivided tissues.

[0087] Step 330 , roughly segment the target image to determine the core tissue area in the target image.

[0088] First of all, it should be noted that the core tissue region refers to the region where the core tissue of the sample is located. For example, if the sample is heart tissue, the core tissue region may be the main structural part of the heart, such as the myocardial tissue region.

[0089] Please refer to Fig.10 , Fig.10 It is the heat map corresponding to the target image. As shown in 10, the illumination intensity of the core tissue area of ​​the sample is almost the same, and there is an obvious difference in illumination intensity between the core tissue area and the soft tissue area, tissue cavity area and other areas.

[0090] Based on this, the target image can be roughly segmented to determine its core tissue area. For rough segmentation, the Ostu algorithm, region growing algorithm, Canny edge detection algorithm, SLIC (Simple Linear Iterative Clustering) algorithm, etc. can be used to achieve it. It can also be achieved by deep learning from the target image through a convolutional neural network, which is not limited here.

[0091] In one possible implementation, such as Fig.11 As shown, the rough segmentation may include the following steps:

[0092] Step 331, obtaining the cumulative probability corresponding to each pixel in the target image.

[0093] Regarding the cumulative probability, the grayscale histogram of the target image can be counted, and the number of pixels at each grayscale level in the histogram is divided by the total number of pixels in the target image to obtain a normalized histogram. Then the cumulative probability distribution of each grayscale level is calculated based on the normalized histogram, which can be achieved through the cumulative histogram, that is, calculating the cumulative sum of pixels from 0 to the current grayscale level.

[0094] Step 333: Calculate variance according to each cumulative probability distribution to obtain multiple between-class variances.

[0095] Regarding the inter-class variance, for each possible threshold t, the image is divided into two parts: foreground (pixel value less than or equal to t) and background (pixel value greater than t). Then, the inter-class variance is calculated based on the segmented foreground and background. The inter-class variance is used to measure the degree of difference between the two segments. Among them, t is the grayscale value range of the target image, that is, from the minimum grayscale level to the maximum grayscale level.

[0096] Step 335 , selecting the largest inter-class variance among the inter-class variances, and the threshold corresponding to the largest inter-class variance is the coarse segmentation threshold.

[0097] As mentioned above, there is a large difference in light intensity between the core tissue area and the soft tissue area, tissue cavity area, etc. Furthermore, the larger the inter-class variance, the greater the difference between the corresponding foreground and background. Therefore, the threshold that maximizes the inter-class variance is selected as the optimal threshold. That is, the threshold that can maximize the difference (light intensity difference) between the segmented foreground (core tissue area) and background (soft tissue area, tissue cavity area, etc.) is selected.

[0098] Step 337 , performing rough segmentation on the target image according to the rough segmentation threshold to obtain the core tissue region of the target image.

[0099] Thus, the core tissue area with the weakest illumination intensity can be obtained. The core tissue area is masked, and the pixels of the core tissue area are 1, and the pixels of other areas except the core tissue area (including background area, tissue cavity area and soft tissue area) are 0, such as Fig.12 As shown, Fig.12 A tissue mask diagram of a core tissue area is shown in FIG. Fig.12 The white area in the figure is the core tissue area, and the black area is other areas.

[0100] However, due to the small difference in illumination between the soft tissue area and the tissue cavity area, the soft tissue area and the tissue cavity area of ​​the target image cannot be segmented by coarse segmentation alone. Therefore, it is necessary to further perform fine segmentation on the target image to distinguish the soft tissue area from the tissue cavity area.

[0101] Step 350 , finely segment the target image using the target threshold to determine the soft tissue area and tissue cavity area in the target image.

[0102] First of all, it should be noted that the soft tissue area is the area where the soft tissue is located in the sample. For example, if the sample is heart tissue, the soft tissue area can be the fat around the heart, such as the fat area. The tissue cavity area is an area with no structure or very little tissue components in the sample. It may be caused by pathological reasons (such as necrosis), technical reasons (sample preparation damage) or image noise. For example, tissue cavities are caused by damage or defects in the sample preparation process. These tissue cavities may be caused by the tearing or destruction of the tissue during the sectioning process, resulting in the lack of actual tissue structure in the tissue cavity area seen under the microscope.

[0103] It should be noted that although the illumination difference between the soft tissue area and the tissue cavity area is small, considering that the illumination intensity of the tissue cavity area is actually approximately equal to the illumination intensity of the background area, the target threshold between the soft tissue area and the surrounding background area can be obtained by comparing the illumination intensity of the soft tissue area and the surrounding background area.

[0104] The target threshold is calculated based on the gray value of each pixel in the target image and is used to distinguish between soft tissue areas and tissue cavity areas.

[0105] In one possible implementation, such as Fig.13a As shown, step 350 may include the following steps:

[0106] Step 351, obtaining a first grayscale value, a second grayscale value, and a third grayscale value of each pixel in the target image.

[0107] The first grayscale value refers to the local maximum grayscale pixel value of the target image, the second grayscale value refers to the local minimum grayscale pixel value of the target image, and the third grayscale value refers to the average grayscale pixel value of the background area in the target image.

[0108] Fig.13b A grayscale pixel frequency diagram of a target image is shown in Fig.13b As shown, the first grayscale value and the second grayscale value can be obtained from the figure. Specifically, the pixel values ​​(x coordinates) corresponding to the local maximum grayscale pixel value and the local minimum grayscale pixel value of the curve are the pixel values ​​corresponding to the position where the slope is 0. At the same time, the grayscale pixel average value of the background area of ​​the target image is calculated.

[0109] Step 353, judging whether there is a soft tissue area in the target image according to the third gray value, and if there is, obtaining a target threshold value based on the first gray value and the second gray value.

[0110] In one possible implementation, such as Fig.14As shown, the determination of soft tissue may include the following steps:

[0111] Step 3531: Use the third grayscale value as a first threshold, compare each pixel in the target image based on the first threshold, and obtain a first tissue mask image.

[0112] The first tissue mask image can be regarded as a mask image that removes the tissue cavity area and retains the soft tissue area and the core tissue area.

[0113] First of all, considering that the illumination intensity of the background area should be approximately equal to the illumination intensity of the tissue cavity area, the grayscale value of the background area can be approximated to the grayscale value of the tissue cavity area. Furthermore, the illumination intensity of the soft tissue area should be smaller than that of the background area and the tissue cavity area, that is, the grayscale value of the soft tissue area should be smaller than that of the background area.

[0114] Among them, the third grayscale value refers to the average grayscale pixel value of the background area in the target image. According to the third grayscale value, the tissue cavity area in the target image can be determined, and the third grayscale value is used as the first threshold to obtain the grayscale value in the target image that is less than the first threshold. It can be understood that the grayscale value in the target image that is less than the first threshold should belong to the soft tissue area or the core tissue area, and the grayscale value in the target image that is greater than or equal to the first threshold should be the tissue cavity area or the background area. Based on the grayscale value less than the first threshold, the first tissue mask image can be obtained.

[0115] Step 3533, reducing the first threshold according to the second set value to obtain the second threshold.

[0116] As mentioned above, the illumination intensity of the background area should be slightly greater than that of the tissue cavity area, that is, the grayscale value of the tissue cavity area should be slightly smaller than that of the background area.

[0117] Then, the second threshold obtained by reducing the first threshold (grayscale pixel average value) can be used to distinguish between soft tissue areas and tissue cavity areas.

[0118] Regarding the first threshold, the tissue cavity area and the background area can be annotated based on multiple target images of the acquired samples, and the corresponding average grayscale values ​​are calculated based on the annotated tissue cavity area and background area, and the first threshold is obtained by the average grayscale value of the two. The machine learning model can also be trained by deep learning methods, and the first threshold is automatically calculated by the soft segmentation model obtained after the training.

[0119] It should be noted that although the soft tissue is thinner and lighter in color, it is darker than the background area. Based on this, the first threshold is reduced by the second setting value to distinguish the soft tissue area of ​​the sample. The second setting value represents the distribution of the pixel difference between the soft tissue area and the background area, which can be obtained by statistically analyzing the pixel difference between the soft tissue area and the background area.

[0120] In a possible implementation, the second setting value may be determined by calculating statistical characteristics such as an average value and a variance of the pixel difference between the soft tissue region and the background region. Specifically, by recording and analyzing the pixel difference between the soft tissue region and the background region of multiple sample images, a second setting value that can most effectively distinguish the soft tissue region is selected according to the distribution of the pixel difference, for example, the second setting value is 10.

[0121] Step 3535, compare each pixel in the target image based on the second threshold to obtain a second tissue mask image.

[0122] The second tissue mask image can be regarded as a mask image that removes the soft tissue area and the tissue cavity area and retains the core tissue area.

[0123] As mentioned above, by reducing the first threshold by the second set value, the soft tissue area of ​​the sample can be distinguished, and of course, the background area can also be removed by the second threshold. Therefore, the soft tissue area and the background area of ​​the target image can be removed by using the second threshold, and a second tissue mask image including only the core tissue area can be obtained.

[0124] Specifically, grayscale values ​​smaller than the second threshold value are obtained in the target image, and the second tissue mask image can be obtained based on the grayscale values ​​smaller than the second threshold value.

[0125] Step 3537, performing IoU calculation on the first tissue mask image and the second tissue mask image to obtain a first IoU result. If the first IoU result satisfies a first set condition, there is a soft tissue area in the target image.

[0126] As mentioned above, the first tissue mask image can be a mask image that removes tissue cavity areas and only includes soft tissue areas and core tissue areas. Then, the overlapping area between the first tissue mask image and the second tissue mask image should be the core tissue area, that is, the difference between the first tissue mask image and the second tissue mask image is the soft tissue area. Based on this, it is possible to determine whether soft tissue exists through the overlapping area between the first tissue mask image and the second tissue mask image.

[0127] Specifically, the first intersection-and-union result can indicate the degree of overlap between the first tissue mask image and the second tissue mask image. The greater the degree of overlap between the first tissue mask image and the second tissue mask image, the greater the degree of overlap between the areas indicated by the first tissue mask image and the second tissue mask image. For example, the closer the first intersection-and-union result is to 1, the greater the overlap between the first tissue mask image and the second tissue mask image.

[0128] For example, the first threshold is 240, and the corresponding grayscale value of the first tissue mask table is less than 240; the second threshold is 230, and the corresponding grayscale value of the second tissue mask table is less than 230; if the target image does not have an area with a grayscale value of 230 to 240, then the first intersection-and-union result is 1, and the degree of overlap between the area indicated by the first tissue mask image and the area indicated by the second tissue mask image reaches 100%, that is, there is a high probability that there is no soft tissue area in the target image.

[0129] The first setting condition can be set according to the size of the soft tissue area. Since the soft tissue sizes of different samples are different, the sizes of the corresponding soft tissue areas are also different, and the requirements for the degree of overlap are also different. In addition, the soft tissue of the sample is usually connected to the core tissue, and the connected areas are intertwined and fused, with unclear boundaries and difficult to distinguish, which may lead to the removal of the core tissue area similar to the soft tissue as soft tissue. In other words, the range setting of the first setting condition can affect the accuracy of image segmentation. If the first setting condition is set too large, the area belonging to the core tissue may be mistaken for the soft tissue area. If the first setting condition is set too small, the soft tissue may be judged as a tissue cavity.

[0130] Furthermore, in order to avoid the above situation, only when the first IoU result satisfies the first set condition will it be considered that soft tissue exists in the target image, thereby avoiding the situation where the core tissue area is mistaken for the soft tissue area.

[0131] In one possible implementation, the first setting condition is that the first intersection-over-union result is less than 0.9, that is, the soft tissue of the sample corresponding to the first setting condition is small, and therefore, the overlap area of ​​the area indicated by the first tissue mask image and the second tissue mask image is large. It should be noted that the first intersection-over-union result is only considered to be a soft tissue area when it is less than 0.9, and soft tissue may also exist when it is greater than 0.9, and it will not be divided at this time. It can be understood that when the first intersection-over-union result is greater than 0.9, there are fewer soft tissue areas in the target image, and even if it is not segmented, the error generated is still within an acceptable range, but it has a great impact on the accuracy of image segmentation.

[0132] In another possible implementation, the first setting condition may be set within a range of 0.95-0.85, and the first setting condition may be adjusted within the above range according to actual image segmentation results, so as to improve the accuracy of soft tissue segmentation.

[0133] Through the above process, it is possible to determine whether there is a soft tissue area in the target image by using the third grayscale value. If not, the soft tissue area is not segmented, thereby reducing the steps of image segmentation and improving image segmentation efficiency.

[0134] If there is a tissue cavity area in the target image, it is removed from the target image. In a possible implementation, such as Fig.15 As shown, the calculation of the target threshold may include the following steps:

[0135] Step 353A, obtaining an adjustment interval based on the first grayscale value and the second grayscale value.

[0136] As mentioned above, the first grayscale value is the local maximum grayscale value of the tissue area, and the second grayscale value is the local minimum grayscale value of the tissue area. That is to say, by using the first grayscale value and the second grayscale value as the adjustment interval, the target image is compared using the grayscale values ​​within the tissue area.

[0137] Step 353C, selecting one of the grayscale values ​​in the adjustment interval as the adjustment threshold, comparing each pixel of the target image based on the adjustment threshold, and obtaining a tissue mask image corresponding to the adjustment threshold.

[0138] Regarding the tissue mask map, the tissue mask map corresponding to the adjustment threshold can be obtained by obtaining the grayscale values ​​in the target image that are smaller than the adjustment threshold and performing mask processing on each grayscale value.

[0139] Step 353F, if the second intersection-and-union result between the tissue mask image and the target tissue mask image satisfies the second set condition, the adjustment threshold corresponding to the tissue mask image is the target threshold; otherwise, the next grayscale value in the adjustment interval is selected as the adjustment threshold until a tissue mask image that meets the second set condition is obtained.

[0140] First of all, it is explained that by taking the grayscale values ​​within the adjustment interval as adjustment thresholds in turn, the soft tissue area and the core tissue area in the target image can be identified, thereby removing the tissue cavity area. In other words, the tissue mask image is a mask image with the tissue cavity area removed.

[0141] The target tissue mask map refers to the tissue mask map corresponding to the target image. The second intersection-over-union result is obtained by calculating the intersection-over-union ratio between the tissue mask map and the target tissue mask map. The second intersection-over-union result can indicate the degree of overlap between the tissue mask map and the target tissue mask map. Then, it can be understood that the difference between the tissue mask map and the target tissue mask map is the core tissue area and the soft tissue area.

[0142] In one possible implementation, the second setting condition is that the second intersection-over-union result is less than 0.85, that is, the overlap between the tissue mask image and the target tissue mask image is less than 0.85, and the adjustment threshold corresponding to the tissue mask image is the target threshold. It should be noted that only when the second intersection-over-union result is less than 0.85 is it considered that there is a tissue cavity area. When it is greater than 0.85, there may also be a tissue cavity area, which will not be divided at this time. It can be understood that when the second intersection-over-union result is greater than 0.85, there are fewer tissue cavity areas in the target image. Even if it is not segmented, the error generated is still within an acceptable range, but it has a great impact on the accuracy of image segmentation.

[0143] In another possible implementation, the second setting condition can be set within the range of 0.80-0.95. It should be noted that if the second setting condition is set too high, the soft tissue area will be mistaken for the tissue cavity area, and if it is set too low, the tissue cavity area will be mistaken for the soft tissue area. Therefore, the second setting condition can be adjusted within the above range according to the actual image segmentation effect to improve the segmentation accuracy of the soft tissue area and the tissue cavity area.

[0144] After obtaining the target threshold, the target threshold can be used to segment the target image. Specifically, grayscale values ​​less than the target threshold are selected from the target image. Based on the selected grayscale values, the target image with the tissue cavity area removed can be obtained, and mask processing is performed on it, such as Fig.16 As shown, Fig.16 , a tissue mask image with tissue cavity areas removed is shown in FIG. , the white part is the soft tissue area and the core tissue area of ​​the target image, and the black part is the background area and the tissue cavity area.

[0145] Furthermore, by using the target tissue mask corresponding to the target image, subtracting the tissue mask with the tissue cavity area eliminated, and then subtracting the core tissue mask corresponding to the core tissue area obtained in step 330, the tissue mask corresponding to the soft tissue area can be obtained.

[0146] Fig.17 A tissue mask image corresponding to a soft tissue area is shown, such as Fig.17 As shown, the white area is the soft tissue area, and the black area is the area other than the soft tissue area.

[0147] It should be noted that if it is determined through the aforementioned step (step 3537) that there is no soft tissue area in the target image, then the grayscale value that satisfies the second set condition can also be used as the target threshold, and the area in the target image that is smaller than the target threshold is regarded as the core tissue area in the target image.

[0148] Through the above process, the target image is finely segmented using the target threshold. The target threshold does not need to be set by itself, but can be calculated based on the grayscale value corresponding to each pixel of the target image, thereby reducing the manual operation steps. In addition, the image training set in the image segmentation model can be quickly expanded, so that the image segmentation model generated based on the image training set can accurately and quickly segment the soft tissue area and the tissue cavity area.

[0149] Step 370, obtaining a segmentation result of the sample based on the core tissue area, the soft tissue area and the tissue cavity area in the target image.

[0150] Regarding the core tissue area, the target image can be roughly segmented to determine the core tissue area in the target image; it can also be determined through the above steps that there is no soft tissue area in the target image, so that the grayscale value that meets the second set condition is used as the target threshold, and the area in the target image that is smaller than the target threshold is regarded as the core tissue area in the target image.

[0151] Furthermore, if there is a soft tissue area, the target tissue mask map corresponding to the target image can be used to subtract the tissue mask map that eliminates the tissue cavity area (obtained in step 353F), and then subtract the core tissue mask map corresponding to the core tissue area obtained in step 330, to obtain the tissue mask map corresponding to the soft tissue area. Based on this tissue mask map, the soft tissue area in the target image can be determined.

[0152] Regarding tissue cavity areas, the first tissue mask image and the second tissue mask image obtained based on the aforementioned steps can remove tissue cavity areas in the target image. Due to problems with the sample during the preparation process (for example, operational problems), tissue cavity areas may be generated in the target image, and the sizes of tissue cavity areas vary. The tissue cavity areas do not contain useful tissue information, so removing them can avoid interference from tissue cavity areas.

[0153] Then, based on the target image of the core tissue area, soft tissue area, and after eliminating the tissue cavity area, the segmentation result of the sample can be obtained. The segmentation result is conducive to the pathology digital scanner to obtain the subdivided tissue more intelligently, thereby providing more detailed information for the subsequent scanning process and improving the scanning quality.

[0154] In combination with the above embodiments, the image segmentation task of the target area is decomposed into coarse decomposition and fine decomposition. The coarse decomposition can quickly segment the core tissue area of ​​the target image, and the fine decomposition can efficiently and accurately complete the segmentation of the soft tissue area and the tissue cavity area based on the target threshold automatically calculated based on each pixel in the target image.

[0155] In addition, by utilizing coarse and fine decomposition of the target image, the efficiency of image segmentation is improved, and accurate segmentation of each area of ​​the target image is achieved, thereby solving the problem of low accuracy in image segmentation of samples existing in related technologies, and further improving the effect of subsequent medical panoramic imaging in subsequent artificial intelligence-assisted diagnosis.

[0156] Fig.18a This is a schematic diagram of a specific implementation of an image segmentation method in an application scenario. In this application scenario, the sample is a pathological tissue.

[0157] Through step 801, a slice image of a pathological tissue is acquired.

[0158] Through step 803, the slice image is processed by the image positioning model trained by the deep learning method to obtain a slice positioning grayscale image of the pathological tissue.

[0159] Through step 805, a light intensity template image obtained by photographing the stage is obtained.

[0160] Through step 807, light intensity correction is performed on the light intensity template image to obtain a light intensity correction weight matrix image.

[0161] Through step 809, the slice positioning grayscale image is corrected by using the light intensity correction weight matrix image to obtain a target image of the pathological tissue.

[0162] like Fig.18b As shown, Fig.18b A schematic diagram of a target image of a pathological tissue is shown in FIG.

[0163] Through step 811, the target image is roughly segmented using the Ostu algorithm to determine the core tissue area.

[0164] Through step 813, the target image is finely segmented using the target threshold to determine the tissue cavity area and the soft tissue area.

[0165] Through step 815, the segmentation result of the pathological image is obtained based on the core tissue area, the soft tissue area and the tissue cavity area of ​​the target image.

[0166] In this application scenario, the core tissue area, soft tissue area and tissue cavity area of ​​the slice image of the pathological tissue can be accurately segmented at low cost, which can save costs and eliminate the need for manual annotation or expansion of the image training set. In addition, the segmentation results of the pathological tissue are conducive to the pathological digital scanner to more intelligently segment each area, provide more detailed information for the subsequent scanning process, and improve the scanning quality.

[0167] The following is an embodiment of the device of the present application, which can be used to perform the image segmentation method involved in the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the method embodiment of the image segmentation method involved in the present application.

[0168] See also Fig.19 In an embodiment of the present application, an image segmentation device 900 is provided, including but not limited to: an image acquisition module 910, a coarse segmentation module 930, a fine segmentation module 950 and an image segmentation module 970.

[0169] The image acquisition module 910 is used to acquire a target image of the sample;

[0170] A coarse segmentation module 930 is used to perform coarse segmentation on the target image and determine the core tissue area in the target image;

[0171] A fine segmentation module 950 is used to perform fine segmentation on the target image using a target threshold value to determine the soft tissue area and the tissue cavity area in the target image; the target threshold value is calculated based on the gray value of each pixel in the target image;

[0172] The image segmentation module 970 is used to obtain the segmentation result of the sample based on the core tissue area, soft tissue area and tissue cavity area in the target image.

[0173] In an exemplary embodiment, the image segmentation device 900 also includes a target image obtained by performing light intensity correction on the original image of the sample using a light intensity correction weight matrix map; the light intensity correction weight matrix map is used to correct the light intensity differences in various regions of the original image.

[0174] In an exemplary embodiment, the image segmentation device 900 is also used to obtain a light intensity template map, which is obtained by photographing a stage without a sample placed thereon; and perform light intensity correction calculation on each pixel in the light intensity template map according to a first set value to obtain a light intensity correction weight matrix map.

[0175] In an exemplary embodiment, the image segmentation device 900 is also used to obtain the first grayscale value, the second grayscale value and the third grayscale value of each pixel in the target image; determine whether there is a soft tissue area in the target image according to the third grayscale value, and if so, obtain the target threshold based on the first grayscale value and the second grayscale value.

[0176] In an exemplary embodiment, the third grayscale value refers to the average grayscale pixel value of the background area in the target image; the image segmentation device 900 is further used to use the third grayscale value as a first threshold value, compare each pixel in the target image based on the first threshold value, and obtain a first tissue mask image; reduce the first threshold value according to a second set value to obtain a second threshold value; compare each pixel in the target image based on the second threshold value to obtain a second tissue mask image; perform an intersection-and-union calculation on the first tissue mask image and the second tissue mask image to obtain a first intersection-and-union result. If the first intersection-and-union result satisfies the first set condition, there is a soft tissue area in the target image.

[0177] In an exemplary embodiment, the first grayscale value refers to a local maximum grayscale value of the target image; the second grayscale value refers to a local minimum grayscale value of the target image;

[0178] The image segmentation device 900 is also used to obtain an adjustment interval based on the first grayscale value and the second grayscale value; select one of the grayscale values ​​in the adjustment interval as the adjustment threshold, compare each pixel of the target image based on the adjustment threshold, and obtain a tissue mask image corresponding to the adjustment threshold; if the second intersection-and-union result between the tissue mask image and the target tissue mask image meets the second set condition, the adjustment threshold is the target threshold; otherwise, select the next grayscale value in the adjustment interval as the adjustment threshold, until a tissue mask image that meets the second set condition is obtained.

[0179] In an exemplary embodiment, the coarse segmentation module 930 is also used to obtain the cumulative probability corresponding to each pixel in the target image; perform variance calculation based on each cumulative probability distribution to obtain multiple inter-class variances; select the largest inter-class variance from each inter-class variance, and the threshold corresponding to the largest inter-class variance is the coarse segmentation threshold; perform coarse segmentation on the target image according to the coarse segmentation threshold to obtain the core tissue area of ​​the target image.

[0180] It should be noted that, when the image segmentation device provided in the above embodiment performs image segmentation, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the image segmentation device will be divided into different functional modules to complete all or part of the functions described above.

[0181] In addition, the image segmentation device and the image segmentation method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs the operation has been described in detail in the method embodiment, which will not be repeated here.

[0182] Fig. 20 A schematic diagram of the structure of an electronic device according to an exemplary embodiment. Figure 1A server 130 in an implementation environment is shown.

[0183] It should be noted that the server is only an example adapted to the present application and cannot be considered to provide any limitation on the scope of use of the present application. The server cannot be interpreted as requiring dependence on or having Fig. 20 One or more components of exemplary server 2000 are shown.

[0184] The hardware structure of the server 2000 may vary greatly due to different configurations or performances, such as Fig. 20 As shown, the server 2000 includes: a power supply 210 , an interface 230 , at least one memory 250 , and at least one central processing unit (CPU) 270 .

[0185] Specifically, the power supply 210 is used to provide operating voltage for each hardware device on the server 2000 .

[0186] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. Figure 1 The interaction between the collection end 110 and the service end 130 in the implementation environment is shown.

[0187] Of course, in other examples adapted by the present application, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, etc. Fig. 20 As shown, this is not intended to be a specific limitation.

[0188] The memory 250 is a carrier for storing resources, which may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 251, an application 253 and data 255, etc. The storage method may be temporary storage or permanent storage.

[0189] Among them, the operating system 251 is used to manage and control the hardware devices and application programs 253 on the server 2000 to enable the central processor 270 to calculate and process the massive data 255 in the memory 250. It can be WindowsServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0190] The application program 253 is a computer-readable instruction that performs at least one specific task based on the operating system 251, and may include at least one module ( Fig. 20(not shown), each module may respectively include computer-readable instructions for the server 2000. For example, the image segmentation device may be regarded as an application 253 deployed on the server 2000.

[0191] The data 255 may be photos, pictures, etc. stored in a disk, or may be target images, etc. stored in the memory 250 .

[0192] The central processor 270 may include one or more processors and is configured to communicate with the memory 250 through at least one communication bus to read the computer-readable instructions stored in the memory 250, thereby realizing the operation and processing of the mass data 255 in the memory 250. For example, the image segmentation method is completed by the central processor 270 reading a series of computer-readable instructions stored in the memory 250.

[0193] In addition, the present application can also be implemented through hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present application is not limited to any specific hardware circuits, software, or a combination of the two.

[0194] See also Fig.21 In an embodiment of the present application, an electronic device 4000 is provided, and the electronic device 400 may include: a desktop computer, a laptop computer, a server, etc.

[0195] exist Fig.21 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0196] The data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.21 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0197] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0198] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0199] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program instructions or codes in the form of instructions or data structures and can be accessed by the electronic device 400, but is not limited to this.

[0200] Computer-readable instructions are stored in the memory 4003 , and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002 .

[0201] The computer-readable instructions are executed by one or more processors 4001 to implement the image segmentation method in the above-mentioned embodiments.

[0202] In addition, an embodiment of the present application provides a storage medium on which computer-readable instructions are stored. The computer-readable instructions are executed by one or more processors to implement the image segmentation method as described above.

[0203] In an embodiment of the present application, a computer program product is provided. The computer program product includes computer-readable instructions. The computer-readable instructions are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the image segmentation method as described above.

[0204] Compared with the related art, firstly, the target image of the sample is roughly segmented to obtain the core tissue area, and the target threshold is used to finely segment the target image to obtain the soft tissue area and the tissue cavity area, and the segmentation result of the sample is obtained based on the core tissue area, soft tissue area and tissue cavity area of ​​the target image; wherein, the target threshold is calculated according to the gray value of each pixel in the target image, and does not need to be set manually, which can reduce the manual processing steps of image segmentation and improve the efficiency of image segmentation;

[0205] At the same time, the target threshold can effectively distinguish the soft tissue area and the tissue cavity area in the target image, thereby effectively solving the problem of low accuracy of image segmentation of samples existing in the related art.

[0206] In addition, using the target threshold to perform fine segmentation on the target image can quickly expand the image training set in the image segmentation model, so that the image segmentation model generated based on the image training set can accurately and quickly segment the soft tissue area and the tissue cavity area.

[0207] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0208] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image segmentation method, characterized in that: include: Acquire a target image of the sample; Performing rough segmentation on the target image to determine the core tissue area in the target image; Obtaining a first grayscale value, a second grayscale value, and a third grayscale value of each pixel in the target image; the first grayscale value refers to a local maximum grayscale value of the target image; the second grayscale value refers to a local minimum grayscale value of the target image; and the third grayscale value refers to an average grayscale pixel value of a background area in the target image; Using the third grayscale value as a first threshold, comparing each pixel in the target image based on the first threshold to obtain a first tissue mask image; reducing the first threshold according to a second set value to obtain a second threshold; Comparing each pixel in the target image based on the second threshold to obtain a second tissue mask image; Performing an intersection-and-union (IoU) calculation on the first tissue mask image and the second tissue mask image to obtain a first IoU result, and if the first IoU result satisfies a first set condition, a soft tissue area exists in the target image; Using a target threshold to segment the target image, and determining a soft tissue area and a tissue cavity area in the target image; The target threshold is calculated based on the grayscale value of each pixel of the target image; Based on the core tissue area, the soft tissue area and the tissue cavity area in the target image, a segmentation result of the sample is obtained.

2. The method according to claim 1, characterized in that The target image is obtained by performing light intensity correction on the original image of the sample using a light intensity correction weight matrix diagram; the light intensity correction weight matrix diagram is used to correct the difference in light intensity of each area of ​​the original image.

3. The method according to claim 2, characterized in that The method further comprises: Acquire a light intensity template image, wherein the light intensity template image is obtained by photographing a stage without the sample placed thereon; According to the first set value, light intensity correction calculation is performed on each pixel in the light intensity template image to obtain the light intensity correction weight matrix image.

4. The method according to claim 1, characterized in that Before finely segmenting the target image using the target threshold to determine the soft tissue area and the tissue cavity area in the target image, the method further includes: If there is a soft tissue area in the target image, the target threshold is obtained based on the first grayscale value and the second grayscale value.

5. The method according to claim 4, characterized in that The obtaining the target threshold value based on the first grayscale value and the second grayscale value includes: Based on the first grayscale value and the second grayscale value, obtaining an adjustment interval; Selecting one of the grayscale values ​​in the adjustment interval as an adjustment threshold, comparing each pixel of the target image based on the adjustment threshold, and obtaining a tissue mask image corresponding to the adjustment threshold; If the second intersection-and-union result between the tissue mask image and the target tissue mask image satisfies the second set condition, the adjustment threshold is the target threshold; otherwise, the next grayscale value within the adjustment interval is selected as the adjustment threshold until a tissue mask image that satisfies the second set condition is obtained.

6. The method according to any one of claims 1 to 5, characterized in that: The step of roughly segmenting the target image to determine the core tissue area in the target image includes: Obtaining the cumulative probability corresponding to each pixel in the target image; The variance is calculated based on the cumulative probability corresponding to each pixel to obtain multiple inter-class variances; Selecting the largest inter-class variance from the inter-class variances, the threshold corresponding to the largest inter-class variance is the coarse segmentation threshold; The target image is roughly segmented according to the rough segmentation threshold to obtain the core tissue area of ​​the target image.

7. An image segmentation device, characterized in that: include: An image acquisition module, used to acquire a target image of a sample; A coarse segmentation module, used to perform coarse segmentation on the target image and determine the core tissue area in the target image; Obtaining a first grayscale value, a second grayscale value, and a third grayscale value of each pixel in the target image; the first grayscale value refers to a local maximum grayscale value of the target image; the second grayscale value refers to a local minimum grayscale value of the target image; the third grayscale value refers to an average grayscale pixel value of a background area in the target image; using the third grayscale value as a first threshold, comparing each pixel in the target image based on the first threshold, and obtaining a first tissue mask image; reducing the first threshold according to a second set value to obtain a second threshold; Comparing each pixel in the target image based on the second threshold to obtain a second tissue mask image; Performing an intersection-and-union (IoU) calculation on the first tissue mask image and the second tissue mask image to obtain a first IoU result, and if the first IoU result satisfies a first set condition, a soft tissue area exists in the target image; A fine segmentation module, used to perform fine segmentation on the target image using a target threshold value, and determine a soft tissue area and a tissue cavity area in the target image; The target threshold is calculated based on the grayscale value of each pixel of the target image; The image segmentation module is used to obtain the segmentation result of the sample based on the core tissue area, soft tissue area and tissue cavity area in the target image.

8. An electronic device, characterized in that: include: at least one processor and at least one memory, wherein: The memory has computer-readable instructions stored thereon; The computer-readable instructions are executed by one or more of the processors so that the electronic device implements the image segmentation method according to any one of claims 1 to 6.

9. A storage medium having computer-readable instructions stored thereon, characterized in that: The computer-readable instructions are executed by one or more processors to implement the image segmentation method according to any one of claims 1 to 6.

Images