Method for Locating Region of Interest in Target Organ Tissue, Electronic Device, and Storage Medium

Abstract

The present invention provides a method for locating a region of interest in a target organ tissue, an electronic device, and a storage medium. The method includes: segmenting the acquired medical image to be located to obtain a candidate target organ tissue region mask image; obtaining position information of a candidate bounding box of the region of interest in the target organ tissue according to the candidate target organ tissue region mask image; performing location on the medical image to be located by using a deep neural network model according to the position information of the candidate bounding box to obtain position deviation information between a predicted bounding box of the region of interest in the target organ tissue and the candidate bounding box; and obtaining position information of the region of interest in the target organ tissue according to the position deviation information and the position information of the candidate bounding box. The present invention can achieve automatic, fast, and accurate location of the region of interest in the target organ tissue, providing a good basis for subsequent precise segmentation of the target organ tissue region.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method for locating an interested region of a target organ tissue, an electronic device, and a storage medium. Background Art

[0002] The segmentation of the aorta plays a very important role in the medical field. In particular, aortic computed tomography angiography (CTA) has a good screening effect on cardiovascular diseases causing chest pain. At the same time, the post-processing technology of CTA imaging can also accurately diagnose and treat diseases, including the location of the diseased part of the patient, the assessment of the severity, the real-time navigation during surgery, and the follow-up evaluation after surgery. One of the key technologies in CTA image post-processing is to accurately extract the aorta completely. Currently, there is a method of threshold segmentation combined with morphological post-processing for aortic segmentation, but this method often consumes a large amount of time to segment one case of the aorta.

[0003] In recent years, with the increasing maturity of deep learning technology, some deep learning-based segmentation methods have emerged in aortic segmentation. However, since the input images are often three-dimensional images, it is difficult to achieve a good balance between accuracy and time. If the aortic CTA images are directly input into the established three-dimensional model, two problems often occur: one is that too much background information will be introduced. The aortic region often only accounts for a part of the overall scanned region, and direct input will lead to poor segmentation results; the other is that since three-dimensional medical images often have a large data volume, the sliding window method needs to be used during deep learning training and inference. If the CTA images are directly input, it will lead to a long sliding window time, and even if the accuracy meets the requirements, it has no practical application value in clinical practice. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for locating an interested region of a target organ tissue, an electronic device, and a storage medium, which can realize the automatic, rapid, and accurate location of the interested region of the target organ tissue, and provide a good basis for the precise segmentation of the subsequent target organ tissue region.

[0005] To achieve the above purpose, the present invention provides a method for locating an interested region of a target organ tissue, including:

[0006] Segmenting the acquired medical image to be located to obtain a candidate target organ tissue region mask image;

[0007] According to the candidate target organ tissue region mask image, obtaining the position information of the candidate circumscribed box of the interested region of the target organ tissue;

[0008] Based on the position information of the candidate bounding box, use a deep neural network model to localize the medical image to be localized, so as to obtain the position deviation information between the predicted bounding box of the target organ tissue region of interest and the candidate bounding box;

[0009] Based on the position deviation information and the position information of the candidate bounding box, obtain the position information of the target organ tissue region of interest.

[0010] Optionally, segmenting the obtained medical image to be localized to obtain a candidate target organ tissue region mask image includes:

[0011] Perform binary processing on the obtained medical image to be localized by using a preset first upper limit value and a first lower limit value to obtain a binary image;

[0012] Denoise the binary image to obtain a candidate target organ tissue region mask image.

[0013] Optionally, denoising the binary image to obtain a mask image of the candidate target organ tissue region of interest includes:

[0014] Perform morphological opening operation on the binary image to obtain a first image;

[0015] Perform connected component analysis on the first image, and use the extracted largest connected component as the candidate region of the target organ tissue region of interest to obtain a candidate target organ tissue region mask image.

[0016] Optionally, the position information includes center point coordinates, length information, width information, and height information.

[0017] Optionally, before segmenting the obtained medical image to be localized, the localization method further includes:

[0018] Perform downsampling on the obtained medical image to be localized to obtain a reduced medical image to be localized;

[0019] Segmenting the obtained medical image to be localized to obtain a candidate target organ tissue region mask image includes:

[0020] Segment the reduced medical image to be localized to obtain a reduced candidate target organ tissue region mask image;

[0021] Perform upsampling on the reduced candidate target organ tissue region mask image to enlarge the reduced candidate target organ tissue region mask image to the original size of the medical image to be localized, and obtain a candidate target organ tissue region mask image.

[0022] Optionally, before localizing the medical image to be localized using a deep neural network model, the localization method further includes:

[0023] Transform the medical image to be localized to a preset size;

[0024] Perform truncation processing on the medical image to be localized that has been transformed to the preset size, so as to adjust the pixel values of each pixel point in the medical image to be localized within a preset range;

[0025] Normalize the pixel values of each pixel point in the medical image to be localized after the truncation processing;

[0026] The localizing the medical image to be localized using a deep neural network model includes:

[0027] Use a deep neural network model to localize the medical image to be localized after the normalization processing.

[0028] Optionally, the performing truncation processing on the medical image to be localized that has been transformed to the preset size includes:

[0029] Transform the candidate target organ tissue region mask image to the preset size;

[0030] Perform a logical AND operation on the candidate target organ tissue region mask image that has been transformed to the preset size and the medical image to be localized that has been transformed to the preset size, so as to obtain a candidate target organ tissue region of interest image;

[0031] Statistically analyze the pixel values of each pixel point in the candidate target organ tissue region of interest in the candidate target organ tissue region of interest image, so as to determine a second upper limit value and a second lower limit value;

[0032] According to the second upper limit value and the second lower limit value, perform truncation processing on the medical image to be localized that has been transformed to the preset size.

[0033] Optionally, the normalizing the pixel values of each pixel point in the medical image to be localized after the truncation processing includes:

[0034] Normalize the pixel values of each pixel point in the medical image to be localized after the truncation processing according to the following formula:

[0035]

[0036] where P′ i is the pixel value of pixel point i in the medical image to be localized after the normalization processing, and P iis the pixel value of the pixel point i in the to-be-localized medical image after truncation processing, is the average value of the pixel values of the to-be-localized medical image after truncation processing, and σ is the standard deviation of the pixel values of the to-be-localized medical image after truncation processing.

[0037] Optionally, the deep neural network model includes a plurality of cascaded residual modules and at least one fully connected module. The residual module is used to extract the target organ tissue features from the input to-be-localized medical image or the output of the previous-level residual module; the fully connected module is used to perform non-linear mapping regression on the extraction result of the target organ tissue features output by the last-level residual module to obtain the position deviation information between the predicted circumscribed box of the target organ tissue region of interest and the candidate circumscribed box.

[0038] Optionally, each residual module includes a dual-channel unit. The dual-channel unit includes a first sub-channel unit and a second sub-channel unit. The first sub-channel unit includes a plurality of cascaded first convolutional layers, and the second sub-channel unit includes a plurality of cascaded second convolutional layers. The second convolutional layers correspond to the first convolutional layers one by one. Among them, the outputs of the first convolutional layer and the second convolutional layer of the previous level are added and used as the inputs of the first convolutional layer and the second convolutional layer of the next level. The input of the dual-channel unit and the output of the dual-channel unit are added and used as the output of the residual module.

[0039] Optionally, the deep neural network model is trained through the following process:

[0040] Obtain a sample set. The sample set includes multiple samples. Each sample includes a sample medical image and a label corresponding to the sample medical image. The label includes the position information of the candidate circumscribed box and the position information of the ground truth circumscribed box corresponding to the sample medical image;

[0041] Train the deep neural network model according to the sample set.

[0042] Optionally, the training of the deep neural network model according to the sample set includes:

[0043] Determine a first proportion of the samples in the sample set as the training set, and determine a second proportion of the samples as the test set;

[0044] Use the training set to train the deep neural network model;

[0045] Use the test set to test the output accuracy of the deep neural network model;

[0046] If it is determined that the output accuracy rate is less than the preset accuracy rate, continue to train the deep neural network model.

[0047] To achieve the above object, the present invention further provides an electronic device, including a processor and a memory, where a computer program is stored on the memory, and when the computer program is executed by the processor, the method for locating the region of interest of the target organ tissue described above is implemented.

[0048] To achieve the above object, the present invention further provides a readable storage medium, where a computer program is stored in the readable storage medium, and when the computer program is executed by a processor, the method for locating the region of interest of the target organ tissue described above is implemented.

[0049] Compared with the prior art, the method for locating the region of interest of the target organ tissue, the electronic device, and the storage medium provided by the present invention have the following advantages: The present invention first segments the acquired medical image to be located to obtain a candidate target organ tissue region mask image; then, according to the candidate target organ tissue region mask image, obtains the position information of the candidate circumscribed box of the region of interest of the target organ tissue; then, according to the position information of the candidate circumscribed box, uses a deep neural network model to locate the medical image to be located to obtain the position deviation information between the predicted circumscribed box of the region of interest of the target organ tissue and the candidate circumscribed box; finally, according to the position deviation information and the position information of the candidate circumscribed box, the position information of the region of interest of the target organ tissue can be obtained. Thus, the present invention first obtains a candidate circumscribed box of the region of interest of a target organ tissue by initially segmenting the image, then uses a deep neural network model to obtain the position deviation information between the predicted circumscribed box of the region of interest of the target organ tissue and the candidate circumscribed box, and finally obtains the position information of the region of interest of the target organ tissue according to the position deviation information and the position information of the candidate circumscribed box, so as to achieve a balance between the accuracy rate and the positioning efficiency. The obtained position information of the region of interest of the target organ tissue can provide a good basis for the precise segmentation of the subsequent target organ tissue region, effectively reducing the time and labor costs of doctor diagnosis. Description of the Drawings

[0050] Figure 1 It is a schematic flowchart of the method for locating the region of interest of the target organ tissue in an embodiment of the present invention;

[0051] Figure 2a It is a schematic diagram of the medical image to be located in a specific example of the present invention;

[0052] Figure 2b is Figure 2a A schematic diagram of the candidate circumscribed box corresponding to the medical image to be located shown;

[0053] Figure 2c For Figure 2a Schematic diagram of the predicted bounding box of the region of interest of the target organ tissue corresponding to the medical image to be located shown in the figure and the candidate bounding box;

[0054] Figure 3 Schematic diagram of the preprocessing process of the medical image to be located in an embodiment of the present invention;

[0055] Figure 4 Schematic diagram of the truncation process in an embodiment of the present invention;

[0056] Figure 5 Schematic diagram of the overall block structure of the deep neural network model in an embodiment of the present invention;

[0057] Figure 6 Schematic diagram of the block structure of the residual module in an embodiment of the present invention;

[0058] Figure 7 Schematic diagram of the training process of the neural network model in an embodiment of the present invention;

[0059] Figure 8 Schematic diagram of the block structure of the electronic device in an embodiment of the present invention.

[0060] Among them, the reference numerals are as follows:

[0061] Candidate bounding box - 11; Predicted bounding box - 12;

[0062] Residual modules - A, B, C, D; Fully connected modules - E, F;

[0063] First sub-channel unit - 21; Second sub-channel unit - 22; First convolutional layers - 211a, 211b, 211c; Second convolutional layers - 221d, 221e, 221f;

[0064] Processor - 31; Communication interface - 32; Memory - 33; Communication bus - 34. Specific embodiments

[0065] The following further elaborates in detail on the method, electronic device, and storage medium for locating the region of interest of the target organ tissue proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the accompanying drawings are in a very simplified form and all use non-precise scales, only for conveniently and clearly assisting in explaining the purpose of the embodiments of the present invention. In order to make the purpose, features, and advantages of the present invention more obvious and understandable, please refer to the accompanying drawings. It should be noted that the structures, scales, sizes, etc. shown in the drawings of this specification are only used to cooperate with the content disclosed in the specification for those familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention. Any modification of the structure, change in the proportional relationship, or adjustment of the size, in the case of being the same or similar to the effects that the present invention can produce and the purposes that can be achieved, should still fall within the scope covered by the technical content disclosed by the present invention.

[0066] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article, or device comprising the said element.

[0067] The core idea of the present invention is to provide a method, electronic device, and storage medium for locating the region of interest of the target organ tissue, which can achieve automatic, rapid, and accurate location of the region of interest of the target organ tissue, providing a good foundation for the subsequent precise segmentation of the target organ tissue region.

[0068] It should be noted that the method for locating the region of interest of the target organ tissue in the embodiments of the present invention can be applied to the electronic device in the embodiments of the present invention. Among them, the electronic device can be a personal computer, a mobile terminal, etc., and the mobile terminal can be a hardware device such as a mobile phone, a tablet computer, etc. with various operating systems. In addition, it should be noted that although the present invention is described by taking the location of the region of interest of the aorta in the aortic CTA image as an example, as can be understood by those skilled in the art, the present invention can also be used to locate the region of interest of other organ tissues other than the region of interest of the aorta in other medical images, and the present invention does not limit this.

[0069] To achieve the above idea, the present invention provides a method for locating an interested region of a target organ tissue. Please refer to Figure 1 , which schematically shows a flowchart of the method for locating an interested region of a target organ tissue provided by an embodiment of the present invention. As Figure 1 shown, the method for locating an interested region of a target organ tissue includes the following steps:

[0070] Step S100: Segment the acquired medical image to be located to obtain a candidate target organ tissue region mask image.

[0071] Step 200: According to the candidate target organ tissue region mask image, obtain the position information of the candidate bounding box of the interested region of the target organ tissue.

[0072] Step S300: According to the position information of the candidate bounding box, use a deep neural network model to locate the medical image to be located, so as to obtain the position deviation information between the predicted bounding box of the interested region of the target organ tissue and the candidate bounding box.

[0073] Step S400: According to the position deviation information and the position information of the candidate bounding box, obtain the position information of the interested region of the target organ tissue.

[0074] Thus, the present invention first obtains a candidate bounding box of an interested region of a target organ tissue by initially segmenting the image, then uses a deep neural network model to obtain the position deviation information between the predicted bounding box of the interested region of the target organ tissue and the candidate bounding box, and finally obtains the position information of the interested region of the target organ tissue according to the position deviation information and the position information of the candidate bounding box, so as to achieve a balance between accuracy and positioning efficiency. The obtained position information of the interested region of the target organ tissue can provide a good basis for the precise segmentation of the subsequent target organ tissue region, effectively reducing the time and labor costs of doctor diagnosis.

[0075] Please continue to refer to Figures 2a to 2c , where Figure 2a schematically shows a diagram of the medical image to be located in a specific example of the present invention; Figure 2b schematically shows a diagram of the candidate bounding box (the dashed rectangular box in the figure) corresponding to the medical image to be located in a specific example of the present invention; Figure 2c schematically shows a diagram of the predicted bounding box (the solid rectangular box in the figure) of the interested region of the target organ tissue corresponding to the medical image to be located in a specific example of the present invention and the candidate bounding box (the dashed rectangular box in the figure). As Figure 2a and Figure 2bAs shown, by segmenting the medical image to be located (for example, an aortic CTA image), a candidate target organ tissue region mask image can be obtained (in this image, the foreground region ( Figure 2b the gray region in) is the candidate target organ tissue region). By extracting the minimum bounding box that contains the entire candidate target organ tissue region in the candidate target organ tissue region mask image, the position information of the candidate bounding box 11 of the target organ tissue region of interest can be obtained (including the center point coordinates, length information, width information, and height information of the candidate bounding box 11). As Figure 2c shown, by inputting the medical image to be located and the position information of the candidate bounding box 11 into the pre-trained deep neural network model, the position deviation information between the predicted bounding box 12 of the target organ tissue region of interest and the candidate bounding box 11 can be obtained. According to the position deviation information and the position information of the candidate bounding box 11, the position information of the predicted bounding box 12 of the target organ tissue region of interest can be obtained (i.e., the position information of the target organ tissue region of interest). It should be noted that although the present invention is described by taking a rectangular box as the bounding box as an example, as can be understood by those skilled in the art, the bounding box in the present invention can also be a structure similar to a rectangular box, such as a quadrilateral box with rounded corners, or a structure that is not similar to a rectangular box, such as other polygon boxes, circular boxes, or elliptical boxes, etc. The present invention does not limit this.

[0076] Specifically, assume that the center point coordinates of the candidate bounding box are (bx, by, bz), the length of the candidate bounding box is bw, the width is bh, and the height is bd, and the center point coordinates of the predicted rectangular box are (fx, fy, fz), the length of the predicted rectangular box is fw, the width is fh, and the height is fd. Then the position deviation between the predicted bounding box and the candidate bounding box satisfies the following relational expressions:

[0077] dz = (fz - bz) / bd Equation (1)

[0078] dy = (fy - by) / bh Equation (2)

[0079] dx = (fx - bx) / bw Equation (3)

[0080] dd = log(fd / bd) Equation (4)

[0081] dh = log(fh / bh) Equation (5)

[0082] dw = log(fw / bw) Equation (6)

[0083] Wherein, dz represents the deviation between the z coordinate of the center point of the predicted rectangular box and the z coordinate of the center point of the candidate rectangular box; dy represents the deviation between the y coordinate of the center point of the predicted rectangular box and the y coordinate of the center point of the candidate rectangular box; dx represents the deviation between the x coordinate of the center point of the predicted rectangular box and the x coordinate of the center point of the candidate rectangular box; dd represents the deviation between the height of the predicted rectangular box and the height of the candidate rectangular box; dh represents the deviation between the width of the predicted rectangular box and the width of the candidate rectangular box; dw represents the deviation between the length of the predicted rectangular box and the length of the candidate rectangular box. Thus, substituting the position deviation information between the predicted bounding box and the candidate bounding box and the position information of the candidate bounding box into the above formulas (1) to (6), the center point coordinates, length information, width information, and height information of the predicted bounding box can be obtained, so as to obtain the position information of the region of interest of the target organ tissue corresponding to the medical image to be located.

[0084] Further, before segmenting the obtained medical image to be located, the positioning method further includes:

[0085] Performing downsampling processing on the obtained medical image to be located to obtain a downsampled medical image to be located.

[0086] Correspondingly, segmenting the obtained medical image to be located to obtain a candidate target organ tissue region mask image includes:

[0087] Segmenting the downsampled medical image to be located to obtain a downsampled candidate target organ tissue region mask image;

[0088] Performing upsampling processing on the downsampled candidate target organ tissue region mask image to enlarge the downsampled candidate target organ tissue region mask image to the original size of the medical image to be located, and obtaining a candidate target organ tissue region mask image.

[0089] Thus, by first performing downsampling on the acquired medical image to be located to obtain a reduced medical image to be located, and then segmenting the reduced medical image to be located, the computational load can be effectively reduced and the segmentation accuracy can be improved. Specifically, assuming that the original size of the medical image to be located is N1×N2×N3, after downsampling, the size of the reduced medical image to be located is (N1 / R)×(N2 / R)×(N3 / R). Then, when segmenting the reduced medical image to be located, the size of the obtained reduced candidate target organ tissue region mask image is (N1 / R)×(N2 / R)×(N3 / R); by performing upsampling on the reduced candidate target organ tissue region mask image with a corresponding magnification of R times, the reduced candidate target organ tissue region mask image can be enlarged to the original size of the medical image to be located (i.e., N1×N2×N3).

[0090] Furthermore, the segmentation of the acquired medical image to be located (the reduced medical image to be located) to obtain a candidate target organ tissue region mask image (the reduced candidate target organ tissue region mask image) includes:

[0091] Using a preset first upper limit value and first lower limit value, performing binary processing on the acquired medical image to be located (the reduced medical image to be located) to obtain a binary image;

[0092] Performing denoising processing on the binary image to obtain a candidate target organ tissue region mask image (the reduced candidate target organ tissue region mask image).

[0093] Specifically, the first upper limit value and the first lower limit value are determined according to the anatomical prior knowledge of the target organ tissue (such as the aorta). For example, when the first upper limit value is 800 and the first lower limit value is 100, by performing binary processing on the acquired medical image to be located (the reduced medical image to be located), the pixel values of the pixel points with pixel values in the range of 100 to 800 in the medical image to be located (the reduced medical image to be located) can be assigned 1, and the pixel values of the pixel points with pixel values less than 100 or greater than 800 can be assigned 0.

[0094] Even further, the denoising processing of the binary image to obtain a mask image of the candidate target organ tissue region of interest (the reduced candidate target organ tissue region mask image) includes:

[0095] Performing morphological opening operation on the binary image to obtain a first image;

[0096] Perform connected component analysis on the first image, and use the largest connected component extracted as the candidate region of the target organ tissue region of interest to obtain a candidate target organ tissue region mask image (shrink the candidate target organ tissue region mask image).

[0097] Thus, by performing morphological opening operation on the binary image, the candidate region of the target organ tissue region of interest can be separated from other regions, and small interfering regions can be removed to obtain the first image. Since the largest connected component in the first image is the candidate region of the target organ tissue region of interest, thus, by performing connected component analysis on the first image to extract the largest connected component, the candidate target organ tissue region mask image (shrink the candidate target organ tissue region mask image) can be obtained.

[0098] In an exemplary embodiment, before using the deep neural network model to localize the to-be-localized medical image, the method further includes: preprocessing the to-be-localized medical image. Please continue to refer to Figure 3 which schematically shows the preprocessing flow diagram of the to-be-localized medical image provided by an embodiment of the present invention. As Figure 3 shown, the preprocessing of the to-be-localized medical image includes:

[0099] Transform the to-be-localized medical image to a preset size;

[0100] Perform truncation processing on the to-be-localized medical image transformed to the preset size to adjust the pixel values of each pixel point in the to-be-localized medical image to a preset range;

[0101] Normalize the pixel values of each pixel point in the to-be-localized medical image after truncation processing.

[0102] Correspondingly, the using the deep neural network model to localize the to-be-localized medical image includes:

[0103] Use the deep neural network model to localize the to-be-localized medical image after normalization processing.

[0104] Since the deep neural network model requires images of a unified size as input, by transforming the medical image to be located to a preset size, the input requirements of the deep neural network model can be met. Specifically, an image interpolation method of the prior art (such as trilinear interpolation) can be used to transform the medical image to be located to the preset size. For more content about the image interpolation method, reference can be made to the prior art, so it will not be elaborated here. By sequentially performing truncation processing and normalization processing on the pixel values of the pixels of the medical image to be located transformed to the preset size, the normalized medical image to be located can resist attacks of geometric transformation, laying a good foundation for obtaining an accurate positioning result subsequently.

[0105] It should be noted that, as can be understood by those skilled in the art, the position information of the candidate bounding box corresponding to the medical image to be located input to the deep neural network model also needs to be subjected to the same transformation, that is, the position of the candidate bounding box corresponding to the medical image to be located input to the deep neural network model corresponds to the candidate target organ tissue region mask image transformed to the preset size. Since the deep neural network model outputs the position deviation information between the predicted bounding box corresponding to the medical image to be located transformed to the preset size and the candidate bounding box, it is necessary to transform the position deviation information to the original size of the medical image to be located. Then, based on the position deviation information in the original size and the position information of the candidate bounding box in the original size, the position information of the region of interest of the target organ tissue can be obtained.

[0106] Please continue to refer to Figure 4 , which schematically shows a flowchart of the truncation processing provided by an embodiment of the present invention. As Figure 4 shown, the truncation processing of the medical image to be located transformed to the preset size includes:

[0107] Transforming the candidate target organ tissue region mask image to the preset size;

[0108] Performing a logical AND operation on the candidate target organ tissue region mask image transformed to the preset size and the medical image to be located transformed to the preset size to obtain a candidate target organ tissue region of interest image;

[0109] Counting the pixel values of each pixel point of the candidate target organ tissue region of interest in the candidate target organ tissue region of interest image to determine a second upper limit value and a second lower limit value;

[0110] Performing truncation processing on the medical image to be located transformed to the preset size according to the second upper limit value and the second lower limit value.

[0111] Specifically, an image interpolation method in the prior art (such as the nearest neighbor interpolation method) can be adopted to transform the candidate target organ tissue region mask image to the preset size. By performing a logical AND operation on the candidate target organ tissue region mask image transformed to the preset size and the medical image to be located transformed to the preset size, the pixel values of the pixel points in the medical image to be located transformed to the preset size corresponding to the foreground region in the candidate target organ tissue region mask image transformed to the preset size can be kept unchanged, while the pixel values of the pixel points in other regions become 0, so as to obtain an image of the candidate target organ tissue region of interest. By statistically analyzing the pixel values of each pixel point in the candidate target organ tissue region of interest in the image of the candidate target organ tissue region of interest, a second upper limit value and a second lower limit value can be determined according to the statistical results. For example, the pixel value distributed at the 99.5th percentile can be used as the second upper limit value, and the pixel value distributed at the 0.5th percentile can be used as the second lower limit value. Thus, according to the second upper limit value and the second lower limit value, truncation processing is performed on the medical image to be located transformed to the preset size, so that the pixel values of the pixel points in the medical image to be located transformed to the preset size within the range of the second lower limit value and the second upper limit value remain unchanged, the pixel values of the pixel points whose pixel values exceed the second upper limit value become the second upper limit value, and the pixel values of the pixel points whose pixel values are less than the second lower limit value become the second lower limit value, so as to obtain a truncated medical image to be located.

[0112] Further, the normalization processing of the pixel values of each pixel point in the truncated medical image to be located includes:

[0113] The pixel values of each pixel point in the truncated medical image to be located are normalized according to the following formula:

[0114]

[0115] where P′ i is the pixel value of pixel point i in the normalized medical image to be located, P i is the pixel value of pixel point i in the truncated medical image to be located, is the average value of the pixel values of the truncated medical image to be located, and σ is the standard deviation of the pixel values of the truncated medical image to be located.

[0116] Thus, by normalizing the pixel values of each pixel point in the truncated medical image to be located according to the above formula (7), an image after normalization with a pixel mean of 0 and a variance of 1 can be obtained.

[0117] In an exemplary embodiment, the deep neural network model includes a plurality of cascaded residual modules and at least one fully-connected module. The residual modules are used to extract the target organ tissue features from the input medical image to be located or the output of the previous-level residual module. The fully-connected module is used to perform non-linear mapping regression on the extraction result of the target organ tissue features output by the last-level residual module to obtain the position deviation information between the predicted bounding box of the target organ tissue region of interest and the candidate bounding box. Since the deep neural network model in the present invention includes a plurality of cascaded residual modules, it can effectively alleviate the problems of gradient disappearance and gradient explosion that are more likely to occur as the network depth increases, ensure the transmission of effective features, and improve the positioning accuracy.

[0118] Please continue to refer to Figure 5 , which schematically shows the block structure diagram of the deep neural network model provided by a specific example of the present invention. As Figure 5 shown, in this example, the deep neural network model includes 3 cascaded residual modules A, 4 cascaded residual modules B, 6 cascaded residual modules C, 3 cascaded residual modules D, 1 fully-connected module E, and 1 fully-connected module F. Among them, the number of output channels of the residual module A is 64, the number of output channels of the residual module B is 128, the number of output channels of the residual module C is 256, the number of output channels of the residual module D is 512, the number of output channels of the fully-connected module a is 512, and the number of output channels of the fully-connected module b is 6. It should be noted that, as can be understood by those skilled in the art, Figure 5 in the deep neural network model shown, the numbers of the residual module A, the residual module B, the residual module C, the residual module D, and the fully-connected module E are all examples and should not be construed as limitations on the embodiments of the present invention. The numbers of the residual module A, the residual module B, the residual module C, the residual module D, and the fully-connected module E can be set according to specific needs.

[0119] Furthermore, the residual module (residual module A, residual module B, residual module C, residual module D) includes a dual-channel unit, the dual-channel unit includes a first sub-channel unit and a second sub-channel unit, the first sub-channel unit includes a plurality of cascaded first convolutional layers, the second sub-channel unit includes a plurality of cascaded second convolutional layers, the second convolutional layers correspond to the first convolutional layers one by one, wherein, the outputs of the upper-level first convolutional layer and second convolutional layer are added and used as the inputs of the next-level first convolutional layer and second convolutional layer, and the input of the dual-channel unit and the output of the dual-channel unit are added and used as the output of the residual module. Since the residual module includes a dual-channel unit, thus, the residual module can extract target organ tissue features at different scales, further improving the positioning accuracy of the present invention. In addition, since the first sub-channel unit includes a plurality of cascaded first convolutional layers and the second sub-channel includes a plurality of cascaded second convolutional layers, thus, deeper target organ tissue feature information can be extracted at different scales, strengthening the extraction of local and global features of the target organ tissue, which is beneficial to improving the positioning accuracy.

[0120] Please continue to refer to Figure 6 , which schematically shows the block structure diagram of the residual module provided by a specific example of the present invention. As Figure 6 shown, in this example, the first sub-channel unit 21 includes 3 cascaded first convolutional layers, namely the first convolutional layer 211a, the first convolutional layer 211b and the first convolutional layer 211c, and the second sub-channel unit 22 includes 3 cascaded second convolutional layers, namely the second convolutional layer 221d, the second convolutional layer 221e and the second convolutional layer 221f, wherein the size of the convolutional kernel of the first convolutional layer 211 is 3×3×3, and the size of the convolutional kernel of the second convolutional layer 221 is 5×5×5. Specifically, the outputs of the first convolutional layer 211a and the second convolutional layer 221d are added and used as the inputs of the first convolutional layer 211b and the second convolutional layer 221e, the outputs of the first convolutional layer 211b and the second convolutional layer 221e are added and used as the inputs of the first convolutional layer 211c and the second convolutional layer 221f, and the outputs of the first convolutional layer 211c and the second convolutional layer 221f are added to the input of the residual module and used as the output of the residual module.

[0121] It should be noted that, as can be understood by those skilled in the art, Figure 6In the residual module shown, the numbers of the first convolutional layer 211 and the second convolutional layer 221 are both examples and should not be construed as limitations on the embodiments of the present invention. The numbers of the first convolutional layer 211 and the second convolutional layer 221 can be set according to specific needs. In addition, it should be noted that since the first convolutional layer 211 and the second convolutional layer 221 have a one-to-one correspondence, in the residual module provided by the embodiments of the present invention, the number of the first convolutional layer 211 included in the first sub-channel unit 21 is equal to the number of the second convolutional layer 221 included in the second sub-channel unit 22.

[0122] Further, as Figure 6 shown, the outputs of the first convolutional layer 211c and the second convolutional layer 221f (i.e., the outputs of the dual-channel unit) are added to the identity mapping of the input of the dual-channel unit (i.e., the input of the residual module) as the output of the residual module. Thus, the present invention can accelerate the training speed of the model and improve the training effect of the model without increasing additional parameters and computational complexity by adding an identity mapping relationship in the residual module. It should be noted that, as can be understood by those skilled in the art, if the dimensions of the input of the dual-channel unit and the output of the dual-channel unit are inconsistent, a linear projection is first performed, and after obtaining consistent dimensions, addition is performed or 0 is filled in the inconsistent part of the dimensions; if the dimensions of the input of the dual-channel unit and the output of the dual-channel unit are consistent, direct addition is performed.

[0123] In an exemplary embodiment, the deep neural network model is trained through the following process:

[0124] Obtain a sample set, the sample set includes multiple samples, the samples include sample medical images and labels corresponding to the sample medical images, and the labels include the position information of the candidate bounding boxes corresponding to the sample medical images and the position information of the ground truth bounding boxes;

[0125] Train the deep neural network model according to the sample set to obtain a trained deep neural network model.

[0126] It should be noted that, as can be understood by those skilled in the art, the position information of the ground truth bounding boxes corresponding to the sample medical images is obtained from the gold standard images obtained by manually segmenting the sample medical images, and the position information of the candidate bounding boxes corresponding to the sample medical images is obtained from the mask images obtained by segmenting the sample medical images using the segmentation method described above.

[0127] Further, the training of the deep neural network model according to the sample set includes:

[0128] Obtain the initial values of the model parameters of the deep neural network model;

[0129] Train a pre-built deep neural network model according to the sample set and the initial values of the model parameters by using the stochastic gradient descent method until a preset end condition is satisfied.

[0130] Since the training process of the neural network model is actually a process of minimizing the loss function, and taking the derivative can quickly and simply achieve this goal, this method of taking the derivative is the gradient descent method. Therefore, by using the gradient descent method to train the deep neural network model in the present invention, the training of the deep neural network model can be quickly and simply realized.

[0131] In addition, since the training purpose of the neural network model is that the position deviation between the candidate circumscribed box and the predicted circumscribed box of the target organ tissue obtained by the model is close to the position deviation between the candidate circumscribed box and the true circumscribed box, that is, the error between the two is reduced to a certain range. Therefore, the preset training end condition can be that the error value between the position deviation between the candidate circumscribed box and the predicted circumscribed box of the sample medical image and the position deviation between the candidate circumscribed box and the true circumscribed box converges to a preset error value. Specifically, assume that the position deviation between the candidate circumscribed box and the predicted circumscribed box of the i-th sample medical image is d i , and the position deviation between the candidate circumscribed box and the true circumscribed box of the i-th sample medical image is Then the loss function adopted in the training process of the deep neural network model is:

[0132]

[0133] It should be noted that as can be understood by those skilled in the art, the training process of the neural network model is a process of multiple cyclic iterations. Therefore, it is possible to set how many iterations to end the training, that is, the preset training end condition can also be that the number of iterations reaches a preset number of iterations.

[0134] Please continue to refer to Figure 7 , which schematically shows a training process flow diagram of the neural network model provided by an embodiment of the present invention. As Figure 7 shown, the training of the pre-built deep neural network model according to the sample set includes:

[0135] Determine a first proportion of samples from the sample set as the training set, and determine a second proportion of samples as the test set;

[0136] Use the training set to train the deep neural network model;

[0137] Test the output accuracy of the deep neural network model using the test set;

[0138] If it is determined that the output accuracy is less than the preset accuracy, continue to train the deep neural network model.

[0139] Specifically, the first ratio and the second ratio can be set according to specific circumstances. For example, 75% of the samples can be selected from the sample set as the training set, and the remaining 25% of the samples can be selected as the test set. For example, 150 out of 200 samples can be used for model training, and the other 50 samples can be used for model testing. The preset accuracy is set according to specific circumstances. For example, the preset accuracy is set to 95%. Thus, when the output accuracy is less than the preset accuracy, continue with the next batch of training. After all batches of training are completed, if the output accuracy is still less than the preset accuracy, the deep neural network model can be continued to be trained by increasing the number of samples in the sample set until the output accuracy is greater than or equal to the preset accuracy.

[0140] Based on the same inventive concept, the present invention also provides an electronic device. Please refer to Figure 8 , which schematically shows the block structure diagram of the electronic device provided by an embodiment of the present invention. As Figure 8 shown, the electronic device includes a processor 31 and a memory 33. A computer program is stored on the memory 33. When the computer program is executed by the processor 31, the above-mentioned method for locating the region of interest of the target organ tissue is implemented. Since the electronic device provided by the present invention and the method for locating the region of the target organ tissue provided by the present invention belong to the same inventive concept, it has all the advantages of the above-mentioned method for locating the region of the target organ tissue, and thus will not be elaborated herein.

[0141] As Figure 8 shown, the electronic device further includes a communication interface 32 and a communication bus 34. Among them, the processor 31, the communication interface 32, and the memory 33 complete mutual communication through the communication bus 34. The communication bus 34 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 34 can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface 32 is used for communication between the above-mentioned electronic device and other devices.

[0142] The processor 31 referred to in the present invention may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor 31 is the control center of the electronic device, and connects various parts of the entire electronic device through various interfaces and lines.

[0143] The memory 33 can be used to store the computer program. The processor 31 realizes various functions of the electronic device by running or executing the computer program stored in the memory 33 and calling the data stored in the memory 33.

[0144] The memory 33 may include non-volatile and / or volatile memory. The non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. The volatile memory may include random access memory (RAM) or an external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0145] The present invention also provides a readable storage medium with a computer program stored therein. When the computer program is executed by a processor, it can implement the method for localizing the region of interest of the target organ tissue described above. Since the readable storage medium provided by the present invention and the method for localizing the region of the target organ tissue provided by the present invention belong to the same inventive concept, therefore, it has all the advantages of the method for localizing the region of the target organ tissue described above, and thus will not be elaborated herein.

[0146] The readable storage medium according to the embodiment of the present invention may adopt any combination of one or more computer-readable media. The readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer hard disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this article, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0147] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0148] The computer program code for performing the operations of the present invention can be written in one or more programming languages or a combination thereof. The programming languages include object-oriented programming languages - such as Java, Smalltalk, C++, and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0149] In summary, compared with the prior art, the method, electronic device, and storage medium for locating the region of interest of target organ tissue provided by the present invention have the following advantages: The present invention first segments the acquired medical image to be located to obtain a candidate target organ tissue region mask image; then, based on the candidate target organ tissue region mask image, obtains the position information of the candidate bounding box of the region of interest of the target organ tissue; then, based on the position information of the candidate bounding box, uses a deep neural network model to locate the medical image to be located to obtain the position deviation information between the predicted bounding box of the region of interest of the target organ tissue and the candidate bounding box; finally, based on the position deviation information and the position information of the candidate bounding box, the position information of the region of interest of the target organ tissue can be obtained. Thus, the present invention first obtains a candidate bounding box of the region of interest of a target organ tissue by initially segmenting the image, then uses a deep neural network model to obtain the position deviation information between the predicted bounding box of the region of interest of the target organ tissue and the candidate bounding box, and finally, based on the position deviation information and the position information of the candidate bounding box, obtains the position information of the region of interest of the target organ tissue, thereby achieving a balance between accuracy and positioning efficiency. The obtained position information of the region of interest of the target organ tissue can provide a good basis for the precise segmentation of the subsequent target organ tissue region, effectively reducing the time and labor costs of doctor diagnosis.

[0150] It should be noted that the devices and methods disclosed in the embodiments herein can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments herein. In this regard, each block in the flowchart or block diagram may represent a module, program, or part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0151] In addition, each functional module in the various embodiments of this article may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0152] The above description is only a description of the preferred embodiments of the present invention and does not limit the scope of the present invention in any way. Any changes and modifications made by those of ordinary skill in the field of the present invention based on the above disclosure belong to the protection scope of the present invention. Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations fall within the scope of the present invention and its equivalent technologies, the present invention also intends to include these modifications and variations.

Claims

1. A method for locating a region of interest in a target organ tissue, characterized in that, Including: Segmenting the acquired medical image to be located to obtain a candidate target organ tissue region mask image; Obtaining position information of a candidate bounding box of the region of interest of the target organ tissue according to the candidate target organ tissue region mask image; Locating the medical image to be located by using a deep neural network model according to the position information of the candidate bounding box to obtain position deviation information between a predicted bounding box of the region of interest of the target organ tissue and the candidate bounding box; Obtaining position information of the region of interest of the target organ tissue according to the position deviation information and the position information of the candidate bounding box; The deep neural network model includes a plurality of cascaded residual modules and at least one fully connected module. The residual module is used to extract target organ tissue features from the input medical image to be located or the output of the previous-level residual module. The fully connected module is used to perform non-linear mapping regression on the extraction result of the target organ tissue features output by the last-level residual module to obtain position deviation information between a predicted bounding box of the region of interest of the target organ tissue and the candidate bounding box; Each residual module includes a dual-channel unit. The dual-channel unit includes a first sub-channel unit and a second sub-channel unit. The first sub-channel unit includes a plurality of cascaded first convolutional layers. The second sub-channel unit includes a plurality of cascaded second convolutional layers. The second convolutional layers correspond to the first convolutional layers one by one. Wherein, the outputs of the first convolutional layer and the second convolutional layer of the previous level are added and used as the inputs of the first convolutional layer and the second convolutional layer of the next level. The input of the dual-channel unit and the output of the dual-channel unit are added and used as the output of the residual module.

2. The method for locating the region of interest of the target organ tissue according to claim 1, wherein The segmenting the acquired medical image to be located to obtain a candidate target organ tissue region mask image includes: Performing binarization processing on the acquired medical image to be located by using a preset first upper limit value and a first lower limit value to obtain a binary image; Performing denoising processing on the binary image to obtain a candidate target organ tissue region mask image.

3. The method for locating a region of interest in a target organ tissue according to claim 2, wherein The performing denoising processing on the binary image to obtain a mask image of the candidate region of interest of the target organ tissue includes: Performing a morphological opening operation on the binary image to obtain a first image; Performing connected component analysis on the first image, and taking the extracted largest connected component as the candidate region of the region of interest of the target organ tissue to obtain a candidate target organ tissue region mask image.

4. The method for localizing the region of interest of the target organ tissue according to claim 1, wherein The position information includes center point coordinates, length information, width information, and height information.

5. The method for locating the region of interest of the target organ tissue according to claim 1, wherein Before segmenting the acquired medical image to be located, the positioning method further includes: Performing downsampling processing on the acquired medical image to be located to obtain a downsampled medical image to be located; The segmenting the acquired medical image to be located to obtain a candidate target organ tissue region mask image includes: Segmenting the downsampled medical image to be located to obtain a downsampled candidate target organ tissue region mask image; Upsample the masked image of the reduced candidate target organ tissue region to enlarge the masked image of the reduced candidate target organ tissue region to the original size of the medical image to be located, and obtain a masked image of the candidate target organ tissue region.

6. The method for localizing the region of interest of the target organ tissue according to claim 1, wherein Before using the deep neural network model to locate the medical image to be located, the positioning method further includes: Transform the medical image to be located to a preset size; Perform truncation processing on the medical image to be located that has been transformed to the preset size to adjust the pixel values of each pixel point in the medical image to be located within a preset range; Normalize the pixel values of each pixel point in the truncated medical image to be located; The step of using the deep neural network model to locate the medical image to be located includes: Use the deep neural network model to locate the normalized medical image to be located.

7. The method for localizing the region of interest of the target organ tissue according to claim 6, wherein The step of performing truncation processing on the medical image to be located that has been transformed to the preset size includes: Transform the masked image of the candidate target organ tissue region to the preset size; Perform a logical AND operation on the masked image of the candidate target organ tissue region that has been transformed to the preset size and the medical image to be located that has been transformed to the preset size to obtain an image of the region of interest of the candidate target organ tissue; Count the pixel values of each pixel point in the region of interest of the candidate target organ tissue in the image of the region of interest of the candidate target organ tissue to determine a second upper limit value and a second lower limit value; According to the second upper limit value and the second lower limit value, perform truncation processing on the medical image to be located that has been transformed to the preset size.

8. The method for localizing the region of interest of the target organ tissue according to claim 6, wherein The step of normalizing the pixel values of each pixel point in the truncated medical image to be located includes: Normalize the pixel values of each pixel point in the truncated medical image to be located according to the following formula: Among them, P i ' is the pixel value of pixel point i in the medical image to be located after normalization, and P i is the pixel value of pixel point i in the medical image to be located after truncation processing, is the average value of the pixel values of the medical image to be located after truncation processing, and σ is the standard deviation of the pixel values of the medical image to be located after truncation processing.

9. The method for localizing the region of interest of the target organ tissue according to claim 1, wherein The deep neural network model is trained through the following process: Obtain a sample set, the sample set includes multiple samples, each sample includes a sample medical image and a label corresponding to the sample medical image, and the label includes the position information of the candidate bounding box and the position information of the ground truth bounding box corresponding to the sample medical image; Train the deep neural network model according to the sample set.

10. The method for localizing the region of interest of the target organ tissue according to claim 9, wherein, The step of training the deep neural network model according to the sample set includes: From the sample set, determine a first proportion of the samples as the training set and a second proportion of the samples as the test set; Use the training set to train the deep neural network model; Use the test set to test the output accuracy of the deep neural network model; If it is determined that the output accuracy is less than the preset accuracy, continue to train the deep neural network model.

11. An electronic device, characterized in that, It includes a processor and a memory, and a computer program is stored on the memory. When the computer program is executed by the processor, the positioning method according to any one of claims 1 to 10 is implemented.

12. A readable storage medium, characterized in that, The readable storage medium stores a computer program, and when the computer program is executed by a processor, the positioning method described in any one of claims 1 to 10 is implemented.

Images